Deploy Quickwit

Deploy and Host Quickwit on Railway

Quickwit is an open-source search engine for observability data — the high-volume, write-once, read-rarely logs and traces that make Elasticsearch clusters expensive. Written in Rust on tantivy, it decouples compute from storage: indexes live as immutable "splits" in an S3-compatible bucket, and stateless searchers read only the bytes a query needs. That drops the replicated hot-disk tier which dominates a log stack's cost, while keeping a real inverted index behind it. It speaks the Elasticsearch search API, ingests OpenTelemetry natively, and is a Jaeger backend.

Self-host Quickwit on Railway with the full distributed topology, not one all-in-one node. This template deploys five Quickwit services — searcher, indexer, control plane, metastore and janitor — each running one role of the same pinned image and finding the others by gossip over Railway's private network. Managed PostgreSQL backs the metastore, an object storage bucket holds every index split, and a Caddy gateway takes the public domain and adds the authentication Quickwit does not ship. Each role then scales and restarts independently, as upstream's own Kubernetes chart lays it out.

Getting Started with Quickwit on Railway

Set GATEWAY_PASSWORD at deploy time, or read the generated one from the gateway's variables — that plus the username quickwit is all you need. Open the public URL, answer the browser prompt, and you land on the query editor. Two indexes already exist, otel-logs-v0_9 and otel-traces-v0_9, so OpenTelemetry data has somewhere to go immediately.

The fastest way to confirm everything works is to create an index and put documents in it. POST a doc mapping to /api/v1/indexes, then send newline-delimited JSON to /api/v1//ingest; add ?commit=force to make documents searchable immediately. Switch the Index ID dropdown to your new index and run something like severity:ERROR AND declined. Then open the Cluster page: it should list five ready nodes, confirming gossip formed. The Indexes page shows each index's document count, split count and the s3:// URI its data lives at.

Point an OpenTelemetry collector at https:///api/v1/otlp/v1/logs with an Authorization: Bearer header — the token is checked ahead of the browser prompt, so exporters, Grafana and curl need no interactive auth.

About Hosting Quickwit

Logs and traces are a different workload from the documents Elasticsearch was designed around: they arrive constantly, are almost never updated, and are usually read once during an incident. Quickwit writes them into immutable splits on object storage and keeps only metadata in a database, so retention is a bucket lifecycle question rather than a disk provisioning one.

• Full-text search and aggregations, including percentiles and histograms
• Elasticsearch-compatible _search API, so existing clients and dashboards work
• Native OpenTelemetry ingest, a Jaeger gRPC backend and a Grafana data source
• Schemaless or strictly mapped indexing, with per-index retention policies

The searcher answers queries, serves the web UI and routes ingest — it takes the gateway's normal traffic. The indexer builds splits, holds the write-ahead log on its volume, and is the one role serving the OpenTelemetry endpoints. The control plane assigns indexing work to indexers. The metastore alone talks to PostgreSQL, serving index, split and shard metadata over gRPC. The janitor applies retention, runs delete tasks and collects unreferenced splits.

Why Deploy Quickwit on Railway

Railway removes the operational work this topology would otherwise need:

• Managed PostgreSQL and object storage arrive with the cluster, already wired up
• Private networking and internal DNS let the nodes find each other
• Each role gets its own health check, logs, metrics and restart policy
• Any single role scales without touching the others

Common Use Cases

• Centralised log search for a fleet of services, with an OpenTelemetry collector shipping to the OTLP endpoint and long retention on cheap object storage
• Distributed tracing backend behind Jaeger's UI, replacing a Cassandra or Elasticsearch trace store
• Elasticsearch cost reduction — moving the log tier off replicated SSD while keeping the _search API existing tooling depends on

Dependencies for Quickwit

• quickwit/quickwit:0.9.0 — the search engine; one image runs all five roles
• caddy:2-alpine — the authenticating gateway on the public domain
• Railway PostgreSQL and an object storage bucket — metastore and splits
• Source repo: github.com/gridalpha/quickwit-railway

Environment Variables Reference

Deployment Dependencies

• Quickwit on GitHub
• Quickwit documentation and its node configuration reference
• quickwit/quickwit on Docker Hub

Hardware Requirements for Self-Hosting Quickwit

One indexer on four cores sustains 20–40 MB/s of ingest, or 1–3 TB a day. Scale searchers first: search is latency-bound and they hold no durable state.

Self-Hosting Quickwit

A single node is enough to try locally, with every role enabled:

docker run --rm -p 7280:7280 \
  -v "$(pwd)/qwdata:/quickwit/qwdata" \
  quickwit/quickwit:0.9.0 run

Then create an index and ingest into it over the REST API:

QW=http://localhost:7280

curl -X POST $QW/api/v1/indexes -H 'content-type: application/json' -d '{
  "version": "0.8", "index_id": "app-logs",
  "doc_mapping": {"mode": "dynamic", "timestamp_field": "timestamp",
    "field_mappings": [
      {"name": "timestamp", "type": "datetime",
       "input_formats": ["unix_timestamp"], "fast": true},
      {"name": "message", "type": "text"}]}}'

curl -X POST "$QW/api/v1/app-logs/ingest?commit=force" \
  -H 'content-type: application/x-ndjson' \
  --data-binary '{"timestamp":1788300000,"message":"hello quickwit"}'

curl "$QW/api/v1/app-logs/search?query=hello"

For a cluster, give each node a distinct QW_NODE_ID and a QW_ENABLED_SERVICES role, point QW_METASTORE_URI at PostgreSQL and QW_DEFAULT_INDEX_ROOT_URI at your bucket, and list a couple of stable peers in QW_PEER_SEEDS.

How Much Does Quickwit Cost to Self-Host?

Quickwit is free and open source under the Apache 2.0 licence, with no paid tier, seat limits or feature gates. Self-hosting on Railway costs only the infrastructure the cluster consumes: the containers, PostgreSQL, the volumes and the object storage the splits occupy. Because splits live in a bucket rather than on replicated SSD, that cost tracks retention rather than cluster size.

FAQ

What is Quickwit? An open-source, Rust-based search engine for logs and traces that keeps its indexes on object storage instead of local disks, giving sub-second search at a fraction of an Elasticsearch cluster's cost.

What does this Railway template deploy? Five Quickwit services — searcher, indexer, control plane, metastore and janitor — plus managed PostgreSQL, an object storage bucket, and a Caddy gateway that adds authentication and takes the public domain.

Why does the template include PostgreSQL and a storage bucket? Quickwit separates metadata from data. PostgreSQL is the metastore, recording which indexes and splits exist and which node owns each shard. The bucket holds the splits, which keeps searchers stateless and makes retention a storage question rather than a capacity-planning one.

Does Quickwit have built-in authentication? No — the open-source build ships no user accounts or API keys, which is why every Quickwit node here stays private behind a gateway. Change GATEWAY_PASSWORD and QUICKWIT_API_KEY before sharing the URL, and give collectors the bearer token rather than the password.

How do I scale self-hosted Quickwit when ingest grows? Raise the searcher count first — searchers are stateless and read from the bucket. Spreading one source across several indexers needs a Kafka source; short of that, add indexes rather than indexers. Loki is cheaper if you only filter by label and Elasticsearch is richer as a general-purpose engine; Quickwit sits between them.