Deploy Apache Druid

Deploy and Host Apache Druid on Railway

Apache Druid is a distributed, column-oriented analytics database built for sub-second queries over event data that keeps arriving. Netflix, Confluent and Reddit run it behind dashboards their engineers refresh all day. The obvious alternatives fall down at that job — a row store like Postgres reads far too many bytes per aggregate, and a batch warehouse answers in minutes when the person staring at the chart wants an answer now. Druid gets there with time-partitioned segments, per-column compression, bitmap indexes on every dimension, and optional roll-up at ingestion time.

Deploy Apache Druid on Railway and you get the real multi-process cluster, not a single-box demo. druid-router is the only public service: it serves the web console and proxies queries. druid-broker fans a query out to the data tier and merges results. druid-coordinator runs as both Coordinator and Overlord, balancing segments and scheduling ingestion. druid-historical holds published segments in a volume-backed cache, and druid-middlemanager forks a task process per ingestion job. zookeeper handles discovery, Postgres is the metadata store, and a Railway bucket is deep storage for segments and task logs.

Getting Started with Apache Druid on Railway

Open the druid-router public URL. The browser prompts for HTTP basic credentials: the username is admin, and the password is the DRUID_ADMIN_PASSWORD value generated on druid-coordinator. There is no signup screen and no default password — the account is created from that variable when the cluster first initialises its metadata tables, so change it afterwards through Druid's authentication API, not by editing the variable.

The console lands on the Query view. Open Services first: six rows — coordinator, overlord, router, broker, historical, middle_manager — confirms every process found ZooKeeper and registered itself. Then use Load data — the "Start a new spec" flow walks you through parsing, timestamps, dimensions and roll-up, and accepts inline JSON. When the task reports SUCCESS, Datasources showing "Fully available" means the historical pulled the new segment out of object storage. Return to Query and run SQL against it; POST /druid/v2/sql is the API your dashboards will use.

About Hosting Apache Druid

Druid splits into three server groups, and this template gives each process its own Railway service so they scale independently: the query group (router, broker) receives requests, the data group (historical, middlemanager) stores segments and runs ingestion, and the master group (coordinator/overlord) decides what goes where. Every process is stateless with respect to your data — the authoritative copy of a segment is the ZIP in deep storage — so a node can be recreated without losing anything.

• Sub-second aggregations over billions of rows, via columnar storage and bitmap indexes
• Streaming ingestion from Apache Kafka or Amazon Kinesis, exactly once
• Ingestion-time roll-up that collapses raw events into pre-aggregated rows
• A full SQL layer, a native JSON query API, and a built-in web console

Self-host Apache Druid when your event volume has outgrown a general-purpose database, when dashboards must be interactive rather than scheduled, or when you want a query layer over a Kafka topic with no ETL job in between.

Why Deploy Apache Druid on Railway

Railway removes the fiddly parts of a seven-service cluster:

• Every process wired to ZooKeeper, Postgres and object storage before first boot
• Private networking keeps everything but the console off the public internet
• Managed Postgres, object storage and volumes included
• Basic authentication on from the first request, with no default password

Common Use Cases

• Product and clickstream analytics — funnel, retention and cohort queries over raw event streams, refreshed continuously rather than nightly
• Observability and network telemetry — high-cardinality metrics and flow records that exhaust a time-series database's dimensions
• Ad tech and real-time bidding — impression and spend rollups behind a customer-facing dashboard
• Operational dashboards on Kafka — point a supervisor at a topic, query it seconds later

Dependencies for Apache Druid

• apache/druid:37.0.0 — all five Druid services run this image with a different role as the start command, and must stay on the same tag.
• zookeeper:3.9 — discovery, leader election, task assignment. Druid requires it.
• Railway managed PostgreSQL — segment records, load rules, task history, user accounts.
• Railway object storage bucket — deep storage for segments and task logs.

Deep storage is not optional here — the middlemanager writes a finished segment there and the historical downloads it.

Environment Variables Reference

Deployment Dependencies

• Source: github.com/apache/druid
• Image: hub.docker.com/r/apache/druid
• Docs: druid.apache.org/docs/latest/design
• Runtime: Java 17, in the image

Hardware Requirements for Self-Hosting Apache Druid

Per Railway service, not the cluster as a whole.

Druid uses far more off-heap than heap memory. Raising a broker or historical means raising DRUID_MAXDIRECTMEMORYSIZE too — it must cover (numThreads + numMergeBuffers + 1) × druid_processing_buffer_sizeBytes — and the historical's segment cache cap must fit its volume.

Self-Hosting Apache Druid

The quickest local look is the single-container image, which runs every process at once against embedded Derby and local storage — fine for a laptop, unsuitable for production:

docker run -d --name druid -p 8888:8888 \
  -e DRUID_SINGLE_NODE_CONF=micro-quickstart \
  apache/druid:37.0.0 router

A real cluster runs each role separately. The repository ships a Compose file starting ZooKeeper, Postgres and five Druid containers:

git clone https://github.com/apache/druid.git
cd druid
docker compose -f distribution/docker/docker-compose.yml up -d

That Compose file shares one host directory between three containers for deep storage — the piece you replace in a real deployment. Set druid_storage_type=s3, point druid_storage_bucket and druid_s3_endpoint_url at your object store, and load druid-s3-extensions.

How Much Does Apache Druid Cost to Self-Host?

Apache Druid is free and open source under the Apache 2.0 licence, with no paid edition, seat limits or feature gates — the version you self-host is the complete product. Commercial offerings exist (Imply sells a managed platform), but nothing here depends on them. On Railway you pay only for the compute, volumes and storage the seven services consume.

FAQ

What is Apache Druid? An open-source real-time analytics database. It stores event data in time-partitioned, column-compressed segments with bitmap indexes, so it answers aggregate queries over very large tables in well under a second while still ingesting new rows.

What does this Railway template deploy? Five Druid roles on apache/druid:37.0.0 — router, broker, coordinator, historical and middlemanager — plus zookeeper, a managed PostgreSQL metadata store and an object-storage bucket for deep storage. Only the router is reachable from the internet.

Why does Apache Druid need ZooKeeper and PostgreSQL? ZooKeeper handles live cluster state — which nodes are up, which is leader, which worker gets a task. PostgreSQL holds durable metadata: the segment catalogue, load rules, task history and user accounts. Druid cannot run a cluster without both.

How do I ingest data from Kafka into self-hosted Druid? The druid-kafka-indexing-service extension is already loaded. Submit a supervisor spec through the console's Supervisors tab or POST /druid/indexer/v1/supervisor with your bootstrap servers and topic; the overlord then keeps ingestion tasks running continuously.

Can I scale Apache Druid beyond one node per role? Yes. Brokers, historicals and middlemanagers are horizontally scalable and coordinate through ZooKeeper. Add another Railway service from the same image with its own druid_host and volume; the coordinator balances segments onto it.