Deploy and Host Apache Kafka on Railway

Apache Kafka is the industry-standard distributed event streaming platform — publish, subscribe, store, and replay streams of records at scale, powering everything from microservice event buses to analytics pipelines handling trillions of events a day. This template deploys a single-node Kafka broker in KRaft mode (no ZooKeeper) bundled with Kafka UI, so you get a working broker and a web interface to see topics, messages, and consumer lag from the first deploy.

What This Template Deploys

Both run on Railway's private network. Kafka is reachable internally via RAILWAY_PRIVATE_DOMAIN at zero egress cost, and externally through Railway's TCP proxy. A persistent volume at /var/lib/kafka/data holds topic logs and metadata.

About Hosting Apache Kafka

Kafka in KRaft mode combines the broker and controller into one process (KAFKA_PROCESS_ROLES=broker,controller), removing the ZooKeeper sidecar that used to double the operational surface. That's the easy part. The hard part on any cloud platform is one setting: KAFKA_ADVERTISED_LISTENERS.

When a client connects, Kafka doesn't just serve data — it replies with the address clients should use for later requests. If that advertised address doesn't match how clients actually reach the broker, they connect once, receive metadata pointing at an unreachable host, and then fail on every real operation. On Railway this is sharper than usual: internal traffic uses the private domain on 9092 while external clients come through the TCP proxy on a different host and port. Both listeners must be advertised correctly, or half your clients silently break.

This template wires dual listeners for exactly that — private networking for Railway-to-Railway services and TCP-proxy access for external clients — and bundles Kafka UI so you can actually see whether it's working. A misconfigured broker looks fine until an app fails; the UI makes topic creation, message flow, and consumer lag visible immediately.

Typical cost: ~$5–15/month on Railway for the broker and UI. Confluent Cloud and other managed Kafka services bill per hour plus per GB of throughput and storage, which climbs fast; a self-hosted single node is flat.

How It Compares

Managed Kafka removes operational burden but bills per usage. RabbitMQ is simpler for task queues but isn't built for replayable high-throughput streams. Self-hosted Kafka wins on flat cost and the full open ecosystem — the trade is you run the node.

Deploy in Under 5 Minutes

1. Click Deploy on Railway — Kafka and Kafka UI build automatically (~2 minutes)
2. Open the Kafka UI service's public URL — it connects to the broker automatically
3. Create a topic in the UI and confirm it appears
4. For internal services, connect to ${{Kafka.RAILWAY_PRIVATE_DOMAIN}}:9092
5. For external clients, enable Railway's TCP proxy and use the proxy host and port it provides

No ZooKeeper to run, no listener wiring by hand, no separate UI to set up.

Common Use Cases

• Microservice event bus — decouple services with a durable, replayable event log instead of direct calls
• Real-time data pipelines — stream events into analytics, search, or a data warehouse via Kafka Connect
• Log and metrics aggregation — funnel high-volume telemetry through a buffer that absorbs spikes
• Stream processing — feed Flink, Spark, or ksqlDB for windowed aggregations and joins
• Change data capture — capture database changes and propagate them to downstream systems

Configuration

KAFKA_ADVERTISED_LISTENERS is the setting that breaks Kafka. It must advertise the private domain for internal clients and the TCP-proxy host and port for external ones. If it's wrong, clients connect once, get bad metadata, and fail on every subsequent request. This template wires it for both paths.

Replication factors must be 1 on a single node. With one broker, any replication factor above 1 prevents internal topics from being created and the broker won't function. These are pre-set correctly.

Dependencies for Apache Kafka Hosting

• Railway account — Hobby plan (~$5–15/month) covers the broker and UI
• A persistent Railway volume at /var/lib/kafka/data (included)
• Railway's TCP proxy enabled if external clients need to connect
• Nothing else — no ZooKeeper, no separate coordination service

Deployment Dependencies

• Apache Kafka Docker Image
• Kafka Documentation
• Kafka UI (Kafbat) Repository
• Railway TCP Proxy Documentation

Implementation Details

The template runs the official apache/kafka image in KRaft combined mode — one process holding both broker and controller roles, so no ZooKeeper service is needed. A persistent volume at /var/lib/kafka/data keeps topic logs and cluster metadata across redeploys.

Two listeners are configured: an internal listener on 9092 reachable at RAILWAY_PRIVATE_DOMAIN for zero-egress traffic between Railway services, and an external listener advertised through Railway's TCP proxy for clients outside Railway. KAFKA_ADVERTISED_LISTENERS is set to advertise both correctly, which is the configuration that determines whether clients can actually use the broker rather than just reach it once.

Single-node replication factors are pinned to 1 for the offsets and transaction-state topics, since a lone broker can't satisfy a higher factor and would fail to initialise those internal topics. Kafka UI runs as a second service pointed at the broker over the private network, giving immediate visibility into topics, throughput, and consumer lag — turning an otherwise opaque broker into something you can inspect and debug.

Frequently Asked Questions

Why no ZooKeeper? This template uses KRaft mode, where Kafka manages its own metadata and controller election internally. ZooKeeper is no longer required and is being removed from Kafka entirely, so KRaft is the current standard.

My client connects then fails — why? Almost always KAFKA_ADVERTISED_LISTENERS. The broker tells clients where to reconnect, and if that address doesn't match how they actually reach it, they fail after the first request. Internal clients need the private domain; external ones need the TCP-proxy address.

How do I connect from another Railway service? Use ${{Kafka.RAILWAY_PRIVATE_DOMAIN}}:9092 as the bootstrap server. Traffic stays on the private network at zero egress cost.

How do I connect from outside Railway? Enable Railway's TCP proxy on the Kafka service and use the host and port it assigns as your bootstrap server. The external listener is already advertised for this.

What does Kafka UI give me? A web dashboard to create and browse topics, inspect messages, monitor consumer groups, and watch lag — without installing a separate tool or running CLI commands against the broker.

Do my topics survive a redeploy? Yes. Topic logs and metadata live on the mounted volume at /var/lib/kafka/data.

Is a single node production-ready? For development, staging, and moderate production, yes. It has no broker redundancy, so high-availability production needs a multi-broker cluster with replication above 1.

How long are messages kept? Seven days by default. Adjust KAFKA_LOG_RETENTION_HOURS, keeping in mind longer retention needs a larger volume.

Why Deploy Apache Kafka on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Kafka on Railway you get event streaming with the two hard parts already handled — dual listeners wired for both private and external access, and a Kafka UI to see whether it's actually working. No ZooKeeper, no manual listener configuration, automatic HTTPS on the UI, and a persistent volume for your topics. Stream events at scale on infrastructure you own.