---
title: "Deploy Kafka [Updated Sep'26]"
description: "Self-host Kafka — event streaming, log replay, no Confluent Cloud bill"
category: "Queues"
url: https://railway.com/deploy/kafka-event-streaming
---

# Deploy Kafka [Updated Sep'26]

Self-host Kafka — event streaming, log replay, no Confluent Cloud bill

**[Deploy Kafka [Updated Sep'26] on Railway](https://railway.com/template/kafka-event-streaming)**

Machine-readable deploy manifest (JSON, validated by TemplateCI): https://railway.com/deploy/kafka-event-streaming/manifest.json

- **Creator:** SB
- **Category:** Queues

## Template content

### Kafka https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQqOKEVxLEPXLVAhlOcrfBnIOTzoCU23BwUMA&s

- **Image:** apache/kafka:latest

## Documentation

# Deploy and Host Apache Kafka on Railway

Apache Kafka is the distributed event streaming platform behind most modern data pipelines — a self-hosted alternative to Confluent Cloud and Amazon MSK. Publish, subscribe, store and replay streams of records, with consumers reading from any offset rather than draining a queue that forgets. This template runs a single-node broker in KRaft mode on the official image, with logs on a volume and listeners wired for private and external clients — a working bootstrap server, not a broker that accepts one connection then fails.

## What This Template Deploys

| Service | Purpose |
| --- | --- |
| `kafka` | Single node in combined broker/controller KRaft mode. Listener on `9092`, controller on `9093`. |
| Volume | Topic partitions, offsets and KRaft metadata. |

One process holds both roles, so there is no ZooKeeper service and no separate quorum to size. Everything durable — partition logs, consumer offsets, the metadata log — lives on the volume, the only thing between a redeploy and an empty broker.

## About Hosting

Kafka fails on a PaaS in ways that produce no useful error, and almost all of them are decided before the first message is produced.

**A regenerated `CLUSTER_ID` bricks an existing volume.** Kafka writes the cluster ID into `meta.properties` on first format. Supply a different ID on a later deploy and the broker finds the mismatch and refuses to start, reporting an inconsistent cluster ID rather than anything pointing at your config. This template fixes the ID so volume and broker stay in agreement.

**`KAFKA_ADVERTISED_LISTENERS` decides whether clients can use the broker at all.** Kafka answers a connection by handing back the address to use afterwards. Advertise the wrong host and clients connect once, get metadata pointing somewhere unreachable, then fail on every produce and fetch. Internal services use the private domain while external clients arrive via TCP proxy on a different host and port — both need advertising.

**The log directory must not be the volume mount root.** Kafka treats every entry in a log directory as a partition, and Railway creates `lost+found` at every volume root. Point `KAFKA_LOG_DIRS` at the mount and the broker aborts on boot. Leave it unset and the image default lands on the ephemeral container layer, where topics vanish on the next deploy without logging anything.

**Single node means replication factor 1 — this is not high availability.** With one broker there is nowhere to replicate to, so the internal offsets and transaction-state topics must be pinned to 1 or they are never created. A restart interrupts consumers and a lost volume is lost data. Fine for development, staging, internal event buses and low-to-medium production; not for anything needing durability across node failure.

**Kafka is a JVM service with a real memory floor.** It is not a $5 sidecar. Budget 1–2 GB of RAM before topics get busy, and treat retention as a disk decision: a week of a chatty topic sizes your volume, not your broker.

Typical cost: **~$20–35/month** for the broker and a 10 GB volume at light traffic, on rates of $10/GB/month RAM, $20/vCPU/month CPU and $0.15/GB/month volumes. Kafka is Apache 2.0 and free.

## How It Compares

| | Kafka (self-hosted) | Confluent Cloud | Amazon MSK | RabbitMQ |
| --- | --- | --- | --- | --- |
| Cost model | Flat infra | Per GB + hourly | Per broker-hour + storage | Flat infra |
| Message model | Replayable log | Replayable log | Replayable log | Queue, consume once |
| Replay from offset | Yes | Yes | Yes | No |
| Self-hostable | Yes | No | No | Yes |

The honest edge: if your workload is task dispatch — one worker takes a job, the job disappears — RabbitMQ is simpler and cheaper, and Kafka is overkill. Kafka wins when you need the log itself: many independent consumers on one stream, replay from an arbitrary offset after a fix, or a buffer that never drops events when a consumer restarts. Against Confluent Cloud and MSK the trade is plainer — you take the operations and keep the bill flat.

## Deploy in Under 5 Minutes

1. Click **Deploy** and pick a workspace. The broker comes up with a fixed `CLUSTER_ID`, a volume and both listeners configured.
2. Wait for the broker to format the log directory on first boot — once, a few seconds.
3. From another Railway service, set `bootstrap.servers` to the broker's private domain on port `9092` — free, and never leaves Railway.
4. For external clients, enable the TCP proxy under **Settings → Networking** and use the host and port it generates.
5. Produce and consume a test message on a throwaway topic to confirm both paths work.

> Verify before you rely on it: redeploy, then consume that topic from the beginning. If the messages are still there, the volume and cluster ID are both correct.

## Common Use Cases

- **Decoupling services** — publish events instead of calling services directly, so a restarting consumer never takes the producer down.
- **Durable event and job logs** — keep a replayable history rather than a queue that forgets; reprocess from any offset after a fix.
- **Streaming pipelines and CDC** — feed change data, clickstreams or telemetry into analytics, search or a warehouse.

## Configuration

| Variable | Required | Description |
| --- | --- | --- |
| `CLUSTER_ID` | Pre-set | Fixed cluster identifier. Never change it once the volume is formatted. |
| `KAFKA_PROCESS_ROLES` | Pre-set | `broker,controller` — combined KRaft mode, no ZooKeeper. |
| `KAFKA_ADVERTISED_LISTENERS` | Required | Addresses clients are told to use. Wrong values break clients after first connect. |
| `KAFKA_LOG_DIRS` | Pre-set | A subdirectory of the volume, never the mount root. |
| `KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR` | Pre-set | `1`. Higher cannot be satisfied by one broker. |
| Storage volume | Pre-set | Persistent volume for partitions, offsets and metadata. |

> **Never change `CLUSTER_ID` on a formatted volume.** The broker compares it against `meta.properties` and refuses to start on a mismatch. Recovery means wiping the volume and losing every topic.

> **Enable the TCP proxy only if you need external clients.** A public PLAINTEXT broker has no authentication. Stay on the private network unless you will configure SASL and TLS yourself.

## Dependencies for Apache Kafka Hosting

- **Railway account** — ~$20–35/month for the broker and a 10 GB volume at light traffic.
- **Bundled services** — none. KRaft mode runs as one process, no ZooKeeper, no external database.
- **Volume** — required. Holds partition logs, offsets and KRaft metadata; without it every topic is lost on redeploy.
- **Optional** — a UI such as Kafbat for browsing topics, and a Prometheus exporter for consumer lag.

### Deployment Dependencies

- [Apache Kafka on GitHub](https://github.com/apache/kafka)
- [Kafka documentation](https://kafka.apache.org/documentation/)
- [KRaft configuration](https://kafka.apache.org/documentation/#kraft)
- [Railway volumes](https://docs.railway.com/volumes)

### Implementation Details

The broker runs the official `apache/kafka` image on a pinned tag in combined KRaft mode, holding both roles in one JVM process. The internal listener serves on `9092`, the controller on `9093`, with the quorum voter list pointing at this node's own private address. Bind addresses leave the host portion empty rather than writing `0.0.0.0`, because Railway's private network is IPv6 and a dual-stack wildcard socket is what makes the broker reachable at its `.railway.internal` name.

Two settings decide whether the deployment survives contact with reality. `KAFKA_LOG_DIRS` points at a subdirectory inside the volume, not the mount root — clear of the `lost+found` entry Railway creates, and off the ephemeral layer the image defaults to. `CLUSTER_ID` is fixed rather than generated, so the value written to `meta.properties` at first format still matches on every later deploy. Offsets and transaction-state replication are pinned to 1, which one broker can satisfy.

For backups the volume is the whole story — partitions, offsets and metadata all live there. Snapshot it on a schedule if the stream is your system of record, and set retention deliberately: the default keeps a week of every topic, which sizes your disk. Durability across node failure needs more brokers and higher replication factors.

## Frequently Asked Questions

**Do I need ZooKeeper?** No. KRaft replaces it with Kafka's own Raft metadata quorum, and Kafka 4.0 removed ZooKeeper entirely. One process holds both roles here.

**How do I connect from another Railway service?** Use the broker's private domain on port `9092` as your bootstrap server. Private networking is free and the traffic never leaves Railway.

**Can I run this in production?** For internal event buses, staging and low-to-medium traffic, yes. One broker means replication factor 1: a restart interrupts consumers and a lost volume is lost data.

**Why won't the broker start after a redeploy?** Almost always a cluster ID mismatch against `meta.properties`, or a log directory pointed at the mount root where Kafka finds `lost+found` and treats it as a malformed partition.

**Is Kafka a drop-in replacement for RabbitMQ?** No. Kafka is a replayable log; RabbitMQ is a queue that removes messages once consumed. For simple task dispatch RabbitMQ is simpler and cheaper. Pick Kafka when you need replay or many consumers on one stream.

## Why Deploy Apache Kafka on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Apache Kafka on Railway you get an event streaming backbone with the traps handled — a fixed cluster ID, a log directory clear of `lost+found`, listeners advertised correctly for private and external clients, and partitions on a volume that survives redeploys.

## Similar templates

- [smoothmq](https://railway.com/deploy/AJv-64) — A drop-in replacement for AWS SQS
- [Kafka UI](https://railway.com/deploy/kafka-ui) — Kafbat UI — Open-source web UI to monitor and manage Apache Kafka clusters
- [Hatchet Lite](https://railway.com/deploy/hatchet-lite) — Hatchet Lite with postgres

Open this page in a browser: https://railway.com/deploy/kafka-event-streaming