Deploy Apache Kafka

Apache Kafka 4.4 in KRaft mode: private plaintext, public SASL, volume

Deploy Apache Kafka

/var/lib/kafka

Deploy and Host Apache Kafka on Railway

Apache Kafka is an open-source distributed event streaming platform. Applications publish records to topics, and other applications read them in order, at their own pace, and can replay them later. It is used for event-driven systems, data pipelines, log aggregation and stream processing.

About Hosting Apache Kafka

This template runs a single Kafka 4.4.0 broker in KRaft mode, so no ZooKeeper is needed. It builds from a public GitHub repository that pins the official apache/kafka image by digest, and stores topic data on a Railway volume. Services in the same project connect over the private network at kafka.railway.internal:9092 without authentication. Clients outside Railway connect through a TCP proxy with SASL/SCRAM and a password generated at deploy time. The broker refuses to start with a public proxy and no password. A small HTTP endpoint on port 8080 reports readiness to Railway's health check. It is a single broker with replication factor 1, suited to development, small production workloads and side projects.

Common Use Cases

  • Event-driven communication between microservices in one Railway project
  • Buffering and replaying events for analytics, ETL or change-data-capture pipelines
  • Collecting logs, metrics or webhook events for later stream processing

Dependencies for Apache Kafka Hosting

  • A Railway volume for topic data (created by the template)
  • A Railway TCP proxy for access from outside Railway (created by the template, optional)

Deployment Dependencies

Implementation Details

Reference the broker from another service in the same project:

KAFKA_BROKERS=${{Kafka.KAFKA_PRIVATE_BOOTSTRAP}}

From outside Railway, use the KAFKA_PUBLIC_BOOTSTRAP variable with these client settings:

security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="kafka" password="YOUR-PASSWORD";

SCRAM never sends the password in clear, but the traffic itself is not encrypted. If you only use Kafka inside Railway, remove the TCP proxy in the service's Networking settings. Broker settings can be added as variables named KAFKA_CFG_SETTING_NAME (upper case, dots as underscores), for example KAFKA_CFG_LOG_RETENTION_BYTES=1073741824 sets log.retention.bytes. The heap defaults to -Xms256m -Xmx1g and can be changed with KAFKA_HEAP_OPTS.

Why Deploy Apache Kafka on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Apache Kafka on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.


Template Content

More templates in this category

View Template
Celery | Web, Worker and Scheduler as Three Services
Web, worker and scheduler wired up: Celery on Redis, outcomes in Postgres.

Templates Guru
0
View Template
Redpanda
Redpanda 26.2: Kafka-compatible streaming, single node, with Kafbat UI.

Agaz Self-Host
2
View Template
smoothmq
A drop-in replacement for AWS SQS

poundifdef
7