Deploy and Host Apache Kafka on Railway

Apache Kafka is the standard distributed event streaming platform: durable, replayable topics that producers and consumers share. This template runs a single Apache Kafka 4.3.1 node in KRaft mode (no ZooKeeper) with a persistent volume, SASL/SCRAM authentication switched on from the first boot, and a public TCP endpoint for Kafka clients.

About Hosting Apache Kafka

This template runs the official apache/kafka image (pinned by digest). One process is both broker and controller. Data and the cluster metadata live on a volume mounted at /var/lib/kafka, so topics, messages, offsets and the login survive redeploys. SASL is enabled at boot: the admin superuser is created with the generated KAFKA_PASSWORD (SCRAM-SHA-256), ACLs are enforced, and unauthenticated clients are rejected. The JVM heap is sized from the container's memory limit (read from the cgroup, capped at 2 GB) and limited to 2 processors so it does not size itself from the host. Two client listeners are advertised: one on the private network (KAFKA_BOOTSTRAP_INTERNAL) for services in the same project, and one through the public TCP proxy (KAFKA_BOOTSTRAP_PUBLIC). Topics are auto-created on first produce with 3 partitions and replication factor 1 (single node). The password is stored in the cluster at first boot; changing KAFKA_PASSWORD later does not change the stored login. Allow about a minute on first boot before clients connect.

Agent contract:

Kafka bootstrap (public): KAFKA_BOOTSTRAP_PUBLIC (TCP proxy, KAFKA_PUBLIC_HOST:KAFKA_PUBLIC_PORT)
Kafka bootstrap (private network): KAFKA_BOOTSTRAP_INTERNAL (kafka.railway.internal:9092 style)
Security: SASL_PLAINTEXT, mechanism SCRAM-SHA-256
Username: admin
Password variable: KAFKA_PASSWORD
No HTTP health path; check readiness with an authenticated kafka-topics.sh --list

cat &gt; client.properties &lt;