
Deploy Simple Kafka
Simple Kafka Setup For Small Projects
Kafka Broker
apache/kafka:latest
Just deployed
Simple Kafka Template
This template provides a straightforward setup for Kafka (with internal and public connectivity), perfect for small projects and lightweight production deployments.
How to Setup
You can connect to Kafka using one of the two provided environment variables:
KAFKA_URL
→ for internal access within your private or production networkKAFKA_PUBLIC_URL
→ for external access (via TCP proxy) from outside the network
In your application, simply use the appropriate environment variable based on your deployment environment.
Example:
# Internal app (same Railway network)
KAFKA_URL=kafka-broker.railway.internal:9092
# External app (internet access)
KAFKA_PUBLIC_URL=random-url.proxy.rlwy.net:12354
After connecting, create your topics in your code — but always check if a topic already exists before creating it.
Services
- Kafka Broker: Uses the
apache/kafka:latest
image. Runs both broker and controller roles in one container for simplicity.
Kafka Broker Environment Variables
KAFKA_NODE_ID="1"
KAFKA_PROCESS_ROLES="broker,controller"
KAFKA_LISTENERS="PLAINTEXT://:9092,PLAINTEXT_HOST://:29092,CONTROLLER://:9093"
KAFKA_ADVERTISED_LISTENERS="PLAINTEXT://${{KAFKA_URL}},PLAINTEXT_HOST://${{KAFKA_PUBLIC_URL}}"
KAFKA_CONTROLLER_LISTENER_NAMES="CONTROLLER"
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP="CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT"
KAFKA_CONTROLLER_QUORUM_VOTERS="1@${{RAILWAY_PRIVATE_DOMAIN}}:9093"
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR="1"
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR="1"
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR="1"
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS="0"
KAFKA_NUM_PARTITIONS="3"
KAFKA_URL="${{RAILWAY_PRIVATE_DOMAIN}}:9092"
KAFKA_PUBLIC_URL="${{RAILWAY_TCP_PROXY_DOMAIN}}:${{RAILWAY_TCP_PROXY_PORT}}"
Explanation of Key Variables
Variable | Description |
---|---|
KAFKA_URL | Internal Kafka endpoint used for private/production connections within the same network. |
KAFKA_PUBLIC_URL | Public TCP proxy endpoint used for external connections (e.g., from local dev or monitoring tools). |
KAFKA_ADVERTISED_LISTENERS | Advertises both internal (PLAINTEXT://${KAFKA_URL} ) and external (PLAINTEXT_HOST://${KAFKA_PUBLIC_URL} ) listeners so Kafka knows how to respond to different clients. |
KAFKA_LISTENERS | Defines the listener ports inside the container: internal (9092 ), external (29092 ), and controller (9093 ). |
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP | Maps each listener to its security protocol — all use PLAINTEXT for simplicity. |
KAFKA_CONTROLLER_QUORUM_VOTERS | Sets the controller configuration for single-broker operation. |
KAFKA_NUM_PARTITIONS | Default partition count for new topics (3 is a balanced default). |
Notes
- Internal services on the same network should always use
KAFKA_URL
. - External clients (like staging or developer laptops) should use
KAFKA_PUBLIC_URL
. - All security protocols are
PLAINTEXT
— not recommended for sensitive data or production-grade clusters without TLS/SASL.
By: Bello Shehu Ango 📧 Email: [email protected] 💻 GitHub: github.com/BelloAngo
Would you like me to add a connection test section (e.g., using kafka-console-producer
and kafka-console-consumer
examples) to verify both internal and public connections? It’s handy for quick debugging.
Template Content
Kafka Broker
apache/kafka:latest