Deploy smg

Deploy and Host SMG on Railway

SMG is a high-performance, engine-agnostic model gateway for large-scale LLM deployments. It provides an OpenAI-compatible HTTP endpoint, routes requests across self-hosted or cloud inference workers, supports streaming APIs, and exposes health and metrics endpoints. Deploy this gateway as a secure, scalable control point for model traffic on Railway.

About Hosting SMG on Railway

Railway runs the published Docker image as a single HTTP service with managed deployment, networking, logs, and a generated public domain. The template preserves SMG's upstream entrypoint and uses Railway's generated secret function for API authentication.

Tech Stack

• Rust model gateway with Python launcher
• Docker image lightseekorg/smg:1.13.0
• OpenAI-compatible HTTP APIs
• Prometheus metrics
• Railway managed networking and deployments

Why Deploy SMG on Railway

Railway provides a fast one-click path to a publicly reachable gateway with automatic TLS, environment variables, deployment logs, and straightforward scaling. The pinned upstream image keeps runtime behavior reproducible while allowing users to connect their own inference workers or external providers.

Common Use Cases

• Centralize access to multiple LLM inference workers
• Route OpenAI-compatible requests across vLLM, SGLang, or other backends
• Provide a stable API endpoint for applications and agents
• Monitor gateway health and Prometheus metrics
• Test model routing and gateway integrations without managing Kubernetes

Deployment Notes

The service listens on container port 30000; Railway routes the generated domain to that port through PORT=30000. SMG_API_KEY is generated with ${{secret(32)}} and should be supplied as an API key when calling protected serving and control-plane routes. The gateway starts without inference workers, so /health returns 200 while model requests return no-models errors until workers are registered or configured. The default in-memory history backend is ephemeral and no volume is attached.

Dependencies for SMG on Railway

SMG is deployed as a standalone gateway. Add or register reachable inference workers separately, then configure worker URLs or use the SMG worker management API. No database or cache is required for the default in-memory history mode.

Deployment Dependencies