Deploy and Host LiteLLM on Railway

LiteLLM is an open-source, OpenAI-compatible LLM gateway. It unifies 100+ providers behind one /v1/chat/completions API, with virtual keys, routing, spend tracking, and an admin UI, so apps talk to a single proxy instead of each vendor SDK.

About Hosting LiteLLM

Hosting LiteLLM means running the proxy next to PostgreSQL so virtual keys, models, and spend live in the database. The official compose file also offers Prometheus; that sidecar is optional and not required to boot. This template uses the official ghcr.io/berriai/litellm:v1.100.0 image (pinned, as upstream recommends), pins PORT=4000 so Railway healthchecks hit /health/liveliness, and points DATABASE_URL at a private Postgres service over Railway's IPv6 private network. Set STORE_MODEL_IN_DB=True so you can add models from /ui without shipping a config.yaml.

Common Use Cases

Unified LLM API: one OpenAI-compatible endpoint in front of OpenAI, Anthropic, Gemini, Bedrock, and local models
Virtual keys and budgets: issue per-app or per-team keys with spend limits
Fallbacks and routing: retry or load-balance across models when a provider fails
Spend tracking: see cost per key, model, and customer from the admin UI
Self-hosted AI gateway: keep provider keys off client apps and laptops

Dependencies for LiteLLM Hosting

PostgreSQL: required for Admin UI, virtual keys, and STORE_MODEL_IN_DB (included)
Provider API keys: optional at boot; add via /ui or env (OPENAI_API_KEY, ANTHROPIC_API_KEY, …)
No Redis required: not in the official two-service compose

Deployment Dependencies

Implementation Details

The image CMD is --port 4000 and the proxy also reads PORT, so this template pins Railway PORT=4000 and keeps the image's default start command. Prisma migrations run automatically on first boot.

PORT=4000
LITELLM_MASTER_KEY=sk-${{secret(32)}}
LITELLM_SALT_KEY=sk-${{secret(32)}}
DATABASE_URL=${{Postgres.DATABASE_PRIVATE_URL}}
STORE_MODEL_IN_DB=True

Do not rotate LITELLM_SALT_KEY after you store provider credentials. After deploy, open /ui and sign in with username admin and LITELLM_MASTER_KEY as the password. Then add a provider key and a model from the UI, and call POST /v1/chat/completions with the master key or a virtual key.

Why Deploy LiteLLM on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying LiteLLM on Railway, you get a pre-configured LLM gateway with Postgres, generated secrets, health checks, managed SSL, and private networking to your application services.