Deploy and Host LiteLLM with Railway

LiteLLM is an open-source AI gateway that puts one OpenAI-compatible endpoint in front of 100+ LLM providers. This template deploys the LiteLLM Proxy Server with Postgres and Redis, giving you virtual API keys, per-key budgets, spend tracking, rate limits, fallbacks and response caching — managed from a web dashboard.

About Hosting LiteLLM

LiteLLM Proxy runs as a Python service backed by Postgres for models, keys, budgets and spend history, plus Redis for response caching and cross-replica rate limiting. Self-hosting it means running that database, keeping migrations applied on every upgrade, generating and safeguarding an encryption key for stored provider credentials, and exposing the gateway over TLS. This template wires all three services together with private networking, generates every secret for you, and pins an exact image version so upgrades are deliberate. There are no configuration files and no third-party accounts to create — you add providers and models from the Admin UI after deploying.

Common Use Cases

One endpoint for every coding agent. Point Claude Code, Cursor, Codex or any OpenAI-compatible client at a single URL and one key.
Spend governance for a team. Issue a virtual key per person with its own monthly budget, and see exactly who spent what.
~100 models from one provider key. Add openrouter/* and every OpenRouter model becomes available immediately.
Cost control through caching and fallbacks. Cache repeated responses in Redis, and fail over automatically when a provider errors.
A private gateway for your own app. Reach it over Railway's private network without exposing it publicly.

Dependencies for LiteLLM Hosting

Postgres — stores models, virtual keys, budgets, teams and spend logs.
Redis — response caching and rate-limit coordination across replicas.

Deployment Dependencies

LiteLLM documentation
LiteLLM proxy configuration reference
LiteLLM GitHub repository
Supported providers
OpenRouter (optional — one key for ~100 models)

Implementation Details

Deploying. Click deploy. Every secret is generated for you and there is nothing to fill in. When the deploy goes green, open the service URL.

Logging in. Go to /ui. Username admin, password is your LITELLM_MASTER_KEY — copy it from the litellm service's Variables tab. That same value is the API key your clients use.

Adding your first model. In the Admin UI choose Models → Add Model, pick a provider, paste your provider key. To get ~100 models from a single OpenRouter key, add a model with the literal name openrouter/* and enable drop_params.

Coding agents. Claude Code:

# both values come from the litellm service's Variables tab
export ANTHROPIC_BASE_URL="https://your-service.up.railway.app"
export ANTHROPIC_AUTH_TOKEN="sk-your-litellm-master-key"

Add models in the Admin UI whose names match what the client requests (for example claude-sonnet-4-5-20250929), routed to whichever provider you prefer. Cursor and any OpenAI-compatible tool use the same URL with /v1 and the same key.

Response caching. In the Admin UI open Response Cache and save — REDIS_URL is already wired, so the form arrives pre-filled. Add at least one model before enabling caching; enabling it on an empty proxy leaves the cache inert until a restart.

Virtual keys and budgets. Keys → Create Key, set max_budget and optionally restrict which models it may use. Spend is tracked per key. With OpenRouter, spend is the provider's exact reported cost rather than an estimate.

Prometheus metrics. This is the one feature that still requires a config file, and the template deliberately ships without one — so it takes two manual steps on your own service. Toggling Prometheus in the Admin UI instead appears to succeed but never creates the endpoint, because the route only exists on the config-file load path.

On the litellm service, add a variable LITELLM_CONFIG_YAML (Railway caps a variable at 32,768 characters, which is far more than enough):

litellm_settings:
  callbacks: ["prometheus"]
  require_auth_for_metrics_endpoint: false

Then set Settings → Custom Start Command so the variable is written to a file at boot:

sh -c 'printf %s "$LITELLM_CONFIG_YAML" > /tmp/config.yaml; exec litellm --config /tmp/config.yaml --port 4000'

A new start command only takes effect on a fresh deployment. If you apply it from the CLI, note that railway redeploy replays the previous configuration — the endpoint will still return 401 and it looks broken. Change any variable (or use the dashboard's deploy button) to trigger a real deploy.

Then scrape /metrics/ — with the trailing slash; without it you get a 307. With require_auth_for_metrics_endpoint: false the scrape needs no credentials and returns 83 metric families.

One warning: an empty LITELLM_CONFIG_YAML combined with that start command is a hard crash loop (Exception: Config cannot be None or Empty), so only set the start command once the variable actually holds YAML.

Scaling. LiteLLM opens 10 database connections per process, so the ceiling is replicas × workers × 10 against Postgres's 100 connections — up to 8 replicas.

Before raising Replicas above 1, add a Pre-deploy Command of litellm --skip_server_startup on the litellm service. Each container applies Prisma migrations at startup, so two replicas booting within ~3s of each other can deadlock on a Postgres advisory lock (ERROR: deadlock detected out of ApplyMigrations) — LiteLLM has no application-level migration lock. The pre-deploy command applies migrations exactly once, in its own container, before any replica starts; the replicas then find nothing pending.

Two things that will bite you.

Never change LITELLM_SALT_KEY. It encrypts stored provider credentials. Rotate it and every saved provider key becomes unreadable — your models silently vanish from the API while the service still reports healthy and logs no errors.
Never bump the Postgres major version. Changing the image tag is not an upgrade; the container refuses to start and crash-loops. Your data is safe and setting the tag back restores service, but a real major upgrade needs a dump and restore.

Plan requirement. LiteLLM idles at roughly 0.78 GB and peaks above 1 GB while applying migrations. It is OOM-killed on a 1 GB limit, so Railway's Free and Trial plans cannot run it — Hobby or better is required. The three-service stack costs roughly $9/month at idle, of which the gateway itself is about 87%.

A note on upgrades. The image is pinned to an exact version and does not auto-update, so your deploy will not change under you. Bumping it is a deliberate act — read the changelog first, and prefer staying within a minor line, since LiteLLM's patch releases have historically carried zero schema migrations while every minor adds some.

One specific minor upgrade needs action. LiteLLM's September 2026 release inverts the meaning of an empty model list: today a key or team created with no models listed can reach every model on the proxy; afterwards it can reach none. Nothing in the proxy warns you — affected keys simply start returning authorization errors on every model.

Before taking that upgrade, audit your keys and teams and give each one an explicit model list. To preserve today's behaviour, grant the reserved entry all-proxy-models rather than leaving the list empty:

# find keys that rely on the old "empty means everything" default
curl -s "https://your-service.up.railway.app/key/list?return_full_object=true" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
| jq -r '.keys[] | select((.models // []) | length == 0) | .token'

# then make the intent explicit for each one
for TOKEN in $(curl -s "https://your-service.up.railway.app/key/list?return_full_object=true" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
| jq -r '.keys[] | select((.models // []) | length == 0) | .token'); do
  curl -s -X POST "https://your-service.up.railway.app/key/update" \
    -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
    -d "{\"key\": \"$TOKEN\", \"models\": [\"all-proxy-models\"]}"
done

Why Deploy LiteLLM on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying LiteLLM on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.