Name: LiteLLM
Availability: InStock
Author: Will

Deploy and Host LiteLLM with Railway

LiteLLM is an open-source AI gateway that puts one OpenAI-compatible endpoint in front of 100+ LLM providers. This template deploys the LiteLLM Proxy Server with Postgres and Redis, giving you virtual API keys, per-key budgets, spend tracking, rate limits, fallbacks and response caching — managed from a web dashboard.

About Hosting LiteLLM

LiteLLM Proxy runs as a Python service backed by Postgres for models, keys, budgets and spend history, plus Redis for response caching and cross-replica rate limiting. Self-hosting it means running that database, keeping migrations applied on every upgrade, generating and safeguarding an encryption key for stored provider credentials, and exposing the gateway over TLS. This template wires all three services together with private networking, generates every secret for you, pins an exact image version so upgrades are deliberate, applies database migrations once per deploy before any replica starts, and gates each rollout behind a readiness healthcheck so a broken version never takes traffic. There are no configuration files and no third-party accounts to create — you add providers and models from the Admin UI after deploying.

Common Use Cases

One endpoint for every coding agent. Point Claude Code, Cursor, Codex or any OpenAI-compatible client at a single URL and one key.
Spend governance for a team. Issue a virtual key per person with its own monthly budget, and see exactly who spent what.
~100 models from one provider key. Add openrouter/* and every OpenRouter model becomes available immediately.
Cost control through caching and fallbacks. Cache repeated responses in Redis, and fail over automatically when a provider errors.
A private gateway for your own app. Reach it over Railway's private network without exposing it publicly.

Dependencies for LiteLLM Hosting

Postgres — stores models, virtual keys, budgets, teams and spend logs.
Redis — response caching and rate-limit coordination across replicas.

Deployment Dependencies

LiteLLM documentation
LiteLLM proxy configuration reference
LiteLLM GitHub repository
Supported providers
OpenRouter (optional — one key for ~100 models)

Implementation Details

Deploying. Click deploy. Every secret is generated for you and there is nothing to fill in. When the deploy goes green, open the service URL.

Logging in. Go to /ui. Username admin, password is your LITELLM_MASTER_KEY — copy it from the LiteLLM service's Variables tab. That same value is the API key your clients use.

Adding your first model. In the Admin UI choose Models → Add Model, pick a provider, paste your provider key. To get ~100 models from a single OpenRouter key, add a model with the literal name openrouter/* and enable drop_params.

Coding agents. Claude Code:

# both values come from the LiteLLM service's Variables tab
export ANTHROPIC_BASE_URL="https://your-service.up.railway.app"
export ANTHROPIC_AUTH_TOKEN="sk-your-litellm-master-key"

Add models in the Admin UI whose names match what the client requests (for example claude-sonnet-4-5-20250929), routed to whichever provider you prefer. Cursor and any OpenAI-compatible tool use the same URL with /v1 and the same key.

Response caching. In the Admin UI open Response Cache and save — REDIS_URL is already wired, so the form arrives pre-filled. Add at least one model before enabling caching; enabling it on an empty proxy leaves the cache inert until a restart.

Virtual keys and budgets. Keys → Create Key, set max_budget and optionally restrict which models it may use. Spend is tracked per key. With OpenRouter, spend is the provider's exact reported cost rather than an estimate.

Prometheus metrics. The template already ships the plumbing: an optional, blank LITELLM_CONFIG_YAML variable and a start command that writes it to a file at boot when it is set. So this is now a single step — paste the YAML.

On the LiteLLM service set LITELLM_CONFIG_YAML (Railway caps a variable at 32,768 characters, far more than enough):

litellm_settings:
  callbacks: ["prometheus"]
  require_auth_for_metrics_endpoint: false

Leaving it blank is safe — the start command is guarded and falls through to the normal boot path, so a default deploy is unaffected.

Toggling Prometheus in the Admin UI instead appears to succeed but never creates the endpoint, because the route only exists on the config-file load path.

The variable only takes effect on a fresh deployment. If you apply it from the CLI, note that railway redeploy replays the previous configuration, so the endpoint keeps returning 401 and looks broken. Changing any variable triggers a real deploy.

Then scrape /metrics/ — with the trailing slash; without it you get a 307. With require_auth_for_metrics_endpoint: false the scrape needs no credentials and returns 83 metric families.

Scaling. LiteLLM opens 10 database connections per process, so the ceiling is replicas × workers × 10 against Postgres's 100 connections — up to 8 replicas.

Replicas are safe to raise, because the template ships a pre-deploy command (litellm --skip_server_startup) that applies Prisma migrations exactly once, in its own container, before any replica starts. Without it, each container migrates at startup and two replicas booting within ~3s of each other deadlock on a Postgres advisory lock (ERROR: deadlock detected out of ApplyMigrations) — LiteLLM has no application-level migration lock. Verified: two replicas against an empty schema produce exactly one migration run and zero deadlocks.

Zero-downtime deploys. A healthcheck on /health/readiness gates every rollout, so a broken version never takes traffic — the previous deployment keeps serving until the new one is live. Draining is set to 120s so in-flight requests finish. Measured: 260/260 requests returned 200 straight through a redeploy cutover.

Two things that will bite you.

Never change LITELLM_SALT_KEY. It encrypts stored provider credentials. Rotate it and every saved provider key becomes unreadable — your models silently vanish from the API while the service still reports healthy and logs no errors.
Never bump the Postgres major version. Changing the image tag is not an upgrade; the container refuses to start and crash-loops. Your data is safe and setting the tag back restores service, but a real major upgrade needs a dump and restore.

Plan requirement. LiteLLM idles at roughly 0.78 GB and peaks above 1 GB while applying migrations. It is OOM-killed on a 1 GB limit, so Railway's Free and Trial plans cannot run it — Hobby or better is required. The three-service stack costs roughly $9/month at idle, of which the gateway itself is about 87%.

A note on upgrades. The image is pinned to an exact version and does not auto-update, so your deploy will not change under you. Bumping it is a deliberate act — read the changelog first, and prefer staying within a minor line, since LiteLLM's patch releases have historically carried zero schema migrations while every minor adds some.

One announced change is worth pre-empting. A draft deprecation list (discussion #32090) proposes inverting the meaning of an empty model list: today a key or team created with no models listed can reach every model on the proxy; afterwards it would reach none, and nothing in the proxy warns you — affected keys would simply start returning authorization errors on every model.

Treat this as announced, not shipped. As of 2026-09-01 the inversion is not present in any release, nor on main, nor in the v1.99.0 / v1.100.0 release candidates, and the list is explicitly marked non-final — a sibling item in the same batch was merged and then reverted. So there is no deadline to race.

It is still worth acting now, because the fix is a no-op on the current version and forward-compatible: granting the reserved entry all-proxy-models means exactly what an empty list means today, so making the intent explicit costs nothing and cannot break:

# find keys that rely on the old "empty means everything" default
curl -s "https://your-service.up.railway.app/key/list?return_full_object=true" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
| jq -r '.keys[] | select((.models // []) | length == 0) | .token'

# then make the intent explicit for each one
for TOKEN in $(curl -s "https://your-service.up.railway.app/key/list?return_full_object=true" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
| jq -r '.keys[] | select((.models // []) | length == 0) | .token'); do
  curl -s -X POST "https://your-service.up.railway.app/key/update" \
    -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
    -d "{\"key\": \"$TOKEN\", \"models\": [\"all-proxy-models\"]}"
done

Why Deploy LiteLLM on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying LiteLLM on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.