---
title: "Deploy LiteLLM"
description: "LiteLLM: LLM router, virtual keys, budgets, cost tracking, guardrails"
category: "AI/ML"
url: https://railway.com/deploy/Lm9gxI
---

# Deploy LiteLLM

LiteLLM: LLM router, virtual keys, budgets, cost tracking, guardrails

**[Deploy LiteLLM on Railway](https://railway.com/template/Lm9gxI)**

- **Creator:** Will Bogusz's Projects
- **Category:** AI/ML
- **Total deploys:** 305

## Template content

### litellm https://bogusz.co/assets/litellm.png

- **Image:** ghcr.io/berriai/litellm:v1.98.0
- **Public domain:** Yes

### Postgres https://devicons.railway.app/i/postgresql.svg

- **Image:** ghcr.io/railwayapp-templates/postgres-ssl:18

### Redis https://devicons.railway.app/i/redis.svg

- **Image:** redis:8.2
- **Start command:** `/bin/sh -c "exec docker-entrypoint.sh redis-server --requirepass $REDIS_PASSWORD --appendonly no --save ''"`

## Documentation

# Deploy and Host LiteLLM with Railway

LiteLLM is an open-source AI gateway that puts one OpenAI-compatible endpoint in front of 100+ LLM providers. This template deploys the LiteLLM Proxy Server with Postgres and Redis, giving you virtual API keys, per-key budgets, spend tracking, rate limits, fallbacks and response caching — managed from a web dashboard.

## About Hosting LiteLLM

LiteLLM Proxy runs as a Python service backed by Postgres for models, keys, budgets and spend history, plus Redis for response caching and cross-replica rate limiting. Self-hosting it means running that database, keeping migrations applied on every upgrade, generating and safeguarding an encryption key for stored provider credentials, and exposing the gateway over TLS. This template wires all three services together with private networking, generates every secret for you, and pins an exact image version so upgrades are deliberate. There are no configuration files and no third-party accounts to create — you add providers and models from the Admin UI after deploying.

## Common Use Cases

- **One endpoint for every coding agent.** Point Claude Code, Cursor, Codex or any OpenAI-compatible client at a single URL and one key.
- **Spend governance for a team.** Issue a virtual key per person with its own monthly budget, and see exactly who spent what.
- **~100 models from one provider key.** Add `openrouter/*` and every OpenRouter model becomes available immediately.
- **Cost control through caching and fallbacks.** Cache repeated responses in Redis, and fail over automatically when a provider errors.
- **A private gateway for your own app.** Reach it over Railway's private network without exposing it publicly.

## Dependencies for LiteLLM Hosting

- **Postgres** — stores models, virtual keys, budgets, teams and spend logs.
- **Redis** — response caching and rate-limit coordination across replicas.

### Deployment Dependencies

- [LiteLLM documentation](https://docs.litellm.ai/docs/proxy/deploy)
- [LiteLLM proxy configuration reference](https://docs.litellm.ai/docs/proxy/config_settings)
- [LiteLLM GitHub repository](https://github.com/BerriAI/litellm)
- [Supported providers](https://docs.litellm.ai/docs/providers)
- [OpenRouter](https://openrouter.ai/) (optional — one key for ~100 models)

### Implementation Details

**Deploying.** Click deploy. Every secret is generated for you and there is nothing to fill in. When the deploy goes green, open the service URL.

**Logging in.** Go to `/ui`. Username `admin`, password is your `LITELLM_MASTER_KEY` — copy it from the litellm service's Variables tab. That same value is the API key your clients use.

**Adding your first model.** In the Admin UI choose **Models → Add Model**, pick a provider, paste your provider key. To get ~100 models from a single OpenRouter key, add a model with the literal name `openrouter/*` and enable `drop_params`.

**Coding agents.** Claude Code:

```bash
# both values come from the litellm service's Variables tab
export ANTHROPIC_BASE_URL="https://your-service.up.railway.app"
export ANTHROPIC_AUTH_TOKEN="sk-your-litellm-master-key"
```

Add models in the Admin UI whose names match what the client requests (for example `claude-sonnet-4-5-20250929`), routed to whichever provider you prefer. Cursor and any OpenAI-compatible tool use the same URL with `/v1` and the same key.

**Response caching.** In the Admin UI open **Response Cache** and save — `REDIS_URL` is already wired, so the form arrives pre-filled. Add at least one model *before* enabling caching; enabling it on an empty proxy leaves the cache inert until a restart.

**Virtual keys and budgets.** **Keys → Create Key**, set `max_budget` and optionally restrict which models it may use. Spend is tracked per key. With OpenRouter, spend is the provider's exact reported cost rather than an estimate.

**Prometheus metrics.** This is the one feature that still requires a config file, and the
template deliberately ships without one — so it takes two manual steps on your own service.
Toggling Prometheus in the Admin UI instead *appears* to succeed but never creates the
endpoint, because the route only exists on the config-file load path.

On the litellm service, add a variable `LITELLM_CONFIG_YAML` (Railway caps a variable at
32,768 characters, which is far more than enough):

```yaml
litellm_settings:
  callbacks: ["prometheus"]
  require_auth_for_metrics_endpoint: false
```

Then set **Settings → Custom Start Command** so the variable is written to a file at boot:

```bash
sh -c 'printf %s "$LITELLM_CONFIG_YAML" > /tmp/config.yaml; exec litellm --config /tmp/config.yaml --port 4000'
```

A new start command only takes effect on a **fresh** deployment. If you apply it from the CLI,
note that `railway redeploy` replays the *previous* configuration — the endpoint will still
return `401` and it looks broken. Change any variable (or use the dashboard's deploy button) to
trigger a real deploy.

Then scrape `/metrics/` — **with** the trailing slash; without it you get a `307`. With
`require_auth_for_metrics_endpoint: false` the scrape needs no credentials and returns 83
metric families.

One warning: an *empty* `LITELLM_CONFIG_YAML` combined with that start command is a hard crash
loop (`Exception: Config cannot be None or Empty`), so only set the start command once the
variable actually holds YAML.

**Scaling.** LiteLLM opens 10 database connections per process, so the ceiling is
`replicas × workers × 10` against Postgres's 100 connections — up to 8 replicas.

Before raising **Replicas** above 1, add a **Pre-deploy Command** of
`litellm --skip_server_startup` on the litellm service. Each container applies Prisma
migrations at startup, so two replicas booting within ~3s of each other can deadlock on a
Postgres advisory lock (`ERROR: deadlock detected` out of `ApplyMigrations`) — LiteLLM has no
application-level migration lock. The pre-deploy command applies migrations exactly once, in
its own container, before any replica starts; the replicas then find nothing pending.

**Two things that will bite you.**

1. **Never change `LITELLM_SALT_KEY`.** It encrypts stored provider credentials. Rotate it and every saved provider key becomes unreadable — your models silently vanish from the API while the service still reports healthy and logs no errors.
2. **Never bump the Postgres major version.** Changing the image tag is not an upgrade; the container refuses to start and crash-loops. Your data is safe and setting the tag back restores service, but a real major upgrade needs a dump and restore.

**Plan requirement.** LiteLLM idles at roughly 0.78 GB and peaks above 1 GB while applying migrations. It is **OOM-killed on a 1 GB limit**, so Railway's Free and Trial plans cannot run it — Hobby or better is required. The three-service stack costs roughly **$9/month** at idle, of which the gateway itself is about 87%.

**A note on upgrades.** The image is pinned to an exact version and does **not** auto-update,
so your deploy will not change under you. Bumping it is a deliberate act — read the changelog
first, and prefer staying within a minor line, since LiteLLM's patch releases have historically
carried zero schema migrations while every minor adds some.

One specific minor upgrade needs action. LiteLLM's **September 2026** release inverts the
meaning of an empty model list: today a key or team created with no models listed can reach
**every** model on the proxy; afterwards it can reach **none**. Nothing in the proxy warns
you — affected keys simply start returning authorization errors on every model.

Before taking that upgrade, audit your keys and teams and give each one an explicit model
list. To preserve today's behaviour, grant the reserved entry `all-proxy-models` rather than
leaving the list empty:

```bash
# find keys that rely on the old "empty means everything" default
curl -s "https://your-service.up.railway.app/key/list?return_full_object=true" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
| jq -r '.keys[] | select((.models // []) | length == 0) | .token'

# then make the intent explicit for each one
for TOKEN in $(curl -s "https://your-service.up.railway.app/key/list?return_full_object=true" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
| jq -r '.keys[] | select((.models // []) | length == 0) | .token'); do
  curl -s -X POST "https://your-service.up.railway.app/key/update" \
    -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
    -d "{\"key\": \"$TOKEN\", \"models\": [\"all-proxy-models\"]}"
done
```

### Why Deploy LiteLLM on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying LiteLLM on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.


## Similar templates

- [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform.
- [stella](https://railway.com/deploy/stella) — Self-host stella with web, API, Postgres, Redis, and object storage.
- [Hermes Agent | OpenClaw Alternative with Dashboard](https://railway.com/deploy/hermes-agent-or-openclaw-alternative-wit) — Self-Hosted Hermes AI Agent for Telegram, Discord & Slack

Open this page in a browser: https://railway.com/deploy/Lm9gxI