---
title: "Deploy LiteLLM Proxy | Open Source OpenRouter Alternative"
description: "Self Host LiteLLM. Cost tracking, gateway for 100+ LLMs, chat UI & more"
category: "Other"
url: https://railway.com/deploy/litellm-proxy
---

# Deploy LiteLLM Proxy | Open Source OpenRouter Alternative

Self Host LiteLLM. Cost tracking, gateway for 100+ LLMs, chat UI & more

**[Deploy LiteLLM Proxy | Open Source OpenRouter Alternative on Railway](https://railway.com/template/litellm-proxy)**

- **Creator:** Heimdall
- **Category:** Other
- **Total deploys:** 78

## Template content

### LiteLLM https://cdn.prod.website-files.com/66f2feef6cd182361acd52e8/67fe1c2809428a7beab6cd9b_litellm.jpg

- **Image:** ghcr.io/berriai/litellm-database:main-stable
- **Health check:** /health/readiness
- **Public domain:** Yes

### Redis https://cdn.sanity.io/images/sy1jschh/production/0ce0bfdcfbdbf69662b1116671f97c2dd788b655-157x157.svg

- **Image:** redis:8.2.1
- **Start command:** `/bin/sh -c "rm -rf $RAILWAY_VOLUME_MOUNT_PATH/lost+found/ && exec docker-entrypoint.sh redis-server --requirepass $REDIS_PASSWORD --save 60 1 --dir $RAILWAY_VOLUME_MOUNT_PATH"`

### Postgres https://devicons.railway.app/i/postgresql.svg

- **Image:** ghcr.io/railwayapp-templates/postgres-ssl:18

## Documentation

![LiteLLM logo](https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQ8mTlU5tAgRWxGkvnWRiW49hr3CjRK-795KA&s)

# Deploy and Host LiteLLM on Railway

LiteLLM is an open-source AI gateway that gives you a single OpenAI-compatible endpoint to call 100+ LLM providers — OpenAI, Anthropic, Azure OpenAI, Bedrock, Gemini, Groq, Cohere, Mistral, Ollama, and more. Point your existing apps at the LiteLLM proxy once, and swap providers, route across models, enforce budgets, track spend, and issue per-team virtual keys without touching app code.

This Railway template deploys a production-shaped LiteLLM stack in one click: the `ghcr.io/berriai/litellm-database:main-stable` proxy, a managed Postgres for persistent model/key/spend state, and a managed Redis for shared rate limits and response caching. You self-host LiteLLM on Railway with the master key, salt key, and `STORE_MODEL_IN_DB=True` already wired so you can add models from the Admin UI without redeploying.

## Getting Started with LiteLLM on Railway

Once the deploy finishes, open your Railway-generated URL. The root `/` serves the LiteLLM Swagger API docs — that's normal for LiteLLM, not a misconfiguration.

1. **Log into the Admin UI** — visit `/ui/` and log in with username `admin` and the `LITELLM_MASTER_KEY` value from Railway variables as the password.
2. **Open the Chat UI** — use `/ui/chat.html` directly (or click "Chat UI" in the sidebar from `/ui/`). A known upstream bug in LiteLLM v1.82.3 makes `/ui/chat` (no `.html`) return 404 on direct navigation — the `.html` path always works. See [BerriAI/litellm#24037](https://github.com/BerriAI/litellm/issues/24037).
3. **Add your first model** — go to Models → Add Model, pick a provider, paste your provider key, save. Encrypted and stored in Postgres via `LITELLM_SALT_KEY`.
4. **Create a virtual key** — Keys → Create Key. Scope it to specific models, set a monthly budget, assign it to a team. Ship the key instead of raw provider keys.

![LiteLLM dashboard screenshot](https://res.cloudinary.com/asset-cloudinary/image/upload/v1776371406/dashboard_jua25o.png)

![LiteLLM adding models](https://res.cloudinary.com/asset-cloudinary/image/upload/v1776371406/ddmodel-dashboard_lnadut.png)

![LiteLLM chat ui](https://res.cloudinary.com/asset-cloudinary/image/upload/v1776371406/ui-chat_dzc3p5.png)

## About Hosting LiteLLM

LiteLLM is a drop-in OpenAI-compatible proxy. Apps keep using the OpenAI SDK; LiteLLM translates, routes, retries, and bills. It solves three things vendors won't: unified auth across providers, unified observability, and unified cost control.

Key features:
- OpenAI-format `/v1/chat/completions`, `/v1/embeddings`, `/v1/images/generations` across all providers
- Virtual keys with per-key budgets, rate limits, model allowlists, TTL
- Team/user hierarchy with aggregate spend caps
- Automatic fallbacks, retries, load balancing across deployments of the same model
- Response caching, rate-limit coordination, and budget sync via Redis
- Full request/spend logs in Postgres for audit and cost attribution

## Why Deploy LiteLLM on Railway

- One-click Postgres + Redis — no separate managed services to wire up
- Private Railway networking between proxy, DB, and cache (no public DB exposure)
- HTTPS public URL out of the box — point apps at it immediately
- Scale the proxy independently from the DB when traffic grows
- Pay only for what runs — no per-token gateway markup

## Common Use Cases for Self-Hosted LiteLLM

- **Drop-in OpenAI proxy** — replace `api.openai.com` with your Railway URL, keep every SDK and code path untouched
- **Multi-provider failover** — route `gpt-4o` requests to OpenAI with Anthropic Claude as fallback when OpenAI rate-limits you
- **Per-team budget enforcement** — issue virtual keys to each squad with a `$500/month` cap; LiteLLM hard-stops when exceeded
- **Cross-team cost attribution** — aggregate spend dashboards by user, team, model, and tag for accurate chargeback

## How to Add Models to Self-Hosted LiteLLM

Because `STORE_MODEL_IN_DB=True` is set, models are managed through the Admin UI and persist in Postgres — no `config.yaml` redeploys needed.

**Via Admin UI (recommended):**
1. `/ui/` → **Models** → **Add Model**
2. Choose provider (e.g. `openai`, `anthropic`, `bedrock`, `azure`, `vertex_ai`, `groq`)
3. Enter `Model Name` (what clients will call it, e.g. `gpt-4o`) and `LiteLLM Model Name` (provider-qualified, e.g. `openai/gpt-4o`)
4. Paste provider API key (encrypted at rest using `LITELLM_SALT_KEY`)
5. Save. The model is live on `/v1/models` immediately.

**Via API with master key:**

```
curl -X POST https://your-app.up.railway.app/model/new \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "gpt-4o",
    "litellm_params": {
      "model": "openai/gpt-4o",
      "api_key": "sk-proj-..."
    }
  }'
```

Then call it in OpenAI format from any app:

```
curl https://your-app.up.railway.app/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Hello"}]}'
```

## Dependencies for LiteLLM on Railway

- `litellm` — `ghcr.io/berriai/litellm-database:main-stable` (proxy + Admin UI on port 4000)
- `Postgres` — `ghcr.io/railwayapp-templates/postgres-ssl:18` (model/key/team/spend store)
- `Redis` — Railway-managed Redis 7+ (rate limits, cache, budget sync)

### Deployment Dependencies

  - **Container image:** [`ghcr.io/berriai/litellm-database:main-stable`](https://github.com/BerriAI/litellm/pkgs/container/litellm-database) on      
  - **Official docs:** [docs.litellm.ai](https://docs.litellm.ai/docs/) — proxy config, supported providers, enterprise features
  - **Production guide:** [docs.litellm.ai/docs/proxy/prod](https://docs.litellm.ai/docs/proxy/prod) — the reference this template was tuned against
  - **Admin UI guide:** [docs.litellm.ai/docs/proxy/ui](https://docs.litellm.ai/docs/proxy/ui)
  - **Supported providers:** [docs.litellm.ai/docs/providers](https://docs.litellm.ai/docs/providers) — 100+ providers including OpenAI, Anthropic, Azure, Bedrock, Gemini, Groq, Cohere, Mistral, Ollama, VLLM

### Environment Variables Reference

| Variable | Purpose |
|---|---|
| `LITELLM_MASTER_KEY` | Proxy root auth token (must start with `sk-`). Not an OpenAI key. |
| `LITELLM_SALT_KEY` | Encrypts provider API keys stored in Postgres. Never rotate without re-entering all provider keys. |
| `DATABASE_URL` | Postgres connection — wired to `${{Postgres.DATABASE_URL}}` |
| `REDIS_HOST` / `REDIS_PORT` / `REDIS_PASSWORD` | Redis connection — split vars are ~80 RPS faster than `redis_url` |
| `STORE_MODEL_IN_DB` | `True` lets Admin UI add/edit models without `config.yaml` |
| `LITELLM_MODE` | `PRODUCTION` disables `.env` loading and tightens defaults |

## Hardware Requirements for Self-Hosting LiteLLM

| Resource | Minimum | Recommended |
|---|---|---|
| CPU | 1 vCPU | 2-4 vCPU |
| RAM | 1 GB | 2-4 GB |
| Postgres Storage | 1 GB | 10 GB (logs grow with request volume) |
| Redis Memory | 128 MB | 512 MB |
| Runtime | Python 3.13 | — |

Traffic above ~100 req/s benefits from a second replica and a larger Redis.

## Self-Hosting LiteLLM with Docker

The template mirrors the upstream `docker-compose.yml`. Outside Railway:

```
docker run -d --name litellm \
  -p 4000:4000 \
  -e DATABASE_URL='postgresql://user:pass@host:5432/litellm' \
  -e LITELLM_MASTER_KEY='sk-...' \
  -e LITELLM_SALT_KEY='...' \
  -e STORE_MODEL_IN_DB=True \
  ghcr.io/berriai/litellm-database:main-stable
```

## How Much Does LiteLLM Cost to Self-Host?

LiteLLM is MIT-licensed and free. A paid Enterprise tier adds SSO (Okta/Azure AD), audit logs, JWT auth, and premium support starting at $250/mo, but the open-source build deployed here covers the full proxy, Admin UI, virtual keys, budgets, and spend tracking. On Railway you pay only for container/DB/Redis usage — typically a few dollars a month at low volume.

## FAQ

**What is LiteLLM and why self-host it on Railway?**
LiteLLM is an open-source AI gateway that unifies 100+ LLM providers behind one OpenAI-compatible API. Self-hosting on Railway keeps your request logs, provider API keys, and spend data in infrastructure you control — no third party in the request path.

**What does this Railway template deploy?**
Three services: the LiteLLM proxy (`ghcr.io/berriai/litellm-database:main-stable`), a managed Postgres, and a managed Redis. All wired together with private networking, master key, salt key, and DB-backed model storage pre-configured.

**Why does this template include Postgres and Redis?**
Postgres persists virtual keys, team/user records, added models, provider keys (encrypted), and spend logs across restarts. Redis coordinates rate limits and budgets across replicas and caches identical prompts — LiteLLM's production docs recommend both.

**Is `LITELLM_MASTER_KEY` the same as my OpenAI API key?**
No. `LITELLM_MASTER_KEY` is LiteLLM's own root auth token — it authenticates clients calling your proxy. Your OpenAI/Anthropic/Bedrock provider keys are separate; you add them per-model via the Admin UI.

**How do I add OpenAI or Anthropic to self-hosted LiteLLM?**
Open `/ui/` → Models → Add Model. Pick the provider, set the public model name clients will use, paste the provider API key, save. The key is encrypted with `LITELLM_SALT_KEY` and stored in Postgres.

**Why does `/ui/chat` return 404?**
Known upstream bug in LiteLLM v1.82.3 — Next.js static export missing `chat/index.html`. Use `/ui/chat.html` directly, or navigate from `/ui/` via the sidebar (client-side routing works).

**Can I use self-hosted LiteLLM as a drop-in for `api.openai.com`?**
Yes. Point your OpenAI SDK's `base_url` at your Railway URL and use a LiteLLM virtual key as the `api_key`. Every OpenAI-format endpoint (chat, embeddings, images, audio) is supported.


## Similar templates

- [Rocky Linux](https://railway.com/deploy/rocky-linux) — [Jul'26] Hosted Rocky Linux 9 workspace with SSH and persistent storage. 🚀
- [Foundry Virtual Tabletop](https://railway.com/deploy/X5tR6G) — A Self-Hosted & Modern Roleplaying Platform
- [Letta Code Remote](https://railway.com/deploy/letta-code-remote) — Run a Letta Code agent 24/7. No inbound ports, just deploy.

Open this page in a browser: https://railway.com/deploy/litellm-proxy