Deploy and Host LiteLLM on Railway

LiteLLM is an open-source proxy that puts one OpenAI-compatible API in front of every model provider — OpenAI, Anthropic, Gemini, Bedrock, Azure, Groq, Ollama and around a hundred more — with keys, budgets, rate limits and spend tracking on top.

About Hosting LiteLLM

This template runs the official ghcr.io/berriai/litellm image on a pinned stable tag, paired with a PostgreSQL 18 instance on a persistent Railway volume. Nothing is rebuilt or forked, so upstream releases and upstream security fixes are what you get.

Everything that matters is stored in Postgres rather than in a config file. STORE_MODEL_IN_DB is enabled, which means models, provider credentials, virtual API keys, teams, budgets and spend logs are all managed from the admin UI and survive every redeploy — on a platform with no editable filesystem, that is the difference between a usable proxy and a static one. Provider keys are encrypted at rest with a salt key generated once at deploy time.

The proxy is reachable two ways at once. Your public Railway domain serves the OpenAI-compatible API and the admin UI, and the same process answers on the private network, so other services in your project can call http://litellm.railway.internal:4000/v1/chat/completions without leaving Railway or paying for egress. Postgres stays private — it has no public endpoint at all.

Common Use Cases

One API for every provider: Point any OpenAI SDK at this proxy and switch between GPT, Claude, Gemini or a local model by changing a string, not a client library.
Keys and budgets for a team: Issue per-developer or per-service virtual keys with their own model allowlists and monthly spend caps, instead of sharing one provider key.
A gateway for the rest of your project: Route every service's LLM traffic through one place that logs spend, applies fallbacks and retries, and caches responses.

Dependencies for LiteLLM Hosting

Deployment Dependencies

LiteLLM 1.83 — the proxy itself
PostgreSQL 18 on a Railway volume — model list, virtual keys, budgets and spend logs
An API key from at least one model provider, added through the admin UI after deploy

Implementation Details

The proxy runs under gunicorn with the bind address [::], and that combination is deliberate. LiteLLM's default uvicorn runner sets IPV6_V6ONLY on its socket, so binding :: there would leave the container reachable only over IPv6 — Railway's healthcheck and edge proxy arrive over IPv4 and would never get an answer. Binding 0.0.0.0 instead fixes the public domain but makes the proxy invisible to *.railway.internal. Gunicorn leaves the socket dual-stack, so both work. The brackets matter too: LiteLLM builds its bind string as host:port, and a bare :: yields the invalid :::4000.

Postgres carries no healthcheck. Railway keeps the previous container running until a new one reports healthy, while Postgres holds an exclusive lock on its data directory — with both on the same volume that is a deadlock, and every redeploy would fail. The database is also the reason the volume is mounted at /var/lib/postgresql rather than deeper: the PostgreSQL 18 image places its cluster in /var/lib/postgresql/18/docker, a subdirectory, so initdb never trips over the lost+found entry Railway creates on a fresh volume.

LITELLM_SALT_KEY encrypts the provider credentials stored in the database. It is generated once at deploy time and must not be changed afterwards — rotating it makes every saved provider key undecryptable. LITELLM_MASTER_KEY is the admin credential for the API; treat it as a root key and issue virtual keys for everything else.

The admin UI lives at /ui and signs in with UI_USERNAME / UI_PASSWORD, both pre-filled. Health endpoints are /health/liveliness and /health/readiness, the latter reporting database connectivity.

Why Deploy LiteLLM on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying LiteLLM on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.

Resources: roughly 1 GB RAM for the proxy and 512 MB for Postgres is a comfortable starting point, with a 1 GB volume that can be grown later without downtime. Spend logs are the part that grows — set a retention policy in the UI if you route heavy traffic through it.