Deploy LiteLLM

Open-source AI gateway: one OpenAI API for 100+ LLMs, keys and budgets

Deploy LiteLLM

/var/lib/postgresql/data

Deploy and Host LiteLLM on Railway

LiteLLM is an open-source AI gateway. It puts one OpenAI-compatible API in front of more than 100 model providers, including OpenAI, Anthropic, Azure, Bedrock, Vertex AI and Ollama, and adds virtual keys, per-key budgets, rate limits, spend tracking, fallbacks and an Admin UI for teams.

About Hosting LiteLLM

This template runs the LiteLLM proxy v1.109.1, built from the public repo aalfath/litellm-railway-template, which pins the official litellm-database image by digest. A Railway Postgres database stores virtual keys, users, teams, budgets, spend logs and models you add in the Admin UI, and database migrations run automatically at start-up. The master key, the salt key that encrypts provider API keys, and the Admin UI password are all generated at deploy time, so there are no default passwords and nothing to fill in. Railway gives the proxy a public HTTPS domain with a health check. New LiteLLM versions arrive as repo updates you choose when to apply.

Common Use Cases

  • One API endpoint and key format for every LLM provider your apps and teams use
  • Issue virtual keys with budgets, rate limits and model access per user, team or app
  • Track spend and usage across providers, with fallbacks and load balancing between models

Dependencies for LiteLLM Hosting

  • LiteLLM proxy (ghcr.io/berriai/litellm-database:v1.109.1)
  • PostgreSQL (Railway Postgres)

Deployment Dependencies

Implementation Details

After the deploy finishes:

  1. Open https://YOUR-DOMAIN/ui and sign in with username admin and the UI_PASSWORD value from the LiteLLM service's Variables tab.
  2. Add a model under Models + Endpoints with your provider's API key. Models added in the UI are saved in Postgres (STORE_MODEL_IN_DB=True).
  3. Create a virtual key under Virtual Keys and point any OpenAI SDK at https://YOUR-DOMAIN with it.

The LITELLM_MASTER_KEY variable is the admin API key (Authorization: Bearer ...). Keep LITELLM_SALT_KEY unchanged after the first deploy, or saved provider keys can no longer be decrypted. Optional extras such as Redis caching, SSO or a config.yaml can be added later by following the LiteLLM docs.

Why Deploy LiteLLM on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying LiteLLM on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.


Template Content

More templates in this category

View Template
Chat Chat
Chat Chat, your own unified chat and search to AI platform.

okisdev
117
View Template
Weaviate
Vector database with hybrid search and an API key-protected API

GitHub Locker
1
View Template
Möbius Personal AI OS
Self-hosted AI agent that builds and runs apps for you, with memory.

bento
1