Deploy and Host LiteLLM on Railway

LiteLLM is an open-source AI gateway that provides a single OpenAI-compatible endpoint for routing requests across multiple LLM providers such as OpenAI, Anthropic, Azure OpenAI, Bedrock, Gemini, Groq, Cohere, Mistral, Ollama, and more.

This template deploys LiteLLM with PostgreSQL and Redis on Railway, giving you a production-ready AI gateway for managing models, virtual keys, provider credentials, request routing, spend tracking, and caching from one centralized proxy.

About Hosting LiteLLM

Hosting LiteLLM gives you a self-hosted LLM gateway that can sit between your apps and multiple AI providers. Instead of wiring each application directly to OpenAI, Anthropic, Gemini, or other providers, your apps can use one OpenAI-compatible API endpoint from LiteLLM.

This setup is useful for teams, internal tools, automation workflows, AI agents, chat interfaces, and applications that need centralized control over model access, API keys, cost tracking, provider routing, and fallback behavior.

This template uses the LiteLLM database image with PostgreSQL for persistent data and Redis for caching, rate limits, and shared gateway state.

Getting Started

Before deploying this template, you only need to input:

UI_USERNAME=your-admin-username
UI_PASSWORD=your-admin-password

After deployment, open your Railway public domain.

The main route displays the LiteLLM API documentation:

https://your-domain.up.railway.app/

This is normal. LiteLLM serves Swagger / OpenAPI docs on the root path.

To access the LiteLLM Admin UI, open:

https://your-domain.up.railway.app/ui

Log in using the UI_USERNAME and UI_PASSWORD values you entered before deployment.

API Base URL

Use this endpoint as the OpenAI-compatible base URL for your applications:

https://your-domain.up.railway.app/v1

Example application environment:

OPENAI_BASE_URL=https://your-domain.up.railway.app/v1
OPENAI_API_KEY=your-litellm-virtual-key

The API key should be a LiteLLM virtual key generated from the Admin UI, not your raw OpenAI, Anthropic, Gemini, or provider key.

How to Add Your First Model

Because this template stores model configuration in PostgreSQL, you can add and manage models directly from the LiteLLM Admin UI without editing a config file or redeploying the service.

Steps:

1. Open the Admin UI at /ui
2. Go to Models
3. Click Add Model
4. Select your provider
5. Enter the model name your apps will use
6. Enter the provider model name
7. Add your provider API key
8. Save the model

Once saved, the model becomes available from the OpenAI-compatible /v1 endpoint.

How to Create a Virtual Key

Virtual keys let you give applications or users access to LiteLLM without exposing your real provider API keys.

Steps:

1. Open the Admin UI at /ui
2. Go to Keys
3. Create a new key
4. Select allowed models
5. Set budget or limits if needed
6. Use the generated key in your app

Example:

OPENAI_BASE_URL=https://your-domain.up.railway.app/v1
OPENAI_API_KEY=sk-your-litellm-virtual-key

Calling LiteLLM from an App

Example request:

curl https://your-domain.up.railway.app/v1/chat/completions \
  -H "Authorization: Bearer sk-your-litellm-virtual-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "Hello from LiteLLM"
      }
    ]
  }'

Common Use Cases

• Self-hosted OpenAI-compatible AI gateway
• Multi-provider LLM routing
• Centralized API key management
• Virtual keys for teams, users, and internal apps
• Spend tracking and budget control
• AI gateway for n8n, OpenWebUI, AnythingLLM, custom apps, and agents
• Fallback routing between providers
• Response caching and rate limit coordination

Dependencies for LiteLLM Hosting

Deployment Dependencies

This template includes:

• LiteLLM database image
• PostgreSQL
• Redis
• Railway public domain
• Admin UI credentials

Environment Variables

The user only needs to provide:

Other required variables are configured by the template:

Important Paths

Why Deploy LiteLLM on Railway?

Railway makes it simple to run LiteLLM with the required database and cache services in one deployment. The LiteLLM service, PostgreSQL, and Redis can run together without manually wiring external infrastructure.

By deploying LiteLLM on Railway, you get a self-hosted AI gateway with a public HTTPS endpoint, persistent database storage, Redis support, and a web-based Admin UI for managing your models and virtual keys.

Hardware Requirements

For small usage, LiteLLM does not need large resources.

For higher traffic, increase LiteLLM service resources and Redis memory.

FAQ

Why does the root domain show Swagger?

That is normal. LiteLLM serves its API documentation on /.

Use /ui to access the LiteLLM Admin UI.

Where is the Admin UI?

The Admin UI is available at:

https://your-domain.up.railway.app/ui

What username and password should I use?

Use the UI_USERNAME and UI_PASSWORD values you entered before deployment.

Is the LiteLLM virtual key the same as my OpenAI API key?

No. A LiteLLM virtual key is generated inside LiteLLM and used by your apps to call the proxy. Your real provider keys are added separately inside the Admin UI.

Why does this template include PostgreSQL?

PostgreSQL stores LiteLLM data such as models, virtual keys, user/team records, provider configuration, and spend tracking.

Why does this template include Redis?

Redis is used for caching, rate limiting, and shared gateway state.

Does LiteLLM need a volume?

No. LiteLLM itself should run as a stateless service. Persistent data is stored in PostgreSQL and Redis.

Can I use this as a drop-in replacement for OpenAI?

Yes. Point your app's OpenAI-compatible base URL to:

https://your-domain.up.railway.app/v1

Then use a LiteLLM virtual key as the API key.