Deploy LiteLLM — AI Gateway

Deploy and Host LiteLLM Model Gateway on Railway

LiteLLM Model Gateway provides a unified OpenAI-compatible API for connecting applications, AI agents, and automation workflows to multiple LLM providers through a single endpoint.

This template deploys LiteLLM with PostgreSQL and Redis, giving you a persistent AI gateway with model management, virtual API keys, usage tracking, routing, authentication, and a built-in administration interface.

About Hosting LiteLLM Model Gateway

LiteLLM acts as a gateway between your applications and AI providers.

Instead of integrating every provider separately, applications can send requests to one LiteLLM endpoint while LiteLLM handles provider routing, authentication, logging, spend tracking, rate limits, and model configuration.

The LiteLLM Proxy exposes OpenAI-compatible APIs, making it easier to switch or combine providers without rewriting application integrations.

PostgreSQL stores persistent gateway data such as models, virtual keys, users, budgets, and usage information. Redis provides a shared backend for features such as caching and distributed rate-limiting when enabled.

Included Architecture

                Applications / AI Agents
                          │
                          │ OpenAI-compatible API
                          ▼
                ┌─────────────────────┐
                │ LiteLLM Model       │
                │ Gateway             │
                │                     │
                │ Routing / Keys      │
                │ Tracking / Limits   │
                │ Admin UI            │
                └───────┬─────┬───────┘
                        │     │
             ┌──────────┘     └──────────┐
             ▼                           ▼
      ┌──────────────┐            ┌──────────────┐
      │ PostgreSQL   │            │    Redis     │
      │ Persistent   │            │ Cache / Rate │
      │ Gateway DB   │            │ Limit State  │
      └──────────────┘            └──────────────┘
                        │
                        ▼
               AI Model Providers

Common Use Cases

• Connect multiple AI providers through one API
• Provide an OpenAI-compatible gateway for applications
• Centralize LLM credentials and provider configuration
• Generate virtual API keys for users, teams, or applications
• Track AI usage and spend
• Apply budgets and rate limits
• Route requests between different models
• Build a shared AI gateway for internal applications
• Connect AI agents and automation platforms
• Standardize LLM access across development teams
• Switch providers without rewriting client integrations

LiteLLM vs Direct Provider Integration

LiteLLM is particularly useful when several applications or teams need access to multiple model providers without managing separate integrations and credentials in every application.

Supported AI Providers

LiteLLM can connect to a broad ecosystem of hosted and self-hosted model providers.

Examples include:

• OpenAI
• Anthropic
• Azure OpenAI
• Google Gemini
• AWS Bedrock
• Vertex AI
• Groq
• Mistral
• Cohere
• Hugging Face
• Ollama
• OpenAI-compatible endpoints
• Other supported LLM providers

Provider credentials and models can be configured after deployment from the LiteLLM administration interface.

Admin UI

After deployment, open:

https:///ui/

The LiteLLM Admin UI provides management for features such as:

• Models
• Providers
• Virtual keys
• Users and teams
• Budgets
• Spend tracking
• Usage
• Gateway configuration

Login credentials are generated by the template.

Open the LiteLLM service Variables page and use:

UI_USERNAME
UI_PASSWORD

to access the administration interface.

Getting Started

1. Deploy the template.
2. Wait until LiteLLM, PostgreSQL, and Redis are online.
3. Open the public LiteLLM URL followed by /ui/.
4. Get UI_USERNAME and UI_PASSWORD from the LiteLLM Variables page.
5. Sign in to the Admin UI.
6. Add your AI provider credentials.
7. Add one or more models.
8. Create a Virtual Key.
9. Configure your application to use the LiteLLM endpoint.
10. Start sending OpenAI-compatible API requests.

No provider API key is required during the initial Railway deployment. Providers can be added later from LiteLLM.

Using the Gateway API

LiteLLM exposes OpenAI-compatible API endpoints.

A typical chat completion endpoint is:

https:///v1/chat/completions

Applications authenticate using a LiteLLM Master Key or a Virtual Key.

The template automatically generates a gateway master key. You can find it in the LiteLLM service Variables page:

LITELLM_MASTER_KEY

For normal applications and users, creating dedicated Virtual Keys from the Admin UI is preferable to distributing the master key.

Virtual Keys

Virtual Keys allow you to provide separate credentials for applications, users, or teams without sharing provider API keys.

They can be used to apply controls such as:

• Model access
• Budgets
• Usage tracking
• Rate limits
• Team access
• Key expiration

This makes LiteLLM useful as a centralized internal AI access layer rather than only a simple reverse proxy.

PostgreSQL Persistence

PostgreSQL provides the persistent application database for LiteLLM.

It can store gateway information such as:

• Models
• Virtual keys
• Users
• Teams
• Budgets
• Spend records
• Gateway configuration

The official LiteLLM Docker Compose setup uses PostgreSQL and enables database-backed model storage for models added through the UI.

The LiteLLM application itself does not require a persistent filesystem volume for this template because important gateway state is stored in PostgreSQL.

Redis

Redis is included as a shared runtime backend.

Depending on the LiteLLM features enabled, Redis can support:

• Response caching
• Shared rate-limit state
• Multi-instance coordination
• Other distributed gateway functionality

Redis communicates with LiteLLM over Railway private networking and does not need to be publicly exposed.

Health Check

The template monitors LiteLLM using:

/health/liveliness

The official LiteLLM Docker Compose configuration uses the same endpoint to verify that the proxy is alive and responding.

Security Considerations

LiteLLM sits between your applications and AI providers, so gateway credentials should be treated as sensitive.

For production environments:

• Keep the LiteLLM Master Key private.
• Use Virtual Keys for applications and users.
• Keep PostgreSQL and Redis on private networking.
• Do not expose database services publicly unless required.
• Use provider credentials with appropriate permissions.
• Apply budgets and rate limits where appropriate.
• Keep the LiteLLM salt key persistent across deployments.

Changing encryption-related secrets after data has already been stored can prevent LiteLLM from decrypting previously encrypted values.

Dependencies for LiteLLM Model Gateway Hosting

• LiteLLM — unified LLM gateway and Admin UI
• PostgreSQL — persistent gateway database
• Redis — shared cache and rate-limit backend
• Railway Private Networking — internal database and Redis connectivity
• Railway Public Networking — HTTPS access to the gateway and Admin UI
• AI Provider — configured by the user after deployment

Deployment Dependencies

• LiteLLM Documentation: https://docs.litellm.ai/
• LiteLLM GitHub: https://github.com/BerriAI/litellm
• LiteLLM Docker Hub: https://hub.docker.com/r/litellm/litellm-database
• Railway PostgreSQL: https://docs.railway.com/databases/postgresql
• Railway Redis: https://docs.railway.com/databases/redis

Why Deploy LiteLLM Model Gateway on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying LiteLLM Model Gateway on Railway, you get a centralized AI gateway with PostgreSQL persistence, Redis support, virtual keys, model routing, spend tracking, health monitoring, and an OpenAI-compatible API for connecting applications to multiple AI providers.