Deploy and Host LLM Stack on Railway

LLM Stack is a production-ready AI application platform that deploys in one click. It provides 7 pre-configured microservices including LiteLLM (proxy for 100+ LLMs), Open WebUI (chat interface), PostgreSQL with pgvector, Redis, Qdrant vector database, R2R RAG framework, and a React client for building AI-powered applications.

About Hosting LLM Stack

Deploying LLM Stack on Railway is a streamlined, one-click process that automatically provisions and configures all seven microservices. Railway handles service discovery, internal networking, and environment variable configuration automatically. The platform auto-provisions PostgreSQL and Redis plugins, while services like LiteLLM, Open WebUI, Qdrant, and R2R are deployed as containerized applications. Setup takes 5-10 minutes with no manual configuration required. All services communicate seamlessly via Railway's internal network, and you can immediately start building AI applications by adding your LLM provider API keys. Railway manages scaling, monitoring, and infrastructure so you can focus on development.

Common Use Cases

• AI-powered chatbots with support for multiple LLM providers (OpenAI, Anthropic, Azure, etc.)
• Document question-answering systems using RAG (Retrieval-Augmented Generation)
• Knowledge base assistants that search and retrieve information from uploaded documents
• Multi-model AI experimentation and comparison platforms
• Rapid prototyping and MVPs for AI-enabled applications

Dependencies for LLM Stack Hosting

• Railway PostgreSQL Plugin (auto-provisioned with pgvector extension)
• Railway Redis Plugin (auto-provisioned for caching and session management)
• LITELLM_MASTER_KEY (user-provided environment variable for API authentication)
• LLM Provider API Keys (optional: OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.)

Deployment Dependencies

• LiteLLM Documentation - Unified API proxy for 100+ LLM providers
• Open WebUI Documentation - Feature-rich chat interface
• R2R Documentation - Complete RAG pipeline framework
• Qdrant Documentation - High-performance vector database

Implementation Details

The LLM Stack uses a microservices architecture where all services communicate via Railway's internal network:

┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│ React Client│────▶│  Open WebUI  │────▶│  LiteLLM    │
└─────────────┘     └──────────────┘     └─────────────┘
                            │                     │
                            ▼                     ▼
                    ┌──────────────┐     ┌─────────────┐
                    │     R2R      │     │  PostgreSQL │
                    │  (RAG Engine)│     │  + pgvector │
                    └──────────────┘     └─────────────┘
                            │                     │
                            ▼                     ▼
                    ┌──────────────┐     ┌─────────────┐
                    │   Qdrant     │     │    Redis    │
                    │ (Vector DB)  │     │  (Caching)  │
                    └──────────────┘     └─────────────┘

After deployment, access your services via their Railway-provided URLs. The React Client provides a custom frontend, while Open WebUI offers a full-featured chat interface with RAG capabilities.

Why Deploy LLM Stack on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying LLM Stack on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.