Deploy and Host Ollama + Open WebUI on Railway

This template pairs Open WebUI — the most popular self-hosted AI chat interface — with Ollama, the open-source local model runner, in one private stack. You get a polished, ChatGPT-style UI you fully own, wired to Ollama for running open models and to any cloud provider (OpenAI, Anthropic, and more) through the same interface. On Railway's CPU compute, Ollama is best for small models and embeddings, or pointed at an external GPU host, while cloud APIs handle heavy generation — so you get a flexible private AI workspace without sending your conversations to a third party.

What This Template Deploys

Service	Purpose
Open WebUI	The AI chat interface — multi-model UI, users, and RAG on port `8080`
Ollama	The local model runner and API, wired to Open WebUI privately

Both connect over Railway's private network — Open WebUI reaches Ollama at its internal URL. Persistent volumes hold Open WebUI's data (chats, users, documents) and Ollama's downloaded models, so everything survives redeploys.

About Hosting

This is a two-service private AI stack, and understanding what runs well where makes it genuinely useful — this template is set up honestly around Railway's compute.

Open WebUI is the private frontend — that's the reliable win. You get a full ChatGPT-style interface you own: multi-model chat, user accounts and roles, conversation history, built-in RAG over your documents, and a tool runner — all self-hosted, with no per-user fees and no data sent to a third party. This works great on Railway regardless of where the models run.

Ollama on Railway suits small models and embeddings — size expectations honestly. Railway provides CPU compute, not GPUs, so Ollama here runs small models (1–3B) and embedding models at usable speeds, which is genuinely handy for lightweight chat, testing, and RAG embeddings. Larger models (7B and up) run slowly on CPU and can exceed memory — for fast local inference on big models, point Open WebUI at an external GPU-backed Ollama host via OLLAMA_BASE_URL, or use a cloud API for heavy generation. This template is honest about that so you configure it for good results rather than hitting a wall.

Mix local and cloud in one interface. Set OPENAI_API_KEY or ANTHROPIC_API_KEY and Open WebUI shows those models alongside anything served by Ollama — run a small local model for quick private tasks and switch to a frontier cloud model for heavy work, in the same chat.

Persist both volumes. Open WebUI's data (chats, users, documents, RAG vectors) lives on its volume, and Ollama's pulled models on theirs — both survive redeploys, so you don't re-download multi-gigabyte models or lose history. Model storage grows with each model you pull.

Set the secret key and lock signups. Set WEBUI_SECRET_KEY to a stable random value so sessions stay valid, and ENABLE_SIGNUP=false after creating your admin account so the public can't register. The first account you create becomes the administrator.

Typical cost: ~$5–15/month on Railway for the two services and model storage, plus any cloud LLM usage. Both tools are free and open source.

How It Compares

	This stack (self-hosted)	ChatGPT Plus	Ollama alone (CLI)	Cloud AI SaaS
Interface	Full web UI	Web UI	Command line	Web UI
Local models	Yes (Ollama)	No	Yes	No
Cloud models	Yes (BYO key)	OpenAI only	No	Vendor
Multi-user + RAG	Yes	Limited	No	Varies
Data ownership	Full — your infra	Vendor	Full	Vendor
Self-hostable	Yes	No	Yes	No

ChatGPT Plus is polished but single-provider and keeps your data. Ollama alone is powerful but command-line only, with no UI, users, or RAG. Cloud AI platforms remove setup but bill monthly and hold your conversations. This stack's edge is a complete private AI workspace — a real web UI over both local Ollama models and cloud APIs, with users and RAG — self-hosted, with your chats on infrastructure you own.

Deploy in Under 5 Minutes

Click Deploy on Railway — Open WebUI and Ollama build and wire together (~3 minutes)
Set WEBUI_SECRET_KEY and confirm Open WebUI points at Ollama via OLLAMA_BASE_URL
Optionally add OPENAI_API_KEY or ANTHROPIC_API_KEY for cloud models
Open your Railway URL, create your admin account, and set ENABLE_SIGNUP=false
Pull a small Ollama model (e.g. a 1–3B model), or select a cloud model, and start chatting

For fast local inference on large models, point OLLAMA_BASE_URL at an external GPU host.

Common Use Cases

Private AI chat — a self-hosted ChatGPT-style workspace for your team, no per-user fees
Small local models — run lightweight open models on Railway for quick, fully private tasks
RAG over your documents — upload files and query them with citations, privately
Open WebUI frontend for a GPU host — use Railway for the UI, an external box for heavy inference

Configuration

Variable	Required	Description
`WEBUI_SECRET_KEY`	Required	Signs Open WebUI sessions — set a stable random value
`OLLAMA_BASE_URL`	Pre-set	Points Open WebUI at Ollama (internal, or an external GPU host)
`OPENAI_API_KEY`	Optional	Adds OpenAI models to the interface
`ANTHROPIC_API_KEY`	Optional	Adds Anthropic Claude models
`ENABLE_SIGNUP`	Recommended	`false` after creating your admin account
Storage volumes	Pre-set	Persist Open WebUI data and Ollama models

Size Ollama use to Railway's CPU. Small models (1–3B) and embeddings run fine; for large models, point OLLAMA_BASE_URL at an external GPU host or use a cloud API key. This template wires the services and is honest about what runs well.

Set the secret key and lock signups. Use a stable WEBUI_SECRET_KEY, set ENABLE_SIGNUP=false after your admin account is created, and add cloud API keys if you want frontier models in the mix.

Dependencies for Ollama + Open WebUI Hosting

Railway account — ~$5–15/month for the two services and model storage
Persistent volumes for Open WebUI data and Ollama models (included)
Optional: an OpenAI or Anthropic API key for cloud models
Optional: an external GPU-backed Ollama host for fast large-model inference

Deployment Dependencies

Implementation Details

The template deploys two services over Railway's private network: Open WebUI (ghcr.io/open-webui/open-webui) serving the chat interface on port 8080, and Ollama (ollama/ollama) serving its model API, with Open WebUI configured to reach Ollama through OLLAMA_BASE_URL on the internal network. Persistent volumes store Open WebUI's SQLite data (chats, users, uploaded documents, RAG vectors) and Ollama's downloaded models, so both survive redeploys.

Because Railway provides CPU compute rather than GPUs, Ollama is best suited here to small models (roughly 1–3B) and embedding models, which run at usable speeds; larger models run slowly and may exceed available memory. For fast local inference on large models, OLLAMA_BASE_URL can be pointed at an external GPU-backed Ollama instance, keeping Open WebUI on Railway as the frontend. Cloud providers are supported in parallel: setting OPENAI_API_KEY or ANTHROPIC_API_KEY surfaces those models in the same interface, so local and cloud models are selectable per conversation.

WEBUI_SECRET_KEY signs sessions and should stay stable, ENABLE_SIGNUP controls public registration, and the first account created becomes the administrator. Open WebUI data and Ollama models on the volumes are the components to back up.

Frequently Asked Questions

Can I run large open models on Railway with this? Not fast — Railway is CPU-only, so Ollama here suits small models (1–3B) and embeddings. For large models at speed, point OLLAMA_BASE_URL at an external GPU host, or use a cloud API key for heavy generation. The UI works the same either way.

What runs well on Railway, then? The Open WebUI interface, small local models, embeddings for RAG, and any cloud model via API. It's a great private frontend; heavy local inference just wants a GPU, which this template lets you attach externally.

Can I use both local and cloud models? Yes — add OPENAI_API_KEY or ANTHROPIC_API_KEY, and those models appear alongside Ollama's in the interface. Switch between a local model and GPT or Claude per message.

Does my data persist? Yes — chats, users, and documents on Open WebUI's volume, and pulled models on Ollama's, both surviving redeploys. Back up both to preserve history and avoid re-downloading models.

How do I keep it private? Set WEBUI_SECRET_KEY, create your admin account, and set ENABLE_SIGNUP=false so no one else can register. Your conversations stay on your instance.

Why Deploy Ollama + Open WebUI on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying this stack on Railway you get a private AI chat workspace with the wiring solved — Open WebUI and Ollama connected, persistent volumes for data and models, and cloud providers optional in the same UI. A polished private frontend for small local models and cloud APIs, with heavy inference attachable via an external GPU host, on infrastructure you own.