Deploy and Host Open WebUI and Ollama on Railway

This template pairs Open WebUI — the most popular self-hosted AI chat interface — with Ollama, behind authentication, as a private AI workspace you fully own. Every conversation stays on your infrastructure: no third-party logging, no per-user fees, and a login wall in front of the whole thing. Connect cloud models (OpenAI, Anthropic) for heavy work, run small local models or embeddings on Ollama, and point Ollama at an external GPU host when you need serious local inference — all through one polished, private ChatGPT-style UI.

What This Template Deploys

Service	Purpose
Open WebUI	The authenticated AI chat interface — multi-model UI, users, RAG (port 8080)
Ollama	The local model runner and API, wired to Open WebUI privately

Both connect over Railway's private network — Open WebUI reaches Ollama at its internal URL, never exposing it publicly. Persistent volumes hold Open WebUI's data (chats, users, documents) and Ollama's models, so everything survives redeploys.

About Hosting

This is a private, authenticated AI workspace, and understanding what runs well where — plus keeping it locked down — is the key. This template is set up honestly around Railway's compute.

Authenticated and private — that's the point. The reason to run this over a public chatbot is control: Open WebUI sits behind a login (set WEBUI_SECRET_KEY and create the first admin, then set ENABLE_SIGNUP=false), and every conversation, document, and model call stays on your Railway infrastructure rather than a vendor's servers. No per-seat fees, no training on your data, no third party in the loop. It's a genuine private ChatGPT alternative for you or your team.

Open WebUI is the reliable win — the private frontend. You get a full ChatGPT-style interface: multi-model chat, user accounts and roles, conversation history, built-in RAG over your documents, and a tool runner — all self-hosted. This works great on Railway regardless of where the models actually run, which is the durable value here.

Ollama on Railway suits small models and embeddings — sized honestly. Railway provides CPU compute, not GPUs, so Ollama here runs small models (1–3B) and embedding models at usable speeds — handy for lightweight chat, testing, and RAG embeddings. Larger models (7B and up) run slowly on CPU and can exceed memory. For fast local inference on big models, point Open WebUI at an external GPU-backed Ollama host via OLLAMA_BASE_URL, or use a cloud API for heavy generation. This template is honest about that so you configure it for good results rather than hitting a wall.

Mix cloud and local in one private UI. Set OPENAI_API_KEY or ANTHROPIC_API_KEY and those frontier models appear alongside anything Ollama serves — run a small local model for quick private tasks, and switch to a cloud model for heavy work, all behind the same login.

Persist both volumes. Open WebUI's data (chats, users, documents, RAG vectors) lives on its volume, and Ollama's models on theirs — both survive redeploys, so you don't lose history or re-download multi-gigabyte models.

Typical cost: ~$5–15/month on Railway for the two services and model storage, plus any cloud LLM usage. Both tools are free and open source.

How It Compares

	This stack (self-hosted)	ChatGPT Plus	Ollama alone (CLI)	Cloud AI SaaS
Private + authenticated	Yes — your infra	Vendor	Local only	Vendor
Local models	Yes (Ollama)	No	Yes	No
Cloud models	Yes (BYO key)	OpenAI only	No	Vendor
Multi-user + RAG	Yes	Limited	No	Varies
Data ownership	Full — your infra	Vendor	Full	Vendor
Self-hostable	Yes	No	Yes	No

ChatGPT Plus is polished but single-provider and keeps your data. Ollama alone is powerful but command-line only, with no UI, users, or RAG. Cloud AI platforms remove setup but bill monthly and hold your conversations. This stack's edge is a private, authenticated AI workspace — a real web UI over both local Ollama models and cloud APIs, with users and RAG — self-hosted, with your chats on infrastructure you own.

Deploy in Under 5 Minutes

Click Deploy on Railway — Open WebUI and Ollama build and wire together (~3 minutes)
Set WEBUI_SECRET_KEY, and confirm Open WebUI points at Ollama via OLLAMA_BASE_URL
Optionally add OPENAI_API_KEY or ANTHROPIC_API_KEY for cloud models
Open your Railway URL, create your admin account, and set ENABLE_SIGNUP=false
Pull a small Ollama model, or select a cloud model, and start chatting privately

For fast local inference on large models, point OLLAMA_BASE_URL at an external GPU host.

Common Use Cases

Private AI chat — a self-hosted, authenticated ChatGPT-style workspace for your team
Small local models — run lightweight open models on Railway for fully private quick tasks
RAG over your documents — upload files and query them with citations, privately
Frontend for a GPU host — use Railway for the private UI, an external box for heavy inference

Configuration

Variable	Required	Description
`WEBUI_SECRET_KEY`	Required	Signs Open WebUI sessions — set a stable random value
`OLLAMA_BASE_URL`	Pre-set	Points Open WebUI at Ollama (internal, or an external GPU host)
`ENABLE_SIGNUP`	Recommended	`false` after creating your admin, so no one else can register
`OPENAI_API_KEY` / `ANTHROPIC_API_KEY`	Optional	Add cloud models to the interface
Storage volumes	Pre-set	Persist Open WebUI data and Ollama models

Lock it down. Set a stable WEBUI_SECRET_KEY, create your admin, and set ENABLE_SIGNUP=false so the workspace stays private. Ollama stays on the private network, never exposed publicly.

Size Ollama to Railway's CPU. Small models (1–3B) and embeddings run fine; for large models, point OLLAMA_BASE_URL at an external GPU host or use a cloud API key. This template wires the services and is honest about what runs well.

Dependencies for Open WebUI and Ollama Hosting

Railway account — ~$5–15/month for the two services and model storage
Persistent volumes for Open WebUI data and Ollama models (included)
Optional: an OpenAI or Anthropic API key for cloud models
Optional: an external GPU-backed Ollama host for fast large-model inference

Deployment Dependencies

Implementation Details

The template deploys two services over Railway's private network: Open WebUI (ghcr.io/open-webui/open-webui) serving the authenticated chat interface on port 8080, and Ollama (ollama/ollama) serving its model API, with Open WebUI configured to reach Ollama through OLLAMA_BASE_URL on the internal network so Ollama is never publicly exposed. Persistent volumes store Open WebUI's SQLite data (chats, users, documents, RAG vectors) and Ollama's models, so both survive redeploys.

Access is gated: WEBUI_SECRET_KEY signs sessions, the first account created is the administrator, and ENABLE_SIGNUP should be set to false after that so the instance stays private. Because Railway provides CPU rather than GPU compute, Ollama is best suited here to small models (roughly 1–3B) and embeddings; larger models run slowly and may exceed memory. For fast local inference, OLLAMA_BASE_URL can point at an external GPU-backed Ollama instance, keeping Open WebUI on Railway as the private frontend. Cloud providers work in parallel: setting OPENAI_API_KEY or ANTHROPIC_API_KEY surfaces those models in the same interface, selectable per conversation. The two volumes are the backup targets.

Frequently Asked Questions

Is it private and secured? Yes — that's the point. Open WebUI sits behind a login (WEBUI_SECRET_KEY plus your admin account, with ENABLE_SIGNUP=false), Ollama stays on the private network, and every conversation and document stays on your infrastructure rather than a vendor's servers.

Can I run large local models on Railway? Not fast — Railway is CPU-only, so Ollama here suits small models (1–3B) and embeddings. For large models at speed, point OLLAMA_BASE_URL at an external GPU host, or use a cloud API key for heavy generation. The private UI works the same either way.

Can I use both local and cloud models? Yes — add OPENAI_API_KEY or ANTHROPIC_API_KEY, and those models appear alongside Ollama's, switchable per message — all behind your login.

Does my data persist? Yes — chats, users, and documents on Open WebUI's volume, and pulled models on Ollama's, both surviving redeploys. Back up both to keep history and avoid re-downloading models.

How do I keep others out? Create your admin account on first visit, then set ENABLE_SIGNUP=false so no one else can register. Your conversations stay on your private instance.

Why Deploy Open WebUI and Ollama on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying this stack on Railway you get a private, authenticated AI workspace with the wiring solved — Open WebUI and Ollama connected privately, a login wall, persistent volumes, and cloud providers optional in the same UI. Your own ChatGPT-style interface for small local models and cloud APIs, with heavy inference attachable via an external GPU host, on infrastructure you own.