Deploy and Host Ollama with Open WebUI on Railway

Ollama runs open-weight language models — Llama, Qwen, Gemma, Mistral, Phi and hundreds more — behind a simple HTTP API. This template pairs it with Open WebUI, a ChatGPT-style interface with real user accounts, so you get a private LLM stack where the model server itself is never exposed to the internet.

About Hosting Ollama

Ollama has no authentication. None. Its API is open to anyone who can reach it, and that is by design — it expects to sit on localhost or a trusted network. Publishing Ollama directly on a public domain, which is the usual shape of a one-click Ollama deployment, means anyone who finds the URL can list your models, pull new ones onto your disk, and run inference that you pay for.

This template does not do that. Ollama runs as a private service with no public domain, reachable only over Railway's internal network. The only thing exposed is Open WebUI, which sits in front with its own account system. The first account to sign up automatically becomes the admin; every signup after that lands in a pending role and cannot use the instance until the admin approves it. Register your own account immediately after deploying — whoever signs up first owns the instance.

Ollama binds to [::] rather than 0.0.0.0, because Railway's private network resolves service hostnames over IPv6 — an IPv4-only bind is unreachable from a sibling container and is the usual reason a private Ollama service appears healthy but nothing can talk to it.

Both services keep state on volumes: downloaded model weights at /root/.ollama (these are large, and re-pulling them on every redeploy is slow and wasteful) and Open WebUI's users, chats, and settings at /app/backend/data.

Inference here is CPU-bound. Small models in the 1B–8B range are practical; large models will run but slowly.

Why Deploy Ollama on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Ollama on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.

• The model server is not public — Ollama has no auth, so this template never gives it a public domain.
• Real accounts in front — Open WebUI provides login; you claim admin by signing up first, and later signups need your approval.
• Correct private-network bind — Ollama listens on IPv6, which is what Railway's internal DNS actually resolves.
• Weights persist — pulled models survive redeploys instead of downloading again every time.
• No per-token billing — open-weight models on flat infrastructure cost, and no prompt leaves your infrastructure.

Common Use Cases

• Private team chat assistant — a shared LLM interface where conversations never reach a third-party provider.
• Internal inference API — other Railway services in the same project can call Ollama directly over the private network.
• Model evaluation — pull several open-weight models and compare them in one interface.

Dependencies for Ollama Hosting

• A volume at /root/.ollama for model weights (included in this template) — size it for the models you intend to pull.
• A volume at /app/backend/data for Open WebUI accounts and chat history (included in this template).
• Enough RAM for your chosen model; an 8B model at 4-bit quantization needs roughly 6GB.

Deployment Dependencies

• Ollama — upstream model server (MIT)
• Open WebUI — the interface this template exposes (BSD-3-Clause)
• Ollama model library — the models available to pull

Implementation Details

Two services on Railway's private network:

Open the public domain and sign up — the first account becomes admin automatically. From there, pull a model from Settings → Models, or from any other service in the project:

curl http://ollama.railway.internal:11434/api/pull -d '{"name": "qwen3:4b"}'

Other services in the same Railway project can use Ollama's API directly over the private network:

curl http://ollama.railway.internal:11434/api/generate \
  -d '{"model": "qwen3:4b", "prompt": "Why is the sky blue?", "stream": false}'

Start with a small model to confirm the stack works before pulling anything large — the first pull also tells you how much volume space you need.