Deploy and Host Ollama with Open WebUI on Railway

Ollama runs open-weight language models — Llama, Qwen, Gemma, Mistral, Phi and hundreds more — behind a simple HTTP API. This template pairs it with Open WebUI, a ChatGPT-style interface with real user accounts, tunes Ollama for the CPU it is actually running on, and downloads a model for you on the first boot — so the deploy finishes with something you can talk to at a usable speed, instead of an empty model list and one token a second.

About Hosting Ollama

Ollama has no authentication. None. Its API is open to anyone who can reach it, and that is by design — it expects to sit on localhost or a trusted network. Publishing Ollama directly on a public domain, which is the shape of several one-click Ollama templates, means anyone who finds the URL can list your models, pull new ones onto your disk, and run inference that you pay for.

This template does not do that. Ollama runs as a private service with no public domain, reachable only over Railway's internal network. The only thing exposed is Open WebUI, which sits in front with its own account system. The first account to sign up automatically becomes the admin; every signup after that lands in a pending role and cannot use the instance until the admin approves it. Register your own account immediately after deploying — whoever signs up first owns the instance.

Ollama's own default is to listen on loopback only, so a private Ollama service reports healthy while the container next to it gets connection refused. This template sets OLLAMA_HOST=::, which binds every interface, v4 and v6.

The CPU tuning, and why it matters more than anything else here

A container reports the host's core count, not the CPU quota it actually has. Ollama's inference runner auto-detects threads from that number, spawns far more than the container can run, and spends most of its time fighting itself. Measured on Railway with llama3.2:1b, same instance, same model, same prompt:

That is roughly 78x, and it applies to every request and every model — not just the preloaded one. This template's Ollama image reads the container's own cgroup CPU quota at startup and sets the thread count from it, so you get the second row without configuring anything. Override LLAMA_ARG_THREADS on the Ollama service if you want a different number.

The first boot pulls llama3.2:1b (about 1.3 GB) behind an already-listening server, so nothing waits on the download — the model simply appears in Open WebUI's model list when it finishes. Change OLLAMA_PRELOAD_MODEL on the Ollama service to preload something else from the Ollama library, or set it to an empty string to start with no model at all.

Both services keep state on volumes: downloaded weights at /root/.ollama (these are large, and re-pulling them on every redeploy is slow and wasteful) and Open WebUI's users, chats, and settings at /app/backend/data.

Inference here is CPU-bound — Railway has no GPU. Models in the 1B–8B range are the practical band; the 1B default is chosen so the first chat responds at a usable speed on a small instance. Larger models will run, more slowly, and need RAM to match: roughly 6 GB for an 8B model at 4-bit quantization.

Why Deploy Ollama on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Ollama on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.

• Tuned for the CPU you are billed for — threads sized to the container's quota instead of the host's core count; ~78x measured against the auto-detected default.
• A model is already there — the deploy ends with a working chat, not an empty model list and a pull screen to go find.
• The model server is not public — Ollama has no auth, so this template never gives it a public domain.
• Real accounts in front — Open WebUI provides login; you claim admin by signing up first, and later signups need your approval.
• Nothing to fill in — every setting is baked into the images, so the deploy form is empty.
• Both versions pinned — Ollama 0.32.5 and Open WebUI 0.11.0, not a floating latest that can change under a running deployment.
• Weights persist — pulled models survive redeploys instead of downloading again every time.

Common Use Cases

• Private team chat assistant — a shared LLM interface where conversations never reach a third-party provider.
• Internal inference API — other Railway services in the same project can call Ollama directly over the private network.
• Model evaluation — pull several open-weight models and compare them in one interface.

Dependencies for Ollama Hosting

• A volume at /root/.ollama for model weights (included in this template) — size it for the models you intend to pull.
• A volume at /app/backend/data for Open WebUI accounts and chat history (included in this template).
• Enough RAM for your chosen model; the preloaded 1B model is comfortable on a small instance, an 8B model at 4-bit quantization needs roughly 6 GB.

Deployment Dependencies

• Ollama — upstream model server (MIT)
• Open WebUI — the interface this template exposes (BSD-3-Clause)
• Ollama model library — the models available to pull
• bon5co/ollama-railway — the two Railway-tuned wrapper images this template deploys

Implementation Details

Two services on Railway's private network:

Open the public domain and sign up — the first account becomes admin automatically. The preloaded model appears in the model picker once its download finishes. To pull another one, use Settings → Models in the UI, or call Ollama from any other service in the project:

curl http://ollama.railway.internal:11434/api/pull -d '{"name": "qwen3:4b"}'