Deploy Ollama Private AI Server with Open WebUI, Zero Config

Run open LLMs on a private Ollama server with a web chat UI.

Deploy Ollama Private AI Server with Open WebUI, Zero Config

Just deployed

/data

Just deployed

/root/.ollama

Deploy and Host Ollama with Open WebUI on Railway

Ollama logo

Ollama is the most popular way to run open-weight large language models on your own server. It downloads models such as Llama, Gemma, Qwen, Mistral, Phi and DeepSeek and serves them through a simple HTTP API that many apps and libraries already support. This template pairs a private Ollama server with Open WebUI, a ChatGPT-style chat interface, and pins both images to tested versions: Ollama 0.40.16 and Open WebUI v0.13.3. There is nothing to fill in at deploy time: secrets are generated, the services are wired together over Railway's private network, and you pull models from the browser. Ollama is MIT licensed with about 187,500 GitHub stars; Open WebUI has about 159,000 stars under its own Open WebUI License. Railway servers have no GPU, so models run on the CPU and small models work best. ChatGPT Plus costs $20 per person per month. This template costs about $10–40/month on Railway, mostly driven by the memory your models need.


What This Template Deploys

ServicePurpose
ollamaollama/ollama:0.40.16, the model server on port 11434, private only. Volume at /root/.ollama for downloaded models. Health check on /.
open-webuighcr.io/open-webui/open-webui:v0.13.3, the chat interface on port 8080 with a public domain. Volume at /data for users, chats, settings and uploads. Health check on /health.

The Ollama API is never exposed to the internet. Open WebUI reaches it at http://ollama.railway.internal:11434, and only signed-in users can send it prompts.


About Hosting Ollama

Every prompt sent to a public AI service leaves your control. Running Ollama yourself keeps conversations, documents and model choice inside your own Railway project, with one flat infrastructure bill instead of a subscription per person.

Pinned, predictable versions. Both images use fixed version tags, so redeploys never pull a surprise upgrade. You choose when to move to a newer release.

Pull models from the browser. Open WebUI's admin settings can download any model from the Ollama library by name, with no shell access needed.

One model at a time by default. OLLAMA_MAX_LOADED_MODELS is 1 and OLLAMA_NUM_PARALLEL is 1, which keeps memory use predictable on CPU-only servers.

Frees memory when idle. OLLAMA_KEEP_ALIVE unloads a model after five idle minutes, so an unused server stays light.

Patient with slow answers. AIOHTTP_CLIENT_TIMEOUT is 600 seconds, so long CPU generations finish instead of timing out.

Private network by design. Ollama listens on [::]:11434, the IPv6 address Railway's private network uses, and has no public domain.


Deploy Ollama in Under 5 Minutes

  1. Click Deploy on Railway. No inputs are needed.
  2. Open the Open WebUI domain and create your account straight away. The first account becomes the administrator.
  3. In Admin Panel, Settings, Models, enter a small model such as gemma3:1b or qwen3:1.7b and download it.
  4. Start a new chat, pick the model at the top and send a message.
  5. In Admin Panel, Settings, General, set new sign-ups to pending or disabled.

Common Use Cases

  • Private team assistant: a ChatGPT-style helper for colleagues without sending data to a public service.
  • Model evaluation: try several open models on your own prompts before choosing one for a product.
  • Document Q and A: upload PDFs and notes and ask questions about them.
  • Coding help: run small code models for explanations and snippets.
  • Backend for your apps: call Ollama from other Railway services over the private network.
  • Teaching and workshops: give a class one shared AI chat with no per-student fees.
  • Prompt libraries: save shared prompts so the team uses consistent instructions.
  • Sensitive drafts: keep confidential writing away from public AI tools.

Configuration

VariableServiceDefaultDescription
OLLAMA_HOSTollama[::]:11434Listen address on the private network
OLLAMA_KEEP_ALIVEollama5mHow long an idle model stays in memory
OLLAMA_NUM_PARALLELollama1Requests each loaded model handles at once
OLLAMA_MAX_LOADED_MODELSollama1Models kept in memory at the same time
OLLAMA_BASE_URLopen-webuiOllama private URLWhere Open WebUI finds Ollama
WEBUI_SECRET_KEYopen-webui🔐 Auto-generatedSigns login sessions; keep it stable
AIOHTTP_CLIENT_TIMEOUTopen-webui600Seconds to wait for a model reply
DATA_DIRopen-webui/dataFolder on the volume for app data

Memory and model size. Without a GPU, a model needs roughly its download size in memory plus some overhead. Models of 1 to 4 billion parameters run well with 4 to 8 GB; 7 to 8 billion parameter models need about 8 to 12 GB and answer more slowly. Raise the ollama service's memory to match the largest model you use.

Disk. Each model is stored on the ollama volume, and a 4B model takes a few gigabytes. Delete unused models from Open WebUI to free space.

More users at once. Raise OLLAMA_NUM_PARALLEL to serve several chats together, and give the service more memory and CPU, since each parallel request uses extra memory.

Keeping a model warm. Set OLLAMA_KEEP_ALIVE to 30m or -1 to avoid reload delays, at the cost of memory held between chats.

Calling Ollama from other services. Any service in the same project can use http://ollama.railway.internal:11434, including OpenAI-compatible clients pointed at its /v1 endpoint.

Hosted models too. Add an OpenAI or other compatible API connection in Open WebUI's settings to use hosted models next to your local ones.

Upgrades. Change the image tags to newer versions, such as Open WebUI v0.13.4, and redeploy. Back up the Open WebUI volume first.

Sign-ups and permissions. Open WebUI lets anyone with the URL register until you change it. Set the default role for new users to pending so an admin approves each one, and use groups to control which models each person can see.

Context length. Longer conversations and documents use more memory. If a model runs out of memory, lower its context size in the model settings or switch to a smaller model.

Backups. Users, chats and uploads live on the Open WebUI volume. Models can always be downloaded again, so that volume is the one to back up.


Picking a Model Size

Model sizeExample modelsMemory to plan forGood for
Under 2BGemma 3 1B, Qwen3 1.7B2–4 GBFast replies, simple tasks
3–4BGemma 3 4B, Qwen3 4B, Llama 3.2 3B4–8 GBEveryday chat and summaries
7–9BLlama 3.1 8B, Qwen3 8B, Mistral 7B8–12 GBBetter reasoning, slower on CPU

Start small, check speed and quality, then move up. The default quantized downloads keep memory low with little loss in quality.


Ollama with Open WebUI vs. Alternatives

FeatureOllama and Open WebUI on RailwayChatGPT PlusLM StudioLocalAI
HostingSelf-hostedHosted SaaSDesktop appSelf-hosted
LicenseMIT and Open WebUI LicenseProprietaryProprietaryMIT
Runs open models✅❌✅✅
Shared web chat for a team✅Per account❌Basic UI
GPU acceleration❌ on Railway✅✅ local GPUOptional
Data stays with you✅❌✅✅
PriceAbout $10–40/month total$20/user/monthFreeHosting cost

ChatGPT Plus offers the strongest models and fastest answers. LM Studio runs models on your own computer's GPU. LocalAI is an OpenAI-compatible model server. This template gives a team a private, zero-config chat app with open models in the cloud.


Dependencies for Ollama Hosting

  • A Railway account with enough memory for your chosen models.

Deployment Dependencies

Implementation Details

Open WebUI stores its database and uploads under DATA_DIR on its volume. Railway restarts either service on failure, and each waits on its own health check before receiving traffic.


Frequently Asked Questions

What does this template run? A private Ollama model server and Open WebUI as a ChatGPT-style chat interface, both pinned to tested versions.

How much does it cost to host? About $10–40/month on Railway, mostly set by the memory your models need.

Do I need to configure anything? No. Deploy, create your admin account, and pull a model from Open WebUI.

Who becomes the admin? The first account created in Open WebUI, so sign up right after deploying.

Why are answers slow? Railway has no GPU, so models run on the CPU. Smaller models answer much faster.

Can my other apps use Ollama? Yes. Services in the same project can call http://ollama.railway.internal:11434 over the private network.


Why Deploy and Host Ollama with Open WebUI on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Ollama with Open WebUI on Railway, you get a private, pinned AI chat server for your team, with your chats and models on your own volumes.


Template Content

More templates in this category

View Template
Chat Chat
Chat Chat, your own unified chat and search to AI platform.

okisdev
117
View Template
Weaviate
Vector database with hybrid search and an API key-protected API

GitHub Locker
1
View Template
Möbius Personal AI OS
Self-hosted AI agent that builds and runs apps for you, with memory.

bento
1