---
title: "Deploy Self-Hosted LLM (Ollama + Open WebUI Private Chat)"
description: "Run open-source LLMs privately: Ollama + Open WebUI chat. No per-seat fees."
category: "AI/ML"
url: https://railway.com/deploy/self-hosted-llm-ollama
---

# Deploy Self-Hosted LLM (Ollama + Open WebUI Private Chat)

Run open-source LLMs privately: Ollama + Open WebUI chat. No per-seat fees.

**[Deploy Self-Hosted LLM (Ollama + Open WebUI Private Chat) on Railway](https://railway.com/template/self-hosted-llm-ollama)**

Machine-readable deploy manifest (JSON, validated by TemplateCI): https://railway.com/deploy/self-hosted-llm-ollama/manifest.json

- **Creator:** blueorange
- **Category:** AI/ML

## Template content

### Open WebUI https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/open-webui-light.svg

- **Image:** ghcr.io/open-webui/open-webui
- **Public domain:** Yes

### ollama https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/ollama-dark.svg

- **Image:** ollama/ollama
- **Start command:** `bash -c "ollama serve & sleep 5 && for model in $(echo $OLLAMA_DEFAULT_MODELS | tr ',' ' '); do ollama pull $model; done && wait"`

## Documentation

# Deploy and Host a Self-Hosted LLM with Ollama on Railway

![Open WebUI, the chat interface for Ollama models](https://raw.githubusercontent.com/open-webui/open-webui/main/banner.png)

Run open-source LLMs on your own server with **Ollama** (184,000+ GitHub stars) and **Open WebUI** (156,000+ stars), the most popular pair for private AI chat. This template deploys an Ollama model server and a ChatGPT-style web interface on Railway, connected over a private network, with Google's Gemma 3 270M model downloaded on first boot. ChatGPT Plus and Claude Pro each cost $20 per user per month. A self-hosted LLM with small models runs for about $10–20/month in total, and your prompts never leave your server.

---

## What This Template Deploys

| Service | Purpose |
|---|---|
| **ollama** | Ollama model server on port `11434`, private network only (no public domain). On start it runs `ollama serve` and then pulls every model listed in `OLLAMA_DEFAULT_MODELS`. |
| **Open WebUI** | The chat interface on port `8080` with a public HTTPS domain: conversations, model switching, document upload and chat with your files, user accounts and an admin panel. |
| **Persistent Volume (ollama)** | Mounted at `/root/.ollama`. Downloaded models survive redeploys, so they are pulled once. |
| **Persistent Volume (Open WebUI)** | Mounted at `/app/backend/data`. Users, chat history, settings and uploaded documents survive redeploys. |

Open WebUI reaches Ollama through `OLLAMA_BASE_URL`, a reference to the ollama service's private domain. The model API is never exposed to the internet; only the web interface is public.

---

## About Hosting a Self-Hosted LLM

Running an LLM yourself usually means a machine with enough memory, installing Ollama, keeping it running, putting a web UI in front of it, adding HTTPS and logins, and making sure gigabytes of model files aren't lost on every restart. On a laptop it stops when the lid closes. On a VPS you maintain it yourself.

On Railway the whole stack is set up from the first deploy. Both services share a private network, the web interface gets an HTTPS domain automatically, and both volumes persist models and chats. To add models, list them in `OLLAMA_DEFAULT_MODELS` or pull them from Open WebUI's admin settings.

**CPU only, so pick small models.** Railway runs this template on CPU, with no GPU. Small models such as Gemma 3 270M (292 MB) and Gemma 3 1B (806 MB) answer at usable speed. A 4B model (3.4 GB) works but is slower, and 12B or 27B models need far more memory and will feel slow. For coding or long reasoning, connect a cloud model through `OPENAI_API_KEY` and keep private chats on the local models.

Typical cost: **~$10–20/month** on Railway with small models. Memory is the main cost at about $10 per GB per month, so larger models cost more. ChatGPT Plus is $20 per user per month; here one deployment serves your whole team.

---

## Deploy a Private LLM in Under 10 Minutes

1. Before deploying, open the Open WebUI service's variables and replace `WEBUI_SECRET_KEY` with a long random string. The template ships a shared placeholder value.
2. Clear `OPENAI_API_KEY`, or paste a real OpenAI key if you also want GPT models. Optionally change `OLLAMA_DEFAULT_MODELS`, for example `gemma3:1b,llama3.2:3b`.
3. Click **Deploy on Railway**. Both services start, and ollama downloads the listed models (a few minutes for small ones).
4. Open the Open WebUI domain right away and create your account. The first account becomes the admin.
5. Pick a model in the selector at the top of a new chat and start talking.

No GPU setup. No Docker commands. No per-message API bill.

---

## Common Use Cases

- **Private ChatGPT alternative.** A chat interface for your team where conversations stay on your own server.
- **Chat with your documents.** Upload PDFs and notes in Open WebUI and ask questions about them, without sending files to a third party.
- **Testing open-source models.** Try Gemma, Llama, Qwen, Mistral and other models side by side before you build on one.
- **One interface for local and cloud models.** Use small Ollama models for everyday questions and a cloud model through your API key for heavy tasks.
- **Internal AI assistant.** Give staff an AI tool with accounts and admin controls, without per-seat subscriptions.
- **Learning and prototyping.** Explore prompts, system prompts and model settings on infrastructure you control.
- **Offline-style privacy.** Sensitive drafts, client notes or code snippets processed by a model that never calls an outside API.

---

## Configuration

| Variable | Service | Required | Description |
|---|---|---|---|
| `OLLAMA_DEFAULT_MODELS` | ollama | Optional | Comma-separated models pulled at boot. Default `gemma3:270m`. |
| `OLLAMA_HOST` | ollama | ✅ Set | `::`, so Ollama listens on Railway's IPv6 private network. |
| `OLLAMA_ORIGINS` | ollama | ✅ Auto-injected | Allowed origins, set from the private domain. |
| `OLLAMA_BASE_URL` | Open WebUI | ✅ Auto-injected | Ollama's private address on port `11434`. |
| `WEBUI_SECRET_KEY` | Open WebUI | ✅ Change it | Signs login sessions. Replace the default with your own random value. |
| `OPENAI_API_KEY` | Open WebUI | Optional | Adds OpenAI models next to your local ones. Clear the placeholder if unused. |
| `CORS_ALLOW_ORIGIN` | Open WebUI | Optional | `*` by default. Set it to your domain to tighten it. |

**Models don't appear in the selector?** The first boot may still be downloading them. Check the ollama service's logs for the pull progress, then refresh Open WebUI. If the logs show a pull error, compare the names in `OLLAMA_DEFAULT_MODELS` with the Ollama library: a misspelled last model can stop the ollama service from starting.

---

## Self-Hosted LLM on Railway vs. Alternatives

| | Ollama + Open WebUI on Railway | ChatGPT Plus | Claude Pro | LibreChat |
|---|---|---|---|---|
| **Monthly cost** | ~$10–20 usage, whole team | $20 per user | $20 per user | Free software, you host it |
| **Self-hosted** | ✅ Yes | ❌ No | ❌ No | ✅ Yes |
| **Prompts stay on your server** | ✅ Yes | ❌ No | ❌ No | ⚠️ Only with local models |
| **Runs models itself** | ✅ Via Ollama | Cloud only | Cloud only | ❌ Needs a model server |
| **Model quality** | ⚠️ Small open models on CPU | ✅ Frontier models | ✅ Frontier models | Depends on provider |
| **Multi-user accounts** | ✅ Built in | ⚠️ Business plan | ⚠️ Team plan | ✅ Built in |
| **License** | ✅ MIT (Ollama), ⚠️ Open WebUI License | ❌ Proprietary | ❌ Proprietary | ✅ MIT |

---

## Dependencies for Self-Hosted LLM Hosting

- Railway account. Hobby ($5/month including $5 of usage) runs the small default model. Larger models need more memory.
- Optional: an OpenAI API key for cloud models alongside local ones.
- A browser. Open WebUI works on desktop and mobile.

### Deployment Dependencies

- [Ollama on GitHub](https://github.com/ollama/ollama): source and releases
- [Ollama model library](https://ollama.com/library): names and sizes for `OLLAMA_DEFAULT_MODELS`
- [Open WebUI Documentation](https://docs.openwebui.com): features, settings and admin guide
- [Railway Volumes Documentation](https://docs.railway.com/volumes): persistence for models and chats

### Implementation Details

The ollama service runs the official `ollama/ollama` image with a custom start command: it starts the server, waits a few seconds, then runs `ollama pull` for each model in `OLLAMA_DEFAULT_MODELS`. Models are stored on the `/root/.ollama` volume, so later boots skip models already downloaded. Open WebUI runs `ghcr.io/open-webui/open-webui` and stores its database and uploads on `/app/backend/data`.

Both images track their latest releases (Ollama 0.40 and Open WebUI 0.12 at the time of writing). Pin version tags once you rely on the setup, and back up the Open WebUI volume before upgrading.

---

## Frequently Asked Questions

**What is the easiest way to self-host an LLM?**
Deploy this template. It runs Ollama to serve open-source models and Open WebUI as the chat interface, so you get a private ChatGPT-style app without installing anything locally.

**How much does it cost compared to ChatGPT Plus?**
About $10–20/month on Railway for the stack with small models, shared by everyone you invite. ChatGPT Plus and Claude Pro cost $20 per user each month.

**Why are responses slow, or why won't a big model load?**
This template runs on CPU. Use models up to about 4B parameters, such as `gemma3:1b` or `llama3.2:3b`. Large models need much more memory and generate text slowly without a GPU.

**Do I lose my models and chats when Railway redeploys?**
No. Models are on the ollama volume at `/root/.ollama`, and users, chats and documents are on the Open WebUI volume at `/app/backend/data`.

**Can I use OpenAI models and local models in the same interface?**
Yes. Add your key to `OPENAI_API_KEY` and both appear in Open WebUI's model selector.

**Is it as capable as ChatGPT?**
Not with small CPU models. They handle summaries, drafting and Q&A well, but frontier cloud models are stronger at complex reasoning and code. Many teams use local models for private data and a cloud model for hard tasks.

---

## Why Deploy and Host a Self-Hosted LLM on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Ollama and Open WebUI on Railway, you get a private AI chat for your whole team, running open-source models on your own server with no per-seat subscription.


## Similar templates

- [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform.
- [Möbius v0.1 Personal AI OS](https://railway.com/deploy/mobius-v01-personal-ai-os) — Self-hosted AI agent that builds and runs apps for you, with memory.
- [stella](https://railway.com/deploy/stella) — Self-host stella with web, API, Postgres, Redis, and object storage.

Open this page in a browser: https://railway.com/deploy/self-hosted-llm-ollama