Deploy and Host Qwen3 on Railway

Run Alibaba's Qwen3 open models privately with a ready-made stack — Ollama as the model server and Open WebUI as a ChatGPT-style interface, deployed in one click. Pick which Qwen3 model to load, and the template handles the download, wires the chat UI to the model server, and persists both your models and your conversations. No manual Ollama setup, no config files. Or point it at a Qwen API to use the largest models — your choice per deployment.

What This Template Deploys

Both run on Railway's private network. Persistent volumes hold the downloaded models (which are gigabytes) and your chat history, so neither is re-downloaded or lost on redeploy.

About Hosting

One thing determines what you can realistically run here, and stating it plainly saves disappointment: Railway is CPU-only — there are no GPUs.

This runs the small Qwen3 models, not the flagship ones. Qwen3 spans a huge range — from tiny 0.6B and 1.7B models up to 30B, 32B, and a 235B MoE flagship. The larger models need serious GPU memory and won't run on a CPU host. What runs well here are the small dense models — qwen3:0.6b, 1.7b, 4b, and (with enough RAM) 8b — which are genuinely capable for chat, drafting, and lightweight reasoning, but respond more slowly on CPU than a GPU-backed service. Deploy expecting a private, self-hosted small model and it delivers; expecting the 235B flagship's speed and depth locally, it can't.

Want the big Qwen models? Skip local inference and point Open WebUI at a Qwen API — Alibaba's DashScope or any OpenAI-compatible Qwen endpoint. Set the API base URL and key in Open WebUI, and you get the full-size models running on the provider's GPUs while Railway hosts your private chat interface and history. The template supports both paths: small models locally via Ollama, or large models via API.

Volumes are essential. Qwen3 models are multiple gigabytes each; without a persistent volume on Ollama, every redeploy re-downloads them — slow and wasteful. Open WebUI needs its own volume for accounts and chat history, and its WEBUI_SECRET_KEY must stay stable or sessions are invalidated. Both are handled here.

Typical cost: ~$10–20/month on Railway depending on model size and RAM — larger local models need more memory. Everything is open source and free; API mode adds the provider's usage cost.

How It Compares

For fully-private local inference, the small Qwen3 models on Ollama keep everything on your infrastructure. For flagship quality, API mode runs inference on the provider's GPUs while keeping your chat interface and history self-hosted. This template gives you both from one deployment.

Deploy in Under 5 Minutes

1. Click Deploy on Railway — Ollama and Open WebUI build automatically (~3 minutes)
2. Confirm volumes are mounted for Ollama models and Open WebUI data
3. Set which Qwen3 model to pull (e.g. qwen3:4b) — the template downloads it on first run
4. Open the Open WebUI domain, create the first account (it becomes admin), and select the model
5. Start chatting — or, for the big models, add a Qwen API key in Open WebUI settings instead

The first model download takes a few minutes depending on size; after that it's cached on the volume.

Common Use Cases

• Private local AI chat — run a small Qwen3 model entirely on your own infrastructure, nothing sent to a vendor
• Self-hosted chat UI for Qwen's API — keep the interface and history private while using the flagship models via API
• Multilingual assistant — Qwen3 has strong multilingual support, useful for non-English and cross-language tasks
• Model experimentation — try different Qwen3 sizes (0.6B–8B) and compare quality versus speed on CPU
• Team AI workspace — Open WebUI's multi-user support gives a team a shared private Qwen interface
• Cost-controlled AI — flat infrastructure for local models, or pay-per-use API mode, chosen per deployment

Configuration

Only small models run locally on Railway. Use qwen3:0.6b, 1.7b, 4b, or 8b. The 30B/32B/235B models need GPUs Railway doesn't have — for those, use API mode with OPENAI_API_BASE_URL pointed at a Qwen endpoint.

Volumes are required. Ollama's model volume prevents multi-gigabyte re-downloads on redeploy; Open WebUI's volume holds accounts and chat history. Without them, models and conversations reset.

Dependencies for Qwen3 Hosting

• Railway account — expect ~$10–20/month depending on model size and RAM
• Persistent volumes for Ollama models and Open WebUI data (included)
• More RAM for larger local models (4B/8B need more than 0.6B/1.7B)
• Optional: a Qwen/DashScope API key for flagship-model access via API mode

Deployment Dependencies

• Ollama GitHub Repository
• Open WebUI GitHub Repository
• Qwen3 Models on Ollama
• Railway Volumes Documentation

Implementation Details

The template runs the official Ollama image (port 11434) and Open WebUI (port 8080) on Railway's private network, with OLLAMA_BASE_URL wiring the chat UI to the model server automatically. On first run, Ollama pulls the Qwen3 model specified in OLLAMA_MODEL onto its persistent volume, so the multi-gigabyte download happens once rather than on every redeploy.

Because Railway is CPU-only, the template targets Qwen3's small dense models (0.6B–8B), which run on CPU — slower than GPU inference but fully private and self-contained. For the large models, Open WebUI can be pointed at a Qwen or DashScope OpenAI-compatible endpoint via OPENAI_API_BASE_URL and OPENAI_API_KEY, running inference on the provider's GPUs while keeping the interface and chat history on your Railway deployment. Open WebUI's own volume persists users, settings, and conversations; WEBUI_SECRET_KEY must stay stable or existing sessions are invalidated.

Frequently Asked Questions

Can I run the 235B Qwen3 flagship on Railway? No — the large Qwen3 models need GPUs and substantial memory, which Railway (CPU-only) can't provide. This template runs the small 0.6B–8B models locally, or connects to a Qwen API for the flagship while keeping your UI self-hosted.

How fast are the local models? The small models run on CPU, so they're slower than a GPU-backed service — fine for many chat and drafting tasks, but not instant. For speed with the large models, use API mode.

What's the difference between local and API mode? Local mode runs a small Qwen3 model on your Railway CPU — fully private, flat cost. API mode sends inference to a provider's GPUs for the flagship models — faster and more capable, but requests leave your infrastructure and cost per use. The template supports both.

Do my models and chats survive a redeploy? Yes, with the volumes mounted. Models are cached on Ollama's volume and chat history on Open WebUI's, so neither is lost or re-downloaded.

Which model should I pick? qwen3:0.6b or 1.7b for the lightest/fastest, 4b or 8b for better quality if you allocate more RAM. Larger models need more memory.

Is Qwen good at multiple languages? Yes — Qwen3 has strong multilingual capabilities, which is a common reason to choose it, including for Chinese and cross-language tasks.

Can a team use it? Yes. Open WebUI supports multiple users with authentication, so a team shares one private Qwen interface.

Why Deploy Qwen3 on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying this Qwen3 stack on Railway you get a private AI chat setup in one click — Ollama and Open WebUI pre-wired, models auto-downloaded to a persistent volume, chat history saved, and a choice of small local models or the flagship via API. Private, self-hosted, and honest about what runs where.