Deploy Ollama AI

Deploy and Host Ollama AI with Open WebUI on Railway

Ollama makes it easy to run open AI models, while Open WebUI provides a modern, self-hosted interface for chatting with and managing those models.

This template combines both into a simple private AI stack with persistent storage, making it easy to run models such as Llama, Qwen, Mistral, Gemma, Phi, and many others from a web interface.

About Hosting Ollama AI

This template deploys two connected services:

• Ollama — runs and manages AI models
• Open WebUI — provides the web interface for chat, model management, users, knowledge, prompts, tools, and integrations

The services communicate over Railway's private network, while only Open WebUI needs to be exposed publicly.

Both services use persistent storage so downloaded models, user accounts, conversations, settings, and application data survive restarts and redeployments.

Included Architecture

                    ┌─────────────────┐
                    │      User       │
                    └────────┬────────┘
                             │
                             ▼
                  Railway HTTPS Proxy
                             │
                             ▼
                    ┌─────────────────┐
                    │   Open WebUI    │
                    │                 │
                    │ Chat / Models   │
                    │ Knowledge       │
                    │ Tools / Users   │
                    └────────┬────────┘
                             │
                    Private Network
                             │
                             ▼
                    ┌─────────────────┐
                    │     Ollama      │
                    │                 │
                    │ Model Runtime   │
                    │ Inference API   │
                    └────────┬────────┘
                             │
                             ▼
                    ┌─────────────────┐
                    │ Persistent      │
                    │ Model Storage   │
                    └─────────────────┘

Ollama exposes its API on port 11434, while Open WebUI connects to the Ollama backend internally. Ollama's official Docker setup also persists model data under /root/.ollama.

Common Use Cases

• Run private AI models on your own infrastructure
• Chat with open-source LLMs through a modern web UI
• Test multiple models from one interface
• Build internal AI assistants
• Create private team AI workspaces
• Use AI without sending prompts to a third-party hosted model provider
• Manage prompts and model configurations
• Work with documents and knowledge bases
• Connect tools and external APIs
• Use Ollama as an inference backend for other applications
• Experiment with local or self-hosted AI models

Ollama AI vs Hosted AI Platforms

This stack is especially useful when you want control over the model runtime, storage, conversations, and infrastructure.

Open WebUI Features

Open WebUI provides much more than a simple chat page.

It supports:

• Chat and conversation history
• Multiple AI models
• Model switching during conversations
• Knowledge and document workflows
• Prompts
• Tools
• User management
• OpenAI-compatible providers
• Ollama models
• API access
• Extensible integrations

Open WebUI officially supports Ollama, OpenAI-compatible APIs, and other model providers from the same interface.

Persistent Storage

Ollama

Downloaded AI models are stored persistently.

The Ollama service uses:

/root/.ollama

This prevents models from needing to be downloaded again after every deployment. Ollama's official Docker documentation uses the same persistent path.

Open WebUI

Open WebUI stores its application data persistently, including:

• User accounts
• Conversations
• Settings
• Model configurations
• Knowledge data
• Application state

This allows the interface to retain its configuration after restarts and redeployments.

Getting Started

After deployment:

1. Open the public Railway domain assigned to the Open WebUI service.
2. Create your first account.
3. The first account becomes the administrator.
4. Open the model selector or model management interface.
5. Download or select an Ollama model.
6. Wait for the model download to complete.
7. Start a new chat.
8. Select the model you want to use.
9. Start chatting.

Open WebUI provides dedicated Ollama integration and model management from its interface.

Choosing a Model

Ollama supports many open models.

Popular choices include:

• Llama
• Qwen
• Mistral
• Gemma
• Phi
• DeepSeek
• Embedding models
• Coding-focused models

Model requirements vary significantly.

Smaller models typically require less memory and start faster, while larger models may require significantly more RAM and compute resources.

For Railway deployments, start with a smaller model first and scale resources as needed.

Resource Considerations

AI model inference can consume considerably more resources than a typical web application.

The primary factors are:

• Model size
• Quantization
• Available RAM
• CPU performance
• GPU availability
• Number of concurrent users
• Context length

Ollama can use GPU acceleration when the underlying environment exposes compatible GPU hardware. Its official Docker documentation describes NVIDIA, AMD, and Vulkan acceleration options.

Without a GPU, Ollama can still run models using CPU, although inference may be slower.

Connecting Other AI Providers

Although this template is configured primarily around Ollama, Open WebUI can also connect to external providers.

Supported connection types include:

• OpenAI
• OpenAI-compatible APIs
• Ollama
• Other supported AI backends

These can be configured later from:

Admin Settings → Connections

Open WebUI documents provider connections directly from its administration interface.

No external API key is required to use the included Ollama service.

Using Ollama as an API

Ollama also exposes an HTTP API for model inference.

Its native API is available on port:

11434

The default API path is:

/api

Other applications in the same Railway project can connect to Ollama through Railway private networking without exposing the Ollama service publicly. Ollama documents its local API at port 11434.

This makes the stack useful not only for chat, but also as an AI backend for:

• AI agents
• n8n workflows
• Internal applications
• RAG systems
• API services
• Automation pipelines
• Development tools

Dependencies for Ollama AI

• Ollama — runs and manages self-hosted AI models
• Open WebUI — provides the web interface for chat, model management, users, knowledge, prompts, and tools
• Ollama Persistent Volume — stores downloaded AI models across restarts and redeployments
• Open WebUI Persistent Volume — stores users, conversations, settings, knowledge data, and application state
• Railway Private Networking — allows Open WebUI to communicate with Ollama without exposing the model API publicly
• Railway Public Networking — provides HTTPS access to the Open WebUI interface

Why Deploy Ollama AI on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Ollama AI with Open WebUI on Railway, you get a self-hosted AI runtime, persistent model storage, a modern web interface, private service networking, and a flexible foundation for running open AI models.