Deploy and Host Ollama on Railway

Ollama runs open language models (Qwen, Llama, Gemma, DeepSeek, Mistral and hundreds more from its library) behind a simple HTTP API, plus an OpenAI-compatible endpoint at /v1, so most AI tools can use it by changing a base URL.

This template runs the official image (pinned to 0.34.4) with an API key in front of it and your models on a volume.

About Hosting Ollama

Ollama has no authentication. A public Ollama lets anyone who finds the address run your models, pull new ones onto your volume or delete them, all on your bill. Here Ollama listens only inside the container, and Caddy in front of it lets a request through only if it carries Authorization: Bearer . The key is generated at deploy. Only / is open, because Railway's healthcheck uses it; it just says "Ollama is running".

Models in OLLAMA_PULL_MODELS are pulled on start (only what's missing, so later starts are quick) and kept on the volume. The default is qwen2.5:0.5b, which fits a 1 GB plan. Models load on the first request and unload after OLLAMA_KEEP_ALIVE (5 minutes) without requests, so an idle service doesn't keep gigabytes of RAM busy.

My first deploy failed its healthcheck with a 403 from Ollama: it rejects requests whose Host header isn't local, which protects against DNS rebinding. Caddy now sends the local address upstream.

Before publishing I tested it on Railway. / answered without a key, /api/tags without a key or with a wrong key got 401, and with the key the model was listed and both /api/generate and /v1/chat/completions answered. After a restart the model was still on the volume and answered again.

Numbers from that test: the first answer took 19.7 seconds (loading the model), later ones 5 to 10 seconds on Railway's CPUs. With the 0.5B model loaded the service used about 1.0 GB, most of it the model file Ollama maps into memory; after the idle unload it dropped to 0.08 GB. Railway has no GPUs, so this is for small models, embeddings and light use. A 7B model needs about 5 to 8 GB of RAM, depending on quantization and context size.

Common Use Cases

A private model endpoint for n8n, LibreChat, Open WebUI, Dify or your own code, without sending data to a model provider
Embeddings for search or RAG with a small embedding model such as nomic-embed-text
Trying open models behind a stable HTTPS address shared by several services in one Railway project

Dependencies for Ollama Hosting

Nothing beyond the template. Clients need to send the API key: in OpenAI-compatible tools set the base URL to OLLAMA_URL/v1 and the API key to OLLAMA_API_KEY; for Ollama's own API add the Authorization: Bearer header.

Deployment Dependencies

Ollama: https://github.com/ollama/ollama
Model library: https://ollama.com/library
Image: ollama/ollama:0.34.4, with Caddy 2.11.4 in front
The wrapper this template builds: https://github.com/dektionstudio/railway-template-images/tree/main/ollama

Implementation Details

curl $OLLAMA_URL/v1/chat/completions \
  -H "Authorization: Bearer $OLLAMA_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "qwen2.5:0.5b", "messages": [{"role": "user", "content": "Hello"}]}'

OLLAMA_API_KEY              # generated; send as Authorization: Bearer 
OLLAMA_PULL_MODELS          # comma-separated, e.g. qwen2.5:0.5b,nomic-embed-text
OLLAMA_KEEP_ALIVE=5m        # unload after idle time; -1 keeps the model loaded
OLLAMA_CONTEXT_LENGTH=4096  # larger windows need more RAM

Why Deploy Ollama on Railway?

You get an authenticated model API on HTTPS in one deploy, reachable from your other Railway services, with models that survive restarts. Memory is billed while a model is loaded, so the idle unload keeps a lightly used endpoint cheap.