--- title: "Deploy Mistral AI" description: "[Jun '26] Self-host Mistral models locally, using Ollama+OpenWebUI." category: "AI/ML" url: https://railway.com/deploy/mistral-ai --- # Deploy Mistral AI [Jun '26] Self-host Mistral models locally, using Ollama+OpenWebUI. **[Deploy Mistral AI on Railway](https://railway.com/template/mistral-ai)** - **Creator:** Heimdall - **Category:** AI/ML - **Total deploys:** 18 ## Template content ### Open-WebUI https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/open-webui-light.svg - **Image:** ghcr.io/open-webui/open-webui - **Public domain:** Yes ### mistral https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/mistral-ai.svg - **Image:** ollama/ollama - **Start command:** `bash -c "ollama serve & sleep 5 && ollama pull $OLLAMA_DEFAULT_MODELS && wait"` ## Documentation ![Mistral AI](https://avatars.githubusercontent.com/u/132372032?s=200&v=4) # Deploy and Host Mistral AI on Railway Mistral is one of the most capable open-source model families available today. Known for strong reasoning, concise outputs, and excellent performance per parameter, the Mistral series includes lightweight base models, instruction-tuned chat variants, and larger 7B+ models that rival proprietary systems. Mistral 7B is widely used for code generation, agents, structured output, and general-purpose chat. ## About Hosting Mistral AI This template deploys **Ollama** as the model runtime and automatically pulls **mistral:7b** on startup, giving you a ready-to-use inference server immediately after deployment. It includes **OpenWebUI** as the frontend, so you can interact with the model, test prompts, and explore system tools without writing a single line of code. Railway handles the networking between services, persistent storage, and environment variables, letting you run Mistral without touching CUDA, GPUs, Dockerfiles, or custom servers. With this template, you get a clean chat interface and a private API endpoint in under a minute. ## Getting Started 1. Deploy the template on Railway. [![Deploy on Railway](https://railway.com/button.svg)](https://railway.com/deploy/mistral-ai?referralCode=QXdhdr&utm_medium=integration&utm_source=template&utm_campaign=generic) 2. Wait for the `mistral:7b` model to download during startup. 3. Open the Railway-generated URL to access OpenWebUI. 4. Start chatting, testing prompts, or switching models. 5. Call the Mistral API internally using `/api/generate` via `OLLAMA_BASE_URL`. ## Common Use Cases • Running Mistral 7B for chat, reasoning, or code generation • Prototyping apps using Mistral via a simple `/api/generate` endpoint • Self-hosting for privacy-sensitive use cases • Agents, automations, and backend inference • Using Mistral as the foundation for RAG pipelines • Quickly testing other models (Mixtral, Mistral-Instruct, etc.) by pulling them with Ollama ## Environment Variables This template includes preconfigured variables for both Ollama and OpenWebUI: Ollama Variables – OLLAMA_HOST: Allows Ollama to listen on all network interfaces. – OLLAMA_ORIGINS: Defines allowed CORS origins when OpenWebUI runs on another host. – OLLAMA_DEFAULT_MODELS: Specifies which model to pull at boot (`mistral:7b`). – Startup command (`ollama serve & sleep 5 && ollama pull ...`): Ensures the runtime starts, waits briefly, and pulls the specified Mistral model before becoming ready. OpenWebUI Variables – OLLAMA_BASE_URL: Points OpenWebUI to the internal Ollama API endpoint. – WEBUI_SECRET_KEY: Secret key that secures user sessions and authentication. – CORS_ALLOW_ORIGIN: Allows the WebUI to be accessed from any origin. Everything is wired to work out-of-the-box on first deployment. ## Dependencies for Mistral AI Hosting **Ollama**: Model runtime that downloads, loads, and serves Mistral 7B. Handles quantization, tokenization, and inference optimization automatically. Requires version 0.1.26+ for Mistral support. **Open WebUI**: Full-featured chatbot interface with conversation threading, markdown rendering, and model selection. Communicates with Ollama via Railway's internal networking—no public API exposure needed. **Mistral 7B Model Files**: Downloaded automatically during container startup from Ollama's library. The 7B variant requires 4.1GB disk space. Mistral also offers larger variants (8x7B Mixture-of-Experts, 8x22B) if you upgrade Railway plans. **Railway Volumes** (Highly Recommended): Persistent storage prevents re-downloading 4.1GB on every deployment. Without volumes, startup time increases from 30 seconds to 4-6 minutes. Costs $1/month for 10GB. ### Deployment Dependencies - **Ollama Library**: [ollama.com/library/mistral](https://ollama.com/library/mistral) – All Mistral model variants and versions - **Open WebUI Documentation**: [docs.openwebui.com](https://docs.openwebui.com) – Customization guides and API references - **Mistral AI Official Site**: [mistral.ai](https://mistral.ai) – Model cards, benchmarks, and technical papers - **Railway Platform Docs**: [docs.railway.app](https://docs.railway.app) – Resource limits, pricing tiers, and volume setup ## Mistral vs Other Open-Source Models Mistral 7B competes with Qwen, Llama, Phi 3, and DeepSeek. Key strengths: – Excellent reasoning and structured output format – Great performance per parameter – Fast inference even on CPU – Reliable coding ability Compared to Llama 3.1, Mistral 7B often produces tighter, more concise responses. Compared to Qwen, it trades multilingual strength for sharper logic and formatting. This makes it a strong fit for agents, tools, and code-heavy applications. | Feature | Mistral 7B (This Template) | Llama 3.2 8B | Qwen3 7B | Gemma 2 9B | |---------|---------------------------|--------------|----------|------------| | **Model Size** | 7.3B parameters | 8B parameters | 7B parameters | 9B parameters | | **Benchmark (MMLU)** | 62.5% | 68.4% | 61.8% | 70.8% | | **Code Performance** | Excellent (HumanEval: 40.2%) | Very Strong (45.1%) | Strong (38.7%) | Good (34.2%) | | **Structured Outputs** | Native JSON mode | Manual prompting | Manual prompting | Manual prompting | | **Context Window** | 8K tokens | 128K tokens | 32K tokens | 8K tokens | | **Speed (tokens/sec)** | 18-25 on Railway starter | 15-20 on Railway starter | 20-28 on Railway starter | 12-18 on Railway starter | | **License** | Apache 2.0 | Llama 3.2 License | Apache 2.0 | Gemma License | ## Railway vs Other Hosting Options Railway gives you extremely fast iteration without DevOps overhead: – No CUDA/GPU setup – No manual Dockerfiles – Zero networking setup between UI and runtime – Simple secrets + environment management – Built-in logs and redeploys Alternatives like AWS or GCP require GPU drivers, containers, networking configuration, and significantly more effort. ## FAQ **Can I use Mistral with my existing application code?** Yes. Ollama exposes an OpenAI-compatible API at `http://${{mistral.RAILWAY_PRIVATE_DOMAIN}}:11434/v1`. Point your OpenAI SDK to this URL instead of `api.openai.com`, and Mistral responds to the same API format. **Can I deploy multiple Mistral models simultaneously?** Yes. Set `OLLAMA_DEFAULT_MODELS="mistral:7b,mistral:7b-instruct,mistral-nemo:12b"` to download multiple variants. Open WebUI's dropdown lets you switch between them. Each model consumes disk space but only loads into RAM when actively used. **How do I restrict Open WebUI access to my team?** Set environment variables `WEBUI_AUTH=true` and `ENABLE_SIGNUP=false` in the Open WebUI service. The first user to access the URL becomes admin and can invite others manually. Pair this with Railway's authentication proxy for SSO. ## Why Deploy Mistral AI on Railway? Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it. By deploying Mistral AI on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway. ## Similar templates - [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform. - [stella](https://railway.com/deploy/stella) — Self-host stella with web, API, Postgres, Redis, and object storage. - [Hermes Agent | OpenClaw Alternative with Dashboard](https://railway.com/deploy/hermes-agent-or-openclaw-alternative-wit) — Self-Hosted Hermes AI Agent for Telegram, Discord & Slack Open this page in a browser: https://railway.com/deploy/mistral-ai