--- title: "Deploy OmniRoute: Self-Hosted LLM Gateway with Auto-Fallback" description: "One OpenAI-compatible endpoint for every model provider, with fallbacks." category: "Other" url: https://railway.com/deploy/omniroute-openai-gateway --- # Deploy OmniRoute: Self-Hosted LLM Gateway with Auto-Fallback One OpenAI-compatible endpoint for every model provider, with fallbacks. **[Deploy OmniRoute: Self-Hosted LLM Gateway with Auto-Fallback on Railway](https://railway.com/template/omniroute-openai-gateway)** Machine-readable deploy manifest (JSON, validated by TemplateCI): https://railway.com/deploy/omniroute-openai-gateway/manifest.json - **Creator:** blueorange - **Category:** Other ## Template content ### omniroute - **Image:** diegosouzapw/omniroute - **Public domain:** Yes ## Documentation # Deploy and Host OmniRoute on Railway ![OmniRoute logo](https://cdn.jsdelivr.net/gh/selfhst/icons/svg/omniroute.svg) OmniRoute is an open source **AI gateway** that puts one OpenAI-compatible `/v1` endpoint in front of every model provider you use. You connect your own accounts for OpenAI, Anthropic, Google Gemini, DeepSeek, Groq, OpenRouter, local Ollama and hundreds of others, then point every app, SDK and coding agent at a single URL. OmniRoute picks the provider, falls back automatically when one is rate-limited or down, translates between OpenAI, Anthropic and Gemini request formats, and tracks tokens and cost per key. It is MIT-licensed, has about 86,800 GitHub stars, and works with Claude Code, Codex, Cursor, Cline and other tools that accept a custom base URL. This template runs the official Docker image on Railway with a persistent volume for its SQLite database, so your providers, keys and usage history survive redeploys. Hosting costs about $5–15/month on Railway; you pay providers directly for tokens. --- ## What This Template Deploys | Service | Purpose | |---|---| | **omniroute** | The `diegosouzapw/omniroute` image on port `20128` with a public HTTPS domain. Serves the dashboard at `/dashboard`, login at `/login` and the API at `/v1`. | | **Volume** | Mounted at `/app/data`, the `DATA_DIR`. Holds the SQLite database with providers, encrypted credentials, gateway keys, usage logs and automatic backups. | There is no separate database service. Three secrets are generated at deploy time: one signs dashboard sessions, one encrypts stored API keys, and one salts machine identifiers. A 24-character `INITIAL_PASSWORD` protects the dashboard. --- ## About Hosting OmniRoute Every model vendor has its own SDK, auth header, error format, rate limits and billing page. Once an app or a team uses more than one, that sprawl turns into duplicated keys, hand-written retries and no clear view of spend. A gateway moves all of that into one place. **One endpoint, many providers.** Any client that speaks the OpenAI API can call OmniRoute. Model names choose the route, so switching from one provider to another is a config change in the dashboard, not a code change. **Combos and fallbacks.** OmniRoute's combos are ordered chains of models. If the first provider returns a rate-limit or server error, the request moves to the next one, so long agent runs keep going. Quota-aware scheduling spreads load across accounts before they hit their limits. **Format translation.** OmniRoute converts between OpenAI, Anthropic and Gemini formats, which lets tools built for one vendor talk to models from another. **Token compression.** OmniRoute can compress prompts and tool output before they reach the model, which the project says cuts token use on long coding sessions. It's optional and set per route. **Bring your own keys.** A fresh gateway has no providers. You add the accounts you're entitled to use, and each provider's own limits and terms still apply. Credentials are encrypted in the database with `API_KEY_SECRET`. **Locked by default.** `REQUIRE_API_KEY=true` rejects any `/v1` request without a gateway key you issue, and stored key values stay masked in the dashboard. Change the initial password after your first login. Typical cost: **~$5–15/month** on Railway for one service and its volume, depending on traffic. The heap is capped at 2 GB by `OMNIROUTE_MEMORY_MB`. Token costs are billed by each provider at their normal rates, with no gateway markup. --- ## Deploy OmniRoute in Under 5 Minutes 1. Click **Deploy on Railway**. The image starts in a minute or two. 2. Copy `INITIAL_PASSWORD` from the service's **Variables** and open the public domain. 3. Sign in, then change the password under **Configuration** and **Security**. 4. Go to **Providers**, add a provider and paste its API key, then open **API Keys** and create a gateway key. 5. Call `https://your-domain/v1/models` with that key, then set the same base URL and key in your OpenAI SDK or coding agent. Requests without a key get `401`, which confirms the gateway is protected. --- ## Common Use Cases - **Coding agents with fallback.** Point Claude Code, Codex, Cursor or Cline at one URL and keep working when a provider throttles you. - **One key per app or teammate.** Issue and revoke gateway keys without sharing provider credentials. - **Shared team endpoint.** Give the whole team one URL with per-person keys and rate limits. - **Cost tracking.** See tokens and spend by key, model and provider in one dashboard. - **Multi-vendor apps.** Use OpenAI, Anthropic and open models from one SDK. - **Local plus cloud.** Route cheap tasks to Ollama and hard ones to a frontier model. - **Model evaluation.** Try a new model by changing a route instead of redeploying apps. - **MCP and A2A tools.** Expose gateway features to AI agents through OmniRoute's MCP server. --- ## Configuration | Variable | Required | Description | |---|---|---| | `INITIAL_PASSWORD` | 🔐 Auto-generated | First dashboard password. Change it in the UI after signing in. | | `JWT_SECRET` | 🔐 Auto-generated | Signs dashboard session tokens. | | `API_KEY_SECRET` | 🔐 Auto-generated | Encrypts stored provider keys. Keep it stable. | | `MACHINE_ID_SALT` | 🔐 Auto-generated | Salt for derived machine identifiers. | | `BASE_URL`, `NEXT_PUBLIC_BASE_URL` | ✅ Auto-set | `https://` plus the Railway domain, so the dashboard shows the right endpoint. | | `REQUIRE_API_KEY` | ✅ Set | `true`, every `/v1` call needs a gateway key. | | `ALLOW_API_KEY_REVEAL` | ✅ Set | `false`, key values stay masked after creation. | | `AUTH_COOKIE_SECURE` | ✅ Set | `true`, secure cookies behind HTTPS. | | `PORT`, `HOST`, `HOSTNAME` | ✅ Set | `20128` on `0.0.0.0`. | | `DATA_DIR` | ✅ Set | `/app/data`, the volume. | | `OMNIROUTE_MEMORY_MB` | ✅ Set | `2048`, the Node.js heap limit. | | `DISABLE_SQLITE_AUTO_BACKUP` | ✅ Set | `false`, so backups are taken before migrations. | | `RAILWAY_RUN_UID` | ✅ Set | `0`, so the app can write to the volume. | **Don't rotate `API_KEY_SECRET` casually.** It decrypts every saved provider key. If it changes, you'll need to re-enter them. **Custom domain.** Add it to the service, then redeploy so `BASE_URL` follows and the Endpoints page shows the new address. --- ## OmniRoute vs. Alternatives | | OmniRoute on Railway | LiteLLM Proxy | OpenRouter | Direct provider APIs | |---|---|---|---|---| | **Monthly cost** | ~$5–15 hosting | Self-host costs | Fee on credits | None | | **Self-hosted** | ✅ Yes | ✅ Yes | ❌ No | ✅ N/A | | **Uses your own provider keys** | ✅ Yes | ✅ Yes | ⚠️ Optional | ✅ Yes | | **Automatic fallback chains** | ✅ Combos | ✅ Yes | ✅ Yes | ❌ You build it | | **Built-in dashboard** | ✅ Yes | ⚠️ Admin UI | ✅ Hosted | ❌ Per vendor | | **Single service to run** | ✅ SQLite | ⚠️ Postgres for keys | ✅ N/A | ✅ N/A | | **License** | ✅ MIT | ⚠️ MIT core, paid enterprise | ❌ Proprietary | ❌ Proprietary | --- ## Dependencies for OmniRoute Hosting - Railway account. Hobby fits personal use; Pro for heavier team traffic. - API keys or accounts for the model providers you want to route to. - Optional: an Ollama or other OpenAI-compatible server for local models. ### Deployment Dependencies - [OmniRoute on GitHub](https://github.com/diegosouzapw/OmniRoute): source code, docs and releases - [Docker Hub image](https://hub.docker.com/r/diegosouzapw/omniroute): image tags - [OmniRoute website](https://omniroute.online): overview and guides - [Railway Volumes](https://docs.railway.com/volumes): how the database persists ### Implementation Details The image has no version tag, so it follows `latest` (currently 3.8.53) and a redeploy picks up new releases. OmniRoute ships often, so pin a tag such as `3.8.53` if you want controlled upgrades. Before each schema migration it backs up the SQLite file on the volume, which makes rolling back easier. Railway terminates HTTPS at its edge and forwards traffic to port 20128. Because all state lives in one SQLite file, run a single replica and scale up with more memory rather than more copies. --- ## Frequently Asked Questions **What is OmniRoute?** An MIT-licensed AI gateway with one OpenAI-compatible endpoint, provider fallbacks, format translation, API keys and cost analytics, all managed from a web dashboard. **Does OmniRoute include any model credits?** No. It's bring-your-own-key. You connect provider accounts you're entitled to use, and each provider bills or limits you directly. **How much does it cost on Railway?** About $5–15/month for the service and volume, plus whatever your providers charge for tokens. OmniRoute adds no markup. **Can I use it with Claude Code or Cursor?** Yes. Set the tool's base URL to your OmniRoute `/v1` endpoint and use a gateway key; OmniRoute handles the format each tool expects. **Is my gateway protected?** Yes. `REQUIRE_API_KEY` is on, so `/v1` returns 401 without a key, and the dashboard needs the password. **Where is my data stored?** In SQLite on the volume at `/app/data`, including encrypted provider keys and usage logs. Back up that volume to keep it safe. --- ## Why Deploy and Host OmniRoute on Railway? Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it. By deploying OmniRoute on Railway, you get a private AI gateway with one endpoint, automatic fallbacks and cost tracking, on infrastructure you control. ## Similar templates - [Rocky Linux](https://railway.com/deploy/rocky-linux) — Hosted Rocky Linux 9 workspace with SSH and persistent storage. 🚀 - [Foundry Virtual Tabletop](https://railway.com/deploy/X5tR6G) — A Self-Hosted & Modern Roleplaying Platform - [Letta Code Remote](https://railway.com/deploy/letta-code-remote) — Run a Letta Code agent 24/7. No inbound ports, just deploy. Open this page in a browser: https://railway.com/deploy/omniroute-openai-gateway