--- title: "Deploy OmniRoute — AI Gateway with Persistent Keys" description: "Self-host an AI gateway — providers & keys survive redeploys" category: "Other" url: https://railway.com/deploy/omniroute-ai-gateway --- # Deploy OmniRoute — AI Gateway with Persistent Keys Self-host an AI gateway — providers & keys survive redeploys **[Deploy OmniRoute — AI Gateway with Persistent Keys on Railway](https://railway.com/template/omniroute-ai-gateway)** - **Creator:** SilverBanana - **Category:** Other - **Total deploys:** 12 ## Template content ### omniroute https://raw.githubusercontent.com/diegosouzapw/OmniRoute/7163081f5ed2a2104e85c31bfd1588033d43c580/public/icon-512.png - **Image:** ghcr.io/bon5co/omniroute-railway ## Documentation # Deploy and Host OmniRoute on Railway OmniRoute is a self-hosted AI gateway that puts one unified API in front of every LLM provider you use — OpenAI, Anthropic, Google, OpenRouter, Ollama, and any OpenAI-compatible endpoint — with your own keys (BYOK), automatic fallback between providers, and prompt compression to cut token spend. The headline of this build: your configured providers, keys, and settings persist on a volume, so they survive redeploys instead of vanishing every time you ship a change. Point your apps at one endpoint and route across models without rewriting integrations. --- ## What This Template Deploys | Service | Purpose | | --- | --- | | **OmniRoute** | The AI gateway — unified API, provider routing, and dashboard on port `20128` | A single service with a persistent volume for configuration, provider definitions, and encrypted keys. Point your applications at OmniRoute's endpoint, and it routes each request to the right provider using your own keys. --- ## About Hosting OmniRoute is a straightforward gateway to run, and this build is specifically about making its state durable — plus the essentials for a secure deploy. **Providers and keys survive redeploys — the whole point of this build.** In many gateway setups, configured providers and keys live in the container and vanish on every redeploy, forcing you to re-enter them each time you ship. This template mounts a persistent volume for OmniRoute's config and encrypted key store, so your providers, routing rules, and keys persist across redeploys, restarts, and crashes. Configure once; it stays configured. **Bring your own keys.** OmniRoute doesn't resell inference — you add your own provider API keys (OpenAI, Anthropic, Google, OpenRouter, Ollama, or any OpenAI-compatible endpoint), and it routes requests using them. You pay each provider directly and keep full control of your credentials, which never leave your infrastructure. **Automatic fallback keeps apps running.** Define a primary provider and fallbacks, and if one is down, rate-limited, or erroring, OmniRoute automatically retries the next — so a single provider outage doesn't take your app down. One endpoint, resilient routing. **Prompt compression cuts your token spend.** OmniRoute can compress prompts before they reach the provider, reducing token usage — and therefore cost — on your own API bills, without you rewriting how your app builds prompts. It's an optional efficiency layer on the requests you already send. **Encrypt the key store and set the master key.** `STORAGE_ENCRYPTION_KEY` (a 256-bit key, `openssl rand -hex 32`) encrypts the stored provider keys at rest — set it and keep it stable, since changing it makes existing stored keys unreadable. Secure the dashboard with strong admin credentials, and keep OmniRoute's endpoint access-controlled since it holds your provider keys. **One endpoint, many models.** Applications call OmniRoute's OpenAI-compatible API and select models across providers without per-provider integration code — useful for redundancy or centralizing routing across a team. Typical cost: **~$5–10/month** on Railway for the gateway, plus whatever you pay your LLM providers for usage. OmniRoute is open source; pin the version (`v3.8.49`) for reproducible deploys. --- ## How It Compares | | OmniRoute (self-hosted) | OpenRouter (hosted) | Direct to each provider | LiteLLM | | --- | --- | --- | --- | --- | | Keys | Your own (BYOK) | Their account | Your own | Your own | | Routing + fallback | Yes | Yes | Manual | Yes | | Prompt compression | Yes | No | No | Varies | | Data ownership | Full — your infra | Vendor | Full | Full | | Cost model | Flat infra + usage | Markup on usage | Usage | Flat infra + usage | | Self-hostable | Yes | No | N/A | Yes | Hosted routers like OpenRouter are convenient but add a markup and route through their account. Calling each provider directly means building and maintaining fallback and routing yourself. LiteLLM is a strong self-hosted proxy in the same space. OmniRoute's angle here is a self-hosted gateway with durable config, BYOK, automatic fallback, and prompt compression — one endpoint across providers, with your keys and routing on infrastructure you own. --- ## Deploy in Under 5 Minutes 1. Click **Deploy on Railway** — OmniRoute builds with a volume for config and keys (~2 minutes) 2. Set `STORAGE_ENCRYPTION_KEY` and strong dashboard admin credentials 3. Confirm the volume is mounted so providers and keys persist 4. Open the dashboard, add your provider API keys, and define routing and fallback rules 5. Point your app at OmniRoute's endpoint and start routing requests Redeploy to confirm your providers and keys are still there — that's the behavior this build guarantees. --- ## Common Use Cases - **One API for every model** — call OpenAI, Anthropic, Google, and more through a single endpoint - **Provider fallback** — keep apps running when one provider is down or rate-limited - **Cut token costs** — apply prompt compression to reduce spend on your own provider bills - **Centralized team gateway** — route and manage LLM access for a team behind one durable endpoint --- ## Configuration | Variable | Required | Description | | --- | --- | --- | | `STORAGE_ENCRYPTION_KEY` | Required | 256-bit key (`openssl rand -hex 32`) encrypting stored provider keys — keep stable | | `PORT` | Pre-set | `20128` — the gateway API and dashboard | | Admin credentials | Required | Strong username/password for the dashboard | | Storage volume | Pre-set | Persistent volume for config, providers, and encrypted keys | | Provider API keys | Required | Your own keys, added in the dashboard (not resold) | > **Keys and providers persist — verify with a redeploy.** The volume keeps your configuration and encrypted keys across redeploys. Set `STORAGE_ENCRYPTION_KEY` and keep it stable, or stored keys become unreadable. > **Secure the endpoint.** OmniRoute holds your provider keys, so use strong dashboard credentials and keep the endpoint access-controlled. You bring your own keys and pay providers directly. --- ## Dependencies for OmniRoute Hosting - Railway account — ~$5–10/month for the gateway, plus your LLM provider usage - Your own provider API keys (OpenAI, Anthropic, Google, OpenRouter, Ollama, or compatible) - A persistent Railway volume for config and keys (included) - Applications that call an OpenAI-compatible endpoint ### Deployment Dependencies - [OmniRoute Documentation](https://docs.railway.com/guides/private-networking) - [OpenAI-Compatible API Reference](https://platform.openai.com/docs/api-reference) - [Railway Volumes Documentation](https://docs.railway.com/volumes) - [Railway Environment Variables](https://docs.railway.com/guides/variables) ### Implementation Details The template runs OmniRoute (pinned to `v3.8.49` for reproducible deploys) as a single service on port `20128`, with a persistent volume holding its configuration, provider definitions, and encrypted key store. The defining behavior of this build is durability: providers, keys, and routing rules are written to the volume rather than ephemeral container storage, so they survive redeploys, restarts, and crashes instead of needing re-entry after every deploy. OmniRoute operates on a bring-your-own-keys model — it does not resell inference. Users add their own provider API keys (OpenAI, Anthropic, Google, OpenRouter, Ollama, or any OpenAI-compatible endpoint), which are encrypted at rest with `STORAGE_ENCRYPTION_KEY`, a 256-bit key that must remain stable or the stored keys become unreadable. Applications call OmniRoute's OpenAI-compatible endpoint and route across providers with automatic fallback, so a single provider outage or rate limit doesn't break the app. An optional prompt-compression layer reduces token usage on outbound requests to lower provider costs. Because the gateway holds provider credentials, use strong dashboard credentials and keep the endpoint access-controlled; Railway provides automatic HTTPS. Config and keys on the volume are the components to back up. --- ## Frequently Asked Questions **Do my providers and keys really survive a redeploy?** Yes — that's the point of this build. Configuration and encrypted keys are stored on a persistent volume, so your providers, routing rules, and keys persist across redeploys, restarts, and crashes. Redeploy once to confirm. **Does OmniRoute resell inference?** No. It's bring-your-own-keys — you add your own provider API keys and pay each provider directly. Your credentials stay on your infrastructure. **How does fallback work?** You set a primary provider and fallbacks; if the primary is down, rate-limited, or erroring, OmniRoute automatically retries the next, keeping your app running through provider issues. **How does prompt compression save money?** It compresses prompts before they reach the provider, reducing token usage on your own API bills, without changing how your app builds prompts. It's optional. **What is `STORAGE_ENCRYPTION_KEY` for?** It encrypts your stored provider keys at rest. Generate it with `openssl rand -hex 32` and keep it stable — changing it makes existing stored keys unreadable. **Which providers work?** OpenAI, Anthropic, Google, OpenRouter, Ollama, and any OpenAI-compatible endpoint, all behind one unified API your apps call. --- ## Why Deploy OmniRoute on Railway? Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it. By deploying OmniRoute on Railway you get a self-hosted AI gateway whose providers and keys survive redeploys — a persistent volume for durable config, BYOK with encrypted key storage, automatic fallback, and optional prompt compression. One endpoint across every model, on infrastructure you own. ## Similar templates - [Rocky Linux](https://railway.com/deploy/rocky-linux) — [Jul'26] Hosted Rocky Linux 9 workspace with SSH and persistent storage. 🚀 - [Foundry Virtual Tabletop](https://railway.com/deploy/X5tR6G) — A Self-Hosted & Modern Roleplaying Platform - [Letta Code Remote](https://railway.com/deploy/letta-code-remote) — Run a Letta Code agent 24/7. No inbound ports, just deploy. Open this page in a browser: https://railway.com/deploy/omniroute-ai-gateway