Deploy OmniRoute — AI Gateway with Persistent Keys

Deploy and Host OmniRoute on Railway

OmniRoute is a self-hosted AI gateway that puts one unified API in front of every LLM provider you use — OpenAI, Anthropic, Google, OpenRouter, Ollama, and any OpenAI-compatible endpoint — with your own keys (BYOK), automatic fallback between providers, and prompt compression to cut token spend. The headline of this build: your configured providers, keys, and settings persist on a volume, so they survive redeploys instead of vanishing every time you ship a change. Point your apps at one endpoint and route across models without rewriting integrations.

What This Template Deploys

A single service with a persistent volume for configuration, provider definitions, and encrypted keys. Point your applications at OmniRoute's endpoint, and it routes each request to the right provider using your own keys.

About Hosting

OmniRoute is a straightforward gateway to run, and this build is specifically about making its state durable — plus the essentials for a secure deploy.

Providers and keys survive redeploys — the whole point of this build. In many gateway setups, configured providers and keys live in the container and vanish on every redeploy, forcing you to re-enter them each time you ship. This template mounts a persistent volume for OmniRoute's config and encrypted key store, so your providers, routing rules, and keys persist across redeploys, restarts, and crashes. Configure once; it stays configured.

Bring your own keys. OmniRoute doesn't resell inference — you add your own provider API keys (OpenAI, Anthropic, Google, OpenRouter, Ollama, or any OpenAI-compatible endpoint), and it routes requests using them. You pay each provider directly and keep full control of your credentials, which never leave your infrastructure.

Automatic fallback keeps apps running. Define a primary provider and fallbacks, and if one is down, rate-limited, or erroring, OmniRoute automatically retries the next — so a single provider outage doesn't take your app down. One endpoint, resilient routing.

Prompt compression cuts your token spend. OmniRoute can compress prompts before they reach the provider, reducing token usage — and therefore cost — on your own API bills, without you rewriting how your app builds prompts. It's an optional efficiency layer on the requests you already send.

Encrypt the key store and set the master key. STORAGE_ENCRYPTION_KEY (a 256-bit key, openssl rand -hex 32) encrypts the stored provider keys at rest — set it and keep it stable, since changing it makes existing stored keys unreadable. Secure the dashboard with strong admin credentials, and keep OmniRoute's endpoint access-controlled since it holds your provider keys.

One endpoint, many models. Applications call OmniRoute's OpenAI-compatible API and select models across providers without per-provider integration code — useful for redundancy or centralizing routing across a team.

Typical cost: ~$5–10/month on Railway for the gateway, plus whatever you pay your LLM providers for usage. OmniRoute is open source; pin the version (v3.8.49) for reproducible deploys.

How It Compares

Hosted routers like OpenRouter are convenient but add a markup and route through their account. Calling each provider directly means building and maintaining fallback and routing yourself. LiteLLM is a strong self-hosted proxy in the same space. OmniRoute's angle here is a self-hosted gateway with durable config, BYOK, automatic fallback, and prompt compression — one endpoint across providers, with your keys and routing on infrastructure you own.

Deploy in Under 5 Minutes

1. Click Deploy on Railway — OmniRoute builds with a volume for config and keys (~2 minutes)
2. Set STORAGE_ENCRYPTION_KEY and strong dashboard admin credentials
3. Confirm the volume is mounted so providers and keys persist
4. Open the dashboard, add your provider API keys, and define routing and fallback rules
5. Point your app at OmniRoute's endpoint and start routing requests

Redeploy to confirm your providers and keys are still there — that's the behavior this build guarantees.

Common Use Cases

• One API for every model — call OpenAI, Anthropic, Google, and more through a single endpoint
• Provider fallback — keep apps running when one provider is down or rate-limited
• Cut token costs — apply prompt compression to reduce spend on your own provider bills
• Centralized team gateway — route and manage LLM access for a team behind one durable endpoint

Configuration

Keys and providers persist — verify with a redeploy. The volume keeps your configuration and encrypted keys across redeploys. Set STORAGE_ENCRYPTION_KEY and keep it stable, or stored keys become unreadable.

Secure the endpoint. OmniRoute holds your provider keys, so use strong dashboard credentials and keep the endpoint access-controlled. You bring your own keys and pay providers directly.

Dependencies for OmniRoute Hosting

• Railway account — ~$5–10/month for the gateway, plus your LLM provider usage
• Your own provider API keys (OpenAI, Anthropic, Google, OpenRouter, Ollama, or compatible)
• A persistent Railway volume for config and keys (included)
• Applications that call an OpenAI-compatible endpoint

Deployment Dependencies

• OmniRoute Documentation
• OpenAI-Compatible API Reference
• Railway Volumes Documentation
• Railway Environment Variables

Implementation Details

The template runs OmniRoute (pinned to v3.8.49 for reproducible deploys) as a single service on port 20128, with a persistent volume holding its configuration, provider definitions, and encrypted key store. The defining behavior of this build is durability: providers, keys, and routing rules are written to the volume rather than ephemeral container storage, so they survive redeploys, restarts, and crashes instead of needing re-entry after every deploy.

OmniRoute operates on a bring-your-own-keys model — it does not resell inference. Users add their own provider API keys (OpenAI, Anthropic, Google, OpenRouter, Ollama, or any OpenAI-compatible endpoint), which are encrypted at rest with STORAGE_ENCRYPTION_KEY, a 256-bit key that must remain stable or the stored keys become unreadable. Applications call OmniRoute's OpenAI-compatible endpoint and route across providers with automatic fallback, so a single provider outage or rate limit doesn't break the app. An optional prompt-compression layer reduces token usage on outbound requests to lower provider costs.

Because the gateway holds provider credentials, use strong dashboard credentials and keep the endpoint access-controlled; Railway provides automatic HTTPS. Config and keys on the volume are the components to back up.

Frequently Asked Questions

Do my providers and keys really survive a redeploy? Yes — that's the point of this build. Configuration and encrypted keys are stored on a persistent volume, so your providers, routing rules, and keys persist across redeploys, restarts, and crashes. Redeploy once to confirm.

Does OmniRoute resell inference? No. It's bring-your-own-keys — you add your own provider API keys and pay each provider directly. Your credentials stay on your infrastructure.

How does fallback work? You set a primary provider and fallbacks; if the primary is down, rate-limited, or erroring, OmniRoute automatically retries the next, keeping your app running through provider issues.

How does prompt compression save money? It compresses prompts before they reach the provider, reducing token usage on your own API bills, without changing how your app builds prompts. It's optional.

What is STORAGE_ENCRYPTION_KEY for? It encrypts your stored provider keys at rest. Generate it with openssl rand -hex 32 and keep it stable — changing it makes existing stored keys unreadable.

Which providers work? OpenAI, Anthropic, Google, OpenRouter, Ollama, and any OpenAI-compatible endpoint, all behind one unified API your apps call.

Why Deploy OmniRoute on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying OmniRoute on Railway you get a self-hosted AI gateway whose providers and keys survive redeploys — a persistent volume for durable config, BYOK with encrypted key storage, automatic fallback, and optional prompt compression. One endpoint across every model, on infrastructure you own.