Deploy OmniRoute — LLM Gateway with Prompt Compression

Deploy and Host OmniRoute on Railway

OmniRoute is an open-source LLM gateway — one OpenAI-compatible endpoint in front of your API keys across many providers (OpenAI, Anthropic, Google, DeepSeek, Groq, Mistral, and more). Point any OpenAI-compatible tool at a single URL, and OmniRoute handles routing, automatic fallback, format translation, and transparent prompt compression that can cut token usage on every request. This template self-hosts it so your keys and traffic stay on infrastructure you own.

What This Template Deploys

A single Node.js service backed by SQLite. A persistent volume at /app/data holds the database — your provider configuration, routing rules, and usage history — so nothing is lost on redeploy. No external database needed.

About Hosting

OmniRoute sits between your tools and your model providers as an OpenAI-compatible proxy, and there are a few things worth setting up correctly.

Bring your own provider keys. OmniRoute is a routing layer, not a model host — you add your own API keys for the providers you use, and requests are billed by those providers directly. It unifies them behind one endpoint so switching or combining providers happens in the dashboard, not in your application code. Point your OpenAI-compatible client at https://your-domain/v1 with a key issued from the dashboard, and it works unchanged.

Transparent compression can lower your token spend. OmniRoute can compress prompts before they reach a provider, reducing the tokens billed on your own keys. It offers several modes from light to aggressive, and the savings vary widely by content — tool-heavy sessions compress far more than short prompts — so treat the range as "up to," not a fixed number, and benchmark against your own traffic. It's transparent to the client: no changes to your app.

Persist the data volume, and encrypt it. All state — provider config, routing, and history — lives in SQLite under DATA_DIR (/app/data). Mount the volume or a redeploy wipes your setup. Because that database holds provider credentials, set STORAGE_ENCRYPTION_KEY (openssl rand -hex 32) to encrypt it at rest, and back the key up — losing it means the encrypted database can't be read.

Secure the dashboard. The instance holds all your provider keys, so protect it. Set a strong dashboard credential and rely on Railway's automatic HTTPS; never leave it openly reachable.

One practical note: OmniRoute is a fast-moving project that ships frequently. Pin a specific image tag rather than latest and upgrade deliberately, so an update doesn't change behavior under you.

Typical cost: ~$5–10/month on Railway for the single service, plus whatever you pay your providers directly. OmniRoute is MIT-licensed and free.

How It Compares

OpenRouter is managed but adds a fee and routes through its cloud. LiteLLM is a capable self-hosted proxy without built-in compression. A single-provider SDK ties you to one vendor. OmniRoute's distinction is a self-hosted, zero-markup gateway that adds transparent prompt compression and automatic fallback on top of provider unification — with your keys and traffic on infrastructure you control.

Deploy in Under 5 Minutes

1. Click Deploy on Railway — OmniRoute builds automatically (~2 minutes)
2. Confirm the volume is mounted at /app/data and set STORAGE_ENCRYPTION_KEY
3. Set a strong dashboard password before the service is publicly reachable
4. Open the dashboard, add your provider API keys, and configure routing and compression
5. Copy the endpoint key and point your tool at https://your-domain/v1

Any OpenAI-compatible client — SDK, CLI, or IDE extension — connects with a single base-URL change.

Common Use Cases

• One endpoint for many providers — unify your OpenAI, Anthropic, Google, and other keys behind a single URL
• Lower your token bill — apply transparent compression to reduce tokens billed on your own provider keys
• Reliable requests — automatic fallback to another configured provider when one is down or rate-limited
• No provider lock-in — switch or combine providers from the dashboard without touching application code
• Format flexibility — translate between OpenAI, Claude, and Gemini API formats so any tool works with any provider
• Centralized key management — keep provider keys in one gateway instead of scattered across projects

Configuration

Persist and encrypt the data volume. All provider config and credentials live in SQLite under /app/data. Mount the volume so it survives redeploys, and set STORAGE_ENCRYPTION_KEY to encrypt it — then back the key up, since losing it makes the database unreadable.

Bring your own keys and secure the dashboard. OmniRoute routes to providers you supply keys for and doesn't mark up usage. The dashboard holds those keys, so use a strong password and rely on Railway's HTTPS — never leave it open.

Dependencies for OmniRoute Hosting

• Railway account — ~$5–10/month for the single service, plus your provider usage
• A persistent Railway volume mounted at /app/data (included)
• Your own API keys for the providers you want to route to
• Node.js 22+ runtime (provided by the image)

Deployment Dependencies

• OmniRoute GitHub Repository
• OmniRoute Environment Reference
• OmniRoute Docker Deployment
• Railway Volumes Documentation

Implementation Details

The template runs the official diegosouzapw/omniroute image on port 20128, serving both the management dashboard and the OpenAI-compatible /v1 API from a single port. OmniRoute uses SQLite (via better-sqlite3) for all persistence, stored under DATA_DIR — set to /app/data and backed by a Railway volume so provider configuration, routing rules, and usage history survive redeploys.

Because the database holds provider credentials, STORAGE_ENCRYPTION_KEY enables AES-256 encryption of the SQLite store at rest; the key must be preserved to read existing data and backups. Requests arrive in OpenAI format and are forwarded to whichever provider you've configured, with optional prompt compression applied transparently before dispatch and automatic fallback to an alternate provider on failure. Format translation lets OpenAI-, Claude-, and Gemini-shaped tools all use the same endpoint.

OmniRoute runs on Node.js 22+ and releases frequently, so pinning a specific image tag and taking a database backup before upgrades is recommended. All routing runs on your infrastructure with the providers you supply, so no traffic passes through a third-party gateway cloud.

Frequently Asked Questions

Does OmniRoute mark up my usage? No. You bring your own provider API keys and pay those providers directly; OmniRoute is a self-hosted routing layer with no usage fee. The only cost is Railway infrastructure.

How much does compression actually save? It varies widely by content — long, tool-heavy sessions compress far more than short prompts. Treat published figures as an upper range and benchmark against your own traffic; compression is transparent to your app either way.

How do I connect my tools? Point any OpenAI-compatible client at https://your-domain/v1 with a key from the dashboard. SDKs, CLIs, and IDE extensions work with just a base-URL change.

Does my configuration persist? Yes, on the /app/data volume. Provider setup, routing, and history live in SQLite there and survive redeploys. Set STORAGE_ENCRYPTION_KEY to encrypt it and back the key up.

Is my traffic private? Yes — self-hosted, requests go only to the providers you configure, on your own infrastructure, with nothing routed through a third-party gateway cloud.

Why should I secure the dashboard? It stores all your provider credentials. On a public Railway domain, use a strong password and HTTPS so no one else can reach your keys.

Should I pin the version? Yes. OmniRoute ships frequently, so pin a specific image tag rather than latest, back up the database before upgrades, and update deliberately.

Why Deploy OmniRoute on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying OmniRoute on Railway you get a self-hosted LLM gateway with the setup handled — the data volume persisted and encryptable, the dashboard and API on one port, and automatic HTTPS. One endpoint for your providers, transparent compression to trim token spend, and automatic fallback, with your keys and traffic on infrastructure you own.