Deploy Deploy Respan Gateway LLM Router
Rust LLM gateway: OpenAI, Anthropic and Gemini APIs to any model
respan-gateway
Just deployed
Deploy and Host self hosted Respan Gateway (LLM Router) on Railway
This template deploys the open-source Respan Gateway as a single Rust service on Railway, with a generated client key and a public domain. It is an independent Railway packaging template, not an official or endorsed listing, and it is not affiliated with the Respan Gateway authors. Railway builds nothing here: it deploys the prebuilt ghcr.io/onepush-dev/respan-gateway:0.3.0 image, which wraps the upstream release binary.
About Hosting Respan Gateway open-source software on Railway (self hosted Respan Gateway template)
This template runs one stateless Respan Gateway service on the upstream 0.3.0 binary, with a public domain on port 8080, automatic HTTPS and a generated client key guarding the API and /ui/. The startup wrapper boots with a placeholder provider so you can reach the UI first; upstream refuses to boot without provider configuration.
Why Deploy Respan Gateway, the open-source LLM gateway on Railway (Railway Free Trial)
If you already run LiteLLM, the main reason to look at Respan Gateway is operational.
Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.
By deploying Respan Gateway on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.
Railway vs Other Hosting Providers and VPS for Respan Gateway self hosting
Respan Gateway is one stateless service; the choice is how much server work you own.
| Provider | What You Get with Railway | What You Get with the Other Provider |
|---|---|---|
| DigitalOcean | A deployed gateway with a public domain and automatic restarts, with no server to patch | A Droplet you secure and update yourself, then install the gateway on it |
| AWS | One template deploy that boots with zero inputs and a generated client key | Broad cloud services, but you wire up networking and the container runtime yourself |
| Hetzner | A managed deploy path with no host operating system to maintain | Low-cost servers where you handle the OS, firewall and gateway process directly |
Common Use Cases for hosted Respan Gateway
- One endpoint for many SDK formats: OpenAI and Anthropic SDK services can share one gateway that translates formats and providers.
- Adding retries and fallback in configuration: A named gateway.toml route can list several models with retries and a hedged second attempt, so apps need no failover code.
- Viewing live request logs: /ui/ shows recent calls with latency, tokens and cost; resets on restart.
- Keeping provider keys out of clients: Provider keys stay in Railway Variables; clients hold only the gateway key.
- Adding MCP tools via gateway.toml: Routed models can call declared MCP tools without client changes.
Dependencies for Respan Gateway Docker hosted on Railway
No volume or database exists, so logs and Responses state reset on restart.
Deployment Dependencies for Managed Respan Gateway Service (one stateless Rust gateway service with no database or volume; provider credentials needed for useful completions after deployment)
| Variable | Set by | Purpose |
|---|---|---|
| RESPAN_CLIENT_KEYS | Template, generated (48 chars) | Gateway API key sent as Bearer, x-api-key or x-goog-api-key; also unlocks /ui/ and the admin API. Comma-separate to add more |
| OPENAI_API_KEY | You, optional | Provider key, auto-detected, needed for useful completions |
| ANTHROPIC_API_KEY | You, optional | Provider key, auto-detected |
| GEMINI_API_KEY | You, optional | Provider key, auto-detected |
| RESPAN_GATEWAY_TOML | You, optional | Full gateway.toml for routes and tools |
| RESPAN_CONFIG | You, optional | Config file path, takes precedence over the TOML variable |
The client key is not a provider key. A single unnamed client key is an admin key, guarding both the API and the web UI. Provider keys are your own pass-through variables.
Implementation Details for Respan Gateway (Using onepush-dev 0.3.0 with unchanged upstream release binary on Debian slim and a startup configuration wrapper)
The image copies the unchanged upstream binary onto debian:bookworm-slim (ca-certificates, tini) as uid 65532.
RESPAN_CONFIG takes precedence over RESPAN_GATEWAY_TOML, and RESPAN_GATEWAY_TOML takes precedence over provider keys found in the environment. Full routes, MCP tools and tracing need a gateway.toml pasted into RESPAN_GATEWAY_TOML, with secrets referenced as env:NAME.
How does Respan Gateway compare against other tools
Respan Gateway vs LiteLLM (Respan Gateway Alternative to LiteLLM)
- Language and footprint: Respan Gateway is one Rust binary; LiteLLM is Python.
- Client formats: Respan Gateway accepts OpenAI Chat Completions, OpenAI Responses, Anthropic Messages and Gemini API clients. Check the LiteLLM docs for the formats your SDKs need.
- Routing and tools: Named routes with fallback, retries and hedged requests live in gateway.toml, and MCP tools attach the same way.
Respan Gateway vs Bifrost (Respan Gateway Alternative to Bifrost)
- Self-hosted fit: Both self-host; Respan Gateway is stateless, no database.
- Live logs: The /ui/ UI shows an in-memory request log with cost, latency and tokens.
- Where Bifrost may win: If you rely on its plugins or ecosystem, moving means rebuilding them.
Respan Gateway vs OpenRouter (Respan Gateway Alternative to OpenRouter)
- Hosting model: OpenRouter is hosted routing; Respan Gateway runs on your own infrastructure.
- Keys and data path: Provider keys sit in Railway Variables; requests go app to gateway to provider.
- Where OpenRouter may win: One hosted account is simpler.
Respan Gateway with Docker vs Respan Gateway on Railway (Railway Respan Gateway Deployment)
- Same binary: The Railway template runs the unchanged upstream binary, behind a different startup wrapper.
- Setup effort: Railway generates a client key and domain automatically; Docker needs host and ports.
- Provider keys: On Railway, add provider keys as Variables and redeploy to pick them up.
How to use Respan Gateway on Railway?
After deploy, place each key.
- Copy the generated RESPAN_CLIENT_KEYS value from the service Variables tab. It is your client key and also opens the web UI.
- Visit your public domain at /ui/ and enter that key.
- Add a provider key such as OPENAI_API_KEY in Railway Variables, then redeploy.
- For OpenAI SDKs, use base URL https://your-domain/v1. For Anthropic or Gemini SDKs, use the bare https://your-domain. Use a model ID from your provider in place of any example name.
- Authenticate requests with the client key as a Bearer token or x-api-key header.
- Run curl https://your-domain/healthz to confirm it is up.
Until a provider key is added, calls return a placeholder 502 error, not useful AI answers.
curl https://your-domain/healthz
curl https://your-domain/v1/chat/completions -H "Authorization: Bearer YOUR_CLIENT_KEY" -H "Content-Type: application/json" -d '{"model":"YOUR_MODEL_ID","messages":[{"role":"user","content":"hi"}]}'
How to self host Respan Gateway on other VPS Services (Respan Gateway self hosting guide)
Clone the Repository
Clone upstream for config examples and docs:
git clone https://github.com/respanai/respan-gateway.git
cd respan-gateway
Install Dependencies
Use the Python package or Docker image:
pip install respan-gateway
# or
docker pull ghcr.io/respanai/respan-gateway:main
Configure Environment Variables
Set a real provider key and a generated client key before binding to a public address.
export OPENAI_API_KEY=your-openai-key
export RESPAN_CLIENT_KEYS=your-generated-client-key
respan-gateway check
Upstream refuses to start with no provider configuration; this Railway wrapper boots with a placeholder so the UI appears. Run respan-gateway check before binding to a public address.
Start the Respan Gateway Application
Start the gateway, then confirm its health route. Replace your-domain with your host:
respan-gateway
curl https://your-domain/healthz
Railway handles domain, HTTPS and restarts; on a VPS you do.
Official Pricing of Respan Gateway (Respan Gateway pricing)
Respan Gateway is open-source (MIT OR Apache-2.0), so self-hosting is free; you pay for infrastructure and providers.
Respan Gateway hosted vs self hosted comparison (Pricing, features, costs, and more)
Respan's managed hosting is not bundled here; optional tracing needs your own RESPAN_API_KEY.
Monthly cost of self hosting Respan Gateway on Railway
Railway bills by usage, so cost depends on RAM, CPU and other services.
System Requirements for Hosting Respan Gateway on a VPS
As one Rust binary, a small VPS with modest memory and a core or two suits light use.
Frequently Asked Questions (FAQs)
Why do calls return 502 when I first deploy?
Until you add a provider key, requests hit a placeholder provider returning 502 "could not be reached." Add one in Railway Variables and redeploy.
Is a provider key the same as the client key?
No. The client key is what your apps send to the gateway. Provider keys are the credentials the gateway uses to call OpenAI, Anthropic or Gemini.
What URL should my SDKs use?
OpenAI SDKs should use your domain with /v1 appended as the base URL. Anthropic and Gemini SDKs use the bare domain.
Where are request logs stored?
Request logs and Responses API state live in memory and are lost on restart; no volume or database.
Can I use routes, fallback, hedging or MCP tools?
Yes, via a full gateway.toml in RESPAN_GATEWAY_TOML.
Is a database required?
No. State is kept in memory.
Template Content
respan-gateway
ghcr.io/onepush-dev/respan-gateway:0.3.0
