
Deploy Portkey AI Gateway
One OpenAI-compatible API for 1,600+ LLMs, with retries and fallbacks
Portkey AI Gateway
Just deployed
Deploy and Host Portkey AI Gateway on Railway
Portkey AI Gateway is an open-source (MIT) gateway that gives you one OpenAI-compatible API for 1,600+ models across OpenAI, Anthropic, Google, Azure, AWS Bedrock, Mistral, Groq, Ollama and many other providers. It adds retries, fallbacks, load balancing, timeouts and guardrails through a per-request config, so your apps call a single endpoint.
About Hosting Portkey AI Gateway
This template runs Portkey AI Gateway 1.15.2 from the official portkeyai/gateway image as one service, built from this GitHub repository. The gateway itself has no login, so a small Caddy front in the same container asks every API call for a generated access key (GATEWAY_API_KEY) in the x-portkey-api-key header, and answers Railway's health check on /healthz. The gateway runs headless: its demo console and its live log stream, which would show request headers to anyone, are off. Your provider API keys are sent per request and never stored. The gateway keeps no data, so there is no database or volume to look after.
Common Use Cases
- One OpenAI-compatible endpoint for your apps, with the model provider picked per request
- Automatic retries, fallbacks to a second provider and load balancing across API keys
- Guardrails, timeouts and request routing in front of self-hosted models such as Ollama or vLLM
Dependencies for Portkey AI Gateway Hosting
- The gateway service (Node.js, from the official
portkeyai/gatewayimage) - An API key for at least one model provider (OpenAI, Anthropic, …), sent with each request
Deployment Dependencies
- Portkey AI Gateway: https://github.com/Portkey-AI/gateway (MIT)
- Gateway docs: https://portkey.ai/docs/product/ai-gateway
- Template source: https://github.com/aalfath/portkey-ai-gateway-railway-template
Implementation Details
Service:
- Portkey AI Gateway (public): the gateway on
127.0.0.1:8787and Caddy onPORT8080. Health check:/healthz(the gateway's own/). No volume, because the gateway keeps no state. If the gateway or Caddy stops, the container exits with an error, so Railway restarts it.
After deploy, copy GATEWAY_API_KEY from the service's Variables tab and call the public URL. Send the key in x-portkey-api-key, the provider in x-portkey-provider and the provider's key in Authorization:
curl https://YOUR-DOMAIN/v1/chat/completions \
-H "Content-Type: application/json" \
-H "x-portkey-api-key: YOUR-GATEWAY-API-KEY" \
-H "x-portkey-provider: openai" \
-H "Authorization: Bearer YOUR-OPENAI-KEY" \
-d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hello"}]}'
With the Portkey SDKs, set baseURL to https://YOUR-DOMAIN/v1 and apiKey to GATEWAY_API_KEY (the SDKs send it as x-portkey-api-key). With the OpenAI SDKs, set base_url to https://YOUR-DOMAIN/v1 and add the two x-portkey-… headers as default headers. Requests without the right key get 401. Routing configs (fallbacks, retries, load balancing) go in the x-portkey-config header, as described in Portkey's docs.
Variables: GATEWAY_API_KEY (generated, letters and digits; you can replace it with your own, using only letters, digits, - and _), PORT (8080). If you empty GATEWAY_API_KEY, the gateway is open to anyone who knows the URL, and the logs warn about it. To reach self-hosted models on Railway's private network, set TRUSTED_CUSTOM_HOSTS to their hostnames (for example ollama.railway.internal), because the gateway blocks private hosts in custom_host by default.
Memory: the gateway uses about 100 MB at rest and fits Railway's 0.5 GB plans (Node's heap is capped at 320 MB).
Updates: the service builds from this template's GitHub repository, which pins the gateway image by version and digest. When a new gateway release is tested here, Railway shows you an update. Read CHANGELOG.md before you apply it.
Why Deploy Portkey AI Gateway on Railway?
Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.
By deploying Portkey AI Gateway on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.
Template Content
Portkey AI Gateway
aalfath/portkey-ai-gateway-railway-template
