Deploy Portkey AI Gateway

One OpenAI-compatible API for 1,600+ LLMs, with retries and fallbacks

Deploy Portkey AI Gateway

Just deployed

Deploy and Host Portkey AI Gateway on Railway

Portkey AI Gateway is an open-source (MIT) gateway that gives you one OpenAI-compatible API for 1,600+ models across OpenAI, Anthropic, Google, Azure, AWS Bedrock, Mistral, Groq, Ollama and many other providers. It adds retries, fallbacks, load balancing, timeouts and guardrails through a per-request config, so your apps call a single endpoint.

About Hosting Portkey AI Gateway

This template runs Portkey AI Gateway 1.15.2 from the official portkeyai/gateway image as one service, built from this GitHub repository. The gateway itself has no login, so a small Caddy front in the same container asks every API call for a generated access key (GATEWAY_API_KEY) in the x-portkey-api-key header, and answers Railway's health check on /healthz. The gateway runs headless: its demo console and its live log stream, which would show request headers to anyone, are off. Your provider API keys are sent per request and never stored. The gateway keeps no data, so there is no database or volume to look after.

Common Use Cases

  • One OpenAI-compatible endpoint for your apps, with the model provider picked per request
  • Automatic retries, fallbacks to a second provider and load balancing across API keys
  • Guardrails, timeouts and request routing in front of self-hosted models such as Ollama or vLLM

Dependencies for Portkey AI Gateway Hosting

  • The gateway service (Node.js, from the official portkeyai/gateway image)
  • An API key for at least one model provider (OpenAI, Anthropic, …), sent with each request

Deployment Dependencies

Implementation Details

Service:

  • Portkey AI Gateway (public): the gateway on 127.0.0.1:8787 and Caddy on PORT 8080. Health check: /healthz (the gateway's own /). No volume, because the gateway keeps no state. If the gateway or Caddy stops, the container exits with an error, so Railway restarts it.

After deploy, copy GATEWAY_API_KEY from the service's Variables tab and call the public URL. Send the key in x-portkey-api-key, the provider in x-portkey-provider and the provider's key in Authorization:

curl https://YOUR-DOMAIN/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: YOUR-GATEWAY-API-KEY" \
  -H "x-portkey-provider: openai" \
  -H "Authorization: Bearer YOUR-OPENAI-KEY" \
  -d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hello"}]}'

With the Portkey SDKs, set baseURL to https://YOUR-DOMAIN/v1 and apiKey to GATEWAY_API_KEY (the SDKs send it as x-portkey-api-key). With the OpenAI SDKs, set base_url to https://YOUR-DOMAIN/v1 and add the two x-portkey-… headers as default headers. Requests without the right key get 401. Routing configs (fallbacks, retries, load balancing) go in the x-portkey-config header, as described in Portkey's docs.

Variables: GATEWAY_API_KEY (generated, letters and digits; you can replace it with your own, using only letters, digits, - and _), PORT (8080). If you empty GATEWAY_API_KEY, the gateway is open to anyone who knows the URL, and the logs warn about it. To reach self-hosted models on Railway's private network, set TRUSTED_CUSTOM_HOSTS to their hostnames (for example ollama.railway.internal), because the gateway blocks private hosts in custom_host by default.

Memory: the gateway uses about 100 MB at rest and fits Railway's 0.5 GB plans (Node's heap is capped at 320 MB).

Updates: the service builds from this template's GitHub repository, which pins the gateway image by version and digest. When a new gateway release is tested here, Railway shows you an update. Read CHANGELOG.md before you apply it.

Why Deploy Portkey AI Gateway on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Portkey AI Gateway on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.


Template Content

More templates in this category

View Template
Chat Chat
Chat Chat, your own unified chat and search to AI platform.

okisdev
117
View Template
Weaviate
Vector database with hybrid search and an API key-protected API

GitHub Locker
1
View Template
Möbius Personal AI OS
Self-hosted AI agent that builds and runs apps for you, with memory.

Tracklayer
1