---
title: "Deploy Portkey AI Gateway"
description: "One OpenAI-compatible API for 1,600+ LLMs, with retries and fallbacks"
category: "AI/ML"
url: https://railway.com/deploy/portkey-ai-gateway-1
---

# Deploy Portkey AI Gateway

One OpenAI-compatible API for 1,600+ LLMs, with retries and fallbacks

**[Deploy Portkey AI Gateway on Railway](https://railway.com/template/portkey-ai-gateway-1)**

Machine-readable deploy manifest (JSON, validated by TemplateCI): https://railway.com/deploy/portkey-ai-gateway-1/manifest.json

- **Creator:** Agaz Self-Host
- **Category:** AI/ML

## Template content

### Portkey AI Gateway https://cfassets.portkey.ai/logo/dew-color.png

- **Source:** https://github.com/aalfath/portkey-ai-gateway-railway-template
- **Health check:** /healthz
- **Public domain:** Yes

## Documentation

# Deploy and Host Portkey AI Gateway on Railway

Portkey AI Gateway is an open-source (MIT) gateway that gives you one OpenAI-compatible API for 1,600+ models across OpenAI, Anthropic, Google, Azure, AWS Bedrock, Mistral, Groq, Ollama and many other providers. It adds retries, fallbacks, load balancing, timeouts and guardrails through a per-request config, so your apps call a single endpoint.

## About Hosting Portkey AI Gateway

This template runs Portkey AI Gateway 1.15.2 from the official `portkeyai/gateway` image as one service, built from this GitHub repository. The gateway itself has no login, so a small Caddy front in the same container asks every API call for a generated access key (`GATEWAY_API_KEY`) in the `x-portkey-api-key` header, and answers Railway's health check on `/healthz`. The gateway runs headless: its demo console and its live log stream, which would show request headers to anyone, are off. Your provider API keys are sent per request and never stored. The gateway keeps no data, so there is no database or volume to look after.

## Common Use Cases

* One OpenAI-compatible endpoint for your apps, with the model provider picked per request
* Automatic retries, fallbacks to a second provider and load balancing across API keys
* Guardrails, timeouts and request routing in front of self-hosted models such as Ollama or vLLM

## Dependencies for Portkey AI Gateway Hosting

* The gateway service (Node.js, from the official `portkeyai/gateway` image)
* An API key for at least one model provider (OpenAI, Anthropic, …), sent with each request

### Deployment Dependencies

* Portkey AI Gateway: https://github.com/Portkey-AI/gateway (MIT)
* Gateway docs: https://portkey.ai/docs/product/ai-gateway
* Template source: https://github.com/aalfath/portkey-ai-gateway-railway-template

### Implementation Details

Service:

* **Portkey AI Gateway** (public): the gateway on `127.0.0.1:8787` and Caddy on `PORT` 8080. Health check: `/healthz` (the gateway's own `/`). No volume, because the gateway keeps no state. If the gateway or Caddy stops, the container exits with an error, so Railway restarts it.

After deploy, copy `GATEWAY_API_KEY` from the service's Variables tab and call the public URL. Send the key in `x-portkey-api-key`, the provider in `x-portkey-provider` and the provider's key in `Authorization`:

```bash
curl https://YOUR-DOMAIN/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: YOUR-GATEWAY-API-KEY" \
  -H "x-portkey-provider: openai" \
  -H "Authorization: Bearer YOUR-OPENAI-KEY" \
  -d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hello"}]}'
```

With the Portkey SDKs, set `baseURL` to `https://YOUR-DOMAIN/v1` and `apiKey` to `GATEWAY_API_KEY` (the SDKs send it as `x-portkey-api-key`). With the OpenAI SDKs, set `base_url` to `https://YOUR-DOMAIN/v1` and add the two `x-portkey-…` headers as default headers. Requests without the right key get `401`. Routing configs (fallbacks, retries, load balancing) go in the `x-portkey-config` header, as described in Portkey's docs.

Variables: `GATEWAY_API_KEY` (generated, letters and digits; you can replace it with your own, using only letters, digits, `-` and `_`), `PORT` (8080). If you empty `GATEWAY_API_KEY`, the gateway is open to anyone who knows the URL, and the logs warn about it. To reach self-hosted models on Railway's private network, set `TRUSTED_CUSTOM_HOSTS` to their hostnames (for example `ollama.railway.internal`), because the gateway blocks private hosts in `custom_host` by default.

Memory: the gateway uses about 100 MB at rest and fits Railway's 0.5 GB plans (Node's heap is capped at 320 MB).

Updates: the service builds from this template's GitHub repository, which pins the gateway image by version and digest. When a new gateway release is tested here, Railway shows you an update. Read `CHANGELOG.md` before you apply it.

## Why Deploy Portkey AI Gateway on Railway?


Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Portkey AI Gateway on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.



## Similar templates

- [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform.
- [Weaviate](https://railway.com/deploy/weaviate-1) — Vector database with hybrid search and an API key-protected API
- [Möbius Personal AI OS](https://railway.com/deploy/mobius-v01-personal-ai-os) — Self-hosted AI agent that builds and runs apps for you, with memory.

Open this page in a browser: https://railway.com/deploy/portkey-ai-gateway-1
