---
title: "Deploy Plano AI"
description: "An open-source AI proxy for routing, guardrails & observability"
category: "AI/ML"
url: https://railway.com/deploy/plano-ai
---

# Deploy Plano AI

An open-source AI proxy for routing, guardrails & observability

**[Deploy Plano AI on Railway](https://railway.com/template/plano-ai)**

Machine-readable deploy manifest (JSON, validated by TemplateCI): https://railway.com/deploy/plano-ai/manifest.json

- **Creator:** Protemplate
- **Category:** AI/ML
- **Total deploys:** 10

## Template content

### Plano https://somi-public-assets.s3.ap-southeast-1.amazonaws.com/railway/plano-logo.svg

- **Source:** https://github.com/protemplate/plano-railway
- **Health check:** /healthz
- **Public domain:** Yes

## Documentation

# Deploy and Host Plano on Railway

Plano is an AI-native proxy built on Envoy that provides a unified, OpenAI-compatible gateway for routing requests across multiple LLM providers. It supports intelligent model selection based on prompt intent, built-in guardrails for safety filtering and jailbreak detection, and automatic OpenTelemetry tracing for observability across all requests.
<img src="https://github.com/katanemo/plano/raw/main/docs/source/_static/img/PlanoTagline.svg" alt="Plano">
<img src="https://github.com/katanemo/plano/raw/main/docs/source/_static/img/plano_network_diagram_high_level.png" alt="Plano">

## About Hosting Plano

Deploying Plano requires a containerized environment with network access to external LLM provider APIs. The service is stateless and needs no persistent storage or database — configuration is generated at startup from environment variables. You provide API keys for one or more LLM providers (OpenAI, Anthropic, Google, Groq, Mistral, DeepSeek, xAI, Together AI), and Plano auto-generates its Envoy proxy configuration. The service exposes a single HTTP port for its OpenAI-compatible gateway endpoint. For advanced setups involving intelligent routing between multiple providers, you can supply a full YAML configuration directly via environment variables.

## Common Use Cases

- **Unified LLM Gateway** — Expose a single OpenAI-compatible `/v1/chat/completions` endpoint that routes to any of 8 supported providers, letting applications switch models without code changes.
- **Cost-Optimized Routing** — Use intelligent routing to automatically direct simple queries to cheaper models (e.g., GPT-4o-mini) and complex reasoning tasks to more capable models (e.g., Claude Sonnet), reducing overall API spend.
- **Multi-Provider Failover** — Configure multiple LLM providers behind one endpoint so that if one provider is unavailable, traffic can be routed to an alternative.

## Dependencies for Plano Hosting

- **At least one LLM provider API key** — One or more of: OpenAI, Anthropic, Google, Groq, Mistral, DeepSeek, xAI, or Together AI.
- **A containerized hosting platform** — Docker-compatible environment with outbound HTTPS access to LLM provider APIs.

### Deployment Dependencies

- [Plano GitHub Repository](https://github.com/katanemo/plano)
- [Plano Documentation](https://docs.planoai.dev)
- [Railway Template Deploy Link](https://railway.com/deploy/1jWZbX)

### Implementation Details

Plano supports three configuration modes, selected automatically based on which environment variables are set:

```bash
# Mode 1: Auto-generate config from individual env vars (recommended)
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
PLANO_DEFAULT_PROVIDER=anthropic

# Mode 2: Full YAML config via env var
PLANO_CONFIG_YAML="version: v0.1.0
listeners:
  egress_traffic:
    address: 0.0.0.0
    port: 12000
    message_format: openai
model_providers:
  - model: openai/gpt-4o
    access_key: \$OPENAI_API_KEY
    default: true"

# Mode 3: Base64-encoded YAML config
PLANO_CONFIG_BASE64=dmVyc2lvbjogdjAuMS4w...
```

### Environment Variables

The following environment variables change Plano's runtime configuration when set in Railway.

#### Required provider keys (set at least one)

| Variable | Purpose | Example |
|----------|---------|---------|
| `OPENAI_API_KEY` | Enables OpenAI as a model provider | `sk-...` |
| `ANTHROPIC_API_KEY` | Enables Anthropic as a model provider | `sk-ant-...` |
| `GOOGLE_API_KEY` | Enables Google Gemini as a model provider | `AIza...` |
| `GROQ_API_KEY` | Enables Groq as a model provider | `gsk_...` |
| `MISTRAL_API_KEY` | Enables Mistral as a model provider | `...` |
| `DEEPSEEK_API_KEY` | Enables DeepSeek as a model provider | `...` |
| `XAI_API_KEY` | Enables xAI (Grok) as a model provider | `xai-...` |
| `TOGETHER_API_KEY` | Enables Together AI as a model provider | `...` |

#### Provider and model selection

| Variable | Purpose | Default / Example |
|----------|---------|-------------------|
| `PLANO_DEFAULT_PROVIDER` | Chooses the default provider when multiple are configured | `openai`, `anthropic`, `google` |
| `PLANO_OPENAI_MODEL` | Overrides the OpenAI model name | `openai/gpt-4o` |
| `PLANO_ANTHROPIC_MODEL` | Overrides the Anthropic model name | `anthropic/claude-sonnet-4-5` |
| `PLANO_GOOGLE_MODEL` | Overrides the Google model name | `gemini/gemini-2.5-flash` |
| `PLANO_GROQ_MODEL` | Overrides the Groq model name | `groq/llama-3.3-70b-versatile` |
| `PLANO_MISTRAL_MODEL` | Overrides the Mistral model name | `mistral/mistral-large-latest` |
| `PLANO_DEEPSEEK_MODEL` | Overrides the DeepSeek model name | `deepseek/deepseek-chat` |
| `PLANO_XAI_MODEL` | Overrides the xAI model name | `xai/grok-3` |
| `PLANO_TOGETHER_MODEL` | Overrides the Together AI model name | `together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo` |

#### Routing, timeout, tracing, and logging

| Variable | Purpose | Default / Example |
|----------|---------|-------------------|
| `PLANO_ROUTING_MODEL` | Enables routing and sets the routing model | `Arch-Router` |
| `PLANO_ROUTING_PROVIDER` | Sets the provider used for routing decisions | `arch-router` |
| `PLANO_TIMEOUT` | Sets request timeout in the generated config | `30s` |
| `PLANO_TRACE_SAMPLING` | Enables OpenTelemetry sampling when greater than 0 | `10` |
| `LOG_LEVEL` | Sets runtime logging level | `info` |
| `PORT` | Sets Plano's listener port | `12000` |

#### Full config override

Use one of these to bypass auto-generated configuration entirely:

| Variable | Purpose |
|----------|---------|
| `PLANO_CONFIG_YAML` | Supplies the full Plano YAML config directly |
| `PLANO_CONFIG_BASE64` | Supplies the full Plano YAML config as base64 |

#### Variables mentioned for integration, not Plano config

These may appear in Railway examples, but they do not directly configure Plano itself:

- `OPENAI_BASE_URL` - Used by another application or service to call Plano.
- `RAILWAY_PRIVATE_DOMAIN` - Railway-provided private networking variable.


For Railway private networking, other services in the same project connect via:

```bash
OPENAI_BASE_URL=http://${{Plano.RAILWAY_PRIVATE_DOMAIN}}:${{Plano.PORT}}/v1
```
or
```bash
API_BASE_URL=http://${{Plano.RAILWAY_PRIVATE_DOMAIN}}:${{Plano.PORT}}/v1
```


## Why Deploy Plano on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Plano on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.


## Similar templates

- [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform.
- [Möbius v0.1 Personal AI OS](https://railway.com/deploy/mobius-v01-personal-ai-os) — Self-hosted AI agent that builds and runs apps for you, with memory.
- [stella](https://railway.com/deploy/stella) — Self-host stella with web, API, Postgres, Redis, and object storage.

Open this page in a browser: https://railway.com/deploy/plano-ai