Deploy Anyray
Self-hosted AI-spend optimizer: gateway, optimizer, and console.
Just deployed
/var/lib/postgresql/data
Just deployed
clickhouse
Just deployed
/var/lib/clickhouse
gateway
Just deployed
/data
optimizer
Just deployed
/data
Just deployed
Just deployed
/data
Redis
Just deployed
/data
Just deployed
Deploy and Host Anyray on Railway
Anyray is a self-hosted AI-spend optimizer. It sits in front of your LLM providers, serves each request as cheaply as it can, and gives you spend visibility and governance — without exposing prompt or response content to humans.
This template deploys the full stack: an OpenAI-compatible gateway, the inference optimizer, an nginx proxy, the Anyray console (Langfuse), and its datastores (Postgres, Redis, ClickHouse, MinIO). All secrets are auto-generated by Railway.
About Hosting Anyray
Hosting Anyray on Railway stands up nine services as one project. The gateway exposes an OpenAI-compatible API; the proxy serves the admin console. After the services deploy, you generate two public domains — one for the proxy (target port 80, the console) and one for the gateway (target port 8787, the API) — add at least one provider key on the gateway, then sign in to the console with the auto-generated ANYRAY_ADMIN_TOKEN. Prompt and response content is encrypted at rest and never logged in plaintext.
Why Deploy Anyray on Railway
Railway provisions and wires all nine services from a single template, auto-generating every secret and datastore password, so there is no manual cluster setup. You keep full control of your data — keys and content stay inside your own Railway project — while Railway handles scaling, networking, and persistence.
Common Use Cases
- Cut the AI-inference spend your employees' coding assistants, agents, and SDK jobs generate.
- Give an organization spend visibility and per-team/per-user attribution for LLM usage.
- Run a privacy-preserving LLM gateway where prompt/response content is never exposed to humans.
Dependencies for Anyray Hosting
- A Railway account/plan able to run nine services (~6-8 GB total, ≥2 GB for the console).
- At least one upstream LLM provider API key (e.g. OpenAI or Anthropic).
Deployment Dependencies
- Anyray docs: https://docs.anyray.ai
- Install repo and template spec: https://github.com/anyrayHQ/install
- Langfuse (console engine): https://github.com/langfuse/langfuse
Implementation Details
Expose only the proxy and gateway services publicly; keep web, worker, optimizer, and the datastores private. Railway's private network is IPv6-only, and the proxy rewrites nginx upstreams to *.railway.internal automatically.
Template Content
