Deploy and Host Anyray on Railway

Anyray is a self-hosted AI-spend optimizer. It sits in front of your LLM providers, serves each request as cheaply as it can, and gives you spend visibility and governance — without exposing prompt or response content to humans.

This template deploys the full stack: an OpenAI-compatible gateway (which also serves the Anyray console), the inference optimizer, an nginx proxy, and a Postgres datastore for the content-free trace and spend store. All secrets are auto-generated by Railway.

About Hosting Anyray

Hosting Anyray on Railway stands up four services as one project. The gateway exposes an OpenAI-compatible API and the admin console; the proxy serves the console on a public domain. After the services deploy, you generate two public domains — one for the proxy (target port 80, the console) and one for the gateway (target port 8787, the API) — add at least one provider key on the gateway, then sign in to the console with the auto-generated ANYRAY_ADMIN_TOKEN. Prompt and response content is encrypted at rest and never logged in plaintext; the Postgres store keeps metadata only.

Why Deploy Anyray on Railway

Railway provisions and wires all four services from a single template, auto-generating every secret and the Postgres password, so there is no manual setup. You keep full control of your data — keys and content stay inside your own Railway project — while Railway handles scaling, networking, and persistence.

Common Use Cases

Cut the AI-inference spend your employees' coding assistants, agents, and SDK jobs generate.
Give an organization spend visibility and per-team/per-user attribution for LLM usage.
Run a privacy-preserving LLM gateway where prompt/response content is never exposed to humans.

Dependencies for Anyray Hosting

A Railway account/plan able to run four services.
At least one upstream LLM provider API key (e.g. OpenAI or Anthropic).

Deployment Dependencies

Anyray docs: https://docs.anyray.ai
Install repo and template spec: https://github.com/anyrayHQ/install

Implementation Details

Expose only the proxy and gateway services publicly; keep optimizer and Postgres private. Railway's private network is IPv6-only, and the proxy rewrites its nginx upstreams to *.railway.internal automatically.