Deploy and Host phoenix (by arize): LLM Evals and Observability - Open Source on Railway

Phoenix is Arize's open-source LLM observability platform. It captures OpenTelemetry traces from every LLM call, runs LLM-as-judge evaluations and experiments over datasets, and gives you trace waterfalls, a prompt playground, and embeddings views — a self-hosted control plane for debugging and evaluating AI applications.

About Hosting phoenix (by arize): LLM Evals and Observability - Open Source

Hosting Phoenix on Railway is a single-container deployment: the official arizephoenix/phoenix image serves the web UI and OpenTelemetry ingest endpoints (HTTP on port 6006, gRPC on 4317). A persistent volume stores your traces, evaluation runs, and datasets in SQLite out of the box — swap in PostgreSQL via environment variables when you outgrow it. Point your applications' OpenInference/OpenTelemetry instrumentors at your Railway URL and traces start flowing immediately. Optional authentication protects the UI and ingest. Scaling is a slider: more RAM/CPU for larger trace volumes, or a lean CPU-only footprint for small teams.

Common Use Cases

Trace every LLM call in OpenAI, LangChain, LlamaIndex, Vercel AI SDK, and custom agents via OpenTelemetry
Run LLM-as-judge evals and experiments over datasets to catch quality regressions before shipping
Debug RAG pipelines span-by-span — retrieval, reranking, and generation in one waterfall
Iterate on prompts in the playground against saved datasets and versioned experiments
Analyze embeddings — drift detection and failure clustering across model versions

Dependencies for phoenix (by arize): LLM Evals and Observability - Open Source Hosting

Phoenix server — official arizephoenix/phoenix image; web UI + OTLP HTTP on 6006, OTLP gRPC on 4317
Persistent volume — SQLite storage for traces, evals, and datasets (PHOENIX_WORKING_DIR=/data)
Optional: PostgreSQL — set PHOENIX_SQL_DATABASE_URL for production-scale storage instead of SQLite
App-side SDKs — arize-phoenix-otel / openinference-instrumentation-* packages, or any OTLP-capable tracer

Implementation Details

Instrument any Python application to stream traces to your Railway-hosted Phoenix:

from phoenix.otel import register

tracer_provider = register(
    endpoint="https://phoenix-yourapp.up.railway.app/v1/traces",  # your Railway domain
    protocol="otlp/http",
)
# OpenInference instrumentors (OpenAI, LangChain, LlamaIndex, ...) plug into this provider

Self-hosting outside Railway with Docker:

docker run -d --name phoenix -p 6006:6006 -p 4317:4317 \
  -v ./phoenix-data:/data \
  -e PHOENIX_WORKING_DIR=/data \
  arizephoenix/phoenix:latest

Optional UI/ingest authentication:

PHOENIX_ENABLE_AUTH=true
PHOENIX_ADMIN_SECRET=

Why Deploy phoenix (by arize): LLM Evals and Observability - Open Source on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying phoenix (by arize): LLM Evals and Observability - Open Source on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.