Deploy arize-phoenix-observability

Deploy and Host Arize Phoenix on Railway

Arize Phoenix is an open-source LLM observability platform — trace, evaluate, and debug your AI applications with OpenTelemetry-standard instrumentation. See every LLM call, retrieval step, and tool use as a span, run evaluations to catch hallucinations and quality regressions, and experiment with prompts in a built-in playground. Because Phoenix speaks OpenTelemetry, you instrument with the open standard instead of a vendor-locked SDK. This template deploys Phoenix with PostgreSQL, authentication enabled, and durable trace storage — so your LLM observability backend is live and secured in minutes.

What This Template Deploys

Both connect over Railway's private network. Phoenix serves the UI and ingests trace spans over OTLP (HTTP on 6006, gRPC on 4317), while PostgreSQL persists your traces, datasets, and experiments — so your observability history survives redeploys.

About Hosting

Phoenix is a capable observability backend, and a few specifics make it a secure, persistent deploy — all handled here.

OpenTelemetry-native — instrument once, no vendor lock-in. Phoenix's defining trait is that it ingests standard OpenTelemetry spans via OpenInference instrumentation, so you trace your app with the open OTel standard rather than a proprietary SDK. Instrument with arize-phoenix-otel and the OpenInference libraries for your framework (OpenAI, LangChain, LlamaIndex, and more), point them at your Phoenix /v1/traces endpoint, and every LLM call, retrieval, and tool step appears as a span — portable across tools.

PostgreSQL for durable storage — not ephemeral SQLite. Phoenix defaults to SQLite, which is fine for a local notebook but loses data on a PaaS. This template wires PHOENIX_SQL_DATABASE_URL to a managed PostgreSQL, so traces, datasets, experiments, and users persist durably beyond a single process — the right setup once observability is shared across a team or environment.

Authentication is enabled — secured from the first deploy. The open-source Phoenix server has no auth by default, so a naive deploy exposes your traces. This template sets PHOENIX_ENABLE_AUTH=True with a strong PHOENIX_SECRET and secure cookies, and bootstraps an admin via PHOENIX_DEFAULT_ADMIN_INITIAL_PASSWORD — sign in as admin@localhost with the generated value and change it from your profile. Mint API keys under Settings to authenticate your instrumented apps.

Memory scales with ingest — size for your span rate. Phoenix queues up to 20,000 spans in memory by default at roughly 50 KiB per span, so RAM usage scales with how fast your app emits traces. A modest app runs comfortably on a small plan; if you see span backpressure in the logs at high ingest, raise the memory. This is the main operational consideration.

Evals and experiments built in. Beyond tracing, Phoenix runs LLM evaluations — hallucination, relevance, toxicity, and custom evals — to score your app's outputs, and includes a prompt-experiment playground and dataset support, so you debug and improve, not just observe.

Typical cost: ~$10–20/month on Railway for Phoenix and PostgreSQL, scaling with trace volume and memory. Phoenix is open source (Elastic License 2.0) and free to self-host.

How It Compares

LangSmith is polished but ties you to its SDK and hosts your traces. Cloud APM tools weren't built for LLM spans, evals, or prompt experiments. Print debugging doesn't scale past a toy. Phoenix's edge is open, OpenTelemetry-standard LLM observability with built-in evals and experiments — self-hosted, so your trace data and prompts stay on infrastructure you own, and your instrumentation stays portable across tools.

Deploy in Under 5 Minutes

1. Click Deploy on Railway — Phoenix and PostgreSQL build and wire together (~4 minutes)
2. Confirm PHOENIX_SQL_DATABASE_URL, PHOENIX_ENABLE_AUTH, and PHOENIX_SECRET are set
3. Open your Railway URL and sign in as admin@localhost with the generated admin password
4. Create a project, then mint an API key under Settings
5. Instrument your app with OpenInference, point it at your /v1/traces endpoint, and watch spans stream in

Change the initial admin password from your profile — the bootstrap value is one-time.

Common Use Cases

• LLM app tracing — see every model call, retrieval, and tool step as an OpenTelemetry span
• RAG debugging — trace retrieval and generation to find where a RAG pipeline goes wrong
• Prompt experiments — test prompt and model variants in the playground against datasets
• Agent observability — follow multi-step agent runs to debug reasoning and tool use

Configuration

OpenTelemetry-native, on durable Postgres. Instrument your app with OpenInference/OTel and point it at your /v1/traces endpoint. This template stores traces in PostgreSQL (not ephemeral SQLite) so your observability history persists.

Auth is on, and memory scales with spans. PHOENIX_ENABLE_AUTH and a strong PHOENIX_SECRET secure the instance from deploy. Phoenix queues up to 20,000 spans in memory (~50 KiB each), so raise RAM if you see span backpressure.

Dependencies for Phoenix Hosting

• Railway account — ~$10–20/month for Phoenix and PostgreSQL, scaling with trace volume
• PostgreSQL (included and wired for durable trace storage)
• An OpenInference/OpenTelemetry-instrumented application to send spans
• Optional: SMTP credentials for team user invitations

Deployment Dependencies

• Phoenix GitHub Repository
• Phoenix Documentation
• Phoenix Self-Hosting Guide
• Railway Private Networking

Implementation Details

The template runs the official arizephoenix/phoenix image serving the UI and OTLP HTTP collector on port 6006 and the OTLP gRPC collector on 4317, backed by a Railway-managed PostgreSQL database. PHOENIX_SQL_DATABASE_URL points at Postgres so traces, datasets, experiments, and users persist durably — the default SQLite backend is unsuitable for a persistent PaaS deployment. PHOENIX_WORKING_DIR sets the working directory.

Phoenix ingests OpenTelemetry spans through OpenInference instrumentation: applications instrumented with arize-phoenix-otel send spans to the /v1/traces (OTLP/HTTP) or gRPC endpoint with a bearer API key, appearing live in the UI. Authentication is enabled by default here — PHOENIX_ENABLE_AUTH=True, a strong generated PHOENIX_SECRET, secure cookies, and an admin bootstrapped via PHOENIX_DEFAULT_ADMIN_INITIAL_PASSWORD (sign in as admin@localhost, then change it) — since the open-source server is otherwise unauthenticated.

An operational note: Phoenix buffers up to 20,000 spans in memory at roughly 50 KiB each, so memory scales with ingest rate and should be increased if span backpressure appears in the logs. Beyond tracing, Phoenix provides LLM evaluations, a prompt-experiment playground, and dataset support. PostgreSQL is the primary backup target.

Frequently Asked Questions

What makes Phoenix different from vendor observability tools? Phoenix uses OpenTelemetry-standard instrumentation via OpenInference, so you trace with the open standard rather than a proprietary SDK — your instrumentation stays portable — and it's self-hosted, keeping your trace data on your own infrastructure.

Why is Phoenix using more memory? It buffers up to 20,000 spans in memory at ~50 KiB each, so memory scales with your trace ingest rate. If you see span backpressure in the logs, raise the plan's RAM.

Can it evaluate my LLM app, not just trace it? Yes — Phoenix runs built-in LLM evals (hallucination, relevance, toxicity, and custom) and includes a prompt-experiment playground and datasets, so you can score and improve outputs, not just observe them.

Why Deploy Arize Phoenix on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Phoenix on Railway you get OpenTelemetry-native LLM observability with the hard parts solved — PostgreSQL wired for durable traces, authentication and a strong secret set, and the OTLP collector ready. Trace, evaluate, and debug your AI apps with the open standard, self-hosted on infrastructure you own.