---
title: "Deploy Langfuse LiteLLM"
description: "Log every LiteLLM proxy call to your own Langfuse"
category: "AI/ML"
url: https://railway.com/deploy/deploy-langfuse-for-litellm-proxy
---

# Deploy Langfuse LiteLLM

Log every LiteLLM proxy call to your own Langfuse

**[Deploy Langfuse LiteLLM on Railway](https://railway.com/template/deploy-langfuse-for-litellm-proxy)**

Machine-readable deploy manifest (JSON, validated by TemplateCI): https://railway.com/deploy/deploy-langfuse-for-litellm-proxy/manifest.json

- **Creator:** onepush
- **Category:** AI/ML

## Template content

### langfuse-worker

- **Image:** langfuse/langfuse-worker:3

### langfuse-web

- **Image:** langfuse/langfuse:3
- **Public domain:** Yes

### ClickHouse

- **Image:** clickhouse/clickhouse-server:24.8

### Postgres https://devicons.railway.app/i/postgresql.svg

- **Image:** ghcr.io/railwayapp-templates/postgres-ssl:18

### Redis https://cdn.sanity.io/images/sy1jschh/production/0ce0bfdcfbdbf69662b1116671f97c2dd788b655-157x157.svg

- **Image:** redis:8.2
- **Start command:** `/bin/sh -c "rm -rf $RAILWAY_VOLUME_MOUNT_PATH/lost+found/ && exec docker-entrypoint.sh redis-server --requirepass $REDIS_PASSWORD --save 60 1 --dir $RAILWAY_VOLUME_MOUNT_PATH"`

## Buckets

- **langfuse-bucket**

## Documentation

# Deploy and Host self hosted Langfuse LiteLLM (Open-Source LLM Gateway Logging) on Railway

This template deploys the self-hosted Langfuse v3 stack (web, worker, Postgres, Redis, ClickHouse, bucket) and shows how to point a LiteLLM proxy at it so every proxied call lands as a traced, costed generation on infrastructure you own.

## About Hosting Langfuse LiteLLM open-source software on Railway (self hosted Langfuse LiteLLM template)

This template runs the whole Langfuse v3 stack inside your own Railway project: langfuse-web on port 3000, langfuse-worker, Postgres, Redis, ClickHouse 24.8 and a Railway Storage Bucket.

## Why Deploy Langfuse LiteLLM, the Helicone alternative on Railway (Railway Free Trial)

Helicone sells you a proxy plus a hosted dashboard. Start on the $5 trial and see the real cost.

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Langfuse LiteLLM on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.

### Railway vs Other Hosting Providers and VPS for Langfuse LiteLLM self hosting

Railway gives langfuse-web a public HTTPS domain for LANGFUSE_OTEL_HOST; a VPS gives the same containers plus more upkeep.

| Provider | What You Get with Railway | What You Get with the Other Provider |
| --- | --- | --- |
| DigitalOcean | Six services deployed, plus a domain LiteLLM reaches over the public net | Droplets and volumes you wire and harden yourself |
| AWS | Managed add-ons by reference, billed per service | ECS, RDS and S3 with IAM and VPC work you own |
| Hetzner | Same containers, no server admin | Cheap VMs where you run Compose, backups and ingress |

The software is the same either way; Railway just skips the ingress and volume chores.

## Common Use Cases for hosted Langfuse LiteLLM

- **Cost per team across providers:** pass the team name as user or tags in request metadata, so one Langfuse view shows spend across OpenAI, Anthropic and Bedrock.
- **Debugging slow or failing upstream models:** each generation keeps duration, time to first token, finish reason and error details, so a 429 from one provider sits next to the same prompt elsewhere.
- **Tagging calls by app with metadata:** send tags and generation_name in request metadata and they land as langfuse.trace.tags and langfuse.generation.name.
- **Sessions for multi-turn chat:** pass session_id in metadata and every proxied turn of a chat groups together, even across models.
- **Comparing routed models:** run one prompt through two model_list entries and compare tokens, cost and latency before changing a default.

## Dependencies for Langfuse LiteLLM Docker hosted on Railway

This template ships only the Langfuse side: langfuse-web (langfuse/langfuse:3, port 3000), langfuse-worker (langfuse/langfuse-worker:3, port 3030), Postgres, Redis, ClickHouse 24.8 at /var/lib/clickhouse, and a Railway Storage Bucket for raw events.

### Deployment Dependencies for Managed Langfuse LiteLLM Service (LLM Observability)

Secrets are generated per deploy: DATABASE_URL, REDIS_CONNECTION_STRING (with ?family=0), CLICKHOUSE_URL, CLICKHOUSE_MIGRATION_URL, CLICKHOUSE_USER, CLICKHOUSE_PASSWORD, CLICKHOUSE_CLUSTER_ENABLED=false, the S3 event and media upload vars, SALT, ENCRYPTION_KEY (64 hex), NEXTAUTH_SECRET, NEXTAUTH_URL.

### Implementation Details for Langfuse LiteLLM (Using Langfuse official docker image)

Official images only, langfuse/langfuse:3 and langfuse/langfuse-worker:3, no build step. If traces stay empty on a first deploy, restart the worker once. Web writes batches to the bucket and Redis; the worker ingests into ClickHouse.

## How does Langfuse LiteLLM compare against other LLM gateway logging and observability tools

All four log a proxied call. They differ on where data lives and how much stack you run.

### Langfuse vs Helicone (Helicone Alternative)

- **Setup speed:** Helicone Cloud takes minutes. This template means six services plus the LiteLLM proxy you add.
- **Data ownership:** Helicone is open source with a hosted tier. Langfuse, ClickHouse and the bucket sit in your Railway project.
- **Where Helicone wins:** lighter ops if you would rather not run ClickHouse.

### Langfuse vs LangSmith (LangSmith Alternative)

- **License:** LangSmith is closed source and self-hosts only on Enterprise. Langfuse core is MIT with no usage limits.
- **Gateway fit:** langfuse_otel captures any OpenAI-compatible request through LiteLLM with no app changes. LangSmith is deepest inside LangChain.
- **Where LangSmith wins:** dataset and evaluation tooling is more mature.

### Langfuse vs Portkey (Portkey Alternative)

- **Scope:** Portkey is a gateway with observability attached. Here LiteLLM stays the gateway and Langfuse does the tracing.
- **Metadata:** LiteLLM metadata keys become langfuse-prefixed span attributes, so session and user filters run on your own data.
- **Where Portkey wins:** one product instead of two services if you have no gateway yet.

### Langfuse vs OpenLIT (OpenLIT Alternative)

- **Approach:** OpenLIT is OTel-native collection. Langfuse accepts OTel too but centers on generations, sessions, scores and prompts.
- **Footprint:** OpenLIT with ClickHouse is lighter than this six-service Langfuse stack.
- **Where OpenLIT wins:** teams that want one collector for every service, not an LLM product.

## How to use Langfuse LiteLLM (the OSS LLM observability platform)?

1. Sign up on your Railway Langfuse domain, create a project, and copy its pk-lf-... and sk-lf-... keys.
2. On the LiteLLM service (port 4000), set `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY` and `LANGFUSE_OTEL_HOST=https://your-langfuse-domain`.
3. Add the callback in `litellm_config.yaml`:

```yaml
litellm_settings:
  callbacks: ["langfuse_otel"]
```

4. Restart LiteLLM, then send a test call:

```bash
curl http://localhost:4000/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"ping"}],"metadata":{"session_id":"s1"}}'
```

5. Open Tracing. Metadata arrives as `langfuse.trace.session_id`, and generations show up after langfuse-worker ingests into ClickHouse.

## How to self host Langfuse LiteLLM on other VPS Services (Langfuse self hosting guide)

On a VPS you run both stacks yourself, then point the proxy at your own domain.

### Clone the Repository

```bash
git clone https://github.com/langfuse/langfuse.git
git clone https://github.com/BerriAI/litellm.git
```

### Install Dependencies

Bring up Postgres, Redis, ClickHouse and the bucket from the Langfuse compose file, then install LiteLLM.

```bash
cd langfuse
docker compose up -d
pip install "litellm[proxy]"
```

### Configure Environment Variables

Set the project keys on the proxy and point it at your domain, no trailing slash.

```bash
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_OTEL_HOST=https://your-domain
```

Add `callbacks: ["langfuse_otel"]` under `litellm_settings` in `litellm_config.yaml`.

### Start the Langfuse LiteLLM Application

Start the proxy, send one chat completion, then check Tracing.

```bash
litellm --config litellm_config.yaml --port 4000
```

TLS, private networking between the two services and the ingestion worker restart are what Railway handles for you.

## Official Pricing of Langfuse LiteLLM (Langfuse pricing)

Langfuse core and LiteLLM are both MIT and free to self host, with no usage limits or per-unit charges. You pay only for the Railway services you run, including the LiteLLM service you add.

## Langfuse LiteLLM cloud vs self hosted comparison (Pricing, features, costs, and more)

Cloud Langfuse meters by units, so proxy traffic adds up fast once every routed call becomes a generation. Self hosting the stack flips that to a flat Railway bill, and LiteLLM stays your gateway either way.

### Monthly cost of self hosting Langfuse LiteLLM on Railway

The six services here (web, worker, Postgres, Redis, ClickHouse, bucket) run roughly $15-40/mo at low traffic. That covers both Langfuse and the LiteLLM calls you route into it. Railway's $5 trial covers the first stretch of that, so you can watch real proxy traffic land before you commit.

### System Requirements for Hosting Langfuse LiteLLM on a VPS

Official self host docs want 4 cores, 16 GiB RAM and 100 GiB disk, because ClickHouse carries the trace load. Smaller boxes work in testing but slow ingestion.

## Frequently Asked Questions (FAQs)

### Do I need the Langfuse SDK in my app?

No. Your app keeps calling LiteLLM in OpenAI format and the proxy exports the generation itself. Add the SDK only for manual spans the proxy never sees.

### Why is cost missing on some generations?

LiteLLM prices a call only when the model name in model_list matches known rates. Aliased or brand-new models arrive with tokens but no dollar figure.

### How do I set user and session?

Pass session_id, user and tags in the request body's metadata field, not as top-level parameters. They export as langfuse.trace.session_id and similar attributes; skip session_id and multi-turn chats split into unrelated traces.

### Can LiteLLM run on Railway next to Langfuse?

Yes, as a second service you add yourself. Point LANGFUSE_OTEL_HOST at the langfuse-web public domain; traces appear once langfuse-worker ingests them into ClickHouse.

### Does logging slow down requests?

Export is asynchronous, so responses don't wait on Langfuse. Any delay you notice is ingestion-side, not client latency.

### What is the difference between the langfuse and langfuse_otel callbacks?

langfuse uses the SDK path; langfuse_otel exports over OpenTelemetry to LANGFUSE_OTEL_HOST plus /api/public/otel, the path the docs recommend for self-hosted instances. Use langfuse_otel.


## Similar templates

- [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform.
- [Möbius v0.1 Personal AI OS](https://railway.com/deploy/mobius-v01-personal-ai-os) — Self-hosted AI agent that builds and runs apps for you, with memory.
- [stella](https://railway.com/deploy/stella) — Self-host stella with web, API, Postgres, Redis, and object storage.

Open this page in a browser: https://railway.com/deploy/deploy-langfuse-for-litellm-proxy