Langfuse is an open source LLM engineering platform. It helps teams collaboratively develop, monitor, evaluate, and debug AI applications. Langfuse can be self-hosted in minutes and is battle-tested.

✨ Core Features

• LLM Application Observability: Instrument your app and start ingesting traces to Langfuse, thereby tracking LLM calls and other relevant logic in your app such as retrieval, embedding, or agent actions. Inspect and debug complex logs and user sessions. Try the interactive demo to see this in action.

• Prompt Management helps you centrally manage, version control, and collaboratively iterate on your prompts. Thanks to strong caching on server and client side, you can iterate on prompts without adding latency to your application.

• Evaluations are key to the LLM application development workflow, and Langfuse adapts to your needs. It supports LLM-as-a-judge, user feedback collection, manual labeling, and custom evaluation pipelines via APIs/SDKs.

• Datasets enable test sets and benchmarks for evaluating your LLM application. They support continuous improvement, pre-deployment testing, structured experiments, flexible evaluation, and seamless integration with frameworks like LangChain and LlamaIndex.

• LLM Playground is a tool for testing and iterating on your prompts and model configurations, shortening the feedback loop and accelerating development. When you see a bad result in tracing, you can directly jump to the playground to iterate on it.

• Comprehensive API: Langfuse is frequently used to power bespoke LLMOps workflows while using the building blocks provided by Langfuse via the API. OpenAPI spec, Postman collection, and typed SDKs for Python, JS/TS are available.

🔌 Integrations

Main Integrations:

Integration	Supports	Description
SDK	Python, JS/TS	Manual instrumentation using the SDKs for full flexibility.
OpenAI	Python, JS/TS	Automated instrumentation using drop-in replacement of OpenAI SDK.
Langchain	Python, JS/TS	Automated instrumentation by passing callback handler to Langchain application.
LlamaIndex	Python	Automated instrumentation via LlamaIndex callback system.
Haystack	Python	Automated instrumentation via Haystack content tracing system.
LiteLLM	Python, JS/TS (proxy only)	Use any LLM as a drop in replacement for GPT. Use Azure, OpenAI, Cohere, Anthropic, Ollama, VLLM, Sagemaker, HuggingFace, Replicate (100+ LLMs).
Vercel AI SDK	JS/TS	TypeScript toolkit designed to help developers build AI-powered applications with React, Next.js, Vue, Svelte, Node.js.
API		Directly call the public API. OpenAPI spec available.

Packages integrated with Langfuse:

Name	Type	Description
Instructor	Library	Library to get structured LLM outputs (JSON, Pydantic)
DSPy	Library	Framework that systematically optimizes language model prompts and weights
Mirascope	Library	Python toolkit for building LLM applications.
Ollama	Model (local)	Easily run open source LLMs on your own machine.
Amazon Bedrock	Model	Run foundation and fine-tuned models on AWS.
AutoGen	Agent Framework	Open source LLM platform for building distributed agents.
Flowise	Chat/Agent UI	JS/TS no-code builder for customized LLM flows.
Langflow	Chat/Agent UI	Python-based UI for LangChain, designed with react-flow to provide an effortless way to experiment and prototype flows.
Dify	Chat/Agent UI	Open source LLM app development platform with no-code builder.
OpenWebUI	Chat/Agent UI	Self-hosted LLM Chat web ui supporting various LLM runners including self-hosted and local models.
Promptfoo	Tool	Open source LLM testing platform.
LobeChat	Chat/Agent UI	Open source chatbot platform.
Vapi	Platform	Open source voice AI platform.
Inferable	Agents	Open source LLM platform for building distributed agents.
Gradio	Chat/Agent UI	Open source Python library to build web interfaces like Chat UI.
Goose	Agents	Open source LLM platform for building distributed agents.
smolagents	Agents	Open source AI agents framework.
CrewAI	Agents	Multi agent framework for agent collaboration and tool use.