Guide to Langfuse and Helicone for LLM Observability

When an LLM is in production, you need to know three things: how fast it responds, how much each call costs, and which prompt generated which response. Langfuse and Helicone are two observability platforms that give you exactly that — end-to-end tracing, cost tracking, latency analysis, and a playground to debug prompts without touching production code.

Pro Tip: Don't wait for a problem to install observability. Put Langfuse or Helicone in place from the first production deploy. Retrofitting it later is far more expensive.

Context

LLM observability is not optional when your application depends on a model that can change behavior without you changing a single line of code. Traditional APM tools (Datadog, New Relic) don't understand prompts, tokens, or tool calls — you need something built for the generative AI stack.

Both Langfuse and Helicone are open-source platforms (with cloud or self-hosted options) that intercept LLM API calls (OpenAI, Anthropic, Gemini, etc.) and record:

• Traces: the complete sequence from user message to final response, including tool calls and retrievals.
• Spans: each individual step within a trace (an API call, a DB query, a tool_call).
• Costs: how many input and output tokens each call consumed and its dollar equivalent.
• Latency: how long each step and the total took.
• Feedback: user ratings or automated evaluations linked to specific traces.

Langfuse offers a more mature product with a rich dashboard, dataset generation for evaluation, and a playground. Helicone is lighter, with a simple API and emphasis on caching and rate limiting.

Note: Both tools work as a proxy or via direct SDK integration. They don't modify the LLM's behavior — they just log what happens.

Setup

Langfuse

Langfuse has SDKs for Python, Node.js, and direct API access.

pip install langfuse

from langfuse import Langfuse
 
langfuse = Langfuse(
    secret_key="sk-lf-...",
    public_key="pk-lf-...",
    host="https://cloud.langfuse.com"  # or your self-hosted instance
)
 
# Trace a call
trace = langfuse.trace(name="chat-completion")
 
span = trace.span(
    name="openai-call",
    input={"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}
)
 
# After the actual OpenAI call...
span.end(
    output={"content": "Hi! How can I help you?"},
    usage={"input": 5, "output": 8}  # tokens
)
 
trace.end()

Native OpenAI integration:

Langfuse integrates directly with the OpenAI SDK:

from langfuse.openai import openai
 
# All openai calls are now automatically traced
response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather?"}]
)

Every call is logged in the dashboard with its prompt, response, tokens, latency, and cost.

Warning: Langfuse's automatic OpenAI integration patches the openai module. If you use third-party patches, verify compatibility.

Middleware for web frameworks (FastAPI):

from langfuse import Langfuse
from langfuse.decorators import observe
 
langfuse = Langfuse()
 
@observe()
def chat_with_user(message: str) -> str:
    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": message}]
    )
    return response.choices[0].message.content

Helicone

Helicone works as a proxy — you point your calls at its endpoint and it forwards them:

pip install helicone

from helicone.openai import openai
 
response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather?"}],
    headers={
        "Helicone-Auth": "Bearer YOUR_API_KEY",
        "Helicone-User-Id": "user-123",
        "Helicone-Property-App": "my-app"
    }
)

Alternatively, point directly to the proxy:

import openai
 
openai.api_key = "sk-..."
openai.base_url = "https://oai.hconeai.com/v1"  # Helicone proxy
 
response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}]
)

Helicone also supports automatic caching: send the same prompt and get a cached response without calling the provider, saving cost and latency.

Pro Tip: Enable Helicone's cache for prompts you know will repeat (e.g., system messages, frequent knowledge base queries). You can reduce costs by up to 40%.

Examples

Example 1: Trace a multi-turn conversation with Langfuse

from langfuse import Langfuse
from langfuse.openai import openai
 
langfuse = Langfuse()
 
# Start a trace per user session
trace = langfuse.trace(
    name="customer-support",
    user_id="user-42",
    session_id="sess-001"
)
 
# Turn 1
generation = trace.generation(
    name="turn-1",
    model="gpt-4o",
    input="I need to cancel my subscription"
)
 
response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "I need to cancel my subscription"}]
)
 
generation.end(
    output=response.choices[0].message.content,
    usage={
        "input": response.usage.prompt_tokens,
        "output": response.usage.completion_tokens
    }
)

In the Langfuse dashboard you see each turn, its cost, latency, and any user feedback you attach.

Example 2: Monitor cost per user with Helicone

Helicone automatically tags requests by Helicone-User-Id, letting you filter costs per user in the dashboard:

headers = {
    "Helicone-Auth": "Bearer YOUR_KEY",
    "Helicone-User-Id": user_id,
    "Helicone-Property-Plan": "premium"
}

In the dashboard: "User-42 spent $12.30 this month with 4,500 calls." Useful for usage-based billing or abuse detection.

Example 3: Evaluation in production with Langfuse

Langfuse lets you attach manual or automated evaluations to traces:

trace.score(
    name="accuracy",
    value=1.0,  # 0.0 to 1.0
    comment="Correct and well-referenced response"
)

This lets you correlate response quality with the prompt, model, and cost of each trace.

Note: Langfuse also includes a playground where you can edit prompts and see how the response changes without touching your code. Useful for hot debugging.

Particularities

Which to choose?

• Choose Langfuse if you need a full dashboard, multi-turn tracing, dataset generation for evaluation, and a playground for debugging. Best for teams building complex conversational apps.

• Choose Helicone if you prioritize simplicity, caching, rate limiting, and minimal setup (just change the endpoint). Ideal for small teams that want observability without adding stack complexity.

Pro Tip: You can use both. Helicone as a proxy for caching and rate limiting, with Langfuse for deep tracing and evaluation. They're not mutually exclusive.

History and evolution