LLM observability — what to record so you can fix it later
The failure mode is not a crash, it is a confident wrong answer. Prices below were checked on each vendor's pricing page in November 2026.
AI Tools Hub Editorial · published · updated
Five things worth recording
- Which prompt version ran
- What retrieval returned — in RAG this is usually the culprit
- Model and parameters
- Token counts and latency
- What the user did next: retried, rephrased, left
The second is the most important and the most commonly missing. Without it you cannot separate a model problem from a retrieval problem, and every fix becomes a guess.
The options
| Langfuse | Helicone | Portkey | Braintrust | |
|---|---|---|---|---|
| Free tier | 50k units/mo, 30-day retention | 10k requests/mo | 10k logs/mo, 3-day retention | $10 model credits |
| Entry paid | $29/mo | $79/mo | $49/mo | $249/mo |
| Open source | Yes | Yes | Yes | Not stated |
| Integration | SDK traces | Gateway | Gateway | SDK |
| OpenTelemetry | Yes | — | — | — |
Two integration shapes
SDK-based tools let you decide what constitutes one trace, which is what you need for multi-step agents. Gateway tools start recording the moment you change a base URL, but they only see the model call — not what happened inside your application.
For single-shot calls, the gateway is faster to adopt. For agents that retrieve, decide and call tools, only tracing lets you find the cause.
Count your units before trusting the free tier
Langfuse counts each trace, observation and score as a unit. A five-step agent produces six to ten units per run, so 50,000 units is roughly 5,000–8,000 runs. A simple chat uses two or three, which is closer to 20,000 runs on the same allowance.
FAQ
Isn't our existing APM enough?
It covers latency and errors. It does not capture prompt version, retrieval results or token counts. Langfuse speaks OpenTelemetry, so it can sit alongside what you already run.
Should we self-host?
Traces contain raw user input. If that cannot go to a third party, self-hosting is the only option. If it can, managed is cheaper once you count the operational time.