On this page
Last updated: September 2026
Written by the team at Flowlines. We build the behavioral detection layer described in the last section. This taxonomy names the failure patterns we see across production agent traces, independent of which tools you use to catch them.
Many agent monitoring stacks start with one question: did this call succeed. That leaves a gap between technical health and user outcome. Traces preserve the execution record, while evaluators detect the behaviors they are configured to score. Cross-session analysis is useful when the recurring pattern, affected population, or changing baseline is the thing you need to find.
This is a working taxonomy of five failure patterns that fall through that gap, what each looks like in a real session, and why status-code and latency monitoring alone do not catch them.
It is the production-side complement to the academic map of this space. Berkeley's MAST taxonomy (Cemri et al., 2025 ↗) catalogs 14 failure modes in multi-agent LLM systems, grouped into specification issues, inter-agent misalignment, and task verification, derived from annotated framework traces. The five patterns below are narrower and different in kind: they are the ones that keep surfacing in production traces of deployed agents, where the user is real and nobody is watching the session.
Quick reference
| Failure mode | What it looks like in a session | What basic monitoring misses |
|---|---|---|
| False success (fabricated completion) | Agent reports a task done; the tool call that would have done it never ran, or ran and returned an error | The call trace still returns 200 and the response text reads as success |
| Loops and repeat failures | The same failure pattern recurs across hundreds of sessions | Each occurrence looks like an isolated, clean trace on its own; the pattern only exists across sessions |
| Silent drift | Behavior degrades gradually over days or weeks; no single session looks broken | Eval suites score against a fixed snapshot, not a moving baseline |
| Intent mismatch | Agent answers a different question than the one the user asked, fluently | No error, no failed tool call, just the wrong target |
| Cohort gaps | Agent works for one user segment and fails another (language, plan tier, region) | Aggregate success-rate metrics average the failure away |
1. False success (fabricated completion)
False success can be especially costly because users may trust the completion claim before discovering it is wrong. The agent tells the user, or the next step in a pipeline, that something happened. The tool call that would have made it happen either never ran, or ran and returned an error the agent's response ignored.
In a customer support flow, that looks like a refund confirmed to the customer while the refund tool itself returned a decline. In a coding agent, it is a confident "tests pass" summary on a run that actually errored, or an edit applied to the wrong file while the agent reports the intended one changed. See how this shows up specifically for customer support agents and coding agents.
2. Loops and repeat failures
A single occurrence of a bug is a bug report. The same failure recurring across hundreds of sessions with nobody connecting them is a pattern nobody is tracking, because each trace looks fine in isolation. Nothing about any individual session trips an alert. The pattern only becomes visible once someone (or something) looks across sessions instead of within one.
This is close to the root cause behind why agents stop improving in production even after teams ship fixes: without cross-session grouping, a fix for one instance of a loop does not tell you whether the underlying pattern actually stopped recurring elsewhere.
3. Silent drift
Silent drift is gradual, not a step change, which is exactly why it survives a deploy review. A prompt change, a model swap, or an upstream data shift nudges behavior slightly worse. No single session after the change looks obviously broken. Days or weeks later, the aggregate is clearly worse, but there is no single commit or session to point at.
Offline eval suites catch regressions against a defined test set before a release. Online evaluators can score production traffic, but slow drift still requires a useful baseline, dimensions, and enough time-series context to compare this week's real sessions with last month's.
4. Intent mismatch
The agent answers fluently, formats the response well, and answers a different question than the one the user actually asked. There is no error anywhere in the trace: the tool calls succeeded, the model returned a well-formed response, latency was normal. The only thing wrong is that the response does not match what the user needed.
This is common in retrieval-grounded agents, where a fluent answer can be built from context that never actually supported it, and in multi-turn agents that lose track of an earlier constraint the user set.
5. Cohort gaps
An agent can have a healthy aggregate success rate while failing consistently for a specific user segment: a language it wasn't tuned for, a plan tier with different tool access, a region with different data availability. Averaged metrics hide this by construction. The team looking at an overall 95% success rate has no visibility into the cohort sitting at 60%.
Multi-session products are especially exposed here, because the gap compounds silently over time instead of showing up in a single bad session. See how this plays out for EdTech and tutoring agents, where a student can disengage across several sessions before anyone notices the trend.
Closing the gap
These patterns are not necessarily infrastructure failures. They may not produce an error, timeout, or failed health check. Traces can contain the necessary evidence and evaluators can detect patterns they are configured to score. Flowlines ↗ adds cross-session analysis that groups recurring production behavior, affected users, and representative sessions. When the application already records usable Langfuse, LangSmith, or OpenTelemetry traces, Flowlines can analyze that evidence without a proprietary application SDK or a second instrumentation pass. MCP servers still need standards-based tool-call spans so the complete journey can be reconstructed.
Flowlines is a detection layer, not a tracing replacement: teams typically keep their existing tracing tool and add Flowlines on top of it. And it is not the right tool for every stage: if your agent has not shipped to production yet, this taxonomy still matters for how you design it, but the right stack for that stage is tracing plus evals, not a behavioral layer. Behavioral observability earns its keep once real users are generating the sessions these five patterns hide in.
Related reading
- The 9 Best AI Agent Observability Tools in 2026: where each layer of the stack, tracing, evals, and behavioral observability, stops.
- What is behavioral observability?
- Why AI agents don't learn in production: the loops-and-drift failure modes in more depth.