The answer
Exception logs help diagnose crashes, but status codes alone do not reveal a fabricated answer, a tool chain that returned nothing, a confident “done” with no work behind it, an answer that drifts off the allowed scope. These can occur even when the HTTP call succeeds.
Flowlines groups behavior issues into categories including looping and stuck behavior, drifting, user frustration, abandonment, result quality, hallucination and trust, and safety and policy.
Open an issue to see what happened, why it matters, where it fires, and the affected sessions. Then inspect the technical detection details before changing the agent.
To check a single task, define what a successful outcome looks like, compare it with the tool result and the final response, and review sessions where a claim of completion lacks evidence. Keep sessions with insufficient evidence as unknown rather than counting them as successes.
How do I look for failures that never raise an error?
- Compare what the agent claimed with what the session shows actually happened.
- Find loops and repeated calls with the same input.
- Look for sessions that end right after a tool result the user did not want.
- Track users who return with the same request.
- Compare outcome rates across cohorts to find groups that quietly fail more often.
Published · Updated