How do I catch silent failures in production?

Start with one task. For an invoice export, success means a usable invoice reached the user, not just that the lookup tool returned successfully. Record the expected result and the evidence that would confirm it.

1. Sample sessions with a completion claim and compare the claim with tool output and the final user response.
2. Label missing results, repeated attempts, and unsupported claims separately. Keep sessions with insufficient evidence as unknown.
3. Track the affected sessions and users against analyzed coverage. Review representative examples before creating a recurring alert.
4. Assign an owner, test the fix against those examples, and compare the next production window with the same task definition.

Flowlines provides the behavior issues and sessions to investigate. Follow the production monitoring walkthrough for a worked denominator, alert rule, and verification checklist, or use the debugging guide to investigate one failure.