The answer
A tool change can be a new description, a new argument, a different response or a new implementation. Each can shift how often models pick the tool and how well it works once picked. Read both effects.
| Measure | Improvement looks like | Could also mean |
|---|---|---|
| Users and sessions using the tool | More people reach it for the intended job. | A traffic or client mix change. |
| Call success rate | Fewer failed calls with a known result. | Fewer calls overall, or more unknown results. |
| Repeated calls per session | Fewer retries to get the right result. | Less pagination for unrelated reasons. |
| Session outcomes | More accomplished, fewer partial results. | Lower outcome coverage. |
Use equal windows that cover the same weekdays, and record the change with service.version so calls are attributable to a release. Flowlines exposes contract changes and time-based comparisons, so you can open the tool view for each window and inspect the sessions that changed.
Published