How do I detect agent drift over time?
Agent drift is the gradual divergence of an agent's behavior from its original instructions, learned user state, or expected outcomes. You detect it by comparing behavioral signal rates across rolling windows and across prompt versions, not by inspecting individual traces.
Per-call traces tell you that a call ran. They don't tell you whether the same agent is acting differently this week than last week. Drift only shows up at the cross-session layer.
Flowlines reads from your existing trace store (Langfuse, Helicone, Arize, or any OTEL pipeline) and computes signal rates, cohort breakdowns, and version comparisons on top. Drift surfaces as a sustained delta in one of those, not as a single anomaly.
The classes of drift worth alerting on:
Constraint drift. The agent starts ignoring rules it used to follow (policy violations, off-topic responses, scope creep). Memory drift. Previously injected user state stops getting referenced. Cost drift. Tokens-per-session climbs without a quality improvement. Behavioral drift. Frustration signals rise on a previously happy cohort.
A 7-day rolling delta on cost_escalation
You ship a new system prompt on Monday. By Wednesday the cost_escalation signal is firing on more sessions than its rolling baseline. On the /versions page, the before/after for that deploy shows tokens-per-session up with no measurable quality lift.
Same agent. Same users. Same intents. The drift is in the prompt, and Flowlines tells you which deploy caused it on the same day it shipped.
Rolling baselines and version pinning
Every signal type maintains a rolling baseline (default 28 days) and a current-window rate (24h, 7d, or 30d). When the current rate exceeds the baseline by more than the configured threshold, the signal is marked as drifting.
On the /versions page, signal rates are pinned to the deployed prompt version, so a deploy-triggered drift shows up as a vertical line on the timeline, not as a noisy gradient.
Last updated 2026-05-28