The Alert Fired. The Agent Kept Spending.

Most of what gets written about agentic AI governance — including most of what we’ve written — is about a call that should never have been allowed: wrong identity, wrong scope, a poisoned tool description, a credential that should have been revoked. This post is about a different failure, one where every individual call was completely legitimate: the right agent, correctly authenticated, correctly authorized, calling an action it was always supposed to be able to call — just far, far too many times.

The failure: authorized, credentialed, and still expensive by lunch

An ops agent is wired up to retry a paid third-party API call on failure, because retrying is the sensible default for a flaky network. The API starts returning slow, ambiguous 5xx responses — not down, just degraded — and the agent’s retry logic, with no idempotency key and no backoff ceiling, does exactly what it was built to do: it tries again. And again. Four hours later someone notices the bill, not because a security control fired, but because finance asked why the usage graph looks like a cliff.

Nothing about this failure is an authentication problem. The agent’s login handshake was fine. Nothing about it is a policy problem in the allow/deny sense either — every one of those calls, evaluated on its own, was a call this agent was authorized to make. The failure lives on an axis neither identity checks nor per-call policy decisions are built to see: not “was this call allowed” but “how many of these is too many, over what window, before someone should have been told.”

Why identity-based governance doesn’t catch volume

Authentication answers who is calling. A policy engine’s allow/deny answers whether this specific call, right now, is within scope. Decision tokens answer whether the permission and the secret used to act on it are still the same answer. All three are evaluated per call, and a runaway loop’s four thousandth call passes every one of them exactly as cleanly as its first — because in isolation, it’s identical to the first. Volume is a property of the sequence, not the individual call, and nothing that only ever looks at one call at a time will ever flag it.

That’s the same blind spot idempotency failures live in, one level up: there, an agent that couldn’t tell if its last call succeeded fired a second refund or a second email. Here, an agent that can’t tell when to stop trying fires the four-thousandth. Both are the predictable output of an agent doing exactly what it was told, at a rate nobody put a ceiling on.

Alerts aren’t the same as a stop

The tempting fix is “add a spend cap” — and it’s the right instinct, but it only closes half the gap. A spend cap that watches cumulative usage and raises an alert at 50%, 80%, and 100% of a monthly budget is genuinely useful: it turns “found out from the invoice” into “found out within minutes of crossing a threshold.” But an alert, by itself, is observability, not traceability applied to cost instead of calls — it tells you a number crossed a line, not that anything actually stopped happening. A monitor that only ever alerts is making a deliberate, defensible choice: it won’t cut off an agent mid-task just because a threshold ticked over, because a payment system improvising a shutdown in the middle of a legitimate operation is its own governance failure. The trade-off is real. It just means the alert can’t be the whole answer — something downstream still has to decide, on purpose, to pull the credential.

That “something” is the same machinery that should already be sitting between the agent and the action it keeps calling. A decision token that expires means a runaway loop’s next call has to be re-authorized, not just re-sent. A kill switch means the moment a spend alert fires, a human — or an automated rule watching the same alert — has something to press that actually stops the agent, not just a dashboard to stare at. A policy engine that can read a usage signal can turn “this owner is at 80% of cap” into a hard deny on the next call, the same way it already turns a scope violation into one. None of these exist because a spend cap fired; they have to already be in place, watching the same broker every call already flows through, so the alert has somewhere to land besides an inbox.

The business case: the same chokepoint that logs the call can watch its rate

This is the argument for putting agentic calls through a broker rather than treating spend monitoring as a billing-team concern bolted on separately from access control:

• Usage and identity share one record. Because every call already passes through the broker for a credential decision, the same event that gets logged for traceability is the event a spend or volume monitor reads — no separate pipeline that has to be kept in sync with what access control actually saw.
• A threshold breach has a real lever to pull. Because the broker already issues short-lived decision tokens and already exposes a kill switch, crossing a spend or call-volume threshold can trigger an actual stop — revoke the credential, deny the next call — instead of an alert with no downstream action wired to it.
• Per-owner, per-workspace visibility, not just a global bill. A runaway agent belongs to one workspace and one owner; a monitor built into the same layer that already scopes credentials and workspaces can flag the one that’s spending, not just the aggregate number on next month’s invoice.
• The stop is deliberate, not accidental. Separating “alert” from “block” means the decision to cut an agent off mid-task is made by a policy or a person looking at the threshold, not by a payment system silently truncating a legitimate operation because a number crossed a line it was never told the context for.

The value isn’t just avoiding a surprise invoice, though that’s real. It’s that “how many times has this agent done this, and is that still normal” becomes a question the same governance layer can answer alongside “was this agent allowed to do this” — instead of two unrelated systems, one watching identity and one watching a bill, that never talk to each other until finance does the talking for them.

FAQ

Isn’t a rate limit the same thing as a spend cap? No. A rate limit protects a route or the platform from abuse in a short window — requests per minute, say. A spend cap tracks cumulative cost for one owner over a billing period, across every call that owner’s agents make. They catch different shapes of runaway behavior; a slow, steady four-hour retry storm can stay under a per-minute rate limit the entire time while still running up a real bill.

Why not just make the spend cap hard by default, and block automatically at 100%? Because a cap that blocks mid-task with no context can strand a legitimate operation half-finished, which is its own incident. The better shape is an alert that reaches a policy or a person fast, paired with a broker that can actually enforce a stop once that judgment is made — deliberate, not automatic-by-surprise.

Does this replace the kill switch post? No — it’s the other half of it. The kill switch is what you press. This is about what tells you, in time, that you need to press it.

Is this really an “AI agentic gone wrong” pattern, or just a billing issue? Both. The mechanism is the same one behind duplicate-action failures: an agent doing exactly what it was told, with no ceiling on how many times “doing what it was told” was supposed to happen. The consequence just shows up on an invoice instead of in a database.

What’s the minimum a team needs to catch this before finance does? Usage visibility scoped per owner and per workspace, alert thresholds that reach someone (or something) able to act, and a way to actually cut the credential off once that judgment is made — not just a dashboard number that keeps climbing after everyone’s seen it.

An agent that never should have been allowed to call something is a policy bug. An agent that was allowed, correctly, and simply never told when to stop is a governance gap with a different shape — and it doesn’t show up until someone’s already paid for it.