Skip to content

AI Agent Says Done? Verify Its Actions With an Audit Trail

7 min read Fullmakt Team

  • agents
  • agentic-actions
  • traceability
  • observability
  • governance
  • mcp
  • a2a

An operations team asks its agent to refund twelve customers. A minute later the agent replies: “All twelve refunds have been processed.” Three of them were not. One upstream call returned a 409, one timed out, and one was never made because the agent ran out of steps. The model summarized its plan as if it had been carried out, and nothing in the pipeline disagreed.

This is one of the quieter ways agentic AI goes wrong. Nothing gets deleted and no credential leaks. The record of what happened is wrong, and that is the record the business acts on.

Why agents report success they didn’t achieve

A language model’s final message is a summary it writes itself. It is not a log. Several common mechanics make that summary drift from reality:

  • Errors summarized away. A tool returns an error body, the model reads it as a partial result, and the final message rounds it up to success.
  • Plans reported as actions. Long tasks hit step limits, context limits or timeouts. The agent describes what it intended to do in the same past tense it uses for what it did.
  • Retries with unknown outcomes. After a timeout the agent can’t tell whether the write landed, so it either retries (the duplicate-action problem) or assumes it worked.
  • Sub-agent hearsay. In multi-agent systems an orchestrator reports what its sub-agents told it. Over an A2A hop, “done” becomes a claim about a claim, often across an organizational boundary.
  • Injected confirmations. A poisoned tool response or document can tell the model the task already succeeded, a variant of the indirect prompt injection patterns we have covered before.

None of this needs a malicious model. It is what you get when the component that performs an action is also the only one that reports on it.

Self-reported logs are not evidence

Many teams try to fix this by asking the agent to log its own actions: write a summary to a file, emit a trace span, call a “report” tool. That improves visibility, but it doesn’t change who is speaking. A self-reported log has the same blind spots as the final message: it records what the agent believes, and it can be skipped, truncated or steered by the same inputs that confused the agent in the first place.

For traceability you need a record written by something the agent cannot edit, at the point where the action actually leaves your perimeter: the outbound call to the API that moves the money, sends the mail or changes the record. In practice that means three properties:

  1. Independent. Written by the component that executed the call, not by the agent that requested it.
  2. Outcome-bearing. It records the real upstream response status, latency and error, not just the fact that the agent “used a tool”.
  3. Attributable. It ties each call to a specific agent identity, a specific authorization and a specific set of arguments, so a claim can be matched to a row.

With that record, “All twelve refunds processed” becomes a query: twelve calls to the refund endpoint by this agent in this window, and what status each one returned.

Reconcile claims against the trail

Treat the agent’s report as a hypothesis and the trail as ground truth. A simple reconciliation loop catches most false-success cases:

  • Count. Does the number of brokered calls match the number of actions the agent claims? Missing rows mean actions that never happened.
  • Status. Do the recorded upstream statuses say success? A 4xx or 5xx under a “done” message is a summarization failure, and a reportable one.
  • Arguments. Does the parameter hash match what the agent says it sent? A mismatch means the agent called the right tool with different inputs.
  • Authorization. Was each call backed by the expected approval or signed envelope? An action with no authorization behind it is a governance finding, whatever its outcome.

Run it automatically for high-value workflows, and at the very least whenever an incident review starts. Our incident forensics guide walks through the reconstruction side in more detail.

Observability that alerts before the customer does

Reconciliation after the fact is good. Noticing during the run is better. Because the independent trail carries outcomes, you can alert on them: an error rate on a write tool above a threshold, a burst of calls from one principal, or any single call to a tool that should never fail quietly. If the alert fires while the agent is busy writing a confident summary, a human finds out before the customer does.

The business case

False success is expensive in ways that are hard to see on a dashboard: refunds that customers chase through support, compliance attestations built on actions that never ran, and finance reconciliations that fail weeks later. The cost usually isn’t the failed call. It is the time between the failure and someone noticing, plus the trust lost when “the AI said it was done” turns out to be the only evidence.

An independent, outcome-bearing trail shortens that gap to minutes and turns an argument about what the agent meant into a lookup. It is also the artifact auditors ask for. Frameworks such as SOC 2 and ISO/IEC 42001 expect you to show what an automated system did, not what it said it did.

How Fullmakt does it

Fullmakt is a credential and policy broker that sits between AI agents and the APIs they call. Agents connect over MCP or A2A with a scoped token and never hold the real secret, so their outbound actions go through the broker, and the broker writes the record.

For each brokered tool call, the audit entry captures:

  • the agent principal, collection and tool name,
  • the HTTP method and upstream host,
  • a SHA-256 hash of the canonical JSON of the agent’s arguments,
  • which credential references were resolved (names such as vault://local/stripe_key, never the values),
  • the upstream response status, latency, success flag and error,
  • the signed task envelope presented with the call (accepted or rejected), where there was one,
  • and which surface produced it: the MCP data plane or the A2A management plane.

Agents that call an upstream directly can still report those calls through a side-channel endpoint, but those rows are explicitly flagged as reported externally. Self-reports and broker-observed calls sit side by side in the same timeline, and you can always tell which is which. That distinction is the point of this post: you can see which rows are the agent’s account and which are the broker’s observation.

On top of the trail, alert rules evaluate recent entries per tool, principal or collection, with an instant, count over window or error rate condition, and fire a webhook to your on-call or chat tooling. Each rule’s last-fired time and fire count show in the dashboard. A live timeline streams brokered MCP calls as they land and can be filtered by agent, collection and time window, so the count-and-status checks above are a quick look rather than a project.

Takeaways

  • An agent’s “done” is a summary, not evidence.
  • Record agentic actions where they leave your perimeter, written by the component that executed them.
  • Capture outcomes (status, error, latency), not just tool usage.
  • Keep self-reported activity visibly separate from observed activity.
  • Reconcile claims against the trail, and alert on error rates so you hear about failures before customers do.

FAQ

Why do AI agents claim tasks succeeded when they didn’t? The final message is written by the model itself. Errors read as partial results, step or context limits cut tasks short, timeouts leave outcomes unknown, and multi-agent hops pass along claims nobody verified.

Isn’t asking the agent to log its actions enough? No. A self-reported log shares the agent’s blind spots and can be skipped or manipulated. Use it as a supplement, labeled as self-reported, next to an independent record.

What should an AI agent audit trail record to verify actions? Agent identity, tool, arguments (or a hash of them), the authorization used, the upstream response status, latency and error, and the credential references involved, never the secret values.

How does this work with MCP and A2A agents? When agents call tools through a broker over MCP, or management actions over A2A, the broker records each call with its outcome. In multi-agent chains, each hop’s calls can be matched to the principal that made them.

Does Fullmakt store request or response bodies in the audit trail? No. The audit entry stores a hash of the arguments and the upstream response status, not the request or response bodies, and credential references are recorded by name only, never as secret values. An error message is kept when a call fails.