Don't Test Your AI Agent With Production Credentials: Rehearse Governance in a Sandbox First
5 min read Fullmakt Team
- agents
- governance
- credentials
- agentic-actions
- traceability
- authentication
- mcp
- a2a
Most agentic AI incidents don’t start in production. They start in a pilot. Someone wires up an agent, the demo needs to touch something real, and a production API key gets pasted into a config file “just to see if it works.” The pilot succeeds, the config file is copied into the next environment, and the key that was only ever meant for an afternoon is now the agent’s standing identity.
The agent in that story didn’t go rogue. The team skipped the step where you find out what the agent does before it can do anything that matters.
The pilot-credential problem
A first agent integration usually asks several questions at once: does the agent call the right tools, does the login handshake work, what happens when a call is slow or fails, and what would a reviewer want to see in the logs. Answering them with real credentials has predictable side effects:
- Real blast radius. A prompt that goes wrong while the agent holds a live write token can send real mail, charge real accounts or delete real records. We’ve covered how destructive actions slip through when nothing separates read from write.
- Secrets in the wrong places. Keys end up in config files, chat threads and agent context windows, the raw material of a leaked-key incident.
- Governance bolted on afterward. Policies, approvals and audit trails get designed after the agent already works, so they get designed around its existing behavior instead of constraining it.
- Untested guardrails. A policy that has never been exercised may not do what its author thinks; a guardrail that silently fails open is worse than none.
What a good rehearsal environment gives you
The goal isn’t a toy. A useful sandbox runs the same control path as production with nothing of value behind it. That means:
- Real mechanics, fake stakes. The broker, policy engine, approval flow and audit trail should be the real ones. Only the upstream systems and secrets are stand-ins.
- Zero-friction start. If trying it needs a procurement cycle, people will paste the production key instead.
- A defined lifetime. Throwaway environments should expire and be cleaned up, so an experiment doesn’t quietly become infrastructure.
- A path to keep your work. Rehearsal is wasted if you must rebuild everything when it’s time to go live.
With those properties a team can answer the pilot’s questions safely: attach a policy and see what it denies, let the agent trip an approval and watch a human decide, then read the audit trail the way an incident responder or auditor would. The staged-autonomy approach starts here: observe, then constrain, then widen.
A checklist for the rehearsal
Before any agent touches a real credential, check that you can:
- Connect it through the real agent handshake (OAuth client, MCP or A2A) with a token that carries only the scope you intend.
- Trigger a deny with a rule and confirm the agent receives a clear refusal rather than an improvised workaround.
- Trigger an approval and confirm a human, not the agent, resolves it.
- Read the trace: agent identity, tool, parameter hash, policy decision and reason, without secrets or response bodies in the log.
- Test failure: an expired token or unreachable upstream should end in a clean error, not an improvised re-login.
Only then swap the stand-in secret for the real one. Because the agent only ever referenced a credential by name, the swap is a configuration change, not a rewrite.
The business case
A sandbox shortens the distance between “interesting demo” and “approved rollout”. Security reviewers get a working system to inspect instead of a slide deck. Engineers find policy mistakes while they’re cheap. And the people who sign off, from compliance to the CFO, see evidence of control before any exposure exists. Teams that skip this step tend to learn the same lessons later, with customers watching.
How Fullmakt does it
Fullmakt is a credential and policy broker between AI agents and the systems they call. Agents hold a scoped token, never the real secret; every call is checked against the collection’s policy, with deny rules, approval rules and behavioral limits, and recorded in the audit trail.
Where the no-signup sandbox is enabled, a visitor gets a real, time-limited session without creating an account (the default lifetime is 72 hours, followed by a cleanup grace period). It runs the genuine broker, vault, policy engine and audit trail, seeded with demo data, and a setup wizard produces an agent principal and OAuth client ready to paste into your agent’s configuration. When you decide to keep it, claiming the sandbox converts it in place into a real account under the same user, so collections, vault entries, principals, clients and audit history carry over.
In practice: open the sandbox, create a collection with a placeholder secret, attach a policy with a rule that requires approval for writes, point your agent at it, and watch the decisions land in the trail. When it behaves, replace the placeholder with the real credential.
Takeaways
- The riskiest credential in an agent project is the one pasted in “just for the pilot”.
- Rehearse with the real control path and fake stakes.
- Exercise deny, approval and failure paths on purpose before go-live.
- Prefer environments that expire and that you can promote, not rebuild.
FAQ
Why shouldn’t I test an AI agent with production credentials? Because an untested agent can take real, hard-to-reverse actions, and the key tends to persist in config files and context long after the test ends.
What should an agent sandbox include? The real policy engine, approval flow and audit trail, a scoped agent token, placeholder secrets and an expiry date.
Does sandbox work carry over to production? In Fullmakt’s sandbox it does: claiming the session turns it into a real account in place, keeping collections, vault entries, principals, clients and history.
Is the sandbox always available? It is a configurable feature, so availability depends on the deployment.