What Is AI Agent Governance? A Practical Guide

Coding agents crossed a line in the last two years. They don’t just suggest code anymore. They open pull requests, run migrations, restart services, and call cloud APIs. Once an agent can act, the interesting question stops being “is the model any good?” and becomes “who said it could do that, and how do we undo it?”

That question is what AI agent governance answers.

A working definition

AI agent governance is the set of controls that determine what an autonomous agent may do in real systems: how it is identified, which operations it can request, which of those require human approval, how every action is attributed and audited, and how changes are rolled back.

Notice what’s not in that definition: model weights, training data, bias audits, or policy PDFs. Those belong to classic AI governance, which governs the model. Agent governance governs the actions. The difference matters, because most “AI governance platforms” stop at documentation and risk registers, while the actual risk now walks through your production environment with a shell and ambient credentials.

Why this became urgent

Three shifts happened at once:

1. Agents got hands. Tools like Claude Code and MCP-connected assistants don’t stop at text. They execute. A modern agent can run terraform apply as easily as it can explain it.
2. Credentials didn’t evolve. The standard way to “give an agent cloud access” is still to expose whatever access keys sit in the environment. The agent inherits your power with none of your judgment. And every prompt injection on the internet becomes a potential caller of your AWS account.
3. Audit stayed API-shaped. CloudTrail can tell you TerminateInstances was called by a role. It can’t tell you which agent called it, in which conversation, at whose request, or what “undo” looks like.

The five pillars

Every serious agent-governance design ends up with the same five layers.

1. Identity: who is acting?

Each agent seat needs its own identity, separate from the human it serves. If your audit trail says admin did X when an agent did X on the admin’s laptop, you have attribution theater, not attribution. In AWS terms this is what sts:SourceIdentity exists for: the acting identity rides along into CloudTrail.

2. Typed capabilities: what can be asked at all?

Free-form shell access is ungovernable by construction. Governance needs a registry of typed operations, like “reboot RDS instance” or “put bucket policy”, each with declared parameters and a risk tier. If it isn’t in the registry, it can’t be requested. If it is, it can be policed.

3. Policy and approvals: what happens automatically?

Risk tiers map to behavior. Read-only observation runs instantly, risky writes pause in an approval queue for a human, and destructive operations can be refused outright. This is the “are you sure?” that raw cloud APIs never had.

4. Attributed audit: what happened?

Every sensitive action lands on an append-only ledger: agent, human, workspace, operation, parameters, policy decision, outcome. Tamper-evidence (hash chaining you can verify) turns the ledger from a log file into evidence.

5. Reversibility: how do we undo it?

Governance that can’t undo is just well-documented regret. Changes should capture the before-state, a rollback envelope, so reversal is a button instead of an incident-response project.

A quick evaluation checklist

If you’re evaluating a platform (or designing your own), ask:

• Does the agent ever hold long-lived cloud credentials? (Correct answer: never.)
• Is enforcement server-side, or is it a system prompt asking nicely?
• Can I start read-only and widen access gradually?
• Can I revoke access on my own, without the vendor’s cooperation?
• Is every action attributed to the agent and the human behind it?
• Can I export the audit trail and verify it hasn’t been altered?
• Is there a rollback story for destructive changes?

How Mesoplane implements this

Mesoplane is a governed middle layer built on exactly these pillars. Agents call a typed operation registry (via skills, HTTP, or MCP). Guardrail policies route high-risk operations to an approvals inbox. Every action lands on a verifiable ledger with per-agent SourceIdentity attribution, and destructive fixes carry rollback envelopes. The cloud connection itself is an AssumeRole handshake with an ExternalId: no stored credentials, and a kill switch you own.

If you’re wiring a coding agent to a real AWS account this quarter, start with our guide on giving Claude Code safe AWS access.

Frequently asked questions

What's the difference between AI governance and AI agent governance?

AI governance usually means governing models: training data, bias, compliance paperwork, acceptable use. AI agent governance governs actions. What can a running agent actually do in your systems, who approves it, how is it audited, and how do you reverse it? You can pass a model-governance review and still have an agent holding your production AWS keys.

Do I need agent governance if I already use IAM?

IAM is necessary but not enough. IAM answers one question: may this principal call this API? It has no idea which agent asked, on whose behalf, whether a human should approve first, or how to undo the change. Agent governance adds those layers on top of IAM, not instead of it.

What should an AI agent audit trail contain?

At minimum: the acting agent and the human it acted for, the workspace or tenant, the exact operation and its parameters, the policy decision (auto-approved, human-approved, or refused), the outcome, and enough before-state to reverse the change. It should also be tamper-evident, so you can prove nobody edited the record after the fact.