Most organisations govern AI with documents. Documents describe intentions. Systems execute permissions, and the two are not the same thing

An organisation that has adopted AI responsibly can usually produce evidence. There is a policy. There is a risk assessment. There is a register of roles and responsibilities, a description of controls, a monitoring commitment, and an audit trail requirement. These documents were drafted carefully, approved at the appropriate level, and circulated.

None of them prevents anything.

A policy is a description of what should happen. It constrains behaviour only through the willingness of people to follow it and the capacity of an institution to notice when they have not. Against a human employee, this works reasonably well most of the time, because people read documents, absorb norms, and fear consequences.

Against a software agent, it does nothing whatsoever. A system that can send an email will send an email regardless of what the communications policy says. A system with write access to a database will write to it. A system with credentials to initiate a payment will initiate one. The policy exists in a document. The permission exists in the system, and only one of those two things determines what actually happens.

This gap has always existed in information security, where it was understood decades ago. It is being rediscovered painfully in AI governance, because organisations are deploying agents with real capabilities while governing them with instruments designed for people.

What Changed

Earlier AI systems produced outputs. A model generated a summary, a score, a draft, a recommendation. Whatever the system produced, a person decided what to do with it. The human being was the mechanism through which anything happened, and governing the human being was therefore sufficient.

Agentic systems remove that step. They are given tools: access to email, calendars, databases, ticketing systems, payment rails, customer records, code repositories. They plan sequences of actions and execute them. The point of the design is that a person is not in the loop for each step, because a person in the loop for each step would defeat the purpose.

The moment that step disappears, every governance instrument that worked by shaping human decisions stops working. Not partially. Entirely.

Two Different Kinds of Control

It is worth being precise about the distinction, because organisations frequently believe they have the second when they only have the first.

Governance that explains consists of policies, standards, risk assessments, role definitions, training, and audit requirements. Its function is to establish expectations, allocate responsibility, and create a record. It is genuinely necessary. Without it, nobody knows what the organisation intends or who is accountable.

Authority that executes consists of the actual permissions a system holds: which credentials it possesses, which resources it can reach, which operations it can perform, which thresholds trigger a required human approval, and what it is technically incapable of doing. This is not a description of intent. It is a fact about the system, and it determines outcomes whether or not anyone has read the policy.

The first without the second produces an organisation that is well-documented and unprotected. Every incident is followed by the discovery that a policy did exist, and was clear, and was not connected to anything.

Designing the Authorisation Layer

If authority is what actually governs, then the design of the authorisation layer is the governance work, and it has a reasonably well-understood shape.

Least privilege. An agent receives access to what its task requires and nothing more. Broad credentials granted for convenience during development have a way of persisting into production.

Explicit approval for consequential actions. The system requests, a person authorises. This is where the tiering matters: retrieving a record needs no approval, sending external correspondence might, initiating a payment or terminating a service certainly does. The threshold should be set by consequence and by reversibility, not by technical difficulty.

Refusal as a designed outcome. An authorisation architecture needs three possible answers, not two. Approved, reviewed, refused. A system where everything is eventually approved because refusing is procedurally difficult has an approval workflow, not an authorisation control.

Immutable logging. Every action recorded in a form the acting system cannot alter, sufficient to reconstruct afterwards what was done and on whose authority.

A functioning stop. Somebody must be able to halt the system quickly, that person must be identified in advance, and using the stop must not be career-threatening. A halt authority that nobody dares exercise is not a control.

None of this is technically hard. It is organisationally inconvenient, which is why it is so often deferred.

The Conscience Argument

There is a reason the Catholic moral tradition is useful here, and it is not decorative.

The tradition holds that conscience is exercised by persons. It cannot be held by an institution, discharged by a document, or transferred to a process. A policy can inform a conscience; it cannot substitute for one. Responsibility attaches to someone who could have decided otherwise.

Applied to autonomous systems, this yields something quite concrete. Every consequential action a system takes must trace back to a person who authorised the capability, understood what it permitted, and can be asked why. Not a committee. Not a role. A person whose name is recorded.

When an organisation cannot identify that person, it has not achieved sophisticated automation. It has arranged for things to happen that nobody chose, and that is a moral condition rather than a technical one.

The phrase to be wary of is “the system did it.” Systems do not act on their own authority. They act on delegated authority, and delegation always has a source. If the source cannot be named, the delegation was not made properly.

What This Asks of Institutions

Three things, in order.

Find out what your systems can actually do. Not what the policy says they may do. An inventory of real permissions, real credentials, and real reach. Most organisations discover a gap here, and the gap is the whole problem.

Close it deliberately. Where a system holds capability that policy did not intend, remove the capability rather than restating the policy.

Name the authorising person for each capability. Written down, current, and known to the person named.

An organisation that has done these three things has governance. An organisation with excellent documents and unexamined permissions has an account of what it meant to do.

The risk was never that AI would act. Acting is what these systems are built for, and much of it is useful. The risk is action nobody authorised, in a form nobody can trace, on behalf of an institution that will discover afterwards that its policy was very clear and entirely disconnected from what its systems could reach.