Out in the wild
A week that will sound familiar.
None of these is a story about a broken agent. In two of the three the agent did something perfectly sensible, which is what makes them hard to talk about afterwards.
Agents without rules means no written line between what an agent decides alone and what it hands to a person.
Two thirds of technology leaders are accountable for AI systems they do not fully control, and 77% say adoption has already passed their governance.1 Companies report an average of 54 agent incidents a year that needed a human to step in.1
The short version
- Fifty-four times a year, on average, an agent does something that needs a person to come and fix it. One in six of those takes more than four hours to contain.1
- Two thirds of tech leaders now answer for systems they do not fully control, and only one in nine feels ready for the number of agents coming next year.1
- The missing rule is almost always a business decision about money, tone or risk that nobody has been asked to make.
- Companies that build the limits into the system rather than watching manually run sixteen times as many agents, with a quarter fewer incidents.1 Rules are what let you go faster here.
- Every time I ask a team what their agent may decide alone, I get different answers from people sitting at the same table. That is the finding, and it takes twenty minutes to fix.
Chances are one of these has happened
- Something was decided in your company's name and you only found out because a customer mentioned it
- You are about to let an agent touch money, contracts or customers, and you would like the boundary settled before it does
- Your legal or compliance lead asked a question you could not answer in the meeting
- You suspect the real rule is whatever the person who set it up happened to type that afternoon
What you take away
- A decision line you can draw in twenty minutes, on one page, per agent
- What actually goes wrong when it is missing, with the numbers behind it
- Why the companies with the tightest limits are the ones running the most agents
- Where this sits in the five AI Culture Levels, and what moves you up
In plain terms
Most leaders meet this as a trust question. It behaves like a delegation question.
The conversation usually gets framed as how much we trust the model, and that framing takes you nowhere. You do not trust a new account manager in the abstract either. You tell them they can discount up to ten per cent, that anything above that comes to you, and that they never promise a delivery date without checking.
Nobody finds that insulting. It is how delegation works when it is done well. An agent gets none of it, because it arrived through a tool and not through a hiring process, so the only real limit on it is what the software happens to allow.
It's like flying a plane at 10,000 feet, being told to climb to 12,000, replace both engines mid-flight and ensure zero turbulence. No one would choose to pilot that plane, but that's exactly what companies are doing today.Afonso Eça, Executive Board Member at Banco BPI1
He is describing the pressure, and the way out of it is duller than people expect. You do not need a governance programme. You need two sentences per agent that a new colleague could read and act on, and one person who says them out loud.
The evidence
Fifty-four times a year, somebody has to come and fix it.
That is the average number of agent incidents companies reported over the past year, where something unintended or harmful needed a human to correct it.1 Roughly one in six of those was serious enough to take more than four hours to contain.
Now the part that surprised me, and the reason this page is not a warning. In the same study, the companies that built limits into the system rather than watching manually were running sixteen times as many agents as everybody else.
Read that the way you would read it about people. The teams with the clearest boundaries are the ones you can hand the most to. It has never worked differently, and it turns out agents are no exception.
Why it happens
The rules were never written, because nothing asked for them.
Writing the rule means making a decision somebody has been avoiding
What is the most this thing may give away without asking. Fifty euros, five hundred, nine hundred. Somebody has to say a number out loud and then live with it. That is uncomfortable in a way that buying a licence is not, so it gets pushed to the next meeting, and the agent keeps running in the meantime with whatever limit the software came with.
The rule you never wrote down is still a rule. It is just whatever the software happened to allow.Paul Musters
The people who could set the line are not in the room
Seven out of ten technology leaders say teams across the business are deploying faster than IT can track.1 An agent goes live in a marketing team on a Wednesday. Legal hears about it in October. By then the question is no longer what should this be allowed to do, it is what has it already been doing.
For CIOs and CTOs, the challenge now is scaling AI systems that operate continuously and autonomously, often within governance models and architectures designed for a far slower, more predictable environment.Matt Lyteson, CIO at IBM1
Your policy is about tools, and this is about decisions
Most AI policies I read describe which tools are approved and what data may go in. Useful, and it answers none of this. An agent that acts needs a different kind of sentence: what it may decide, what it must hand over, what it may never touch. Only a third of companies have formally adopted a policy for agents at all.2 The rest have something about tools, which the agent has never read.
Before you draw the line, find out where you stand
Three minutes, six questions, anonymous. It gives you your level, the price of staying on it, and what moves at the next one.
One thing to try
Three columns. Twenty minutes.
Take one agent that is running today. Get the person who owns it and the person who carries the risk in the same room, which is usually two people and not a committee. Fill in three columns.
Decides alone
Things you would be comfortable reading about after the fact. Put a number on anything with money in it.
- Answers a question the customer could have found on the site
- Books a slot in a calendar that is already free
- Refunds up to fifty euros on an order under thirty days old
Hands to a person
Things where a person adds judgement rather than a click. Name who, not which team.
- Anything above that amount, or with a complaint attached
- A promise about a date the company has not confirmed
- Anything a customer would call unfair even if it is correct
Never
The short list. If it is long, you are describing a job the agent should not have.
- Contracts, notice periods, anything with a signature
- Anything involving someone's employment
- Writing to a customer in a dispute
Example lines. Yours will be different, and the middle column is where the real conversation happens.
Two rules about the exercise. Every line in the middle column needs a name, because “escalate to the team” means escalate to nobody. And if you cannot fill in the first column without a debate, you have found a decision the business never made, which the agent has been making for you in the meantime.
In the five levels
Written rules are a Level 3 habit.
In the five AI Culture Levels we use with clients, this is the step where things stop living in people's heads. Level 3 is where the way of working gets written down, and an agent's boundaries are part of that.
01Campfire60%
02Wild West25%
03Blueprint10%Rules get written here
04Engine4%
05Ecosystem1%
At Level 2 the limits live in the head of whoever built the thing. That works until they are away, and it stops working entirely once a second person can change the agent. At Level 3 the way of working is written down, so a new colleague can read what the agent may do and act on it in their first week. That is the whole difference, and it is smaller than most transformation programmes make it sound.
Level 4 is where the limits stop being a document and start being part of the system, so the agent cannot exceed them even if somebody wants it to. That is what the sixteen times number is about. Companies at that level are not more cautious. They can hand over more precisely because the edges hold.
And most companies sit on more than one level at once. Your engineering team may be at 3 while the commercial side is at 1, and the agent with the loosest boundary is usually not in the team you are watching.
What a measurement adds to a rule
An Operating Profile in use. Personality type and AI level in one profile, with the agents that fit it.
A rule tells the agent where to stop. It does not tell you whether the person on the other side of that line can handle what arrives there. Escalating to somebody who will glance at it and click approve is not oversight, it is a delay with a signature on the end.
We measure it at the level of the person. How somebody thinks and works, and where they are with AI. Then the middle column of your decision line goes to a name that can actually carry it. In practice the right name is often not the most senior one in the room.
About emaho
emaho measures one Operating Profile per person: personality type and AI level in a single profile. On that we build a personal set of AI agents that fit how that person works, inside the tools they already use. Fifteen minutes to complete, first profile free, built for companies between 20 and 500 people.
Fifteen minutes per person. No credit card, no strings.
What to do
Five steps, and the hard one is second.
Start with the agent that touches money or customers, not the one that is easiest. Draw the three columns for that one. Twenty minutes with two people beats a quarter with a working group, and you will learn more from the argument than from the document.
Then put a number on the middle column. This is the step everyone slides past. Somebody has to say the amount, the date range, the tone. If the room cannot agree, do not paper over it with a phrase like “use judgement”. Write down that it is unresolved and put a name and a date against it, because right now the agent is resolving it for you every day.
Make escalation land on a person, not a channel. A queue nobody owns is where escalations go to be ignored, and after a month people stop sending them there. One name, and a second name for when the first is away.
Then move what you can into the system itself. A limit that is written in a document depends on everyone remembering it. A limit that is coded into the agent holds on a Sunday. That is where the difference between the companies with a quarter fewer incidents comes from.1
And read the exceptions once a quarter. What got escalated, and what should have been and was not. That half hour tells you more about whether your line is in the right place than any dashboard will.
Drawn your three columns? Send me the middle one
The middle column is where I can actually tell you something. Send it and I will tell you what I would move and what I would tighten. No deck, no call. One message back.
What waiting costs
You find out what it may decide on the day it decides it.
The visible cost is the incidents themselves, and they are more mundane than the word suggests. Of the serious ones, over a third ended in data exposure or a security breach and a third caused failures that spread into other systems.1 Four hours to contain, on a day you had planned to do something else.
Underneath that sits a slower cost. Once a team has been surprised twice, they start checking everything the agent produces. Nobody asked them to. They do it because nobody told them where the edge was, and checking everything is the only safe answer to that. At that point you are paying for the agent and paying for the human review, and the gain you built the thing for has quietly gone.
And there is the one that shows up in a board meeting. Somebody asks what the agent is allowed to do and you need a week to answer. That week tells them how much of the business is now running on decisions nobody wrote down, and once a board sees that, the appetite for the next agent goes to nothing. More than 40% of agent projects are expected to be cancelled before the end of 2027, and inadequate risk controls sit in that same sentence.3
What people search for
What people ask about agent rules
What should an AI agent be allowed to decide on its own?
Whatever you would be comfortable reading about after it happened, with a number attached wherever money is involved. A workable line has three columns: what it decides alone, what it hands to a named person, and what it may never touch. Contracts, anything about someone's employment and messages to a customer in dispute belong in the third column for almost every business.
What are guardrails for AI agents?
Guardrails are the limits an agent cannot exceed, ideally built into the system rather than written in a document. The difference matters: a documented limit depends on everyone remembering it, while a coded limit still holds on a Sunday night. IBM found in 2026 that organisations embedding control in their systems had 25% fewer incidents than those relying on manual oversight.
How often do AI agents actually cause problems?
More often than most leadership teams assume. IBM's 2026 study of 2,000 technology executives found an average of 54 agent incidents per organisation in the past year, where something unintended or harmful needed a human to correct it. Around 17% of those were serious enough to take more than four hours to contain.
How do you know which people can decide what an agent may do on its own?
That call needs somebody who understands both the work and the tool, and those two rarely sit in the same place on an org chart. emaho measures one Operating Profile per person, personality type and AI level in a single profile, so you can see who has the judgement for it rather than who has the job title for it.
What goes wrong in a serious AI agent incident?
Of the high severity incidents in IBM's 2026 study, 37% caused data exposure or a security breach, 33% caused failures that cascaded into other systems, and 17% triggered compliance problems. The pattern is rarely dramatic. It is usually an agent doing something reasonable in a situation nobody had thought about.
Who should decide what an agent may do, IT or the business?
The business, with IT in the room. The missing rule is almost never technical: it is a decision about money, tone or risk that somebody has been avoiding. IT can tell you what is possible and can build the limit in, but the number in the middle column has to come from whoever carries the commercial or legal consequence.
What does emaho do about AI agent governance?
emaho starts one level down from policy. Each person gets an Operating Profile, and on that we build a personal set of agents that fit how they work, with the limits written into the agent rather than into a document nobody opens. The first profile is free, fifteen minutes per person, built for companies between 20 and 500 people.
Does writing rules for agents slow us down?
The evidence points the other way. IBM found that organisations which build control into their AI systems run 16 times more agents than those relying on manual governance, with 25% fewer incidents and 18% higher operating margins. Clear boundaries are what let you hand over more, which is how it has always worked with people too.
How do we keep control as the number of agents grows?
Move limits out of documents and into the systems, and review the exceptions rather than the volume. Technology executives expect a 38% increase in deployed agents by 2027, while only 11% feel fully ready for the scale coming in the next year alone. Manual oversight does not survive that arithmetic; built-in limits do.
How does this relate to AI maturity levels?
Written rules are Level 3 behaviour in the five emaho AI Culture Levels, where the way of working gets documented instead of living in people's heads. About one in ten organisations is there. At Level 4 the limits are part of the system rather than the paperwork, which is why those organisations can safely run far more agents than the rest.
Close by
Rules are the second question. Here is the first, and the two that follow.
A boundary only works if somebody is behind it and somebody is still looking. These three sit either side of this one.
01
Nobody owns the agent
A rule with no name behind it is a suggestion. Before you draw the line, somebody has to be responsible for what sits on either side of it.
Read this one 18Oversight quietly stops
You wrote the rule and put a person behind it. Six months later that person approves everything within a minute, because it always looked fine before.
Read this one 07Shadow AI
Rules for the agents you know about do nothing for the ones running in personal accounts. Almost half of employees use AI nobody approved.
Read this oneAll 25 AI challenges, and what to do about each
Twenty-five things that break inside a company once people start using AI, each with the research behind it and the level where it starts to bite. This one starts to bite at Wild West, and 8 of the others start there too.
Getting going
Draw the line this week. Then check who is standing behind it.
The three columns take twenty minutes. What they cannot tell you is whether the person the middle column escalates to can judge what lands there. That is the part we measure, starting with you.
- Twenty minutes with the three columnsMay decide, must ask, never touches. Written down once, it settles arguments for a year.
- Find out who the middle column points atEscalation only works if the person at the end of it can judge what lands there. One profile each shows you whether they can.
- Put the rule inside the agentA rule in a document gets read once. A limit built into the agent still holds on a Friday afternoon.
First profile free · no credit card · built for companies of 20 to 500 · you decide what your team gets to see
Not ready to put your team in anything yet? Start with the level of the company instead. The Culture Level scan is six questions, three minutes, and asks nothing of you.
Fifteen years of leadership and team development in Dutch scale-ups. That practice now sits in software: one Operating Profile per person, with agents that actually fit. He writes these pages from what he runs into with clients, not from a research summary.
Sources and numbers used on this page
- IBM Institute for Business Value with Oxford Economics, published 8 June 2026. 2,000 senior technology executives across 33 geographies and 19 industries, surveyed January to April 2026. Source of: two thirds accountable for AI systems they do not fully control, 77% saying adoption outpaces governance, 11% fully ready for the scale of agent deployment expected next year, 80% under a CEO-driven AI mandate, 70% saying teams deploy faster than IT can track, an average of 54 agent incidents needing human correction in the past year, 17% of those high severity taking over four hours to contain, of which 37% caused data exposure or a security breach, 33% cascading system failures and 17% compliance issues, and the comparison showing organisations that embed control run 16 times more agents with 25% fewer incidents and 18% higher operating margins. Quotes from Afonso Eça (Banco BPI) and Matt Lyteson (IBM) come from the same release.
- Cloud Security Alliance, Enterprise AI Security Starts with AI Agents, April 2026. Commissioned by Zenity, 445 responses from IT and security professionals. Source of: 31% have formally adopted a policy for agent usage, 53% have had an agent exceed its permissions and 8% say that never happens.
- Gartner, press release, 25 June 2025. Over 40% of agentic AI projects expected to be cancelled by the end of 2027, because of escalating costs, unclear business value or inadequate risk controls.
- Moffatt v. Air Canada, British Columbia Civil Resolution Tribunal, 14 February 2024. A company held liable for what its website chatbot told a customer, after arguing unsuccessfully that the chatbot was a separate entity responsible for its own statements.
- emaho AI Culture Levels. Share of organisations per level, calibrated against BCG 2025 and McKinsey 2025.
Numbers are quoted as published. Where a figure is described as around or roughly, that is how the source states it. Nothing on this page is legal advice.