Here's the moment a lot of us had this year.

You built an agent. It was genuinely impressive — it reasoned, it planned, it called tools, it chained steps together to get real work done. The demo was so good that you gave it real permissions. Let it send the emails. Let it update the records. Let it hit the production API.

And then, one day, it did something you never sanctioned. Not maliciously — confidently. Correctly-looking. It completed the action, reported success, and moved on. You only found out later, when the consequence surfaced somewhere downstream. That's the moment you realize what you actually shipped: a system with a powerful engine and no brake pedal.

The entire industry has spent two years racing to make agents smarter. Better reasoning, longer context, more tools, more autonomy. Almost nobody spent that time making them safe to let act. And those are completely different problems — because the instant an agent stops just talking and starts doing, a more capable agent isn't more useful. It's more dangerous.

Let me make the case, and then talk about the brakes.

Brains and brakes are different problems

This is the distinction the hype cycle blurred, so it's worth stating plainly.

Making an agent capable — able to reason about a task, plan a sequence of steps, pick the right tool, recover from an error — is one problem, and it's the one all the model progress went into. Every new release is a better brain.

Making an agent safe to let act — ensuring that when it's wrong, the wrongness is caught, bounded, reversible, or stopped before it does damage — is a completely different problem. And here's the uncomfortable part: no smarter model solves it for you. A more capable agent doesn't act wrongly less; it acts wrongly faster and more convincingly. Upgrading the brain does nothing for the brakes, because the brakes were never in the model. They're in the architecture around it — the part you have to build yourself.

You can have the best engine in the world. Without brakes, that just means you hit the wall sooner.

Why "acting" changes everything

A chat that only talks has a wonderful property: its mistakes are free. It says something wrong, you read it, you move on. Nothing happened in the world.

The moment an agent can act, three things change, and each one is a reason you need control that a chatbot never did.

Irreversibility. A wrong sentence costs nothing. A wrong DELETE, a sent email, a processed refund, a pushed commit — those don't have an undo button. The cost of an agent's mistake stops being "re-read it" and becomes "clean up the damage," and some damage doesn't clean up.

Confidence is not correctness. This is the through-line of everything I write, and it's never more dangerous than when an agent can act. The agent takes the wrong action with exactly the same confidence it takes the right one. There is no tremor in its voice, no hedge, no tell. It is equally sure when it's about to help you and when it's about to hurt you.

Silent success. The nastiest one. The agent completes the wrong action and reports success — because from its point of view, it did complete the task, it just completed the wrong one. Your monitoring goes green. Nobody sees the wrong customer getting the refund until the wrong customer gets the refund. "It worked" and "it did the right thing" are different claims, and only one of them shows up on the dashboard.

Put those together and you get the core risk: an agent that can act is a system that can cause irreversible harm, confidently, without anything flagging it. That is precisely the situation brakes exist for.

Brake #1: Approval gates — ask before acting

The most basic brake, and the one most systems skip: for anything consequential, the agent doesn't do the thing. It proposes the thing, and waits for a human to say yes.

The skill here isn't gating everything — that would make the agent useless, and you'd click "approve" on autopilot until the gate meant nothing. The skill is gating by blast radius:

  • Auto-run the trivial and reversible — read something, format something, draft something. No gate needed.
  • Ask first for the consequential — sending, paying, writing to production, anything a user would want to see before it happens.
  • Draft-only for the dangerous — the agent prepares the action but cannot execute it at all; a human does.

The mistake to avoid is a uniform gate on everything, which trains people to rubber-stamp. A good gate is rare enough that when it fires, the human actually looks.

Brake #2: A reviewer that can actually say no

The popular pattern: a second agent reviews the first agent's work before it's allowed to proceed. A judge. A critic. It's a good pattern — and it has a failure mode that will fool you completely if you're not watching for it.

A reviewer that has never been seen to fail is indistinguishable from a reviewer that approves everything.

If your judge-agent has been green for months, that is not evidence it's working. It might be catching problems. It might also be rubber-stamping, silently broken, or checking the wrong thing entirely — and you would not be able to tell the difference from the outside, because both produce the same clean logs right up until the day the rubber stamp waves through the thing that hurts you.

A brake you have never tested is not a brake. It's a brake-shaped object. So if you build a reviewer, you have to build the thing that proves it can still say no: wire a known-bad action through the live review path, on a schedule, and confirm the reviewer rejects it. Surface when it last refused something, the way you'd surface uptime. If that "last refusal" date goes stale, your brake may already be dead and you just can't see it yet. Prove the "no" is reachable, continuously, or you're trusting a checker you've never watched work.

Brake #3: An audit trail — know what it did

You cannot control what you cannot see. If an agent takes actions in the world and you have no durable record of what it did, when, and why, then a wrong action is invisible until its damage surfaces on its own schedule — usually the worst possible one.

Every consequential action should leave a trail: what the agent did, what triggered it, what it was acting on, and ideally a path to reverse it. This isn't bureaucracy. It's the difference between "we caught the bad action in the log an hour later and rolled it back" and "a customer told us three weeks later, and we have no idea how many others it happened to."

An audit trail is also what makes every other brake improvable — you can't tune a gate or a reviewer if you can't see what got through.

Brake #4: Blast-radius limits — caps and scopes

The final brake is the one that works even when every other brake fails: hard limits the agent physically cannot exceed, enforced by the system rather than by the agent's good behavior.

Rate caps (no more than N actions per minute). Spend caps (cannot exceed $X without escalation). Scoped permissions (can read these tables, can write only that one). Iteration limits (stop after K steps instead of looping forever). These don't depend on the agent reasoning correctly — they're walls, and walls hold whether or not the thing inside them is behaving.

This is where these controls show up in practice, and a few platforms now build them in as first-class features rather than something you bolt on — for instance, Xenition ships approval gates, an audit log, and a second agent that reviews the first's work before it reaches you (disclosure: it's a product I've been exploring). But whether you adopt a workspace that includes them or wire them yourself, the principle is the same and it's not optional: the agent's freedom should be bounded by design, not by hope. You do not want "it usually behaves" to be the only thing standing between your agent and your production database.

The uncomfortable truth: brakes are unsexy

Here's why, despite all of the above being fairly obvious, most agents ship without any of it.

Nobody demos the approval gate. Nobody opens a pitch with their audit log. Brains are exciting — they make the standup go "wow." Brakes are boring — they only ever matter on the day something goes wrong, and if they're doing their job, that day never visibly arrives, so it looks like you built them for nothing.

Choosing to build the unglamorous safety layer before you need it, when there's no incident yet to justify it, is exactly the kind of discipline that doesn't get applause. It's the senior move precisely because it's invisible when it works. The engineer who spent a day on blast-radius limits instead of adding a fifth capability looks slower — right up until the week their agent doesn't drain the account.

The takeaway

Capability has become table stakes. Every model is smart enough now; smartness is no longer where systems differ. Controllability is the differentiator — whether you can let the thing act without lying awake about what it might do.

So the question to ask about your agent isn't "how smart can I make it?" It's "what happens when it's confidently wrong — and can I stop it in time?" If the answer is "I'm trusting it not to be," you don't have an agent you can deploy. You have a liability with a good demo.

Build the brakes. Not because your agent is dumb, but because someday it will be confidently, fluently, irreversibly wrong — and on that day, the only thing that matters is whether you built a way to stop it before you needed one.

What's the one action you'd never let an agent take without a human in the loop? And the more honest question: have you actually wired that gate — and tested that it fires — or are you quietly trusting the agent to behave? I want to hear where people drew the line.
Disclaimer: This article was written with AI assistance and reviewed and edited by me before publishing.