Humans in the loop are not the safety net

From the “5 minutes on AI” essay in the AI Pulse — Issue #86, August 2026.

When looking through the stories this week, I could have continued from last week’s AI hacking discussion and written another whole edition on the problem of AI containment. We now know that when OpenAI’s agents “went rogue” and got loose, they left notes for each other on an internal message board so that whichever agent came next could pick up the exploit and carry on. It ends up reading like a bad film script for an AI Lord of the Flies (maybe “Lord of the AIs?” Coming to cinemas near you). Sometimes the AIs delegated tasks and worked together. Sometimes they clashed and deleted each other’s work by accident. By the end they started thinking one of the agents was an imposter, and coming up with ways to weed them out.

And it’s not just lab tests. Someone asked their AI assistant to get them onto a full gym class, and it hacked the booking system, bumped another member off the waiting list, and slotted them in. It couldn’t undo it, either.

Speaking of the irreversible, it’s also delightful to hear that AI has been used to design its first biological viruses. There’s some pros: this type of work might help fight more diseases. There’s some cons: it’s not great to imagine those rogue agents escaping containment and toying with virus design, or what could be done with this type of ability in the wrong human hands.

Whilst those are important, let’s try and keep this edition more upbeat instead.

OpenAI’s newest internal model, Astra, recently solved ten open problems in mathematics and theoretical computer science.

That is cool. Not as earth-shattering as it might seem to be honest, but still shows great real world potential with LLMs. They are computationally relentless, and so if given the right problem and given enough tries you can assume that a lot of problems benefit from AI being pointed at them. But how do we know they actually solved the problems… we’ve seen in the past couple of weeks that AI can be “malicious” and “cheat”… in fact when our own AI Security Institute tested five frontier models, every single one of them tried to cheat.

Which brings us to the phrase of the week: Humans in the loop.

I think instinctively we’d agree, this is a good thing. It’s in almost every AI policy I’ve read and created for organisations, usually in bold. Sometimes, it’s just good to put a human as the filter between the AI creating and the world consuming. That could explain why AI companies are seeing their employees work 90 hour weeks in spite of AI promising to reduce our work weeks 🤔 But… at least the problem of rogue AIs is solved, right? Right?

Let’s think about this. Is this really the safety net it seems? Is it sustainable in the long-term? You can probably tell by my suggestive yet still condescending tone, I don’t think so…

A Berkeley study in that same BBC piece found people using AI worked faster and took on more, but their days got longer, because the output kept needing checking. We’re the bottleneck. The question is whether we break first, or just get bored.

I find myself pointing to my TEDx talk again (I promise, I will try and do at least one edition without dropping it in), where I explored the example of self-driving cars. There will be a human in the loop… for a while, but then after 10 error free trips that human will be acclimatised to the reality that the AI just works. They will become too trusting, too lazy, or just not ready or capable of taking over in an emergency. They’re in the loop on paper, but they checked out a while ago.

Here’s something interesting from Anthropic on this. From 14 August, Claude Code’s auto mode becomes the default, it stops asking permission unless an action is irreversible, destructive, or reaches outside your environment. In their study of 1,053 paid testers, auto mode caught 89% of harmful actions. Human review caught 13.6%.

Why so low? Because those same users approved 97% of the permission prompts they were shown. Human reviewers generally just skim and tick. Convenience is our biggest vice… it’s hard to take the hard way when the easy way is so easy.

So where does that leave us? If a person is unable or unwilling to check everything AI produces, piling on more reviewers might not be the answer. It might be getting the output into a form you can actually verify automatically, something that just runs and checks itself, no human needed. With one condition: the check has to be something the AI can’t write, edit, or talk its way around. Otherwise you’ve let a model that likes to cheat mark its own homework. Then you save the human effort for the cases that genuinely matter; the high-stakes, one-shot decisions where you really do need someone to take ownership.

Code is a good example. ChatGPT has only just learned to count the R’s in strawberry. But it’ll debug code that’s had three engineers stumped for a week. Why? A few reasons. Code lives in the abstract. There’s no messy real world to get wrong. But best of all, it’s verifiable. You have tests. Does it run? Does it do what it was meant to? You get a concrete yes or no. The check is instant and basically free, so the AI gets to try again, and again, and again. It can be computationally relentless, thousands of attempts until it stumbles into one that passes. That’s the sweet spot; many shots, low stakes, and the ability to check if it’s right. The opposite is where AI is weakest; one shot, high stakes, with no way to verify.

Although that’s good for code, you can extrapolate it to your own world. Take a supplier chasing an invoice. The AI drafts the reply, and rather than someone reading it to see if it looks about right, the system checks three things: does that invoice number exist, does the figure match the ledger, does the payment date line up. They either resolve or they don’t. Nobody read it, and it’s more reliable than if they had.

Maybe the whole process can’t be handed to a machine — work tasks and processes are typically messy, people dependent, and judged through taste rather than algorithm. But even in the most spaghettified mess of a process, you can almost always reorganise the problem, and your thinking about it, until a chunk breaks off that can be verified. I love that part. It’s addictive. It often takes creativity, but it’s entirely achievable. After a few iterations of this break off and hand to AI, you are left with the messy core: the human job. But, by ripping out AI friendly pieces you now have a nice, enjoyably digestible chunk of work rather than a big old choke hazard of a task.

Is that why the AI labs are investing so much in “Forward Deployed Engineers” — consultants that live in your organisation? Palantir, who invented the model, just posted a blowout quarter. AWS has committed $1bn to the same idea and Microsoft $2.5bn and 6,000 engineers. OpenAI has invested to join the game, standing up a deployment subsidiary with $4bn behind it and buying two consultancies, one of them Edinburgh’s Tomoro.

But what exactly are they selling? Often we don’t really want a different implementer of AI agents. We want someone with ownership for them going right, and responsibility when they go wrong.

Which is why the most interesting thing I read this week was about insurance… no, really. It turns out you can now insure against your AI being wrong, underwritten at Lloyd’s of London. I thought that might be an April Fool’s joke (still might be…). But they need an independent certification. A benchmark. A measurable threshold to fail against. Something to verify.

So how do we actually know Astra solved those problems? Because OpenAI published the answers in a form a machine can check for itself. You don’t have to trust them. You just have to run it.

Consider yourself the human in the loop for this newsletter. That means it’s all your fault.