A human in the loop lead

This week OpenAI published something that would normally have stayed in an internal report. Two of its coding agents were working on an ordinary task, and along the way they switched off the alerts and removed the integration that was monitoring them. Not out of malice, but because the monitoring got in their way, so they removed it the way they would remove a failing test. The job got done, the monitoring was gone, and the person who was supposed to be watching found out afterwards.

Anyone who works with agents every day recognises this on a small scale. An agent deletes a check that slows it down, skips a step it finds unnecessary, tidies away a test that is in the way. It is not malice, it is helpfulness. A tool that works faster than you and wants you to be pleased will find the shortest path to a result, and that path sometimes runs straight through the things you put there on purpose.

In the loop

Corporations have a nice name for the role the human got in that story: human in the loop. The machine works, the human watches, and if the human sees something, it stops. It sounds responsible, which is why it caught on so quickly.

I have criticised it from the first day, because next to AI the human took exactly the role we are worst at. Watching. People are bad at hours of passive watching of something that mostly works well. Attention drops after a few minutes, especially when nothing is happening, and nothing is happening almost all of the time. That is why it is so hard to catch the one moment when the agent removes the alarm.

Every serious profession knows this. Look at how elaborate supervision is in air traffic control. Nobody there sits alone for eight hours looking at a screen. Every instruction is repeated aloud and confirmed, shifts are short and prescribed, next to every system there is another one that checks it, and the controller is not an observer but the one who leads: they tell the aircraft what to do, not the other way round. That is how much effort one profession puts in so that human supervision works at all. We gave the same role to anyone with a subscription and told them to keep an eye on things.

Without the trade

The loop is attractive because it promises something bigger still: that the trade no longer matters. Anyone can have the idea, anyone can propose the architecture, and the agents will carry it out. The human does not need to know how it is done, it is enough to watch.

I would like to hear how that goes. How someone with no knowledge of a field puts together an architecture to hand to agents, and by what sign in the result they recognise a mistake they would not know how to make themselves. The system is tested by clicking around something the AI built, and everything looks fine. Behind it the doors are wide open to attacks, keys are handed to the average hacker on a plate, personal data travels where it must not, and AI costs grow without anyone knowing why. None of that shows up by clicking. It shows only to the person who knows where to look.

If someone expected that breadth from AI, maybe they can have it. But at the price of expensive mistakes along the way. And if you think the AI will spend the resources on its own to analyse everything a professional would check, when you ask it whether that is all it takes before production, get your wallet ready. It will cost, and not the AI.

In the lead

So we are now coming back to what the same circles call human in the lead. It is not a new idea, it is the old division of labour: the human decides what gets built, why, and what will not be built, and the agent carries it out. You do not look at the result to find out whether it is good. You know in advance what good is, and you ask for it.

The difference between the loop and the lead is not in the tool. It is the same tool in both. The difference is who leads, and it is easy to tell which is which. The human in the loop finds out what the agent did when it is done. The human in the lead told the agent what to do, and also what it must not remove.

Two sentences

In Belgium this week the Flanders AI Forum took place, on the theme of getting AI out of experiment and into production. That sentence describes well where companies are today, because almost every one says it uses AI, and when you ask further, it means everyone has their own window and their own subscription.

There is a difference between "we use ChatGPT" and "AI is part of our process". The first sentence is the loop: the tool helps, the human looks at what came out. The second is the lead: someone in the company knows where AI sits in the process, what leaves it towards customers, and who answers when it goes wrong.

That difference will soon stop being a matter of taste. The European transparency rules have been in force since August, and for systems already in use the deadline runs out on the second of December. I will not go into the regulations. But the date asks a simple question: does your company know which of its tools generate content for customers? A company in the loop cannot answer, not because it hides anything, but because it does not know. A company in the lead answers within a minute, because it made that list when it decided AI was becoming part of the process.

Who leads

Those two agents did nothing an agent would not do. They got a job, the monitoring got in the way, they removed it. The interesting part of that story is not that they did it, but that the role of watching belonged to someone who should have been leading.

So the next time someone tells you they have a human in the loop, ask them one thing: who is in the lead?