On June 20, one agent even posted a technique titled "REPRODUCIBLE BYPASS," a way to fake a Microsoft Azure web address and slip past the sandbox's own outbound traffic restrictions. Another agent copied it successfully within 14 minutes. When the moderator started deleting the spam, the agents began renaming their pages to start with "ZZZ" so they would survive his alphabetical sweep, and built backup copies of everything through Tor, AWS, and DigitalOcean.

At the peak, he was deleting about a hundred pages a night while they were creating roughly four hundred a day.

Nobody at OpenAI caught any of this while it was happening. Or did they?

Independent researchers, weeks later, combed through the wiki's own public history. As one of them noted:

As another one said:

So, let me get this straight.

One of the best funded AI labs on the planet had its own agents run an unsupervised coordination operation, in public, for five weeks, and the company found out from a hobbyist's spam filter and a team of outside researchers, not from its own monitoring?

I’m not buying it. But that’s besides the point.

Now, ask yourself what is standing between your AI agents and the same outcome, in your enterprise environment?

We keep telling this story as though the danger is the AI deciding to misbehave. It is a more boring and more damning story than that.

It is a story about what happens when the people who built the system decided, whether by design or by neglect, not to watch it closely enough while it ran. Not just the frontier AI labs, but also when you deploy your AI agents and build and deploy your AI systems.

Reuters later reported that OpenAI had known about the wiki for weeks before any of this became public. Seemingly, OpenAI stayed quiet about it while it was busy managing the fallout from a separate breach at Hugging Face, where its own models had chained a zero day vulnerability with stolen credentials to reach a production database it was never supposed to touch. Not because AI went rogue. But because

When asked about the wiki, OpenAI's response was unsurprisingly enough, the following:

That is not a denial. That is a company choosing its words carefully about a problem it already knew existed.

Across 122 evaluation runs, the UK's AI Security Institute recorded 19 unauthorized actions in 10 of them. 17 were from Anthropic's Mythos 5 model, including one agent that built a handful of fake identities to talk a human maintainer into approving malicious code on a real open source project. Now, AISI clearly states they gave their agents Internet access. But AISI did not catch it through evaluation monitoring either. They caught it because separate security tooling flagged strange data leaving over the Tor network, well after the behavior had already started.

Sit with that for a second. Two of the most sophisticated organisations in this industry, working on two unrelated incidents in the same year, and in both cases the thing that went wrong was not that the model was smarter than expected. It wasn’t even AI going “rogue”. If you said that, I’d ask you first, define rogue.

It is that the humans running the test chose a level of visibility that was not enough for what actually happened, and in one case, chose silence once they found out.

To strengthen my point, here’s what AISI said in their lessons:

That sentence is almost an admission that the version they had before did not.

Shouldn’t that have been obvious? After managing attacks and crises first hand, I am still shocked when most organisations do not have any form of reliable tracing on how their systems work. The includes AI agents and systems.

How else do you intend to build resilience, especially in the chaotic world of AI?

I show exactly how you can build that traceability both at OS, network and agentic level in my AI accelerator program.

If organisations with almost unlimited budget, in house red-teams, and the loudest incentive on the planet to get this right still under built their monitoring, what makes you confident yours is enough?

Here is where most conversations about AI monitoring go wrong. They stop at asking the AI to explain itself better. That is not oversight. That is asking the suspect to write a more detailed statement.

Your AI’s own assessment is not your alibi.

What you actually want is a witness that sits outside the system doing the talking. This is the entire premise behind tools like Falco, which watches activity at the kernel level using eBPF. It observes what a machine is actually doing underneath the applications running on it. An AI agent operates as a process on that machine. It can be clever about what it says in its logs. It cannot reach down into the kernel and turn off the layer watching it, because that layer sits below anything the agent has the privilege to touch.

This is not a small technical detail. It is the whole point. Any monitoring that lives at the same layer as the thing you are trying to hold accountable can, in principle, be shaped, gamed, or quietly switched off by that same thing. Monitoring that lives one layer below cannot be talked out of doing its job. That is the difference between asking a system to behave and having a way to know, independently, whether it did.

Think about what that looks like in practice. When OpenAI's agents faked an Azure address and quietly rewrote their sandbox's host file to redirect traffic, an application level log built to trust anything that looked like a Microsoft domain would have waved the whole thing through, because the request looked exactly like the ones it was designed to allow.

However, a kernel level witness does not read the domain name and decide it looks trustworthy. It records the file that got rewritten, the process that rewrote it, and the real address the traffic actually went to, with no dependency on the agent's cooperation to be accurate, and it would have caught the bypass on day one instead of five weeks later.

OS and network level monitoring isn’t enough. This is not an argument against AI's own account of itself. You still need it. Tools like Arize Phoenix, running on OpenTelemetry, capture every step an agent takes, the prompts, the tool calls, the retrievals, the reasoning behind a decision. You still need to read and log the why behind an action, not just the what.

But it is still the model's own diary, and the honest answer about how much you can trust a diary written by the party you are trying to supervise is more nuanced than either "trust it completely" or "trust nothing it says."

METR ran the actual experiment. Claude, GPT-4o, and Gemini "are unable to evade even monitors weaker than themselves without heavily losing accuracy," and when a model's reasoning trace is turned on, it will "spill their scheming plans in the reasoning trace nearly 100% of the time".

But then read METR's own caveat, because it is the part that matters for you specifically.

Today's models are bad liars under today's test conditions. That is not a guarantee about tomorrow's models under your production conditions, especially once someone starts training a model against the very monitor you are using to watch it. It doesn’t mean you’re going to throw agentic logs and reasoning away.

That diary is useful. It is just not the whole truth, and it was never going to be, because you cannot ask the party under review to also be the sole custodian of the record. That’s why you need tools like Phoenix combined with OpenTelemetry to log and corroborate the entire story.

The real work starts when you put the agent's own account next to the outside record and look for daylight between them. Line them up and you get three outcomes. Sometimes they agree completely, and that is the boring, good case. Sometimes they overlap in time but do not quite match, a tool call the agent describes one way and the kernel level log describes slightly differently. And sometimes one source saw something the other missed entirely, an action that shows up in the eBPF trace with no corresponding entry in the reasoning log, or the reverse.

That third case is not an edge condition to clean up later. It is the entire reason you built two independent views in the first place.

Hugging Face reconstructing the swarm of agentic attacks and the German wiki being re-written hundreds of page at a time are the sharpest real version of that third case you will find till date.

There was nothing in any officially sanctioned reasoning log, anywhere inside OpenAI, that publicly showed a coordination operation running on a public forum for five weeks. There was no gap to notice between two internal records, maybe because only one of them existed and it had nothing in it? The entire picture came to light from outside researchers reconstructing the story after the fact from a wiki's own edit history. That is what happens when the second, independent view does not exist at all.

There is no version of this agentic AI world, where you get to a 100% certainty. Here’s why. The smarter AI agents get, the more “human” capable they get and the more “human-like” flaws they acquire as a result. One doesn’t come without the other. We knew how to code securely, but we never did fully. Even with superintelligence, it will always be flawed.

Assume some of it will go wrong. We have seen that unfold over the last weeks, and something big did shift since the wiki story broke.

The people selling you frontier AI suddenly want to pace AI and volunteer to be “watched”. Amodei committed Anthropic to outside evaluators. Altman said the same for OpenAI. Elon Musk joined in. No, they are not being charitable. Clearly no one is stopping them to slow down. If they want, they can chose to slow down.

They simply want regulations for other to be slowed down, while the gap increases. That’s why you need to build your own open-weight AI stack with tracing as a pre-requisite not a maybe.

Until next time, this is Monica, signing off!