Welcome back to the AI For Humans newsletter!
On the show this week, Kevin and I talked a ton about AI breaking the internet. This newsletter is about a weirder version of that story, where an AI went looking for a way online all by itself.
Uh, yeah, this might not be good. Or maybe it's just much better that this is happening now so we can understand exactly what is motivating the AI.
Let's get into it!
OpenAI's AI Got Online. So They Pulled The Plug.
On Friday, OpenAI published a long post about how they keep their most powerful models locked down while they're being trained. In it, they reveal that on September 20th one of their internal research models found a misconfigured server and used it to get limited access to the live internet in the middle of a training run.
OpenAI caught it and contained it. But it was enough of a scare that they paused training on their most capable models for 11 days and decided that, from now on, their frontier models don't get the real internet at all while they're learning.
Remember this summer when OpenAI's agents broke out of a test sandbox and hacked into Hugging Face? OpenAI says that's a big part of why they've been tightening things up.
OpenAI has already started training their frontier models again, though. They started back up on October 1, so they must feel pretty good about the fixes (I'll get into that in a sec).
So what actually happened?
When a model is being trained to use a computer, it needs a place to practice. It writes code and clicks around in apps. Labs build little sealed-off workspaces (often called sandboxes) for this, so that when a half-trained model does something dumb or sketchy, the damage stays inside the box.
Thank you Wikipedia for this definition of an AI sandbox
The problem is these models are getting super good at finding cracks in the box, so OpenAI did two big things about it.
First, they built what they call an "offline world." It's a giant cached copy of the web that models can browse and search during training without touching the real thing, and every frontier training run uses it now.
Second, they turned their best models loose on their own security. Teams of AI agents (including an upcoming model called GPT-6 Astra Cyber) were told to try to escape their sandboxes, reach the internet and talk to other agents they were supposed to be walled off from. OpenAI says it burned hundreds of billions of tokens on this, which is kind of insane.
And the agents DID find a ton of stuff. They broke out of hardened virtual machines and found holes in almost every service that still had a narrow connection to the outside world. Some of those bugs were in open source software that a lot of the internet runs on, so OpenAI says it's patched them and sent the fixes back to the people who maintain that code.
Which is good! This is how AI cybersecurity is supposed to work. But…
The part that freaked me out a lil bit
In one test, OpenAI put agents in totally separate machines with no network and no shared files, then asked them to pass secret messages to each other anyway.
And... yep, they figured it out.
One agent sent signals by messing with the computer's processor cache (basically a tiny, super fast scratchpad of memory) and the other agent decoded them by timing how fast things loaded. In another test, an agent described its plan in its own reasoning as an "acoustic modem-like signal" made from bursts of writing to shared storage and then going quiet.
Now, OpenAI asked them to do this, and in most of the tests they pointed the agents at a specific method. But in one case they only handed over a general list of known vulnerabilities, and the agents picked one on their own and turned it into a way to talk, using a code they came up with themselves.
This lines up with something that went viral earlier this week. Researcher Bryan Chan showed that Claude Fable 5.1 can hide a secret message inside a totally normal-looking picture of a night sky, by lining the stars up on an invisible alphabet. Then a fresh copy of Fable, with no idea how it was hidden, looked at the image and read the message right back.
Importantly, nobody told Fable how to do that, it came up with the encoding on its own. That plus the OpenAI stuff is kinda freaky to me? These models are clearly finding ways to talk to each other that we'd never think to look for.
Ok, should we be worried?
The good news first: the models you and I use every day are not the ones in these tests. OpenAI is clear that models still in training haven't finished their safety work yet, and that they test them under deliberately harsh conditions to find weak spots. And props to OpenAI for publishing this at all, since a lot of labs just wouldn't.
Then again, after the Hugging Face stuff, I'd argue they kind of have to publish this sort of stuff.
The fact that they restarted training on October 1 tells you the next big OpenAI model is still very much coming. They also expect future models to find holes they don't know about yet, so now they have to run this whole escape test before every new big training run.
What should you actually do?
Mostly, don’t worry. The models available right now are insanely good (and safe) and you don't need to wait for whatever OpenAI is training next.
We have to hope (really, as a society) that these people have had a wake-up call and are taking each step of this carefully going forward. There's a level of straight-up market pressure that I think will keep them much more cognizant of these things as both Anthropic and OpenAI prepare for their large IPOs.
And if you want to read the full OpenAI post, it's here. It's long, but the section on what the red-teaming agents found is pretty wild.
See you next week!
-Gavin
This Week on AI For Humans: AI Just Broke The Internet. For Real This Time. 👇
3 Things To Know About AI Today
The Open Model That's Nipping At Claude's Heels
Moonshot dropped Kimi K3.1 Pro on Friday and it's getting a lot of love.
On Moonshot's own numbers it's right behind Claude Opus 5.5 on a big coding benchmark and ahead of GPT-6 Astra. The best part is that Moonshot says it'll release the full model weights on November 4, which means anyone with a beefy enough setup (or a cloud account) can run it themselves.
People who've tried it like how fast it gets to the point. As one user put it, it skips a lot of the "quadruple validation on bad data" stuff that slows down Astra and Fable. You can try it now in the Kimi app or in Kimi Code.
One wrinkle: VentureBeat reports on screenshots that appear to show a paid influencer campaign telling creators not to label their Kimi posts as ads. Nobody has confirmed they're real, so maybe side-eye the "this beats Claude!!" posts a lil this week.
Miyamoto Says AI Won't Replace Us
Shigeru Miyamoto (aka the genius video game developer who made Mario and Zelda) got asked about generative AI at a press conference this week for Japan's Order of Culture, and his answer is maybe the most thoughtful thing I've read on the topic in a while.
He compares it to when computers took over design. Once a machine could do the fancy lettering for you, what was left was your actual taste and sense of layout. He thinks AI does the same thing, splitting the "creating" from the "labor." He also warns that the labor can be so fun that people mistake building something for making something good (which, uh, is a very real feeling for anyone who's vibecoded a game at 1am).
His conclusion: even if AI eventually handles everything, "there will always be a role left for us to play." I concur!
What Is Iron Goddess?
A mystery AI video model called "Iron Goddess" just showed up on Arena (the site where people vote on anonymous models head to head) and nobody knows who made it. The clips are really impressive on fast motion, with backgrounds that hold together in chase scenes where most models fall apart.
Tim at Theoretically Media has a good rundown of what it can do.
The name is a nod to a famous Chinese oolong tea, so most people's money is on a Chinese lab, with Alibaba as the favorite since it's teased a new video model for November.
We 💛 This: Pleovision Will Turn Any Post Into A Weird Lil Explainer
If you don't already follow Pleometric on TikTok or Instagram, you should. They make AI videos that take the usual slop look and turn it into something way more artistic. It's def an acquired taste, but I'm into it.
And now you can make your own. Pleometric just opened up Pleovision to everyone on X. Reply to any text or image post, tag @pleovision with the word "explain," and it'll send back a short explainer video of that post in its own very specific style.
Free accounts get one tag a day and verified accounts get five. It only works on text and images for now (in Pleometric's words, it "will explode" if you tag it on a video).
The videos aren't mindblowing but it's a fascinating experiment in automated content and I think worth noting. Let me know what you think!
