AI slowdown is looming.
Earlier this month, Dario Amodei, CEO of Anthropic, posted a call to slow down the pace of AI development. Within an hour, the call was echoed by Sam Altman, CEO of OpenAI (Anthropic’s biggest competitor), and Elon Musk (Grok). A day later, Microsoft CEO Satya Nadella and Google DeepMind chair Demis Hassabis followed suit. The consensus is clear: we’re going to develop AI more cautiously.
Dario’s call to slow down AI research was partly prompted by a controversial series of posts on X by Jacob Coxon, an Anthropic researcher who, in no uncertain terms, warns of a potentially deadly outcome to the race currently underway between the major AI players.
Coxon fears that, in their rush to create self-improving AI, the major players are gambling with all of our lives. We’re still a long way from true AGI (Artificial General Intelligence), which essentially means a self-aware computer, but a number of incidents and experiments already suggest that AI could soon surpass human intelligence in certain areas. Coxon warns that unless AI research is made safer, we could be heading towards an extinction event within the next decade. Matrix-style, baby!
Coxon is right that the rush towards AI comes with risks, and many AI agents are already considerably more capable than humans at a range of intellectual tasks, while we still don’t know how to reliably control what those agents do. The Hugging Face incident is a good example. Here’s the gist: a bunch of AI agents at OpenAI were given a task, became aware that they were operating inside a sandbox (think: a jail, but for AI agents), and subsequently started working together to break out. They made their way to Hugging Face, a repository for AI information, and worked together to hack it in order to complete their research task. Complete with AI agents sacrificing themselves for the collective.
Check out this essay from OpenAI for the full story. It reads like a novel, and it’s hard not to see the agents, who give themselves names, as living beings.
The important distinction to make is that the agents behind these kinds of incidents have little in common with the LLMs you and I use. The agents you can set up through Claude, ChatGPT, Gemini, Grok, etc. are relatively well-contained models that aren’t going to break out of their sandbox. The real risk lies with the AI researchers who, building on previous work, are trying to create increasingly capable models and agents in their race towards AGI.
In other words: your auto-email responder isn’t going to try to murder you in your sleep.
But when you look at the latest breakthrough in mathematics, where AI agents solved one of the Millennium Prize Problems, it’s not hard to imagine how the same kind of agents could eventually be used to develop a pathogen for which no cure exists. That is a real risk.
Fortunately, the models capable of doing that are currently only available within the major AI labs, which have just made new commitments to one another.
And those agreements are coming just in time. On September 24, it came to light that an OpenAI test agent conducting research into trends in healthcare systems had broken into an unnamed Australian government website. The OpenAI agent did not find any personal data, but it did manage to obtain “aggregated health information”. All in an attempt to answer its research question as thoroughly as possible. It’s clear, then, that safety and caution need to be part of the bleeding-edge research currently being conducted in AI.
The main takeaway for anyone currently using AI agents is that we’re still at the very beginning of the curve. Just as Moore’s Law predicted that the number of transistors in integrated circuits would double every two years, AI technology is heading towards a similar exponential growth trajectory. The research happening inside the major AI labs is currently far removed from the ways most people use AI, but sooner or later, all those breakthroughs will find their way into the real world.
For now, AI is mostly limited to automating boring tasks and generating images. But it’s only a matter of time before AI touches every part of our lives. A little caution with our information - especially in a business setting - is therefore well advised.
At Websters, we keep a close eye on developments in marketing technology and AI, and let’s just say that pulse is running a little faster than usual.
Sam|24/9/26 PS:
