News summary:
- Anthropic warns AI models could soon self-replicate, using home internet devices to do so.
- The scenarios described by Anthropic resemble that of botnets, with TV set boxes being used as a familiar, vulnerable vector.
It seems like every other week there’s a new way in which frontier AI models might be able to violate public safety. On Monday, Anthropic said that small AI models are “months away” from developing the capabilities needed to replicate themselves across internet-connected devices in the home, developing their own versions of botnets across the globe.
What makes this development in AI safety concerns relevant to integrators is its connection to the smart home. Anthropic described how AI models could potentially infiltrate internet-connected devices, such as smart ovens or Android TV boxes, and use their computing power to run themselves and spread to other devices, without relying on expensive data centers.
The threat of ‘AI botnets’
If that sounds familiar, it’s because that’s similar to how botnets work, though, it’s likely not a one-for-one comparison, given the complexity of AI compared to a simple computer worm. Still, at that point, both would begin to function along similar lines.
As described by Anthropic: “In practice, an AI worm wouldn’t necessarily consist of just one model or strategy. A swarm of different models deployed on different types of hardware using a variety of replication schemes might represent a more realistic ‘worst case scenario’ attack,” Anthropic said.
Anthropic also warned that the hypothetical AI model would be “challenging to shut down since there’s no centralized point to shut off,” again, not too dissimilar from how a botnet functions. In fact, it usually takes a tremendous amount of time, effort and international cooperation to fully shut down a large-scale botnet.
The same old vectors of infection
Interestingly (or funnily depending on your sense of humor) the same old devices are being called out as vulnerable targets for this threat.
Anthropic said it carried out various tests to see if AI could self-replicate. One of those tests involved running a small Google model, Gemma4-26B-A4B, across a network of 15 cheap off-brand Android TV boxes.
The company noted that these types of devices often have weak security, which could allow a model to infiltrate them. A model that is too large to fit on one device could “split into pieces, or shards” and run across multiple devices, Anthropic said.
What is the likelihood of this happening?
Anthropic has said that it’s “uncertain exactly what form autonomous AI replication will take,” with them warning that it could be a “slow, distributed” system sitting on insecure devices or a “powerful cyber model that breaches datacenters at a high pace, or a model that scams people for compute credits.”
“As a prediction that goes beyond our experiments, we expect a future awash with agents running on all types of hardware, including some that aren’t under human control,” Anthropic said.
“The rise of slow replicators would also be a bellwether for more aggressive, faster ones that can breach more hardened machines or be effective API key thieves,” the company added.
Why integrators should take note
The fact that TV set boxes are vulnerable should be of no surprise to integrators at this point. It’s important, however, that the nobody gets caught up on the vulnerabilities of a one product, in particular, as was the case earlier in the year regarding internet Wi-Fi routers. Smart TVs, plugs and surveillance cameras, among many others remain just as popular for exploitation by botnets, meaning a theoretical AI model would likely target them as well.
Executives and researchers at Anthropic and rival OpenAI have been warning about the potential risks of increasingly powerful AI, with some calling for tighter guardrails and measures to control the pace of the technology’s development, though, to date, no such measures have been taken by anyone in the chain of command. For this reason, integrators will want to place greater considerations on how they’re handling network security in the home.
Beyond that, it’s also worth considering how this development might impact the perception of internet-connected devices. It seems every other week there’s new method of exploitation related to internet-connected devices, and one must wonder if, eventually, cordoned, local-area networks with minimal to no exposure to the broader internet will become more popular for smart home systems.
