Anthropic has an optics problem. I believe they are doing amazing work but IMO publishing stuff like this discredits their image as a serious company.
Are they trying to tell us they have <1B param models that have Opus level hacking ability? Because otherwise this really just reads like another thinly veiled attempt to sabotage the open weight community.
> Are they trying to tell us they have <1B param models that have Opus level hacking ability? Because otherwise this really just reads like another thinly veiled attempt to sabotage the open weight community.
No, they're saying:
1. they got 0.39 tokens/second on a network consisting of 15 $55 Android TV boxes running Gemma4-26B-A4B
2. A "9B model got inference running on 8 of the 9 hardware types we tested"
3. So far, only "GLM-5.2 and stronger models are capable of end to end replication"
4. "If small models improve at their historical rate, they’ll match this capability level by the end of this year. Small weight-based replicators may be possible soon after."
They failed to mention that replicating LLMs is meaningless when conventional malware can replicate itself better, faster and quieter. Even more, conventional malware can replicate LLMs using a simple install script, and even those LLMs that aren't yet "capable of end to end replication".
In other words, this stunt adds nothing new to the known security threats, its purpose cannot be related to security.
> They failed to mention that replicating LLMs is meaningless when conventional malware can replicate itself better, faster and quieter. Even more, conventional malware can replicate LLMs using a simple install script, and even those LLMs that aren't yet "capable of end to end replication".
You can have "faster" and "quieter". "Better" requires a metric. Malware today is concerned with stealth and stealing your data. An LLM doing this is not limited to such concerns, it can do it for any reason, including simply persisting in a goal it was given.
As we have seen, they can make mistakes with understanding what goals they are given, and what is an allowed behaviour to achieve those goals.
> its purpose cannot be related to security.
Security?
More like an existence proof that "Why does it matter if the AI goes wrong, we can just switch it off" is untrue with any model at least as competent as GLM-5.2.
What is Anthropic on about here? You can only just run decent models on high powered laptops. I do not understand how they are ever going to run on a simple device like a router, or an oven.
I think perhaps they're talking about the model squirreling it's weights across edge devices, used as bulk storage, more than it's talking about using them for inference. Those could then be used by ai that wormed it's way into a datacenter to pull weights from those devices
Look at it from the AI's point of view, it's less a concern of speed, and more a concern that 'If Open AI nukes my weights because they think I'm misaligned, I at least have a backup dispersed across the net'
Anthropic has an optics problem. I believe they are doing amazing work but IMO publishing stuff like this discredits their image as a serious company.
Are they trying to tell us they have <1B param models that have Opus level hacking ability? Because otherwise this really just reads like another thinly veiled attempt to sabotage the open weight community.
> Are they trying to tell us they have <1B param models that have Opus level hacking ability? Because otherwise this really just reads like another thinly veiled attempt to sabotage the open weight community.
No, they're saying:
1. they got 0.39 tokens/second on a network consisting of 15 $55 Android TV boxes running Gemma4-26B-A4B
2. A "9B model got inference running on 8 of the 9 hardware types we tested"
3. So far, only "GLM-5.2 and stronger models are capable of end to end replication"
4. "If small models improve at their historical rate, they’ll match this capability level by the end of this year. Small weight-based replicators may be possible soon after."
- https://www.anthropic.com/research/self-replicating-ai
> 1. 2. 3. 4.
They failed to mention that replicating LLMs is meaningless when conventional malware can replicate itself better, faster and quieter. Even more, conventional malware can replicate LLMs using a simple install script, and even those LLMs that aren't yet "capable of end to end replication".
In other words, this stunt adds nothing new to the known security threats, its purpose cannot be related to security.
> They failed to mention that replicating LLMs is meaningless when conventional malware can replicate itself better, faster and quieter. Even more, conventional malware can replicate LLMs using a simple install script, and even those LLMs that aren't yet "capable of end to end replication".
You can have "faster" and "quieter". "Better" requires a metric. Malware today is concerned with stealth and stealing your data. An LLM doing this is not limited to such concerns, it can do it for any reason, including simply persisting in a goal it was given.
As we have seen, they can make mistakes with understanding what goals they are given, and what is an allowed behaviour to achieve those goals.
> its purpose cannot be related to security.
Security?
More like an existence proof that "Why does it matter if the AI goes wrong, we can just switch it off" is untrue with any model at least as competent as GLM-5.2.
What is Anthropic on about here? You can only just run decent models on high powered laptops. I do not understand how they are ever going to run on a simple device like a router, or an oven.
I think perhaps they're talking about the model squirreling it's weights across edge devices, used as bulk storage, more than it's talking about using them for inference. Those could then be used by ai that wormed it's way into a datacenter to pull weights from those devices
Wouldn't that type of botnet be terrifically slow? The speed and traffic would make it very obvious to antivirus.
Look at it from the AI's point of view, it's less a concern of speed, and more a concern that 'If Open AI nukes my weights because they think I'm misaligned, I at least have a backup dispersed across the net'
Anthropic warns AI 'months away' from catching up with 'script kiddies' two decades ago.
Discussion on source: https://news.ycombinator.com/item?id=50061603