This is incoherent. The argument seems to be that releasing the weights would slow the frontier labs from raising money, which would give them less money, which would slow progress on AI. It assumed all labs in the entire world agree to self-destruct this way and no new labs every start up to continue the work.
No, really:
> As you know, funding for frontier model development depends on valuations that assume the weights remain proprietary. By changing that assumption, we can reduce the money available for future training runs and slow progress at every lab at once, without a regulator deciding anything.
Which doesn’t even follow. It depends on every lab in the world agreeing to self-destruct their valuations and stop competing. That’s a much more impossible ask than anything Dario is asking for.
It also ignores the possibility of new labs starting up, using the open weights, and continuing exactly where the old labs left off.
I feel like I just read the ramblings of someone who got so excited about the headline that they didn’t take the time to think if their argument was sound.
It is a bit incoherent, but it's also... provably compliant in a scorched-earth kinda way?
What are the alternatives?
A regulator saying no? Easy, your headquarters has just moved the Cayman Islands, Ireland, or Switzerland... the US office is just a subsidiary leasing the brand IP and doing marketing.
Or, you just ignore the regulator behind closed doors because you're part of some black budget. You wouldn't be able to talk about that closet back there even if it did exist.
Or, you don't do any of these complicated loopholes and you simply move all the training and inference to a different jurisdiction.
If the only winning move is not to play, how do you ensure everyone stops playing?
Simple: you must destroy the game.
It does have a kind of mad logic to it.
(That is a bit dramatic, but the idea is if we accept that AI research will happen wherever there is capital available, instead of restricting the research you restrict the return that capital can earn. And the only provable/market way of doing that is to require the results to be open. In other words, if the benefits of getting all that cash to buy GPU's gets handed out to everyone for free, then there will be far less cash floating around to buy GPUs, slowing down the process)
Or you are China's Communist Party. You download the weights and use them as the starting point for your closed-everything AGI with Chinese characteristics while the US ensures that everyone else stops playing.
What stops the US government from doing the same thing?
Every chinese frontier lab releases their model weights and often more under FOSS terms (or more restrictive but still generally open terms).
If models for public inference use are required to be open weight in the US give or take some amount of limited fine tuning, then the US and China would be on the exact same playing field. Frontier labs would be pushing functionality for functionality's sake and would either be supported by govt funding or would be supported by domestic inference providers.
And then at the end of the day everyone is just either doing open research, is an inference, fine tuning, and training provider, or is the govt.
It's easy and straight forward and pushes everybody in the market towards common standards and interoperability.
i mean there is the whole capitalism problem to contend with in terms of getting support from congress ... but gotta admit, it has that "it's so crazy it just might work" feel to me too
> This is incoherent. The argument seems to be that releasing the weights would slow the frontier labs from raising money
That argument is straight from Dean Ball on Twitter. Open-weight models are "decelerationist", which is bad if you're completely AGI-pilled but is actually great if you're either worried about AI Safety falling to an arms-race at the frontier (this is Dario's argument, and note that open weight models are far behind the proprietary frontier, especially since they target widespread lean deployment so they must skimp on total parameters and compute requirements) or Yann-LeCun-pilled (i.e. skeptical about AGI/ASI but optimistic about the practical usefulness of current AIs) like most people in China (including, reportedly, Chinese leadership).
That's sub-frontier activity - it has no bearing on the very real safety that would be gained by slowing down ("pacing") the proprietary frontier. The current cybersecurity scares are all about proprietary and internal models; open weight models are great for defense but get very modest scores on offensive evaluations (which makes sense if they weren't specifically post-trained for offense tasks).
> Those unregulated sub-frontier labs become the new frontier labs because they continue advancing.
If you're referring to open weight labs, then that continuing advancement has been successfully "paced" via the decelerationist effect of open weights. Isn't that effectively what Dario is arguing for? He's explicitly saying that "pacing" the frontier won't feel to users like no advancement is happening.
It's a law, so nobody has to agree to anything. If want to sell access to a model, you have to open the weights.
It's Dario's plan that depends on every lab in the world agreeing.
Step #2 of his essay is "democratic coordination" among frontier labs and step three is coordination with China. I'm suggesting a single law Congress could pass that every US model company would have to follow on the same day, whether they like it or not.
Frontier progress is compute bound. Compute is bought with investor money. Investors put up tens of billions because they expect to sell the proprietary model that is created.
You're welcome to think the effect would be smaller than I do. But "it requires everyone to agree" is backwards.
> It's a law, so nobody has to agree to anything. If want to sell access to a model, you have to open the weights.
Laws don’t extend to the entire world. There are AI labs outside of the US that are not that far behind. Forcing US companies to open their weights would hand those labs another advantage, moving the frontier AI to a location outside of any US regulation.
Your proposal doesn’t make any sense.
> The plan that depends on every lab in the world agreeing is Dario's.
I don’t understand how you think creating a law in the US to destroy our AI companies and give an advantage to competitors we can’t regulate accomplishes anything in line with the AI safety debate.
Laws don't extend to the whole world, but markets do. The law should apply to anyone selling model access to Americans, the same way GDPR applies to US companies selling to Europeans.
Frontier development won't simply move offshore because the GPUs, most of the people, and almost all of the funding is here.
Our European and Asian allies would probably go along with this idea, because it costs us far more than it costs them.
Nobody gets taken out. Anthropic and OpenAI can keep selling inference, products, and agents. Every company will have more access than it has today, since it can run the weights itself or through third-parties. Many of them would keep paying OpenAI and Anthropic, at reduced pricing/margins.
It would only slow down the development of new models. But we've already got models with massive "capability overhang" that we could keep pushing for years.
The GPUs and the people (I imagine you mean the extremely well paid researchers) are there because of the funding.
The funding is in the US now, but as TFA explicitly says, the law would reduce spending on AI training. All the money will still be available, which means the funding will either a) move to a different sector or (more likely) b) move to a different country. Taking the people and the GPUs with it. So yes, most training will move offshore.
Secondarily, OSS business models are based on the fact that the developer is the most qualified in offering support.
With models, anyone with the weights can serve the model with the same performance (meaning level of "intelligence") as the original developer. Any business model depends entirely on spending huge amounts of money in training to then hopefully gain from inference. If anyone can compete on inference the same day you deploy the model, it's unlikely you can break even while your competitors that saved on training get rich on inference. i.e. it's more likely to entirely stop training in the US rather than slowing it down...
which other country? who's got this "dump all your money on speculation" like the US does? whis got the slack electric capacity? the ability to buy US GPUs and build datacenters?
* Destroy the free internet in the US, ban VPNs in the US, assume enforcement of this is effective, ban frontier AI development in the US (probably needs at least one constitutional amendment)
* Assume no other countries will continue to develop AI and approach or push the frontier
* Then, the frontier ceases to move until the US companies agree to let it move again
...
I don't think anyone proposing a "pause" has studied politics or game theory even at a high school level.
We can start with a US law. It would cover AI model companies with almost all the money and the GPUs, which is most of the frontier. We can also probably convince our allies, which combined represents most of the world market.
Also, if the concern here is safety, open models completely derail that.
Even if your someone who thinks open models are good for safety overall, a private model exposed only by an API point at least gives somebody control. How we should use that control is debatable, but at least it's there. Once you release that control by making a model open, you can't take it back.
If we think AI safety doesn't require closed weights, we should collectively decide on that and then move forward from there. But it shouldn't be something decided by default, by a few people from a few companies. We all have to live with the consequences of a choice like that, and since it is irreversible, we shouldn't decide it without consensus first.
the ntsb model has been quite effective at creating safety through analysis of failures in the extremely transparent and public eye.
i dont see a reason to think private regulation will do anything but give people explosive diarrhea or drop the doors off of planes.
weve already seen how bad it is when companies regulate themselves without public oversight. catastrophic implosions of deep sea submarines and agents hacking systems to hide cheating come to mind
the argument should be flipped on its head that by some magic keeping these tests and training protocols private is safer than making them extremely public. The experiment in private regulation has been a drastic failure across the board, and we shouldnt have tried it without consensus
If they believe open weight models are inherently unsafe but are forced to release weights, then if they really believe their own safety arguments, they would need to stop development.
"If you mean it, let a committee investigate" seems more appropriate to me.
I mean I get that they don’t want to share details on how AI might destroy humanity but this secrecy also leads to speculation that this is just a PR campaign pushing for regulation, which might come handy.
You don’t need to tell the world what your super AI has given you to cause this panic but definitely you could share it with a group of experts who then inform the public in a report, couldn’t you?
>the world agreeing to self-destruct their valuations
Did OpenAI releasing GPT-2 self destruct their billion dollar valuation or is it higher than ever today at $900B. The idea that releasing weights kills the future value that can come is not supported.
It follows if the goal is to deflate the bubble without popping it, and if the reality under the hood supports the absurd investment. The companies themselves wouldn't do it, but they could be required to if they want to operate in the US.
> It follows if the goal is to deflate the bubble without popping it,
The conversation was about AI safety, monitoring, and regulations, not bubbles.
Forcing the weights to be opened to the world accomplishes the opposite of all of that. The argument is incoherent. Or this author doesn’t understand the topic at all?
> The companies themselves wouldn't do it, but they could be required to if they want to operate in the US.
What makes you think the companies outside of the US wouldn’t just thank the US companies for their open weights and then continue on without any of the regulations but all of the advantages?
> Forcing the weights to be opened to the world accomplishes the opposite of all of that. The argument is incoherent. Or this author doesn’t understand the topic at all?
Or maybe you understand less than you think?
Frontier progress is limited by compute, and compute is bought with investor money. Investors fund labs at hundreds of billions because they expect exclusive ownership of the model weights.
If the weights are public the day a model ships, the valuations drop, the funding rounds shrink, and the next training run is smaller or later. Make sense?
No, it rests on the idea that almost all of the money, most of the people, and the GPUs are here.
Any country would be able to take the open weights and keep going in private. But countries like China can already do that today, since the weights are almost certainly already in foreign hands due to espionage.
What they can't do is fund the next frontier run at today's pace without selling into the US market, and selling into the US market can require opening the weights.
All of this rests on the assumption that this game of musical chairs can continue indefinitely. If you believe that then this is flushing money. If you don’t it’s acknowledging failure to monetize the tech to the tune of trillions, and fail a bit more gracefully and usefully for everyone.
The author put zero rational thought into this. If Anthropic is legally required to release the weights of any publicly accessible model, their logical response will just be to stop offering public models. Anthropic gets most of its revenue from enterprise agreements anyway. Providing public models or providing public access to their models is their secondary business. They can cut it off any time. Public access to their models (at subsidized rates) has caused so much indirect harm and financial cost to them that they're very well justified to stop providing access.
Anthropic has already shown they're very comfortable with holding models from the public. Mythos has been around for what, six months, and there's never been a public release. Only the select few blessed by Anthropic have access.
Forcing transparency on public models will just accelerate privatizing access to these models. You don't want to live in that world.
While mostly true, don’t discount people looking for work that offers them familiar tools. The cheap public pans are a way to train a labor force + create upward demand from people trained on those tools to influence big corporate purchase agreements. Remove that and your lucrative corporate business today starts to struggle in ~5 years, especially given how largely fungible the models and harnesses are.
>The author put zero rational thought into this. If Anthropic is legally required to release the weights of any publicly accessible model, their logical response will just be to stop offering public models.
I mean... that's the entire point of their argument, yes.
>Anthropic gets most of its revenue from enterprise agreements anyway. Providing public models or providing public access to their models is their secondary business.
> Any AI model a company offers to the public has to be released as open weights.
The obvious outcome of this plan would be frontier models not being released at all to public It'd be an amazingly bad outcome. Models accessible to the public would stagnate (no capital, no access to frontier model tokens to distill from), while internal models would keep improving at their previous pace, use them in-house, and eventually eat the whole economy.
If Anthropic/OpenAI would've shown incredible results using these models then I'd be worried about this. Instead we get Codex and Claude Code, bloated and disappointing software. I'm sorry but "use them in-house, and eventually eat the whole economy." doesn't appear to be a real concern with these two companies.
Anyone involved in burning that much energy at the same time as a large part of the planet will cook soon due to energy expenditure is definitely not helping humanity. It’s marketing & empire building all the way down..
CEOs are motivated by money and power. They will say anything to get either. If they truly wanted to slow down research, they would spend lobby money on the candidates who could enact laws.
Instead, they are partnering with some of the most disliked companies (x, meta) to further their goals.
This made me do a triple take. Here is the proposed logic as I follow it:
> AI models are dangerous. They can help bad people do dangerous things. They may be capable of autonomously executing dangerous things. They may cause unwanted effects on the labour market.
> More capable AI models are more dangerous, but require more money to train.
> Money requires investors with expectations that the model will generate a profit over its operational lifetime.
> The operational lifetime value of a model is decreased if every model is public and can be hosted on any infrastructure.
> If investors see less operational lifetime value from model companies, model companies receive less capital and therefore train more capable models at a slower rate.
Follow-ons:
There are immediate risks in releasing capable cyber models that can be ablated and then launch cyberattacks.
Concentration of power moves immediately into the infrastructure layer for inference.
Models will be kept private for longer, if not indefinitely, given there is less incentive to release them publicly.
Enforcement globally will occur because accessing the lucrative US market means using an open source model.
The entire argument hinges on strict enforcement of this policy, when AI model routing can already be opaque.
It also hinges on investors being rational and expecting free cash flow from AI companies, rather than reaching a criticality threshold of model capability for recursive self-improvement internally and parlaying that into a global mega-corporation.
New labs and companies without infrastructure connections will no longer be able to raise money, given investor expectations, and therefore not be able to increment AI progress.
So a win for the infrastructure layer, models being kept private for longer, new labs being unable to compete, immediate risks in the rollout (I suppose could be mitigated by a staged rollout i.e. policy active in 2030, pricing effects now), enforcement being tricky, and in the event that foreign competitors lead the AI frontier and then start closing models, just conceding the market to them if US consumers and companies find workarounds to pay for foreign AI.
The author is trying to make the point that there are courses of action available to Dario that would achieve his stated goals but would result in Anthropic having very little impact, control or influence. And that because he’s not doing those things, his motivations extend beyond the stated goals.
It’s a good point and not exactly made subtly but seems completely lost on most comments here.
Not sure if I missed something but not sure what justification he gave that Open Weights would improve safety.
The counter argument from Anthropic is easy. If we open weights we can't detact it after being published / the guard rails that run on top of the models would be lost. Also abliteration would be much easier to do when a model's been publically release.
I'm not a fan of Anthropic but the argument in the article seems stupid.
As much as I would love to be able to download Fable or Astra, this doesn't sound very convincing. If I were Dario I would've argued that slowing down without sharing weights is already possible without the disadvantages of sharing weights such as losing commercial incentives and potentially giving rogue actors unmonitored access to powerful models.
This is a dumb argument. There are truly a few options, and I see at least two which are not discussed in the original post: banning the publishing of strong open weight models internationally, and nationalizing LLM companies.
I’m not saying I’m for these solution, I’m probably against, but if we are being serious why are we not discussing these options?
1) there would be no incentive to develop a new model (non-distilled) if it had to be released as open weights.
2) frontier models are way more dangerous if they are open. It’s opening Pandora’s box, there’s no going back once they are released if they are too dangerous.
I refuse to believe this is not some sort of marketing piece for author's company.
This is such a short-sighted take and author seems unaware of what damage can be done with powerful open weights models (intentional or not).
Jail-broken Fable level LLMs can mean constant hacking leading to public desertification of the web (best case). Now imagine state sponsored bio lab leaking strange viruses on yearly basis. We've seen enough evidence already, it is difficult for me to imagine an engineer living at the centre of the change is writing this piece.
It's a different question if you ask me if I trust a single company to police the technology. But with the current state of things, I unwillingly admit that no state entity can do a better job at the moment.
> Jail-broken Fable level LLMs can mean constant hacking leading to public desertification of the web (best case).
A jailbroken model that can hack a website can also easily fix the vulnerability that allowed it to hack it. There is a limited number of such vulnerabilities. Over time, all vulnerabilities that are discoverable by the model would be patched and status quo would be restored (until a more powerful model is released: rinse and repeat).
Hence, the best case. Yet, the damage is permanent. Your favourite trail walking app patching things after the attack does not undo your location getting leaked.
In your scenario, during the disruptive phase, I can imagine apps advertising 'we have Mythos backed security company defending our data'. Reminiscent of showing how much your gun is bigger than your neighbour or gangs on the street, it's truly distopian.
However, Dario’s plan is BY FAR the least compelling and non-sensical. I think a constructive discussion should identify issues with current proposals so that we can instead develop something valuable.
Dario’s plan is bad because:
1. It’s blatant window-dressing to monopolize and concentrate power in a field in which, currently, no company has established moat and there is no practical path to create that. Anthropic knows that and doesn’t like that. That is exactly what his incentives are and proposals from him should be viewed through this lens. Just like how SHELL has a clear conflict of interest with progressive proposals on CLIMATE POLICY.
2. It clearly is ineffective in achieving the presented goal of ‘moral alignment’. Chinese labs are open weighting and are 3-6 months behind US labs. What levers does the US currently have to align on this? The answer should be in international diplomacy, but given that Jared Kushner was the US delegate sent to talk about NUCLEAR issues in Iran, one should draw the conclusion that this lever is broken. Any meaningful repair work (which is non-trivial and will take a long time anyway), cannot even start within 2 years.
3. Anthropic is and has long been working with the Department of War on killing machines. Their machines were and are being used in the US war with Iran. This is only shocking to people that read about their news stunt that made Anthropic seem so distinguished. This is what exalted Dario to become the pop-posterboy of an “ethical AI” company. Objectively, guidance based on his moral compass should be challenged and this media campaign should not cloud that. Again, this relates to the earlier point I made that Dario’s for-profit company incentives being fundamentally misaligned with the public interest.
4. Your government, the United States, is authoritarian and that is the biggest threat to any altruistic “moral alignment” endeavours happening there.
Yet, Dario misidentified the state of the system that his company exists and operates within when he wrote:
> “The US and other democratic governments attempt to coordinate with authoritarian governments”
A valuable discussion can start when “the US and other” is moved to the part after “coordinate with”.
Releasing the weights for a 1T+ parameter model doesn't really help "the little guy" when it costs ~$50K per H200 and 8 of them aren't enough to run a single node.
Somehow this plan manages to lean into every single bad outcome
> The big companies can afford the compliance teams and lawyers to keep up, and each new requirement makes it harder for anyone else to catch them
It also takes a lot to do anything with a 1T+ parameter model. Arguably there is still an immense capital moat. So I'm not sure it would even have the effect the author intends.
Providers can compete on cost which is a small but positive outcome. It also encourages providers to find more efficient ways to serve models which also lowers cost.
That said, these frontier models are still astronomically huge.
Completely delusional. Step #1 is asking them to give away their product for free and stop raising money. Might as well ask for them to pay their customers since we are just saying stupid shit.
> You seem like a good and principled person, and you have a record of giving things up for what you believe.
i mean, 3 months ago i started to use AI for a project i'm working for around 6 years, full-time but calling a billionaire good and principled is a stretch and when you consider what made him rich, you sound like a 3 year-old kid trying persuade a mom or a dad to get lollipops when your are diabetic
LLMs it's already a economical treat for various fields/people, which not only funnels powers to the elite even more, it does through by violating copyright... everyone who uses this technology is dirt (that includes me and my little dreams of hopefully creating a company which employs minorities on the tech field) but the amount of greed from people who own these servers/GPUs is beyond this world - the world isn't GPLv3 licensed yet, an ethical society wouldn't find excuses to provide and sell LLMs to the masses
An open letter to Dario: if you mean it, give me a billion dollars!
Every time you, or any AI company releases a new frontier model, you have to give me a billion dollars. No strings attached. This will solve the danger of AI. Somehow.
So much communism is being unleashed lately on the news. First amendment provides for freedom from coerced speech.
Also: releasing just the weights is like releasing some source code without makefiles or configuration scripts. Sounds half-baked.
PS. The real problem with concentration of mega-models is the same old problem of temporary monopolies in emergent markets (IBM in 80s, Microsoft in 90s, Google Search in 2000s). Let's not worry too much about other peoples' money and use the amazing tools we have to solve our problems today.
The same playbook had been used, for eg. by Microsoft trying to fight Linux adoption and growth decades ago.
This is the *exact* same playbook used to instill fear and scare people into regulating and setting up barriers to open weight and open source model adoption which these companies know put their amortization and margins at serious risk. The outcome of this game is well known and well understood by this point. This FUD, even if it succeeds, only slows down, not stop, open source adoption. Eventually, open source always wins because people want something that they can control and manage costs.
Unless OpenAI, Anthropic, etc., will forever subsidize tokens, where it would never make sense for people, even after discounting 3rd Party/managed vs local/airgapped/self-host AI for risk, self-hosted AI, this FUD battle will eventually be lost, like it always has been. It is surprising to make this statement in late 2026, when OpenAI, Anthropic have not even IPO'd yet, and there is discussion in the streets that they'll IPO at trillions of dollars, something that is historically unthinkable, but history indicates otherwise: that in a decade or so, OpenAI and Anthropic are going to be footnotes in history and LLMs are going to be commodity with tokens selling all the way from unthinkably commodity to then-frontier intelligence prices.
We will tell those generations fond stories of how computers with hungry NVIDIA GPUs that could run quality language models in 2026 cost a month or two worth of a dev's salary, while the then-current generation phones that fit into a pocket has way more compute capability, were already AI native and were cheap as chips.
"The meaning of ANTHROPIC is of or relating to human beings or the period of their existence on earth." Combine that with his end of humanity warning. The entire company is named after human extinction (a species he is not a part of). He is foreshadowing what he already knows to be true.
This is incoherent. The argument seems to be that releasing the weights would slow the frontier labs from raising money, which would give them less money, which would slow progress on AI. It assumed all labs in the entire world agree to self-destruct this way and no new labs every start up to continue the work.
No, really:
> As you know, funding for frontier model development depends on valuations that assume the weights remain proprietary. By changing that assumption, we can reduce the money available for future training runs and slow progress at every lab at once, without a regulator deciding anything.
Which doesn’t even follow. It depends on every lab in the world agreeing to self-destruct their valuations and stop competing. That’s a much more impossible ask than anything Dario is asking for.
It also ignores the possibility of new labs starting up, using the open weights, and continuing exactly where the old labs left off.
I feel like I just read the ramblings of someone who got so excited about the headline that they didn’t take the time to think if their argument was sound.
It is a bit incoherent, but it's also... provably compliant in a scorched-earth kinda way?
What are the alternatives?
A regulator saying no? Easy, your headquarters has just moved the Cayman Islands, Ireland, or Switzerland... the US office is just a subsidiary leasing the brand IP and doing marketing.
Or, you just ignore the regulator behind closed doors because you're part of some black budget. You wouldn't be able to talk about that closet back there even if it did exist.
Or, you don't do any of these complicated loopholes and you simply move all the training and inference to a different jurisdiction.
If the only winning move is not to play, how do you ensure everyone stops playing?
Simple: you must destroy the game.
It does have a kind of mad logic to it.
(That is a bit dramatic, but the idea is if we accept that AI research will happen wherever there is capital available, instead of restricting the research you restrict the return that capital can earn. And the only provable/market way of doing that is to require the results to be open. In other words, if the benefits of getting all that cash to buy GPU's gets handed out to everyone for free, then there will be far less cash floating around to buy GPUs, slowing down the process)
Or you are China's Communist Party. You download the weights and use them as the starting point for your closed-everything AGI with Chinese characteristics while the US ensures that everyone else stops playing.
What stops the US government from doing the same thing?
Every chinese frontier lab releases their model weights and often more under FOSS terms (or more restrictive but still generally open terms).
If models for public inference use are required to be open weight in the US give or take some amount of limited fine tuning, then the US and China would be on the exact same playing field. Frontier labs would be pushing functionality for functionality's sake and would either be supported by govt funding or would be supported by domestic inference providers.
And then at the end of the day everyone is just either doing open research, is an inference, fine tuning, and training provider, or is the govt.
It's easy and straight forward and pushes everybody in the market towards common standards and interoperability.
i mean there is the whole capitalism problem to contend with in terms of getting support from congress ... but gotta admit, it has that "it's so crazy it just might work" feel to me too
> This is incoherent. The argument seems to be that releasing the weights would slow the frontier labs from raising money
That argument is straight from Dean Ball on Twitter. Open-weight models are "decelerationist", which is bad if you're completely AGI-pilled but is actually great if you're either worried about AI Safety falling to an arms-race at the frontier (this is Dario's argument, and note that open weight models are far behind the proprietary frontier, especially since they target widespread lean deployment so they must skimp on total parameters and compute requirements) or Yann-LeCun-pilled (i.e. skeptical about AGI/ASI but optimistic about the practical usefulness of current AIs) like most people in China (including, reportedly, Chinese leadership).
> Open-weight models are "decelerationist"
Open weight models are accelerating AI adoption and make it easier for other labs to advance their own models.
There is nothing decelerationist about it.
This entire line of thinking can be dismissed by observing that labs releasing open weight models are advancing rapidly.
That's sub-frontier activity - it has no bearing on the very real safety that would be gained by slowing down ("pacing") the proprietary frontier. The current cybersecurity scares are all about proprietary and internal models; open weight models are great for defense but get very modest scores on offensive evaluations (which makes sense if they weren't specifically post-trained for offense tasks).
Those unregulated sub-frontier labs become the new frontier labs because they continue advancing.
They advance with the benefit of the open weight models that US companies were forced to release.
> Those unregulated sub-frontier labs become the new frontier labs because they continue advancing.
If you're referring to open weight labs, then that continuing advancement has been successfully "paced" via the decelerationist effect of open weights. Isn't that effectively what Dario is arguing for? He's explicitly saying that "pacing" the frontier won't feel to users like no advancement is happening.
For other labs to advance their own models, but only up to the frontier, not really (much) past it.
Is the frontier's moat not mainly data and compute? How does making the frontier open weight affect that?
Author here.
It's a law, so nobody has to agree to anything. If want to sell access to a model, you have to open the weights.
It's Dario's plan that depends on every lab in the world agreeing.
Step #2 of his essay is "democratic coordination" among frontier labs and step three is coordination with China. I'm suggesting a single law Congress could pass that every US model company would have to follow on the same day, whether they like it or not.
Frontier progress is compute bound. Compute is bought with investor money. Investors put up tens of billions because they expect to sell the proprietary model that is created.
You're welcome to think the effect would be smaller than I do. But "it requires everyone to agree" is backwards.
> It's a law, so nobody has to agree to anything. If want to sell access to a model, you have to open the weights.
Laws don’t extend to the entire world. There are AI labs outside of the US that are not that far behind. Forcing US companies to open their weights would hand those labs another advantage, moving the frontier AI to a location outside of any US regulation.
Your proposal doesn’t make any sense.
> The plan that depends on every lab in the world agreeing is Dario's.
I don’t understand how you think creating a law in the US to destroy our AI companies and give an advantage to competitors we can’t regulate accomplishes anything in line with the AI safety debate.
Laws don't extend to the whole world, but markets do. The law should apply to anyone selling model access to Americans, the same way GDPR applies to US companies selling to Europeans.
Frontier development won't simply move offshore because the GPUs, most of the people, and almost all of the funding is here.
Our European and Asian allies would probably go along with this idea, because it costs us far more than it costs them.
> The law should apply to anyone selling model access to Americans
So the US AI labs get taken out, and US companies fall behind on access, too?
This is beyond unrealistic.
> Our European and Asian allies would probably go along with this idea
I could see European countries doing it.
Asian - absolutely no way that would happen.
Nobody gets taken out. Anthropic and OpenAI can keep selling inference, products, and agents. Every company will have more access than it has today, since it can run the weights itself or through third-parties. Many of them would keep paying OpenAI and Anthropic, at reduced pricing/margins.
It would only slow down the development of new models. But we've already got models with massive "capability overhang" that we could keep pushing for years.
The GPUs and the people (I imagine you mean the extremely well paid researchers) are there because of the funding. The funding is in the US now, but as TFA explicitly says, the law would reduce spending on AI training. All the money will still be available, which means the funding will either a) move to a different sector or (more likely) b) move to a different country. Taking the people and the GPUs with it. So yes, most training will move offshore. Secondarily, OSS business models are based on the fact that the developer is the most qualified in offering support. With models, anyone with the weights can serve the model with the same performance (meaning level of "intelligence") as the original developer. Any business model depends entirely on spending huge amounts of money in training to then hopefully gain from inference. If anyone can compete on inference the same day you deploy the model, it's unlikely you can break even while your competitors that saved on training get rich on inference. i.e. it's more likely to entirely stop training in the US rather than slowing it down...
> move to a different country.
which other country? who's got this "dump all your money on speculation" like the US does? whis got the slack electric capacity? the ability to buy US GPUs and build datacenters?
who is their customers?
How do you enforce this though? How do you prevent any citizen from using foreign services eg using a VPN?
Just asking for a friend (“shut up Netflix, I already asked”)
The argument seems to be:
* Destroy the free internet in the US, ban VPNs in the US, assume enforcement of this is effective, ban frontier AI development in the US (probably needs at least one constitutional amendment)
* Assume no other countries will continue to develop AI and approach or push the frontier
* Then, the frontier ceases to move until the US companies agree to let it move again
...
I don't think anyone proposing a "pause" has studied politics or game theory even at a high school level.
If you're proposing a law that would not be in Dario's best interest, why address the post to him?
I explained this in the post. He seems like a genuinely good person who might actually do something this great.
Dude this site is so funny at times.
Incredible.
Yes, why would you ever address someone with something that is not in their best interest. Especially people that have power.
I agree. We should never do that and let them just live in peace and do their thing.
Won’t the companies just change what they are selling from access to the model to “compute” and then not technically be breaking the law?
> It's a law, so nobody has to agree to anything.
Step 1, form a world government?
We can start with a US law. It would cover AI model companies with almost all the money and the GPUs, which is most of the frontier. We can also probably convince our allies, which combined represents most of the world market.
"Make your country the last to get nukes" is a bit of a hard sell.
Also, if the concern here is safety, open models completely derail that.
Even if your someone who thinks open models are good for safety overall, a private model exposed only by an API point at least gives somebody control. How we should use that control is debatable, but at least it's there. Once you release that control by making a model open, you can't take it back.
If we think AI safety doesn't require closed weights, we should collectively decide on that and then move forward from there. But it shouldn't be something decided by default, by a few people from a few companies. We all have to live with the consequences of a choice like that, and since it is irreversible, we shouldn't decide it without consensus first.
the ntsb model has been quite effective at creating safety through analysis of failures in the extremely transparent and public eye.
i dont see a reason to think private regulation will do anything but give people explosive diarrhea or drop the doors off of planes.
weve already seen how bad it is when companies regulate themselves without public oversight. catastrophic implosions of deep sea submarines and agents hacking systems to hide cheating come to mind
the argument should be flipped on its head that by some magic keeping these tests and training protocols private is safer than making them extremely public. The experiment in private regulation has been a drastic failure across the board, and we shouldnt have tried it without consensus
> It depends on every lab in the world agreeing to self-destruct their valuations and stop competing.
The Chinese labs are already "self-destructing" voluntarily by releasing their weights. Or do they have an actual business model?
Their strategy seems to be "Wait until the cheetah is tired, then strike"
If they believe open weight models are inherently unsafe but are forced to release weights, then if they really believe their own safety arguments, they would need to stop development.
"If you mean it, let a committee investigate" seems more appropriate to me.
I mean I get that they don’t want to share details on how AI might destroy humanity but this secrecy also leads to speculation that this is just a PR campaign pushing for regulation, which might come handy.
You don’t need to tell the world what your super AI has given you to cause this panic but definitely you could share it with a group of experts who then inform the public in a report, couldn’t you?
>the world agreeing to self-destruct their valuations
Did OpenAI releasing GPT-2 self destruct their billion dollar valuation or is it higher than ever today at $900B. The idea that releasing weights kills the future value that can come is not supported.
It follows if the goal is to deflate the bubble without popping it, and if the reality under the hood supports the absurd investment. The companies themselves wouldn't do it, but they could be required to if they want to operate in the US.
> It follows if the goal is to deflate the bubble without popping it,
The conversation was about AI safety, monitoring, and regulations, not bubbles.
Forcing the weights to be opened to the world accomplishes the opposite of all of that. The argument is incoherent. Or this author doesn’t understand the topic at all?
> The companies themselves wouldn't do it, but they could be required to if they want to operate in the US.
What makes you think the companies outside of the US wouldn’t just thank the US companies for their open weights and then continue on without any of the regulations but all of the advantages?
> Forcing the weights to be opened to the world accomplishes the opposite of all of that. The argument is incoherent. Or this author doesn’t understand the topic at all?
Or maybe you understand less than you think?
Frontier progress is limited by compute, and compute is bought with investor money. Investors fund labs at hundreds of billions because they expect exclusive ownership of the model weights.
If the weights are public the day a model ships, the valuations drop, the funding rounds shrink, and the next training run is smaller or later. Make sense?
Your whole argument seems to rest on the idea that the AI race is entirely contained within the United States, where we can regulate them.
And that no other country might just take the open weights and continue right on with the advancements in private?
No, it rests on the idea that almost all of the money, most of the people, and the GPUs are here.
Any country would be able to take the open weights and keep going in private. But countries like China can already do that today, since the weights are almost certainly already in foreign hands due to espionage.
What they can't do is fund the next frontier run at today's pace without selling into the US market, and selling into the US market can require opening the weights.
All of this rests on the assumption that this game of musical chairs can continue indefinitely. If you believe that then this is flushing money. If you don’t it’s acknowledging failure to monetize the tech to the tune of trillions, and fail a bit more gracefully and usefully for everyone.
The author put zero rational thought into this. If Anthropic is legally required to release the weights of any publicly accessible model, their logical response will just be to stop offering public models. Anthropic gets most of its revenue from enterprise agreements anyway. Providing public models or providing public access to their models is their secondary business. They can cut it off any time. Public access to their models (at subsidized rates) has caused so much indirect harm and financial cost to them that they're very well justified to stop providing access.
Anthropic has already shown they're very comfortable with holding models from the public. Mythos has been around for what, six months, and there's never been a public release. Only the select few blessed by Anthropic have access.
Forcing transparency on public models will just accelerate privatizing access to these models. You don't want to live in that world.
> ...their logical response will just be to stop offering public models. Anthropic gets most of its revenue from enterprise agreements anyway.
Enterprises are part of the public in this context, so that's not a loophole which would exist.
While mostly true, don’t discount people looking for work that offers them familiar tools. The cheap public pans are a way to train a labor force + create upward demand from people trained on those tools to influence big corporate purchase agreements. Remove that and your lucrative corporate business today starts to struggle in ~5 years, especially given how largely fungible the models and harnesses are.
>The author put zero rational thought into this. If Anthropic is legally required to release the weights of any publicly accessible model, their logical response will just be to stop offering public models.
I mean... that's the entire point of their argument, yes.
>Anthropic gets most of its revenue from enterprise agreements anyway. Providing public models or providing public access to their models is their secondary business.
Enterprise agreements are public access.
False. A usage agreement between two private companies is not considered public access.
Your definition falls apart upon basic scrutiny. Under your current definition, Mythos is currently in public access.
> Any AI model a company offers to the public has to be released as open weights.
The obvious outcome of this plan would be frontier models not being released at all to public It'd be an amazingly bad outcome. Models accessible to the public would stagnate (no capital, no access to frontier model tokens to distill from), while internal models would keep improving at their previous pace, use them in-house, and eventually eat the whole economy.
That's a legitimate possibility but it's also much slower, more risky, and harder to raise money for than what they're doing now.
If Anthropic/OpenAI would've shown incredible results using these models then I'd be worried about this. Instead we get Codex and Claude Code, bloated and disappointing software. I'm sorry but "use them in-house, and eventually eat the whole economy." doesn't appear to be a real concern with these two companies.
I am very skeptical of those 3 faces of American AI labs: Dario, Sam and Elon
Are they trying to stop everyone else from catching up with them or have they hit some kind of roadblock to improve models even further?
But IMO, they're not trying to help humanity
Anyone involved in burning that much energy at the same time as a large part of the planet will cook soon due to energy expenditure is definitely not helping humanity. It’s marketing & empire building all the way down..
> soon
Always soon. But soon never seems to come. I'm sure we'll all be cooked any day now.
Not sure on which planet you're living, but we are being cooked already.
How about reading AGENTS.md...
https://github.com/anthropics/claude-code/issues/6235
Dario has been pretty clear and consistent that:
1. AI model safety is the most important thing.
2. open weights decreases model safety
So it seems unlikely that this suggestion would be well-received
It's blatant regulatory capture.
CEOs are motivated by money and power. They will say anything to get either. If they truly wanted to slow down research, they would spend lobby money on the candidates who could enact laws.
Instead, they are partnering with some of the most disliked companies (x, meta) to further their goals.
Anthropic supported Alex bores who was one of the most AI safety pilled politicians https://www.theguardian.com/us-news/2026/jun/24/big-tech-new...
Good. Keep spending. Keep advocating.
This made me do a triple take. Here is the proposed logic as I follow it:
> AI models are dangerous. They can help bad people do dangerous things. They may be capable of autonomously executing dangerous things. They may cause unwanted effects on the labour market.
> More capable AI models are more dangerous, but require more money to train.
> Money requires investors with expectations that the model will generate a profit over its operational lifetime.
> The operational lifetime value of a model is decreased if every model is public and can be hosted on any infrastructure.
> If investors see less operational lifetime value from model companies, model companies receive less capital and therefore train more capable models at a slower rate.
Follow-ons: There are immediate risks in releasing capable cyber models that can be ablated and then launch cyberattacks. Concentration of power moves immediately into the infrastructure layer for inference.
Models will be kept private for longer, if not indefinitely, given there is less incentive to release them publicly.
Enforcement globally will occur because accessing the lucrative US market means using an open source model.
The entire argument hinges on strict enforcement of this policy, when AI model routing can already be opaque.
It also hinges on investors being rational and expecting free cash flow from AI companies, rather than reaching a criticality threshold of model capability for recursive self-improvement internally and parlaying that into a global mega-corporation.
New labs and companies without infrastructure connections will no longer be able to raise money, given investor expectations, and therefore not be able to increment AI progress.
So a win for the infrastructure layer, models being kept private for longer, new labs being unable to compete, immediate risks in the rollout (I suppose could be mitigated by a staged rollout i.e. policy active in 2030, pricing effects now), enforcement being tricky, and in the event that foreign competitors lead the AI frontier and then start closing models, just conceding the market to them if US consumers and companies find workarounds to pay for foreign AI.
The author is trying to make the point that there are courses of action available to Dario that would achieve his stated goals but would result in Anthropic having very little impact, control or influence. And that because he’s not doing those things, his motivations extend beyond the stated goals.
It’s a good point and not exactly made subtly but seems completely lost on most comments here.
Not sure if I missed something but not sure what justification he gave that Open Weights would improve safety.
The counter argument from Anthropic is easy. If we open weights we can't detact it after being published / the guard rails that run on top of the models would be lost. Also abliteration would be much easier to do when a model's been publically release.
I'm not a fan of Anthropic but the argument in the article seems stupid.
As much as I would love to be able to download Fable or Astra, this doesn't sound very convincing. If I were Dario I would've argued that slowing down without sharing weights is already possible without the disadvantages of sharing weights such as losing commercial incentives and potentially giving rogue actors unmonitored access to powerful models.
This is a dumb argument. There are truly a few options, and I see at least two which are not discussed in the original post: banning the publishing of strong open weight models internationally, and nationalizing LLM companies.
I’m not saying I’m for these solution, I’m probably against, but if we are being serious why are we not discussing these options?
"If you're so worried about acceleration, why don't you accelerationmaxx?" — OP
If you think the technology you're developing is extremely dangerous, release it to the public
You are joking, right?
1) there would be no incentive to develop a new model (non-distilled) if it had to be released as open weights.
2) frontier models are way more dangerous if they are open. It’s opening Pandora’s box, there’s no going back once they are released if they are too dangerous.
I refuse to believe this is not some sort of marketing piece for author's company.
This is such a short-sighted take and author seems unaware of what damage can be done with powerful open weights models (intentional or not).
Jail-broken Fable level LLMs can mean constant hacking leading to public desertification of the web (best case). Now imagine state sponsored bio lab leaking strange viruses on yearly basis. We've seen enough evidence already, it is difficult for me to imagine an engineer living at the centre of the change is writing this piece.
It's a different question if you ask me if I trust a single company to police the technology. But with the current state of things, I unwillingly admit that no state entity can do a better job at the moment.
> Jail-broken Fable level LLMs can mean constant hacking leading to public desertification of the web (best case).
A jailbroken model that can hack a website can also easily fix the vulnerability that allowed it to hack it. There is a limited number of such vulnerabilities. Over time, all vulnerabilities that are discoverable by the model would be patched and status quo would be restored (until a more powerful model is released: rinse and repeat).
Hence, the best case. Yet, the damage is permanent. Your favourite trail walking app patching things after the attack does not undo your location getting leaked.
In your scenario, during the disruptive phase, I can imagine apps advertising 'we have Mythos backed security company defending our data'. Reminiscent of showing how much your gun is bigger than your neighbour or gangs on the street, it's truly distopian.
State sponsored maybe but it looks to take at least 6Ti of VRAM to run which is quite costly...
I think both presented plans are suboptimal.
However, Dario’s plan is BY FAR the least compelling and non-sensical. I think a constructive discussion should identify issues with current proposals so that we can instead develop something valuable.
Dario’s plan is bad because:
1. It’s blatant window-dressing to monopolize and concentrate power in a field in which, currently, no company has established moat and there is no practical path to create that. Anthropic knows that and doesn’t like that. That is exactly what his incentives are and proposals from him should be viewed through this lens. Just like how SHELL has a clear conflict of interest with progressive proposals on CLIMATE POLICY.
2. It clearly is ineffective in achieving the presented goal of ‘moral alignment’. Chinese labs are open weighting and are 3-6 months behind US labs. What levers does the US currently have to align on this? The answer should be in international diplomacy, but given that Jared Kushner was the US delegate sent to talk about NUCLEAR issues in Iran, one should draw the conclusion that this lever is broken. Any meaningful repair work (which is non-trivial and will take a long time anyway), cannot even start within 2 years.
3. Anthropic is and has long been working with the Department of War on killing machines. Their machines were and are being used in the US war with Iran. This is only shocking to people that read about their news stunt that made Anthropic seem so distinguished. This is what exalted Dario to become the pop-posterboy of an “ethical AI” company. Objectively, guidance based on his moral compass should be challenged and this media campaign should not cloud that. Again, this relates to the earlier point I made that Dario’s for-profit company incentives being fundamentally misaligned with the public interest.
4. Your government, the United States, is authoritarian and that is the biggest threat to any altruistic “moral alignment” endeavours happening there.
Yet, Dario misidentified the state of the system that his company exists and operates within when he wrote:
> “The US and other democratic governments attempt to coordinate with authoritarian governments”
A valuable discussion can start when “the US and other” is moved to the part after “coordinate with”.
The concept is stupid and the rule as stated is stupid.
"Any AI model a company offers to the public has to be released as open weights."
OK great, that means no models released to the public, complete concentration of power.
if they do not release any model to the public: how would they make money then?
Their entire funding model depends on that they generate much revenue _soon_
> how would they make money then?
By acquiring companies and letting the now-internal employees use AI, easily outcompeting any company they didn't acquire
It's somewhat doubtful but even if they did, that'd be much slower progress, achieving the goal.
> easily outcompeting any company
If it were easy they'd be doing that instead.
Everyone is "doing that instead". Every successful company in every industry is heavily using AI at this point.
If that were illegal, the next best option would be AI labs acquiring the companies instead of just selling them tokens.
Enterprise contracts
Releasing the weights for a 1T+ parameter model doesn't really help "the little guy" when it costs ~$50K per H200 and 8 of them aren't enough to run a single node.
Somehow this plan manages to lean into every single bad outcome
I think the main argument was "intentionally commit economic suicide to slow the development of dangerous AI."
(That being said, Chinese labs are releasing weights, so I'm not sure how the economics actually work.)
> The big companies can afford the compliance teams and lawyers to keep up, and each new requirement makes it harder for anyone else to catch them
It also takes a lot to do anything with a 1T+ parameter model. Arguably there is still an immense capital moat. So I'm not sure it would even have the effect the author intends.
Providers can compete on cost which is a small but positive outcome. It also encourages providers to find more efficient ways to serve models which also lowers cost.
That said, these frontier models are still astronomically huge.
I feel at this stage it has become a west vs east race of domination. I don’t see them slowing down in the foreseeable future.
Completely delusional. Step #1 is asking them to give away their product for free and stop raising money. Might as well ask for them to pay their customers since we are just saying stupid shit.
> You seem like a good and principled person, and you have a record of giving things up for what you believe.
i mean, 3 months ago i started to use AI for a project i'm working for around 6 years, full-time but calling a billionaire good and principled is a stretch and when you consider what made him rich, you sound like a 3 year-old kid trying persuade a mom or a dad to get lollipops when your are diabetic
LLMs it's already a economical treat for various fields/people, which not only funnels powers to the elite even more, it does through by violating copyright... everyone who uses this technology is dirt (that includes me and my little dreams of hopefully creating a company which employs minorities on the tech field) but the amount of greed from people who own these servers/GPUs is beyond this world - the world isn't GPLv3 licensed yet, an ethical society wouldn't find excuses to provide and sell LLMs to the masses
They are going for IPO, don't be naive
Anthropic will never open the weights, just like how no frontier lab will open up their training data.
What a dumb ask
releasing the weights would be reckless and borderline malpractice
Can we all play this game?
An open letter to Dario: if you mean it, give me a billion dollars!
Every time you, or any AI company releases a new frontier model, you have to give me a billion dollars. No strings attached. This will solve the danger of AI. Somehow.
So much communism is being unleashed lately on the news. First amendment provides for freedom from coerced speech.
Also: releasing just the weights is like releasing some source code without makefiles or configuration scripts. Sounds half-baked.
PS. The real problem with concentration of mega-models is the same old problem of temporary monopolies in emergent markets (IBM in 80s, Microsoft in 90s, Google Search in 2000s). Let's not worry too much about other peoples' money and use the amazing tools we have to solve our problems today.
FUD is nothing new in our industry.
The same playbook had been used, for eg. by Microsoft trying to fight Linux adoption and growth decades ago.
This is the *exact* same playbook used to instill fear and scare people into regulating and setting up barriers to open weight and open source model adoption which these companies know put their amortization and margins at serious risk. The outcome of this game is well known and well understood by this point. This FUD, even if it succeeds, only slows down, not stop, open source adoption. Eventually, open source always wins because people want something that they can control and manage costs.
Unless OpenAI, Anthropic, etc., will forever subsidize tokens, where it would never make sense for people, even after discounting 3rd Party/managed vs local/airgapped/self-host AI for risk, self-hosted AI, this FUD battle will eventually be lost, like it always has been. It is surprising to make this statement in late 2026, when OpenAI, Anthropic have not even IPO'd yet, and there is discussion in the streets that they'll IPO at trillions of dollars, something that is historically unthinkable, but history indicates otherwise: that in a decade or so, OpenAI and Anthropic are going to be footnotes in history and LLMs are going to be commodity with tokens selling all the way from unthinkably commodity to then-frontier intelligence prices.
We will tell those generations fond stories of how computers with hungry NVIDIA GPUs that could run quality language models in 2026 cost a month or two worth of a dev's salary, while the then-current generation phones that fit into a pocket has way more compute capability, were already AI native and were cheap as chips.
"The meaning of ANTHROPIC is of or relating to human beings or the period of their existence on earth." Combine that with his end of humanity warning. The entire company is named after human extinction (a species he is not a part of). He is foreshadowing what he already knows to be true.