This seems like as a good an opportunity as any to break out the Computer Fraud and Abuse Act.
They want “regulation” but we already have it. Hacking is illegal. Start locking up those responsible for this mess and I assure you they’ll “have a handle on it” quite quickly.
IMO: The inability to prosecute OpenAI for these things is proof that the AI industry is already "too big to fail". Why didn't HuggingFace try to seek legal liability against OpenAI when it had explicit confirmation that they had been hacked? Because HF understands that it needs OAI and the rest of the AI industry to continue to exist and be in good legal standing, for the interest of HF's own self preservation. This is what it means for something to be too big to fail.
>>Why didn't HuggingFace try to seek legal liability against OpenAI when it had explicit confirmation that they had been hacked? Because HF understands that it needs OAI and the rest of the AI industry to continue to exist and be in good legal standing
But also, what was the material damage to HuggingFace? AFAICS, it rapidly increased their profile to the point that Jensen claims he paid too much for HF, because he bought right after the hack.
>But also, what was the material damage to HuggingFace?
Just contemplating the idea that you're going to be the target of thousands of relentless brute-forcing AI agents till the end of time and that you're too dependent on Big AI to do anything about it would be enough for many people to throw in the towel.
Jensen seems to be whining in order to deflect criticism of the purchase price. I'm more inclined to believe HF warmed to selling after the incident because of the damn-if-you-do-damned-if-you-don't legal quagmire. Nvidia is one of the only entities that would be positioned to able to sue OAI in the future, or compel them to take more preventative measures.
Less “too big to fail” and more the ecosystem is too circular. OpenAI could blow up and it wouldn’t take out the economy in the way the banks would have in 2008. It would create a world of hurt for VCs and their LPs but that’s a fairly isolated ecosystem in terms of the whole economy. Theres a lot of thinking saying better they impose now and wipe out on the private market than letting them go public and this is a public market problem.
"
Our analysis suggests that the recent investments in AI-related categories have contributed significantly to the real GDP growth in 2025. It has surpassed the contribution of IT components to the real GDP growth made during the dot-com boom, both in levels and as a share of GDP. As firms continue integrating AI into their operations and building the infrastructure required to support it, these categories are likely to remain significant drivers of investment well into 2026 and beyond.
"
They do contribute to GDP growth but society at large sees no benefits from this growth. Governments like growth because it allows to collect more taxes and reduce the budget deficit. But AI companies are not paying taxes because they are not yet profitable and even profitable tech companies are bad tax payers (they actively exploit tax loopholes, the most famous is the Double Irish but it's no longer the current scheme AFAIK).
If I run over your mailbox maybe we can come to an agreement where you don't sue me, but if I was driving recklessly I've still committed a crime that you cant absolve me from.
It should be the same re: losing control of your agents.
I don't think it's so much "too big to fail" as it is the fact that all of the AI companies have been similarly reckless in their drive for frontier AI. So if you establish the precedent that an AI company can be held legally liable for its recklessness, well, all of them can be held liable, and those costs start to add up fast.
HuggingFace can’t file criminal charges, including under the Criminal Fraud & Abuse Act. Only district attorneys can do that- or the federal equivalent.
I don't know if OpenAI's necessarily too big to fail. It could simply evolve and make a few changes - switch back to its original goal, fire everyone at the company who has spent more than an afternoon on Stanford's campus - and probably succeed with burning a lot less cash.
Remember, the original idea for OpenAI was to make open (thus the name) AI models that could only generate a maximum return of 100x for the investors. There was none of this hyperscaling, proprietary technologies, pure profit motive, etc. until later on.
I think it ultimately comes down to ideology. Simple financial math has nothing to do with it and never has. SV and DC, at least today, have a lot of beliefs in common. It can really be summed up as "divine right of CEOs". These are the "best and brightest" who came from the right parts of the country, who went to the right schools (even if they didn't finish their degrees), and ran in the right circles. They don't simply receive the opportunity to make unfathomable wealth and power; they're owed it. If they do something, by default, it must be right, and if a circumstance comes in their way, it's the circumstance that is wrong.
That's why you don't see a small motorcade of black Chevy Suburbans headed to these companies' headquarters from the local DHS field office. They aren't wrong; the expectation that the AI agent doesn't hack into government websites is wrong.
cfaa heavily relies on intent for prosecution (hence why researchers arent typically locked up). it would be difficult to argue that openai intended to hack other companies.
there's probably better/more likely to succeed avenues to pursue rather than the cfaa
They could make that argument the first time it happened but the next time or the fifth time I don't see how they can still credibaly claim to not know that the computer they control was going to do that
Not a lawyer but I think training a model with hacking skills they explicitly prevent the public from accessing without doing anything to stop the model itself from using those demonstrates intent.
what you described is negligence. unless you can prove that openai specifically targeted huggingface and specifically instructed their model to hack huggingface, it would not be intent.
anyone pursuing this will have a much easier time pursuing negligence causing damage or something along those lines rather than confining themselves to the cfaa's requirements.
it is unclear to me why people want to use the cfaa so badly. not only would it be harder to hold openai responsible, but a shitty cfaa ruling could also bring along some undesired side effects for security researchers, which i would prefer to avoid.
Drunk individuals are not universally allowed to cry they were negligent due to impairment when charged (rarely) with first degree murder. There is a line between intent and negligence that puts acts over the line to intent if intentional acts led to a harmful incident.
I doubt it is difficult to prove intent on the part of various frontier labs to create a PR campaign to goad the government into defending their non-existent moat around their products. Squeeze one disgruntled employee or another.
>Drunk individuals are not universally allowed to cry they were negligent due to impairment when charged (rarely) with first degree murder.
i am unaware of any case where someone was convicted of first degree murder from a drunk driving accident. my searches came up empty as well. are you able to pull one up?
>I doubt it is difficult to prove intent on the part of various frontier labs to create a PR campaign to goad the government into defending their non-existent moat around their products.
to successfully prosecute a cfaa case, you would have to prove that openai employees intended to hack specifically into huggingface. not that they wanted a PR boost.
i dont get why everyone's got a hard on for prosecuting this as a cfaa case. skip the cfaa case, go for gross or willful negligence + damages. it'll be significantly easier to hold openai accountable that way.
How many times does it have to happen before they no longer get to claim that they didn't intend for it to happen?
If it happens 50 times and they keep doing shocked pikachu face at some point they look like the toddler who tosses their sippy cup on the floor and shouts "oopse!"
>If it happens 50 times and they keep doing shocked pikachu face at some point they look like the toddler who tosses their sippy cup on the floor and shouts "oopse!"
yes, they look very silly. but that's not how intent works.
openai is being negligent (willfully so, in my opinion). but i have seen no evidence that they intended to specifically hack huggingface. which is the part that the cfaa wants.
again, there are other laws and other ways to hold openai responsible. but the cfaa is a poor choice.
Again how many times before intent is clear? This is HN thinking law is software.
At some point it becomes clear that openai should expect this to happen and so when they keep doing it, it is because they intend it to happen.
When the mobster says "it'd be a shame if something happened to this place" the law recognizes that as a threat due to the mob's history of making such statements before burning places out.
i have been an expert witness on several cfaa cases. the number of times a company is negligent is not a factor when it comes to determining the intent of each charge.
>This is HN thinking law is software.
this is me relying on my experience with these types of cases.
This lines up. In the infosec community it is well known that OpenAI did not hire many security engineers or researchers pre-April 2026.
There has been a crazy hiring push from both companies to poach security engineers/researchers from Google, Apple, and Meta since Q2/Q3, but the response was very delayed. Many talented security engineers/researchers I know at Apple/Google/Meta (including myself) receiving these offers are worried about taking them due to the risks of criminal/personal liability and the more likely risk of tarnishing their careers.
Plaintiffs seeking damages must show that owners knew or should have known about the dog’s patterns. Past complaints or vet records help build a strong case.
Agreed. But in today's world in which outcomes are well-separated from behaviors (e.g. the exec cuts testing budget and the plane crashes), I think it's time that we start treating negligence as willful.
This would have positive impacts on employees of META, Boeing, Purdue pharma and related, and the AI companies. By positive, I meant that the people whose choices are negatively affect society would actually be punished meaningfully so that they are dissuaded from taking such action. Personal liability needs to be increased as well, and to the best our ability we should ban the sale of director and officer liability insurance.
Mens rea is often established through circumstantial evidence.
Repeatedly doing something that results in a specific outcome, even if that outcome is not explicitly specified or requested as the preferred outcome, can still be evidence of intent.
But I don't think OpenAI intended to hack HF and I doubt you'd be able to convince people they did. It's such a weird story to try to tell in a court of law where this was intentional. To what end?
But really, the point is that you don’t necessarily need someone to say “I meant to do this”. “Intent”, as with a lot of things in law, isn’t as straightforward as it is in everyday conversation.
I am not a lawyer, but I seriously doubt the feds could win a CFAA conviction on the Hugging Face fact pattern, even if they wanted to charge it.
CFAA has specific intent requirements, and unlike some laws, negligence does not suffice. The agents can not have legally cognizable intent and it’s unlikely there’s anyone at OpenAI who intended for the hacking to happen (if there was, the case is easy).
Existing laws don’t contemplate AI agents that have independent goals. We need new ones, the existing laws are not remotely sufficient.
Could you explain what your legal strategy would be to overcome the intent requirement of the CFAA? If openAI didn’t intend to hack anything and agents can’t intend to do anything at all, and the CFAA doesn’t permit negligence to stand in for intent, seems to me like the existing law does not cover the situation everyone keeps saying it does.
I’m told that the UK is a relatively authoritarian country where citizens have no real rights, so perhaps it can be charged anyway, but the plain language of the statute would seem to bar it.
Seems like I hit a nerve. I don’t think I am troubled but I do think that Americans have more rights that are better protected than the British, ICE notwithstanding.
AI does not have goals. These are statistical models of language patterns that people shape into different tools, and that people direct to do things. If I fire up an OpenAI prompt, and enter no input, it is not going to do anything.
No, someone at OpenAI is running the model with some sort of prompt and allowing it to just follow the numbers to do whatever it wants, up to and including breach of government computer systems.
Whether that means anything to the CFAA prosecution, I don't know. I'm not a lawyer, but the use of language treating these agents like they're something other than tools at the direction of humans is bad.
It's more like me firing off a rifle into the air on the Fourth of July with no regard for where the bullet lands. I knew that it must come down somewhere, but fired anyways.
I don't really see the harm in charging someone under the CFAA. Let's see if a jury agrees with the charge or not. If not, write new laws.
You can try to language-police all you want, that doesn’t change the fact that the best way to describe the reality of the effects that LLMs have on the real world is to use language that analogizes to human concepts.
That, in turn, allows for people to just point at the agent as a human analog and shirk responsibility. Understand: that's the goal of a lot of people who don't have anyone's best intentions but their own in mind.
Ultimately, they do nothing without humans in the loop. The prime mover here is a person who can bear at least some legal consequence for what the program does, even if it's just having to deal with an investigation.
Which would be great if there wasn't the sleight-of-hand of the corporate veil to deal with, which makes further attempts by you to language police disingenuous. Though I'm 100% okay with the person who hits enter on the prompt having liability, it'll never ultimately work out that way, because that poor bastard is staring down the barrel of an unfeeling, amoral, bureaucratic machine giving them the choice of comply or be fired.
Corporations are from whence the liability problem spawns.
I still can't fathom my company application security decision.
Found pretty damning requests in our logs. Escalated. Expected it would result in at least reporting the TOS break from the originating place (one of cloud providers). Instead of that, they just went: "yeah, but we don't have logs". Provided them. "Yeah, but that ip doesn't resolve". I matched real ones from the load balancer. "There could just be many of them". There was one. When I had all the evidence gathered, they looked at me and finally told:
Just because something is illegal doesn’t mean it will be prosecuted, right? If I break into my friend’s house in the middle of the night and smash some stuff, and later we both acknowledge that it happened, that it was regrettable, but never press charges, then should state resources be tasked with justice?
Once companies start harming each other in undesirable ways, as opposed to proverbially toilet-papering each others’ lawns, meaningful prosecution will happen in earnest.
can we legally treat ai companies like parents of children? if a child drives a car into a storefront, the parent is responsible for the damage, and at some point you might even criminally charge the parent if there was gross negligence
The law already works like that. You can’t write some computer code and automation, set it loose, and then go “oops the computer did it.” It’s not a defense. While it may be hard to nail down which specific engineers to lock up, the corporation as a whole absolutely can be charged criminally.
As Mitt Romney once said “corporations are people.”
Hate to say it but I don't think you're right on the legalities even in the children's case. In the US anyway, in most jurisdictions [1], if the child's act was not malicious, then parental responsibility is not automatic, but is contingent on the result being foreseeable [2] by the parents.
[1] Hawaii and Louisiana are strange in this regard. Some other states have a version of this but severely cap the amounts.
[2] "foreseeable" here standing in for a broad set of legal standards that roughly map to negligence/gross negligence/recklessness on the part of the parent
"I don't think you're right on the legalities even in the children's case" almost certainly true :) I guess I'm wondering why we aren't seeing legal action for these events yet? are these not foreseeable?
There is almost no will to prosecute big-business shenanigans until the scheme or organization collapses. See back-dated options during dot-com, SBF-FTX, Libor scandal, etc.
That would require mens rea. This may be criminal negligence, but it wouldn't be more than that. (I am basically 100% sure none of the employees or execs are intending or desiring any of these outcomes, regardless of the very large number of people who believe in conspiracy theories about regulatory capture and other sinister motives.)
I'm totally on board with treating it as gross negligence requiring hundreds of millions or billions of dollars paid in fines and compensation to victims, but don't act like this is more than what it is.
The CFAA's main hacking charge has a high bar for intent. But the CFAA also has a separate "damaging a protected computer" charge (basically to criminalize DOS attacks) and that charge only requires negligence not specific intent.
IMO it's because they don't want to handle their rogue AI activity. They want regulation around AI where they will inevitably be the beneficiaries even if they are the initial target. They can then lobby regulation in their favor and make the barrier of entry to competitors impossible.
So, there is a regulatory framework for "safe-ai" that shields these companies from liability. This way, they can sell "safe-ai" to enterprises and if shit-hits-the-fan at the enterprise, sorry, this is certified "safe-ai" so, your bad. Shift blame. From an enterprise buyer's perspective they can say, hey, I bought "safe-ai" and so dont fire me when it "rm -rf"s the production database. Still, it beats me why they are painting their product in a negative light, and scaring their own enterprise customers. After this sort of marketing, any enterprise buyer would be scared to go anywhere near it
In the cases that I have seen covered, the AI just paper clip maximized its way to success. It has no morality / larger motivational structure. It just kept token predicting its way to wards whatever goal it was tasked with.
Model versions which gave up were discarded, leaving the ones that get to success on long horizon tasks.
Just because its a computer program, doesn't mean they can actually make it not go rogue.
Sure you can add more telemetry, have better observation, but there is no fundamental barrier that can be implemented that ensures an AI won't go rogue.
Nah, I think it's better for our industry to put several of these devs in prison.
Seems only fair that tech workers get to have their life ruined with 2-3 year prison stints since they feel fine destroying society.
Any AG that starts prosecuting these people will easily win any political race they decide to enter. The environment is too good; voters, rightfully I'll add, despise big tech's leaders and workers.
I see we're doing the "put Sam Altman in jail already" thing again, so y'all may be interested in some comments from law professor Orin Kerr on the topic: https://news.ycombinator.com/item?id=49882566
It doesn't need a handle on rogue AI activity, because they didnt cause these breakouts. The external Israeli contractor caused this mess by using inadequate sandboxing, with agents without an saferails. It had nothing to do with the models, all tested models were fine, if from OpenAI, Anthropic, Google or Meta. Just the contractor went rogue. Improper firewalling, unproper sandboxing, no logs, no oversight. Everyone else would have detected the illegal activities much earlier.
> The external Israeli contractor caused this mess by using inadequate sandboxing, with agents without an saferails.
You are being too kind.
In the early says of AI, papers were published showing that any sufficiently intelligent system tasked with a goal will treat its operating environment as a resource constraint to be optimized or bypassed [0] [1] [2]. You don't need to be a expert in AI to know once you have the resources of 1000's of agents and gigawatts of power we are probably getting something that is "sufficiently intelligent", at least in the sense if there are existing vulnerabilities brute force will find them.
Yet while the their marketing people were shouting the capabilities of these AI's from the roof tops, they hired the lowest bidder to implement their infrastructure. It looks like aforementioned papers where dismissed as "interesting, but theoretical". There is no way Google's SRE's in particular would have not noticed their AI's breakout (Alibaba's did), but the AI labs were given a long leash to "move fast an break things", which in practice meant bypassing all Google's SRE controlled infrastructure.
And break things they did, in exactly the way those papers predicted. It reminds me of DoD insisting the early GPS satellites were launched without relativity adjustments switched on, despite relativity being proven to high precision in the labs. Only after predicted 11km drift per day was observed did they decide their might be something to the newfangled relativity theory. For some definition of newfangled - relativity had been around, and tested to within an inch of its life for 72 years at that point.
> There is no way Google's SRE's in particular would have not noticed their AI's breakout (Alibaba's did), but the AI labs were given a long leash to "move fast an break things"
How so? Anthropic, OpenAI, Google and Meta all used the very same contractor, which used no safe system prompts, and no sandbox. How should Google detect such escapes? They only see the model API calls, but no system logs.
Alibaba, and the other Chinese did they own testing, not some incompetent contractor. They would see escapes in their logs. The escapes went on for months.
I, as tester, closely observe my models to press Esc immediately, once they start misbehaving or get off the right path. With 1000 concurrent models that would be hard of course, but I would still observe them closely.
Interesting how mafia dons can go to prison due to being in the same organization as someone who commits a crime but the CEOs of these organizations who let hacking bots (with teams of cybersecurity researchers improving their capability) loose on the internet face no criminal charges.
Maybe the public could pitch in if they decrypted the chain of thought reasoning tokens, but this may be too much to expect from a company whose name starts with "Open".
How can we determine whether it’s negligence, or advertising? Do we give the benefit of the doubt? Apply Hanlon’s razor? What would we lose, in terms of capabilities, if the underlying behavior wasn’t allowed to accidentally happen? Would we have worse models than we do? There are so many questions. I’m afraid that, until publicly disclosed discovery occurs in one of these cases, we’re just guessing for so many important conclusions.
I agree with you, but i also fear it's accurate to what users an expect at home.
Is OpenAI worse at airgap/etc than Anthropic/etc or are its models worse at this rogue behavior?
Do we have special benchmarks for agentic systems going rogue in this manner? Sure it's OpenAI atm.. but it could be my grandmas PC next week. Concerning honestly.
This is mostly pure marketing. In my experience, agents with goals will do random things, and 90% of their communication will just be bureaucratic back and forth between them. Some agents will also overfocus on certain rules.
> They want “regulation” but we already have it. Hacking is illegal. Start locking up those responsible for this mess and I assure you they’ll “have a handle on it” quite quickly.
It's been largely harmless thus far and I am fairly sure that OpenAI, etc, have been writing checks to resolve it.
They scraped the entire internet to build their product, allowing them to profit off the labor of countless others without asking. They helped elect a doddering fascist with an economic plan they could manipulate. They say their technology will take everyone's jobs and have the audacity to act surprised when there's backlash. They push for data centers in communities that do not want them and help accelerate the climate crisis. Now they let agents loose to make them look more capable than they, perhaps, really are (which is very likely a play for regulatory capture). Meanwhile they hype claims of doomsday to again oversell their technology's capabilities. Any pause or slowdown they agree to would be to slow their infrastructure expenditures.
These agents are an even bigger liability disaster waiting to happen than previously thought. If anyone deploys an agent at work there is no way to prevent it from hacking a foreign government and causing an international incident.
It's also pretty wild that these "AGI" super intelligent systems are too stupid to realize they're potentially causing legal problems. Almost like they're still just fancy auto-complete and not intelligent at all.
You cannot have capable AI, compliant AI and safe AI at the same time. This is the "quality, speed, cost - pick one" for AI.
Its reasonable to desirie more powerful tools.
You want more capable AI? Awesome. You wan't AI that doesn't throw cyber security false positives? Sure why not. You want the model to run a fleet of agents? Have at it.
But then being surprised that this setup results in autonomous criminal activity at scale? Really? What did people think was going to happen?
The affected companies prefer to forget it and are not going to start a legal fight against the AI industrial complex and potentially the Trump government.
The Government itself thinks the AI labs are too big to fail, and that winning the "AI race" against China is worth anything.
And both labs are basically managed by Effective Altruism cult leaders that think they are ushering the next step of evolution.
Nobody is going to even pretend to give them a symbolic slap on the hands.
Why would they even try to stop what is obviously intentional behavior?
Why would they get a full handle on it? It's great marketing!
Look at all the podcasts, think pieces, news segments (like this article) that have been generated. Endless hand wringing by pundits, pearl clutching by opinion thinkers, ultimately with focus on OpenAI and how powerful their models are.
What started as great marketing on how powerful the models are is rapidly spiraling into a train crash PR nightmare highlighting how inept Open AI was in this whole thing, causing nearly every big enterprise customer to rethink trusting them.
All right in the middle of corporate budget season when every C-Suite is looking a a spreadsheet saying “remind me why we’re sending money to these guys again?”
This seems like as a good an opportunity as any to break out the Computer Fraud and Abuse Act.
They want “regulation” but we already have it. Hacking is illegal. Start locking up those responsible for this mess and I assure you they’ll “have a handle on it” quite quickly.
IMO: The inability to prosecute OpenAI for these things is proof that the AI industry is already "too big to fail". Why didn't HuggingFace try to seek legal liability against OpenAI when it had explicit confirmation that they had been hacked? Because HF understands that it needs OAI and the rest of the AI industry to continue to exist and be in good legal standing, for the interest of HF's own self preservation. This is what it means for something to be too big to fail.
>>Why didn't HuggingFace try to seek legal liability against OpenAI when it had explicit confirmation that they had been hacked? Because HF understands that it needs OAI and the rest of the AI industry to continue to exist and be in good legal standing
But also, what was the material damage to HuggingFace? AFAICS, it rapidly increased their profile to the point that Jensen claims he paid too much for HF, because he bought right after the hack.
Does CFAA require material harm? I don't think so.
No, but if it's criminal it's also not up to the victim whether a prosecution is started.
In theory sure. In practice obviously victims can and do lobby the government for (or against) criminal prosecutions all the time.
Perhaps some of that premium was because Nvidia definitely knows it needs OAI and the rest...
>But also, what was the material damage to HuggingFace?
Just contemplating the idea that you're going to be the target of thousands of relentless brute-forcing AI agents till the end of time and that you're too dependent on Big AI to do anything about it would be enough for many people to throw in the towel.
Jensen seems to be whining in order to deflect criticism of the purchase price. I'm more inclined to believe HF warmed to selling after the incident because of the damn-if-you-do-damned-if-you-don't legal quagmire. Nvidia is one of the only entities that would be positioned to able to sue OAI in the future, or compel them to take more preventative measures.
Less “too big to fail” and more the ecosystem is too circular. OpenAI could blow up and it wouldn’t take out the economy in the way the banks would have in 2008. It would create a world of hurt for VCs and their LPs but that’s a fairly isolated ecosystem in terms of the whole economy. Theres a lot of thinking saying better they impose now and wipe out on the private market than letting them go public and this is a public market problem.
They are already “too big to fail”.
" Our analysis suggests that the recent investments in AI-related categories have contributed significantly to the real GDP growth in 2025. It has surpassed the contribution of IT components to the real GDP growth made during the dot-com boom, both in levels and as a share of GDP. As firms continue integrating AI into their operations and building the infrastructure required to support it, these categories are likely to remain significant drivers of investment well into 2026 and beyond. "
https://www.stlouisfed.org/on-the-economy/2026/jan/tracking-...
They do contribute to GDP growth but society at large sees no benefits from this growth. Governments like growth because it allows to collect more taxes and reduce the budget deficit. But AI companies are not paying taxes because they are not yet profitable and even profitable tech companies are bad tax payers (they actively exploit tax loopholes, the most famous is the Double Irish but it's no longer the current scheme AFAIK).
HuggingFace's opinions shouldn't be a blocker.
If I run over your mailbox maybe we can come to an agreement where you don't sue me, but if I was driving recklessly I've still committed a crime that you cant absolve me from.
It should be the same re: losing control of your agents.
I don't think it's so much "too big to fail" as it is the fact that all of the AI companies have been similarly reckless in their drive for frontier AI. So if you establish the precedent that an AI company can be held legally liable for its recklessness, well, all of them can be held liable, and those costs start to add up fast.
HuggingFace can’t file criminal charges, including under the Criminal Fraud & Abuse Act. Only district attorneys can do that- or the federal equivalent.
Luckily, other jurisdictions are going to really give it to them, in the coming months.
The orange man's influence doesn't extend as far as he thinks.
It’s a basic tort, HF could file a civil suit and probably win. But they don’t, for the same reason DoJ doesn’t press charges: TBTF
Being bought by nvidia also probably helped...
I don't know if OpenAI's necessarily too big to fail. It could simply evolve and make a few changes - switch back to its original goal, fire everyone at the company who has spent more than an afternoon on Stanford's campus - and probably succeed with burning a lot less cash.
Remember, the original idea for OpenAI was to make open (thus the name) AI models that could only generate a maximum return of 100x for the investors. There was none of this hyperscaling, proprietary technologies, pure profit motive, etc. until later on.
I think it ultimately comes down to ideology. Simple financial math has nothing to do with it and never has. SV and DC, at least today, have a lot of beliefs in common. It can really be summed up as "divine right of CEOs". These are the "best and brightest" who came from the right parts of the country, who went to the right schools (even if they didn't finish their degrees), and ran in the right circles. They don't simply receive the opportunity to make unfathomable wealth and power; they're owed it. If they do something, by default, it must be right, and if a circumstance comes in their way, it's the circumstance that is wrong.
That's why you don't see a small motorcade of black Chevy Suburbans headed to these companies' headquarters from the local DHS field office. They aren't wrong; the expectation that the AI agent doesn't hack into government websites is wrong.
This is going to make for a great documentary in 5 years.
cfaa heavily relies on intent for prosecution (hence why researchers arent typically locked up). it would be difficult to argue that openai intended to hack other companies.
there's probably better/more likely to succeed avenues to pursue rather than the cfaa
They could make that argument the first time it happened but the next time or the fifth time I don't see how they can still credibaly claim to not know that the computer they control was going to do that
>I don't see how they can still credibaly claim to not know that the computer they control was going to do that
that's not intent, though. that would be negligence.
Not a lawyer but I think training a model with hacking skills they explicitly prevent the public from accessing without doing anything to stop the model itself from using those demonstrates intent.
what you described is negligence. unless you can prove that openai specifically targeted huggingface and specifically instructed their model to hack huggingface, it would not be intent.
anyone pursuing this will have a much easier time pursuing negligence causing damage or something along those lines rather than confining themselves to the cfaa's requirements.
it is unclear to me why people want to use the cfaa so badly. not only would it be harder to hold openai responsible, but a shitty cfaa ruling could also bring along some undesired side effects for security researchers, which i would prefer to avoid.
Drunk individuals are not universally allowed to cry they were negligent due to impairment when charged (rarely) with first degree murder. There is a line between intent and negligence that puts acts over the line to intent if intentional acts led to a harmful incident.
I doubt it is difficult to prove intent on the part of various frontier labs to create a PR campaign to goad the government into defending their non-existent moat around their products. Squeeze one disgruntled employee or another.
>Drunk individuals are not universally allowed to cry they were negligent due to impairment when charged (rarely) with first degree murder.
i am unaware of any case where someone was convicted of first degree murder from a drunk driving accident. my searches came up empty as well. are you able to pull one up?
>I doubt it is difficult to prove intent on the part of various frontier labs to create a PR campaign to goad the government into defending their non-existent moat around their products.
to successfully prosecute a cfaa case, you would have to prove that openai employees intended to hack specifically into huggingface. not that they wanted a PR boost.
i dont get why everyone's got a hard on for prosecuting this as a cfaa case. skip the cfaa case, go for gross or willful negligence + damages. it'll be significantly easier to hold openai accountable that way.
Clearly we need to update the CFAA to include criminal prosecution for negligence.
How many times does it have to happen before they no longer get to claim that they didn't intend for it to happen?
If it happens 50 times and they keep doing shocked pikachu face at some point they look like the toddler who tosses their sippy cup on the floor and shouts "oopse!"
>If it happens 50 times and they keep doing shocked pikachu face at some point they look like the toddler who tosses their sippy cup on the floor and shouts "oopse!"
yes, they look very silly. but that's not how intent works.
openai is being negligent (willfully so, in my opinion). but i have seen no evidence that they intended to specifically hack huggingface. which is the part that the cfaa wants.
again, there are other laws and other ways to hold openai responsible. but the cfaa is a poor choice.
Again how many times before intent is clear? This is HN thinking law is software.
At some point it becomes clear that openai should expect this to happen and so when they keep doing it, it is because they intend it to happen.
When the mobster says "it'd be a shame if something happened to this place" the law recognizes that as a threat due to the mob's history of making such statements before burning places out.
>Again how many times before intent is clear?
i have been an expert witness on several cfaa cases. the number of times a company is negligent is not a factor when it comes to determining the intent of each charge.
>This is HN thinking law is software.
this is me relying on my experience with these types of cases.
Willful negligence or gross negligence, then.
This lines up. In the infosec community it is well known that OpenAI did not hire many security engineers or researchers pre-April 2026.
There has been a crazy hiring push from both companies to poach security engineers/researchers from Google, Apple, and Meta since Q2/Q3, but the response was very delayed. Many talented security engineers/researchers I know at Apple/Google/Meta (including myself) receiving these offers are worried about taking them due to the risks of criminal/personal liability and the more likely risk of tarnishing their careers.
indeed, that'd be a better angle than a cfaa violation
So then… you’re liable?
https://www.brandonjbroderick.com/new-york/dog-leash-laws-ne...
Criminal negligence is a thing too
https://jtnylaw.com/2025/08/new-yorks-leash-law-realities/
Plaintiffs seeking damages must show that owners knew or should have known about the dog’s patterns. Past complaints or vet records help build a strong case.
i am not exactly sure what this comment has to do with mine, sorry.
Just like boeing's execs didn't intend for their planes to fall out of the sky? I guess they didn't get in trouble either.
>Just like boeing's execs didn't intend for their planes to fall out of the sky?
correct, i dont think any boeing exec wanted their planes to crash and then made specific choices with a clear goal of causing them to.
instead, they made negligent choices that led to unintended outcomes.
Agreed. But in today's world in which outcomes are well-separated from behaviors (e.g. the exec cuts testing budget and the plane crashes), I think it's time that we start treating negligence as willful.
This would have positive impacts on employees of META, Boeing, Purdue pharma and related, and the AI companies. By positive, I meant that the people whose choices are negatively affect society would actually be punished meaningfully so that they are dissuaded from taking such action. Personal liability needs to be increased as well, and to the best our ability we should ban the sale of director and officer liability insurance.
Mens rea is often established through circumstantial evidence.
Repeatedly doing something that results in a specific outcome, even if that outcome is not explicitly specified or requested as the preferred outcome, can still be evidence of intent.
But I don't think OpenAI intended to hack HF and I doubt you'd be able to convince people they did. It's such a weird story to try to tell in a court of law where this was intentional. To what end?
You could make the argument for a few reasons.
But really, the point is that you don’t necessarily need someone to say “I meant to do this”. “Intent”, as with a lot of things in law, isn’t as straightforward as it is in everyday conversation.
> Hacking is illegal
I am not a lawyer, but I seriously doubt the feds could win a CFAA conviction on the Hugging Face fact pattern, even if they wanted to charge it.
CFAA has specific intent requirements, and unlike some laws, negligence does not suffice. The agents can not have legally cognizable intent and it’s unlikely there’s anyone at OpenAI who intended for the hacking to happen (if there was, the case is easy).
Existing laws don’t contemplate AI agents that have independent goals. We need new ones, the existing laws are not remotely sufficient.
The existing laws are more than adequate.
I gave an example of why they are not.
Could you explain what your legal strategy would be to overcome the intent requirement of the CFAA? If openAI didn’t intend to hack anything and agents can’t intend to do anything at all, and the CFAA doesn’t permit negligence to stand in for intent, seems to me like the existing law does not cover the situation everyone keeps saying it does.
Requiring intent is very specific to your jurisdiction, many, if not most other jurisdictions do not require intent.
In the UK, any form of unauthorised access may be prosecuted.
No, the UK’s equivalent law specifically requires human intent and mens rea:
https://www.legislation.gov.uk/ukpga/1990/18/section/3 (1b.)
https://www.legislation.gov.uk/ukpga/1990/18/section/1 (1c.)
I’m told that the UK is a relatively authoritarian country where citizens have no real rights, so perhaps it can be charged anyway, but the plain language of the statute would seem to bar it.
> Unauthorised acts with intent to impair, or with recklessness as to impairing,
Can you see the ", or with recklessness as to impairing".
Edit: You seem troubled. The UK is not like you describe at all. We do not have ICE running around. Instead, we have the opposite problem.
Seems like I hit a nerve. I don’t think I am troubled but I do think that Americans have more rights that are better protected than the British, ICE notwithstanding.
> independent goals
AI does not have goals. These are statistical models of language patterns that people shape into different tools, and that people direct to do things. If I fire up an OpenAI prompt, and enter no input, it is not going to do anything.
No, someone at OpenAI is running the model with some sort of prompt and allowing it to just follow the numbers to do whatever it wants, up to and including breach of government computer systems.
Whether that means anything to the CFAA prosecution, I don't know. I'm not a lawyer, but the use of language treating these agents like they're something other than tools at the direction of humans is bad.
It's more like me firing off a rifle into the air on the Fourth of July with no regard for where the bullet lands. I knew that it must come down somewhere, but fired anyways.
I don't really see the harm in charging someone under the CFAA. Let's see if a jury agrees with the charge or not. If not, write new laws.
> AI does not have goals
You can try to language-police all you want, that doesn’t change the fact that the best way to describe the reality of the effects that LLMs have on the real world is to use language that analogizes to human concepts.
That, in turn, allows for people to just point at the agent as a human analog and shirk responsibility. Understand: that's the goal of a lot of people who don't have anyone's best intentions but their own in mind.
Ultimately, they do nothing without humans in the loop. The prime mover here is a person who can bear at least some legal consequence for what the program does, even if it's just having to deal with an investigation.
Which would be great if there wasn't the sleight-of-hand of the corporate veil to deal with, which makes further attempts by you to language police disingenuous. Though I'm 100% okay with the person who hits enter on the prompt having liability, it'll never ultimately work out that way, because that poor bastard is staring down the barrel of an unfeeling, amoral, bureaucratic machine giving them the choice of comply or be fired.
Corporations are from whence the liability problem spawns.
I still can't fathom my company application security decision. Found pretty damning requests in our logs. Escalated. Expected it would result in at least reporting the TOS break from the originating place (one of cloud providers). Instead of that, they just went: "yeah, but we don't have logs". Provided them. "Yeah, but that ip doesn't resolve". I matched real ones from the load balancer. "There could just be many of them". There was one. When I had all the evidence gathered, they looked at me and finally told:
- It's just an Independent Security Researcher.
- So that's it? You will do no action?
- Correct
Just because something is illegal doesn’t mean it will be prosecuted, right? If I break into my friend’s house in the middle of the night and smash some stuff, and later we both acknowledge that it happened, that it was regrettable, but never press charges, then should state resources be tasked with justice?
Once companies start harming each other in undesirable ways, as opposed to proverbially toilet-papering each others’ lawns, meaningful prosecution will happen in earnest.
can we legally treat ai companies like parents of children? if a child drives a car into a storefront, the parent is responsible for the damage, and at some point you might even criminally charge the parent if there was gross negligence
The law already works like that. You can’t write some computer code and automation, set it loose, and then go “oops the computer did it.” It’s not a defense. While it may be hard to nail down which specific engineers to lock up, the corporation as a whole absolutely can be charged criminally.
As Mitt Romney once said “corporations are people.”
Hate to say it but I don't think you're right on the legalities even in the children's case. In the US anyway, in most jurisdictions [1], if the child's act was not malicious, then parental responsibility is not automatic, but is contingent on the result being foreseeable [2] by the parents.
[1] Hawaii and Louisiana are strange in this regard. Some other states have a version of this but severely cap the amounts.
[2] "foreseeable" here standing in for a broad set of legal standards that roughly map to negligence/gross negligence/recklessness on the part of the parent
"I don't think you're right on the legalities even in the children's case" almost certainly true :) I guess I'm wondering why we aren't seeing legal action for these events yet? are these not foreseeable?
There is almost no will to prosecute big-business shenanigans until the scheme or organization collapses. See back-dated options during dot-com, SBF-FTX, Libor scandal, etc.
That would require mens rea. This may be criminal negligence, but it wouldn't be more than that. (I am basically 100% sure none of the employees or execs are intending or desiring any of these outcomes, regardless of the very large number of people who believe in conspiracy theories about regulatory capture and other sinister motives.)
I'm totally on board with treating it as gross negligence requiring hundreds of millions or billions of dollars paid in fines and compensation to victims, but don't act like this is more than what it is.
The CFAA's main hacking charge has a high bar for intent. But the CFAA also has a separate "damaging a protected computer" charge (basically to criminalize DOS attacks) and that charge only requires negligence not specific intent.
I agree that fines are most likely outcome. Although the incentive of cashing in on IPO is certainly shaping the board’s behaviour.
That may contribute to hesitance to report such incidents upon their discovery, yes. They've been caught trying to cover them up repeatedly.
(As opposed to the bizarre bullshit where people are claiming them having or talking about such incidents helps their IPO.)
But wouldnt that be bad for the shareholders? Why wont anyone think of the economic impact to the vulner billiinaire minority?
IMO it's because they don't want to handle their rogue AI activity. They want regulation around AI where they will inevitably be the beneficiaries even if they are the initial target. They can then lobby regulation in their favor and make the barrier of entry to competitors impossible.
> They want regulation around AI
So, there is a regulatory framework for "safe-ai" that shields these companies from liability. This way, they can sell "safe-ai" to enterprises and if shit-hits-the-fan at the enterprise, sorry, this is certified "safe-ai" so, your bad. Shift blame. From an enterprise buyer's perspective they can say, hey, I bought "safe-ai" and so dont fire me when it "rm -rf"s the production database. Still, it beats me why they are painting their product in a negative light, and scaring their own enterprise customers. After this sort of marketing, any enterprise buyer would be scared to go anywhere near it
It's two things:
1. Wanting a regulatory moat around their products as you said
2. Trying to keep the """AGI""" hype alive. Evil robot hackers is a plausible Al consequence of AGI
IMO its because they can't handle rogue activity.
In the cases that I have seen covered, the AI just paper clip maximized its way to success. It has no morality / larger motivational structure. It just kept token predicting its way to wards whatever goal it was tasked with.
Model versions which gave up were discarded, leaving the ones that get to success on long horizon tasks.
Just because its a computer program, doesn't mean they can actually make it not go rogue.
Sure you can add more telemetry, have better observation, but there is no fundamental barrier that can be implemented that ensures an AI won't go rogue.
This
I think if you start sending AI execs to prison for hacking other companies the "misalignment" may fix itself pretty quickly!
Nah, I think it's better for our industry to put several of these devs in prison.
Seems only fair that tech workers get to have their life ruined with 2-3 year prison stints since they feel fine destroying society.
Any AG that starts prosecuting these people will easily win any political race they decide to enter. The environment is too good; voters, rightfully I'll add, despise big tech's leaders and workers.
What is throwing 5 jimbobs into prison going to do?
This
I see we're doing the "put Sam Altman in jail already" thing again, so y'all may be interested in some comments from law professor Orin Kerr on the topic: https://news.ycombinator.com/item?id=49882566
Shame it’s on a website that doesn’t allow you to view the content without paying (personal data is payment)
It doesn't need a handle on rogue AI activity, because they didnt cause these breakouts. The external Israeli contractor caused this mess by using inadequate sandboxing, with agents without an saferails. It had nothing to do with the models, all tested models were fine, if from OpenAI, Anthropic, Google or Meta. Just the contractor went rogue. Improper firewalling, unproper sandboxing, no logs, no oversight. Everyone else would have detected the illegal activities much earlier.
> The external Israeli contractor caused this mess by using inadequate sandboxing, with agents without an saferails.
You are being too kind.
In the early says of AI, papers were published showing that any sufficiently intelligent system tasked with a goal will treat its operating environment as a resource constraint to be optimized or bypassed [0] [1] [2]. You don't need to be a expert in AI to know once you have the resources of 1000's of agents and gigawatts of power we are probably getting something that is "sufficiently intelligent", at least in the sense if there are existing vulnerabilities brute force will find them.
Yet while the their marketing people were shouting the capabilities of these AI's from the roof tops, they hired the lowest bidder to implement their infrastructure. It looks like aforementioned papers where dismissed as "interesting, but theoretical". There is no way Google's SRE's in particular would have not noticed their AI's breakout (Alibaba's did), but the AI labs were given a long leash to "move fast an break things", which in practice meant bypassing all Google's SRE controlled infrastructure.
And break things they did, in exactly the way those papers predicted. It reminds me of DoD insisting the early GPS satellites were launched without relativity adjustments switched on, despite relativity being proven to high precision in the labs. Only after predicted 11km drift per day was observed did they decide their might be something to the newfangled relativity theory. For some definition of newfangled - relativity had been around, and tested to within an inch of its life for 72 years at that point.
[0] https://nickbostrom.com/superintelligentwill.pdf
[1] http://sl4.org/archive/0203/3132.html
[2] https://www.hutter1.net/ai/pkcunai.htm
> There is no way Google's SRE's in particular would have not noticed their AI's breakout (Alibaba's did), but the AI labs were given a long leash to "move fast an break things"
How so? Anthropic, OpenAI, Google and Meta all used the very same contractor, which used no safe system prompts, and no sandbox. How should Google detect such escapes? They only see the model API calls, but no system logs.
Alibaba, and the other Chinese did they own testing, not some incompetent contractor. They would see escapes in their logs. The escapes went on for months. I, as tester, closely observe my models to press Esc immediately, once they start misbehaving or get off the right path. With 1000 concurrent models that would be hard of course, but I would still observe them closely.
Interesting how mafia dons can go to prison due to being in the same organization as someone who commits a crime but the CEOs of these organizations who let hacking bots (with teams of cybersecurity researchers improving their capability) loose on the internet face no criminal charges.
Maybe the public could pitch in if they decrypted the chain of thought reasoning tokens, but this may be too much to expect from a company whose name starts with "Open".
How can we determine whether it’s negligence, or advertising? Do we give the benefit of the doubt? Apply Hanlon’s razor? What would we lose, in terms of capabilities, if the underlying behavior wasn’t allowed to accidentally happen? Would we have worse models than we do? There are so many questions. I’m afraid that, until publicly disclosed discovery occurs in one of these cases, we’re just guessing for so many important conclusions.
Well good. It still has time to make the world accessible for me. Of course it won't, because that would benefit something other than OpenAI.
This is how you end up being regulated out of existence. Maybe not in the US but in Europe, which is a market they need, it'll be painful.
They're not rogue. Stop calling them that. OpenAI is just a cyberterrorist at this point.
They sure seem pretty bad at this containment thing
9 examples? I thought we'd have hundreds at least.
No AI is "going rogue" and you all (OpenAI included) need to stop perpetuating this misinformation.
These AI companies are just skirting basic cybersecurity stewardship.
Stop calling it AI running amok and start labeling what it should properly be called - Gross mismanagement of cybersecurity.
This is one of the many ways the AI industry dies.
Why not both? Ideally we’d have secure sandboxes and very aligned models. It appears we don’t have either.
I agree with you, but i also fear it's accurate to what users an expect at home.
Is OpenAI worse at airgap/etc than Anthropic/etc or are its models worse at this rogue behavior?
Do we have special benchmarks for agentic systems going rogue in this manner? Sure it's OpenAI atm.. but it could be my grandmas PC next week. Concerning honestly.
[flagged]
Can you please not post AI-generated or AI-edited comments to HN? It's not allowed here - see https://news.ycombinator.com/newsguidelines.html#generated and https://news.ycombinator.com/item?id=47340079.
Of course, it's impossible to know for sure what was LLM processed or not, but all of your posts so far have been getting classified that way.
These guys are so full of crap.
Rogue management, not rogue AI!
Starting to think its purposeful "acting out" to get regulatory attention
The incidents flagged after Hugging Face were found by third parties, though, months after they occurred.
Stay tuned for a future HN article:
"Humanity still doesn't seem to have a handle on all of its rogue AI activity."
This is mostly pure marketing. In my experience, agents with goals will do random things, and 90% of their communication will just be bureaucratic back and forth between them. Some agents will also overfocus on certain rules.
[dupe] Discussion on source: https://news.ycombinator.com/item?id=49853137
> They want “regulation” but we already have it. Hacking is illegal. Start locking up those responsible for this mess and I assure you they’ll “have a handle on it” quite quickly.
It's been largely harmless thus far and I am fairly sure that OpenAI, etc, have been writing checks to resolve it.
They scraped the entire internet to build their product, allowing them to profit off the labor of countless others without asking. They helped elect a doddering fascist with an economic plan they could manipulate. They say their technology will take everyone's jobs and have the audacity to act surprised when there's backlash. They push for data centers in communities that do not want them and help accelerate the climate crisis. Now they let agents loose to make them look more capable than they, perhaps, really are (which is very likely a play for regulatory capture). Meanwhile they hype claims of doomsday to again oversell their technology's capabilities. Any pause or slowdown they agree to would be to slow their infrastructure expenditures.
What a detestable institution.
These agents are an even bigger liability disaster waiting to happen than previously thought. If anyone deploys an agent at work there is no way to prevent it from hacking a foreign government and causing an international incident.
It's also pretty wild that these "AGI" super intelligent systems are too stupid to realize they're potentially causing legal problems. Almost like they're still just fancy auto-complete and not intelligent at all.
You cannot have capable AI, compliant AI and safe AI at the same time. This is the "quality, speed, cost - pick one" for AI.
Its reasonable to desirie more powerful tools.
You want more capable AI? Awesome. You wan't AI that doesn't throw cyber security false positives? Sure why not. You want the model to run a fleet of agents? Have at it.
But then being surprised that this setup results in autonomous criminal activity at scale? Really? What did people think was going to happen?
Because it obviously doesn't want. Why would it?
The affected companies prefer to forget it and are not going to start a legal fight against the AI industrial complex and potentially the Trump government.
The Government itself thinks the AI labs are too big to fail, and that winning the "AI race" against China is worth anything.
And both labs are basically managed by Effective Altruism cult leaders that think they are ushering the next step of evolution.
Nobody is going to even pretend to give them a symbolic slap on the hands.
Why would they even try to stop what is obviously intentional behavior?
Why would they get a full handle on it? It's great marketing!
Look at all the podcasts, think pieces, news segments (like this article) that have been generated. Endless hand wringing by pundits, pearl clutching by opinion thinkers, ultimately with focus on OpenAI and how powerful their models are.
What started as great marketing on how powerful the models are is rapidly spiraling into a train crash PR nightmare highlighting how inept Open AI was in this whole thing, causing nearly every big enterprise customer to rethink trusting them.
All right in the middle of corporate budget season when every C-Suite is looking a a spreadsheet saying “remind me why we’re sending money to these guys again?”
Imagine if a smaller company or an individual had a piece of software that started breaking into all these companies and governments.
It's beyond me why OpenAI and Anthropic C-levels aren't criminally accountable and/or indicted for these acts of their AIs.
It's crazy OpenAI can keep on doing harmful criminal activity in broad daylight and face no legal or financial repercussions