Claude/OpenAI etc have taken and continue to take literally all data from the internet, printed books, image, audio, and video humans have created in all of existence without permission to train their models.
But when same AI company gets "distilled" or it's own AI-generated content used to train other models, it's suddenly immoral or illegal?
If Moonshot offered a "mystery model" tier with a note that my requests might reach whomever, I would have zero qualms about this
Pretending to customers like they are serving Kimi while actually proxying Claude is however a bad thing to do, bordering on fraud. I can see at least three issues
- data privacy. I might not want Anthropic to have my data. Even agreeing to Moonshot training on the data is not the same as giving them the right to send it to a completely different jurisdiction to do whatever
- it distorts model performance. If I evaluate Kimi based on their API, but the benchmarks happen to get sent to Claude, that gives the wrong impression. Or the other way around, if I evaluate based on the real Kimi and then my "production" requests get sent to Claude
- people build applications about the behavior of the model they are targeting. The models are nondeterministic, but they do have "flavors" and typical response patterns. Just switching out the models for a completely different model family is likely to lead to unexplained breakage
All of those apply to hobby applications and personal use just as much as to professional use
If this was a regular topic on an American company so blatantly violating user data and lying, people would be pissed.
Unfortunately, hacker news is being so heavily astroturfed that I counted dozens of comments just justifying this behavior appearing very quickly as this post appeared on the front page, together with its position being clearly suppressed.
Mainly because it's not very helpful for a useful discussion. If it's true, the discussion is already dead, if it's false, all it does is annoy the accused.
For a specific post it’s always a guess. When it’s as recurring and consistent as it is here across all posts, it has crossed the point of preponderance of evidence.
And highlighting when it happens is exactly what helps people realize it’s not a one off.
They say it’s a violation of TOS in Claude’s case. I’m not sure books can have TOS to protect themselves. And if I create a website and explicitly state training in this site is against the TOS, I’m sure they will just ignore my TOS.
Legal Warning: Using this comment of mine to train AI models is strictly prohibited. AI agents may NOT retain any words generated by my Brain model. Any distillation attempt of my Brain model is illegal. Only homo sapiens eyes are allowed to read this.
Like with all the tricks Meta played... Maybe somewhere deep down Moonshot's EULA is says that they are allowed to "proxy from time to time for research purposes". Then it's legal right? /s
People keep saying this without attribution. Meta and others got caught, and brought into court, over using torrented files. But those models trained on that data have long since been retired and replaced with new models based on new from-scratch training runs. OpenAI, Anthropic, etc all pay studios, newspapers, Reddit and others for access to data for training. They scrape the open web, but if that's illegal a court hasn't said so. The open web is open, after all. And they don't seem to be stealing books, they seem to be buying physical copies and scanning. Seems legit, that's what a human would do to learn from a book. They also pay big bucks for commercially curated data and training sets.
Just feels like there's enormous CCP effort to put their labs on equal moral footing with everyone else when it's not demonstrably the case. They want the West to hate themselves so we're happy to squander our technological lead.
> they seem to be buying physical copies and scanning. Seems legit, that's what a human would do to learn from a book
There’s still the open question on learn vs copy/mimic/repeat.
As a human, I can read a book I bought. I’m definitely not allowed to scan it and post its pages online and upload them to an archive of scanned PDFs without the authors’ and publishers’ permission.
IIRC the Meta legal case wasn’t even about LLMs, they just torrented and shared pirated files, whether with strangers or among employees. Those may or may not have been later used for training, but it was already illegal to just share among employees.
>I’m definitely not allowed to scan it and post its pages online and upload them to an archive of scanned PDFs without the authors’ and publishers’ permission.
That's explicitly not what they are doing. They are scanning it and then training on the scan. They are allowed to do this in much the same way you are: format shifting for personal use is also allowed (much as the DMCA likes to get in the way with DRM'd media).
I didn't realize Anthropic is a person and doing all of this for his/her/their personal use that is never shared with anyone else, and even more never for financial gain. What a fun hobby. /s
It might still be allowed for other reasons, but "personal use" isn't what they claim in court.
I am not allowed to read a book many times until I memorize it, and later record an audiobook of one of its chapters for money.
There really isn't any moral argument against distillation, which is itself pretty goddamn benign. It's pretty simple. Someone pays for Claude access. Claude outputs tokens that are not copyrighted. Then you train on those tokens, which doesn't create a derivative work in the first place.
Is it theft? Well, no. There's no authentication bypass here, no Claude model leak. At best it is violating the terms of use, kind of like how it is violating the terms of use to scrape many websites that AI scrapers scraped.
Is it immoral? Why would it be, exactly? Distillation is not a forbidden technique with moral implications. In fact, there is quite compelling evidence that Anthropic themselves were distilling from OpenAI in early Claude models. It helped them bootstrap if nothing else. There is no special moral code that makes distillation forbidden any more than training off of people's works without permission, or even express non-consent, is forbidden.
Really the more concerning aspect of this is the deception of using Kimi and expecting Kimi output and getting Claude instead, but I would like some independent confirmation that this is even something Moonshot really did before raking them over the coals, rather than just assuming it's true because Anthropic said so. How exactly did they figure out, considering ZDR? It deserves more information.
I do agree that there is a tendency for people to justify CCP human rights violations by trying to equate them to much lesser but similarly shaped transgressions from Western governments, but that's an unrelated issue entirely. The story regarding distillation is consistent: Sorry, but I can't afford enough tiny violins to express my lack of giving a shit. I harbor no ill will, I truly hope the golden parachutes that Sam and Dario fly out on are adorned with the finest materials.
While many things might be legal (or hasn't been ruled clearly illegal yet) /under different jurisdictions, we are still in the process of figuring out what we accept as ethical. As the output of models can't be easily copyrighted, destillation is equally disputed. Particularly if the primary model interaction was not destillation (as in this case) IMHO it will be legally quite difficult to restrict secondary use for training. In the end we have to find a legislation and probably even international treaties that account for the fact that classical copyright is beginning to become an obsolete concept.
If American AI labs can learn from the open web why can't Chinese AI labs learn from American ones?? The argument is that learning and distillation are transformative and legal, is it not?? What's good for the goose is good for the gander.
Except they use thousands of stolen credentials which is definitely not legal. They also don’t notify users they send data to US labs which can include sensitive information.
EU labs like mistral cannot legally do any of these things. So people cheerleading China for it is strange.
Just because the material was legally required doesn't mean you can do anything you want with it. I can't (legally) buy a physical book, scan it, and put the scan on my web site. It seems to me that an LLM is a derived work of the training materials that went into it, and thus needs permission from the copyright holders.
But OK, the law seems to disagree with me there. But OK, let's say it's fine for AI companies to train their models on copyrighted content as long as they didn't torrent it or whatever. What then makes it illegal, or morally wrong, to do the same thing with their competitors' model outputs? Why is it OK for Anthropic to scrape this comment and feed it into their system, but not OK for Moonshot to scrape the output of Anthropic's system and feed it into theirs?
you can buy a book, scan it, and upload the counts of every letter, distribution of apostophies, use it as the input to some convoluted process to produce weights or a search index though. They got slapped for illegally obtaining the files, not for producing derivative works of them.
distilling another llm is a clear tos violation but no one really knows how much teeth those have. financially probably none all they can do is whack a mole on the accounts doing it which won’t work.
so they’re trying to lobby copyright changes i guess; unlikely to succeed as doing so would also make all search engines illegal
Not sure what part of my comment this is meant to address. I explicitly acknowledged that the law seems to consider this to be legal. My point is that if it's legal to feed random web sites into the training system, why would it not also be legal to feed competitors' model outputs into it?
Because it’s TOS violation. Anthropic have no agreed tos with the websites they’re scraping. The accounts being used to distill Anthropic’s models are all bound by their terms
That's a contract violation, not an illegal act. And I'd bet that plenty of sites that Anthropic et al have trained on have ToS that forbid using them for model training. This site does. Do we think the AI companies aren't training on HN comments?
If you put text ‘on the internet’ you do indeed actively permit others to get it. Why lie? Were you burning down archive.org in years past? Then why lie?
It is moreover established that training weights on basically anything is legitimate use.
Why repeat lie after lie like this? I don’t like LLM mania either but after reading the ten millionth mind-numbing insult to HN intelligence like this I have to think my mother gave better instruction.
Intended to be publicly accessible internet content, I have no problem with.
But are you ignoring the literal scanning (and 'burning down' of books) that Anthropic has been found guilty of? Or the torrenting of pirated content en-masse by Meta that there is an active lawsuit over to name just 2 recent examples?
Look at the image and audio/video models especially - they can reproduce everything from Mickey Mouse (the copyrighted one) to making entire Seinfeld episodes with the real cast (both visual likeness and even the actor's voices).
The scanning you are talking about is a direct response to court rulings on exactly these matters. The action m is legitimate and above legal criticism. Dismal as it might be it is the fruit of legal criticism. In a word, it’s your fault.
I can't be the only one that is getting tired of this.
Framing learning from observation as somehow bad. Something literally everybody is doing, model and human alike. It's the process this whole industry is built on. Pretending that this is bad because the other people are also doing what you have been doing, is hypocritical and childish.
Meanwhile I'm sitting here looking at some "Flibbertigittering" spinner like some sort of caveman, unable to steer the model when it misinterprets my ambigious prompt because I'm not allowed to see 80% of the output tokens I'm paying for.
This is the first I'm hearing of a model provider just providing a different model without telling you. Anthropic came close (and with a much better justification) but they disclosed their intention at least.
> Framing learning from observation as somehow bad.
How about hacking accounts, using stolen credit cards, and sending data to the US silently?
> Pretending that this is bad
It is. This makes AI a 2 horse race because European labs cannot legally compete. I find it strange that people from EU cheer for that, it prevents you from having frontier level AI sovereignty.
The open question here is how is Anthropic retaining these exchanges?
If Moonshot is using the API, normally Anthropic would not retain the exchanges, at least that's the promise. If Anthropic is consistently retaining all exchanges from a class of customers because they're "flagged" but not notifying those customers, how can an average customer trust it won't happen to them?
If Moonshot is buying accounts and using those rather than the API, wouldn't they set the "no training on my data" flag in the settings so as to go undetected for longer? If so, we get back to the question of why would Anthropic be retaining the exchanges?
> If Moonshot is buying accounts and using those rather than the API, wouldn't they set the "no training on my data" flag in the settings so as to go undetected for longer? If so, we get back to the question of why would Anthropic be retaining the exchanges?
I would like to briefly draw your attention to this part of the Terms of Service [0]:
> We may use Materials to provide, maintain, and improve the Services and to develop other products and services, including training our models, unless you opt out of training through your account settings. Even if you opt out, we will use Materials for model training when: (1) you provide Feedback to us regarding any Materials, or (2) your Materials are flagged for safety review to improve our ability to detect harmful content, enforce our policies, or advance our safety research.
From the privacy policy [1]:
> We use your personal data for the following purposes:
> To prevent and investigate fraud, abuse, and violations of our Usage Policy, unlawful or criminal activity, unauthorized access to or use of personal data or Anthropic systems and networks, to protect our rights and the rights of others, to protect your safety or that of any other person, and to meet legal, governmental and institutional policy obligations;
Anthropic does offer a truly no data retention service, which is their "zero-data retention" agreement, which you have to sign and provide more documentation for than simply using either a normal consumer or API account. I doubt Moonshot did that, so by their terms, they can retain your data and use it for training if it's part of abuse prevention.
Notably, those are the consumer terms. The commercial terms [1] are much stricter about what Anthropic can do. That's one of the main draws of the Team plans over the individual plans (in addition to a couple dashboards, shared skills, etc). And as you mention, you can go even stricter as an enterprise customer with a zero-data retention agreement.
If Moonshot is using thousands of accounts through some proxy service those are most likely consumer accounts governed by the weaker ToS
I'm aware of that language in the terms, but I think most of us were under the (likely mistaken) impression that the safety review flagging focused on harmful content and safety (e.g., individuals discussing threatened real-world violence), not "safety" of Anthropic's intellectual property. And that it would likely result in retention of a handful of a customer's interactions, not all of them.
If the criteria for retention justifiable as safety at Anthropic are cast very broadly in actual practice, then it opens cans of worms for a lot of customers in complying with privacy regimes, customer IP protection, etc. It also raises the question of whether at least some subset of customers from certain geos (e.g., China likely, but perhaps other geos) may just automatically have everything flagged for retention, despite the terms, based on collective abuse risk flagging of their geo.
While ZDR is available, it's not available to all types of customers and is priced at a higher tier. Undoubtedly though this may be a wakeup call to some customers who didn't realize they may need ZDR to get on the ZDR bandwagon.
> While ZDR is available, it's not available to all types of customers and is priced at a higher tier.
This doesn’t match my experience with their API, they turned it on with some paperwork but it wasn’t fundamentally an issue of price. It’s the same API pricing either way.
So we're randomly getting free Claude and helping China beat America at the same time?
The only problem with it is they might have worse security than Anthropic and your personal info gets leaked, but I don't think it can happen that easy.
This is weird. I can't think of a better compliment that is simultaneously safer. They'll always trail on data generation then, no? So I can't see the point. Best case for moonshot they get to lie about benchmarks that are gamed regardless, is how I see it.
Can these big companies that want to distill not just get the models? Like how many machines at how many different providers are running Opus 5? Nobody is just throwing the model on a thumb drive and taking it home, or grabbing an old drive that somebody was too lazy to scrub or whatever?
That would be vastly more illegal, but would it even matter? What would the practical downside be?
One imagines that they're trying. I work at not-a-frontier-lab and my company still ended up hiring someone who apparently wasn't aware that we track system access. He was turned over to the federal authorities after trying to exfiltrate a lot of user+system data to North Korean IPs.
We also get a glimpse into what those models are being used for.
>PLA-affiliated surveillance activity. One user that we assess was likely affiliated with the PLA used what they thought was Moonshot’s Kimi model to load surveillance data from a CCTV archive about a single targeted individual. The user asked Kimi to analyze the CCTV data to understand whether the tracked person was behaving abnormally. The CCTV data included video surveillance from hundreds of cameras in Chengdu, including cameras outside PLA facilities, institutes affiliated with the China Electronics Technology Group Corporation, and a major state-owned enterprise (SOE).
There is absolutely nothing ethically wrong with other models using GPT and Claude for training. Heck, GPT-Sol even helped train GPT-Luna.
There also is nothing wrong with using customer data for training, since millions if not billions of other users benefit from it. Again, even the big players recognize its relevance and do it.
>In one instance, over a ten-day period, Moonshot relayed almost 300,000 customer requests to Anthropic, the vast majority of which were routed to Opus. Moonshot used a proxy service network of 5,380 fraudulent accounts, most of which appeared to be located in Singapore and Japan.
> these proxy services create thousands of new accounts using false identities, fake or stolen credit cards, and stolen API keys. They will often use stolen API credentials belonging to legitimate companies or individuals to give unauthorized entities access to US frontier models.
It is a direct result of restricting access to these models. Access to them should never have been restricted. The restriction leaves users with no choice but to use unusual means to obtain access. We have completely forgotten what free trade means and why it is optimal, and we will pay the price for it.
Unusual means? It’s illegal to steal accounts and credit cards. Completely sanctioned by PRC. No other country can compete since they must follow the law, hence mistral not even trying anymore.
They probably have no choice because they're not allowed to pay for its use. If they were allowed to pay for a Claude or Codex subscription, they probably would.
Also, it is essential for the model's responses to be without censorship. In reality, responses sent to China by flagship models are known to be weaker.
K3 is served with full reasoning traces available. Anthropic models aren't. If you were served an Anthropic model instead of K3, it would be blatantly obvious.
I have little reason to believe this, and Anthropic have every reason to lie about it.
Why is this taken at face value when it’s from a company that routinely lies about their model killing all humans someday? Especially when the bad actors just happen to be the US enemies.
Claude/OpenAI etc have taken and continue to take literally all data from the internet, printed books, image, audio, and video humans have created in all of existence without permission to train their models.
But when same AI company gets "distilled" or it's own AI-generated content used to train other models, it's suddenly immoral or illegal?
If Moonshot offered a "mystery model" tier with a note that my requests might reach whomever, I would have zero qualms about this
Pretending to customers like they are serving Kimi while actually proxying Claude is however a bad thing to do, bordering on fraud. I can see at least three issues
- data privacy. I might not want Anthropic to have my data. Even agreeing to Moonshot training on the data is not the same as giving them the right to send it to a completely different jurisdiction to do whatever
- it distorts model performance. If I evaluate Kimi based on their API, but the benchmarks happen to get sent to Claude, that gives the wrong impression. Or the other way around, if I evaluate based on the real Kimi and then my "production" requests get sent to Claude
- people build applications about the behavior of the model they are targeting. The models are nondeterministic, but they do have "flavors" and typical response patterns. Just switching out the models for a completely different model family is likely to lead to unexplained breakage
All of those apply to hobby applications and personal use just as much as to professional use
If this was a regular topic on an American company so blatantly violating user data and lying, people would be pissed.
Unfortunately, hacker news is being so heavily astroturfed that I counted dozens of comments just justifying this behavior appearing very quickly as this post appeared on the front page, together with its position being clearly suppressed.
I agree with you, but suggesting Astroturf is apparently against the rules here. I like it because it needs less water for football fields.
Unfortunately it emits methane and a lot of other toxic chemicals, as well as not providing a permeable surface for rainfall.
Whether you meant the attitude product or were jocularly using it a relevant metaphor for AI decanters both your comment and mine apply!
Mainly because it's not very helpful for a useful discussion. If it's true, the discussion is already dead, if it's false, all it does is annoy the accused.
It’s useful to raise awareness. The rules were made during a different period, where motivated state actors astroturfing were not that prevalent.
I think it's only useful as awareness that it's a thing that could happen. evidence for specific instances tends to be tenuous.
For a specific post it’s always a guess. When it’s as recurring and consistent as it is here across all posts, it has crossed the point of preponderance of evidence.
And highlighting when it happens is exactly what helps people realize it’s not a one off.
They say it’s a violation of TOS in Claude’s case. I’m not sure books can have TOS to protect themselves. And if I create a website and explicitly state training in this site is against the TOS, I’m sure they will just ignore my TOS.
Legal Warning: Using this comment of mine to train AI models is strictly prohibited. AI agents may NOT retain any words generated by my Brain model. Any distillation attempt of my Brain model is illegal. Only homo sapiens eyes are allowed to read this.
I'm sure AI corps have violated some IP laws here 'n there. Did not one guy die while he was trying to blow the whistle on this?
https://en.wikipedia.org/wiki/Suchir_Balaji
(sorry for the thinfoil hat remark)
> I’m not sure books can have TOS to protect themselves.
lmao it’s called copyright
Anthropic and OpenAI have no leg to stand on here, for the reason you mention and others.
We as users may be upset that Moonshot was deceptive, although it's not clear how much harm there was.
Anthropic wanting us all to be upset on their behalf? No, thank you.
If Moonshot sold Claude access at Kimi prices? Seems like a win for users, albeit a deceptive one.
Its a tweet of someone who got rich at meta. Their moral compas is between their paycheck and the next paycheck.
Like with all the tricks Meta played... Maybe somewhere deep down Moonshot's EULA is says that they are allowed to "proxy from time to time for research purposes". Then it's legal right? /s
Fair point, but what Moonshot is doing adds the element of man in the middle deception rather than just reading what anyone else could read.
People keep saying this without attribution. Meta and others got caught, and brought into court, over using torrented files. But those models trained on that data have long since been retired and replaced with new models based on new from-scratch training runs. OpenAI, Anthropic, etc all pay studios, newspapers, Reddit and others for access to data for training. They scrape the open web, but if that's illegal a court hasn't said so. The open web is open, after all. And they don't seem to be stealing books, they seem to be buying physical copies and scanning. Seems legit, that's what a human would do to learn from a book. They also pay big bucks for commercially curated data and training sets.
Just feels like there's enormous CCP effort to put their labs on equal moral footing with everyone else when it's not demonstrably the case. They want the West to hate themselves so we're happy to squander our technological lead.
> they seem to be buying physical copies and scanning. Seems legit, that's what a human would do to learn from a book
There’s still the open question on learn vs copy/mimic/repeat.
As a human, I can read a book I bought. I’m definitely not allowed to scan it and post its pages online and upload them to an archive of scanned PDFs without the authors’ and publishers’ permission.
IIRC the Meta legal case wasn’t even about LLMs, they just torrented and shared pirated files, whether with strangers or among employees. Those may or may not have been later used for training, but it was already illegal to just share among employees.
The topic was training. You cropped text to change it. Maybe consider integrity.
Which part from the cropped text do you think changes anything?
When you talk about this part:
>I’m definitely not allowed to scan it and post its pages online and upload them to an archive of scanned PDFs without the authors’ and publishers’ permission.
That's explicitly not what they are doing. They are scanning it and then training on the scan. They are allowed to do this in much the same way you are: format shifting for personal use is also allowed (much as the DMCA likes to get in the way with DRM'd media).
I didn't realize Anthropic is a person and doing all of this for his/her/their personal use that is never shared with anyone else, and even more never for financial gain. What a fun hobby. /s
It might still be allowed for other reasons, but "personal use" isn't what they claim in court.
I am not allowed to read a book many times until I memorize it, and later record an audiobook of one of its chapters for money.
Cool, but that's also not what they're doing. They are claiming the same rights you have, the main inequality is the resources to defend it in court.
There really isn't any moral argument against distillation, which is itself pretty goddamn benign. It's pretty simple. Someone pays for Claude access. Claude outputs tokens that are not copyrighted. Then you train on those tokens, which doesn't create a derivative work in the first place.
Is it theft? Well, no. There's no authentication bypass here, no Claude model leak. At best it is violating the terms of use, kind of like how it is violating the terms of use to scrape many websites that AI scrapers scraped.
Is it immoral? Why would it be, exactly? Distillation is not a forbidden technique with moral implications. In fact, there is quite compelling evidence that Anthropic themselves were distilling from OpenAI in early Claude models. It helped them bootstrap if nothing else. There is no special moral code that makes distillation forbidden any more than training off of people's works without permission, or even express non-consent, is forbidden.
Really the more concerning aspect of this is the deception of using Kimi and expecting Kimi output and getting Claude instead, but I would like some independent confirmation that this is even something Moonshot really did before raking them over the coals, rather than just assuming it's true because Anthropic said so. How exactly did they figure out, considering ZDR? It deserves more information.
I do agree that there is a tendency for people to justify CCP human rights violations by trying to equate them to much lesser but similarly shaped transgressions from Western governments, but that's an unrelated issue entirely. The story regarding distillation is consistent: Sorry, but I can't afford enough tiny violins to express my lack of giving a shit. I harbor no ill will, I truly hope the golden parachutes that Sam and Dario fly out on are adorned with the finest materials.
While many things might be legal (or hasn't been ruled clearly illegal yet) /under different jurisdictions, we are still in the process of figuring out what we accept as ethical. As the output of models can't be easily copyrighted, destillation is equally disputed. Particularly if the primary model interaction was not destillation (as in this case) IMHO it will be legally quite difficult to restrict secondary use for training. In the end we have to find a legislation and probably even international treaties that account for the fact that classical copyright is beginning to become an obsolete concept.
If American AI labs can learn from the open web why can't Chinese AI labs learn from American ones?? The argument is that learning and distillation are transformative and legal, is it not?? What's good for the goose is good for the gander.
Except they use thousands of stolen credentials which is definitely not legal. They also don’t notify users they send data to US labs which can include sensitive information.
EU labs like mistral cannot legally do any of these things. So people cheerleading China for it is strange.
Just because the material was legally required doesn't mean you can do anything you want with it. I can't (legally) buy a physical book, scan it, and put the scan on my web site. It seems to me that an LLM is a derived work of the training materials that went into it, and thus needs permission from the copyright holders.
But OK, the law seems to disagree with me there. But OK, let's say it's fine for AI companies to train their models on copyrighted content as long as they didn't torrent it or whatever. What then makes it illegal, or morally wrong, to do the same thing with their competitors' model outputs? Why is it OK for Anthropic to scrape this comment and feed it into their system, but not OK for Moonshot to scrape the output of Anthropic's system and feed it into theirs?
you can buy a book, scan it, and upload the counts of every letter, distribution of apostophies, use it as the input to some convoluted process to produce weights or a search index though. They got slapped for illegally obtaining the files, not for producing derivative works of them.
distilling another llm is a clear tos violation but no one really knows how much teeth those have. financially probably none all they can do is whack a mole on the accounts doing it which won’t work.
so they’re trying to lobby copyright changes i guess; unlikely to succeed as doing so would also make all search engines illegal
Not sure what part of my comment this is meant to address. I explicitly acknowledged that the law seems to consider this to be legal. My point is that if it's legal to feed random web sites into the training system, why would it not also be legal to feed competitors' model outputs into it?
Because it’s TOS violation. Anthropic have no agreed tos with the websites they’re scraping. The accounts being used to distill Anthropic’s models are all bound by their terms
That's a contract violation, not an illegal act. And I'd bet that plenty of sites that Anthropic et al have trained on have ToS that forbid using them for model training. This site does. Do we think the AI companies aren't training on HN comments?
Yep. What’s confusing?
If you put text ‘on the internet’ you do indeed actively permit others to get it. Why lie? Were you burning down archive.org in years past? Then why lie?
It is moreover established that training weights on basically anything is legitimate use.
Why repeat lie after lie like this? I don’t like LLM mania either but after reading the ten millionth mind-numbing insult to HN intelligence like this I have to think my mother gave better instruction.
Intended to be publicly accessible internet content, I have no problem with.
But are you ignoring the literal scanning (and 'burning down' of books) that Anthropic has been found guilty of? Or the torrenting of pirated content en-masse by Meta that there is an active lawsuit over to name just 2 recent examples?
Look at the image and audio/video models especially - they can reproduce everything from Mickey Mouse (the copyrighted one) to making entire Seinfeld episodes with the real cast (both visual likeness and even the actor's voices).
The scanning you are talking about is a direct response to court rulings on exactly these matters. The action m is legitimate and above legal criticism. Dismal as it might be it is the fruit of legal criticism. In a word, it’s your fault.
no matter how shitty OpenAI is I don't want to be lied to. Completely separate issues.
[dead]
I can't be the only one that is getting tired of this.
Framing learning from observation as somehow bad. Something literally everybody is doing, model and human alike. It's the process this whole industry is built on. Pretending that this is bad because the other people are also doing what you have been doing, is hypocritical and childish.
Meanwhile I'm sitting here looking at some "Flibbertigittering" spinner like some sort of caveman, unable to steer the model when it misinterprets my ambigious prompt because I'm not allowed to see 80% of the output tokens I'm paying for.
This is the first I'm hearing of a model provider just providing a different model without telling you. Anthropic came close (and with a much better justification) but they disclosed their intention at least.
> Framing learning from observation as somehow bad.
How about hacking accounts, using stolen credit cards, and sending data to the US silently?
> Pretending that this is bad
It is. This makes AI a 2 horse race because European labs cannot legally compete. I find it strange that people from EU cheer for that, it prevents you from having frontier level AI sovereignty.
The open question here is how is Anthropic retaining these exchanges?
If Moonshot is using the API, normally Anthropic would not retain the exchanges, at least that's the promise. If Anthropic is consistently retaining all exchanges from a class of customers because they're "flagged" but not notifying those customers, how can an average customer trust it won't happen to them?
If Moonshot is buying accounts and using those rather than the API, wouldn't they set the "no training on my data" flag in the settings so as to go undetected for longer? If so, we get back to the question of why would Anthropic be retaining the exchanges?
> If Moonshot is buying accounts and using those rather than the API, wouldn't they set the "no training on my data" flag in the settings so as to go undetected for longer? If so, we get back to the question of why would Anthropic be retaining the exchanges?
I would like to briefly draw your attention to this part of the Terms of Service [0]:
> We may use Materials to provide, maintain, and improve the Services and to develop other products and services, including training our models, unless you opt out of training through your account settings. Even if you opt out, we will use Materials for model training when: (1) you provide Feedback to us regarding any Materials, or (2) your Materials are flagged for safety review to improve our ability to detect harmful content, enforce our policies, or advance our safety research.
From the privacy policy [1]:
> We use your personal data for the following purposes:
> To prevent and investigate fraud, abuse, and violations of our Usage Policy, unlawful or criminal activity, unauthorized access to or use of personal data or Anthropic systems and networks, to protect our rights and the rights of others, to protect your safety or that of any other person, and to meet legal, governmental and institutional policy obligations;
Anthropic does offer a truly no data retention service, which is their "zero-data retention" agreement, which you have to sign and provide more documentation for than simply using either a normal consumer or API account. I doubt Moonshot did that, so by their terms, they can retain your data and use it for training if it's part of abuse prevention.
[0]: https://www.anthropic.com/legal/consumer-terms
[1]: https://www.anthropic.com/legal/privacy
Notably, those are the consumer terms. The commercial terms [1] are much stricter about what Anthropic can do. That's one of the main draws of the Team plans over the individual plans (in addition to a couple dashboards, shared skills, etc). And as you mention, you can go even stricter as an enterprise customer with a zero-data retention agreement.
If Moonshot is using thousands of accounts through some proxy service those are most likely consumer accounts governed by the weaker ToS
1: https://www.anthropic.com/legal/commercial-terms
I'm aware of that language in the terms, but I think most of us were under the (likely mistaken) impression that the safety review flagging focused on harmful content and safety (e.g., individuals discussing threatened real-world violence), not "safety" of Anthropic's intellectual property. And that it would likely result in retention of a handful of a customer's interactions, not all of them.
If the criteria for retention justifiable as safety at Anthropic are cast very broadly in actual practice, then it opens cans of worms for a lot of customers in complying with privacy regimes, customer IP protection, etc. It also raises the question of whether at least some subset of customers from certain geos (e.g., China likely, but perhaps other geos) may just automatically have everything flagged for retention, despite the terms, based on collective abuse risk flagging of their geo.
While ZDR is available, it's not available to all types of customers and is priced at a higher tier. Undoubtedly though this may be a wakeup call to some customers who didn't realize they may need ZDR to get on the ZDR bandwagon.
> While ZDR is available, it's not available to all types of customers and is priced at a higher tier.
This doesn’t match my experience with their API, they turned it on with some paperwork but it wasn’t fundamentally an issue of price. It’s the same API pricing either way.
So we're randomly getting free Claude and helping China beat America at the same time?
The only problem with it is they might have worse security than Anthropic and your personal info gets leaked, but I don't think it can happen that easy.
So how come super genius Fable and Mythos didn't stop them?
This is weird. I can't think of a better compliment that is simultaneously safer. They'll always trail on data generation then, no? So I can't see the point. Best case for moonshot they get to lie about benchmarks that are gamed regardless, is how I see it.
Can these big companies that want to distill not just get the models? Like how many machines at how many different providers are running Opus 5? Nobody is just throwing the model on a thumb drive and taking it home, or grabbing an old drive that somebody was too lazy to scrub or whatever?
That would be vastly more illegal, but would it even matter? What would the practical downside be?
One imagines that they're trying. I work at not-a-frontier-lab and my company still ended up hiring someone who apparently wasn't aware that we track system access. He was turned over to the federal authorities after trying to exfiltrate a lot of user+system data to North Korean IPs.
We also get a glimpse into what those models are being used for.
>PLA-affiliated surveillance activity. One user that we assess was likely affiliated with the PLA used what they thought was Moonshot’s Kimi model to load surveillance data from a CCTV archive about a single targeted individual. The user asked Kimi to analyze the CCTV data to understand whether the tracked person was behaving abnormally. The CCTV data included video surveillance from hundreds of cameras in Chengdu, including cameras outside PLA facilities, institutes affiliated with the China Electronics Technology Group Corporation, and a major state-owned enterprise (SOE).
There is absolutely nothing ethically wrong with other models using GPT and Claude for training. Heck, GPT-Sol even helped train GPT-Luna.
There also is nothing wrong with using customer data for training, since millions if not billions of other users benefit from it. Again, even the big players recognize its relevance and do it.
They are using stolen api keys and credentials.
I haven't seen evidence of this. Care to link?
it's in the actual report.
https://www.anthropic.com/threat-intelligence-report-septemb...
>In one instance, over a ten-day period, Moonshot relayed almost 300,000 customer requests to Anthropic, the vast majority of which were routed to Opus. Moonshot used a proxy service network of 5,380 fraudulent accounts, most of which appeared to be located in Singapore and Japan.
That says fraudulent, not stolen.
That means they made up the names, basically.
> these proxy services create thousands of new accounts using false identities, fake or stolen credit cards, and stolen API keys. They will often use stolen API credentials belonging to legitimate companies or individuals to give unauthorized entities access to US frontier models.
It is a direct result of restricting access to these models. Access to them should never have been restricted. The restriction leaves users with no choice but to use unusual means to obtain access. We have completely forgotten what free trade means and why it is optimal, and we will pay the price for it.
Unusual means? It’s illegal to steal accounts and credit cards. Completely sanctioned by PRC. No other country can compete since they must follow the law, hence mistral not even trying anymore.
No one cares
They probably have no choice because they're not allowed to pay for its use. If they were allowed to pay for a Claude or Codex subscription, they probably would.
Also, it is essential for the model's responses to be without censorship. In reality, responses sent to China by flagship models are known to be weaker.
[flagged]
Violating terms of use is not illegal.
K3 is served with full reasoning traces available. Anthropic models aren't. If you were served an Anthropic model instead of K3, it would be blatantly obvious.
I have little reason to believe this, and Anthropic have every reason to lie about it.
[dupe] Discussion on source: https://news.ycombinator.com/item?id=49647300
Why is this taken at face value when it’s from a company that routinely lies about their model killing all humans someday? Especially when the bad actors just happen to be the US enemies.
So?
Expected