When Timnit Gebru described LLMs as "stochastic parrots", she was making a deep point about what she believed to be the limitations of the technology. Her argument claimed that while we perceived legible text coming from ChatGPT, we were actually being fooled by our innate pattern-matching instincts. Notably, at the time she wrote this, it wasn't uncommon to see LLMs collapse into piles of gibberish, lending credence the the idea that there was never any actual meaning in the output other than what we readers brought to it.
It's safe to say that argument has collapsed utterly. Modern LLMs virtually never collapse into endless babbling loops. Far more importantly, they solve objectively hard problems. An LLM didn't merely inspire a mathematician to spot a counterexample to the Jacobian Conjecture that had been in front of his nose the whole time; it found the counterexample. These things are unquestionably generating meaningful outputs.
(That obviously doesn't mean that you can simply trust LLM outputs!)
At this point, trying to dunk on frontier models by calling them "stochastic parrots" is saying more about you than about the models. Moreover, it suggests you're not only out on a limb about the capabilities of LLMs, but also on what even the skeptic literature about it was saying.
You should know what "stochastic parrot" actually meant; you shouldn't be using the term just because you think it sounds snazzy.
It's interesting that you'd be choosing this canned phrase of all canned phrases, given just how hard that is linked to emotion-driven facebook comment fights that is.
This doesn't read like something that belongs into an engineering-driven discussion.
I see them collapse when faced unusual problems. They’ll go down a weird rabbit hole and then just keep going, even if their idea is completely irrational and not working.
They’re incapable of changing their mind, because they’re a Markov chain. They’re cool and they’re good at doing things, but they’re not sentient in the slightest.
Not at all the same idea. People also rathole into doomed approaches. What we're talking about here is literally LLMs producing line noise, or English words that don't fit into sentences.
I've seen this reflex quite a few times now. To diminish the impressive capabilities of the latest LLMs, one resorts to the assertion that they're not sentient, or not truely intelligent, or various other semantic diversions. No one claimed that. GP was all about usefulness.
Then GP misrepresented the "On the Dangers of Stochastic Parrots" paper, which spoke about meaning. But GP also said "meaning" several times, so no.
There's some haziness here about the difference between comprehensible output and comprehending. To put it mildly. I mean that's been the whole question for five or six years.
LLMs collapsing into piles of gibberish is a misrepresentation, too. The paper doesn't mention it and the argument doesn't hinge on it, or even benefit from it, really. But the other bit:
> Her argument claimed that while we perceived legible text coming from ChatGPT, we were actually being fooled by our innate pattern-matching instincts
This is an accurate summary, I think, and the point the authors are making sits somewhere in the coordinate space of a simplified reframing of the Chinese Room, Grice's account of intention in meaning, Wittgenstein's "Say 'It's cold here' and mean 'It's warm here'" challenge, and a whole bunch of the hermeneutic tradition.
Very roughly, you might say that an LLM doesn't mean anything when it emits text, but that the text still has meaning (which I think is all tptacek wants from "meaningful outputs", but you could take that further as culturally or scientifically 'meaningful', i.e. 'significant').
Philosophically, there are people who hold the position that something has to have been meant to have meaning, or that something has to have meaning in the first place to be meant, and so on. There were always interesting edge cases to these accounts (actors delivering lines, or the Dada/Burroughs cut-up technique), but LLMs are challenging in a way that e.g. Racter wasn't.
"Stochastic parrot" is a fitting term for this because the parrot's words have meaning, but the parrot itself doesn't mean anything by them. Not literally, anyway. And that's it. That's the whole analogy.
Have some people watered the term down to "LLM dumb"? Yes. Is that what Gebru meant in the first place? Not in my reading. Is it incorrect to use the term today? Depends, uh, on what you mean by it.
I really wish the article in question hadn’t used that phrase, because you’re right. But the rest of the article is also correct in the dangers facing us from this technology as an accelerant regardless, and I think it’s much more interesting to talk about that: but this entire comment thread will dunk on the stochastic parrot paragraphs instead.
The funny thing is though, I think it’s an in group signalling mechanism. You need the anti-AI crowd to know that you’re anti-AI, and this is a rapid way of doing so, to get it upvoted here.
> I mean, technically speaking, a video game from 1980 was likely to contain more logical branching points than a modern LLM's kernel, and was likely to be a lot more interesting to read.
Are you really saying that the "AI" in 1980 is _Really_ much more interesting than today's LLM? The author is either dishonest or ignorant.
The article is extremist nonsense where the author argues billions are going to die due to climate change in the next decades, and that Altman and Musk are hoping AI can until then replace human labor in order to keep their living standard in such a future.
It's not like the "stochastic parrot" thing was the only problem here.
LLM's have chat templates that sit at the lowest levels of how we feed tokens to them. It's quite integral to how they are trained to process tokens and rises up through the tech stack.
Go to a huggingface repo, look at the files, they have a literal chat_template.jinja
I don’t see people driving engines around, since that is impossible. You actually need the rest of the car around the engine to have it function as a drivable vehicle.
Of course. But most people don't buy an engine, ever. They don't have one laying around. They buy a whole car, every time.
So, since apparently I may need to clarify my actual point: For most people, an LLM-plus-agent-harness is just "LLM" or "AI". If you're trying to build one, you care about the distinction. Ditto if you're a pedant. But most people are not either of those things.
"a metaphor that frames large language models as systems that statistically mimic text without real understanding". I suppose you're defending them as having understanding. It's not "safe to say that argument has collapsed utterly", you're part of a campaign to reject the meme. On the other side, the meme is popular as a way to be rude about AI, because it's a cutting insult.
It means whatever you want it to mean, that's the best part. To me, it's best used as it was intended, but primarily in the contexts where the stochastic parrots are plowing through the ivory towers we built mostly for idiot savants.
I don't even think the claim (about what she was saying) is true. It's still used with the original meaning, which was not about being fooled into perceiving legible text, but fooled into perceiving understanding.
LLMs get better and better and do more and more things. But I still get the impression "their thinking isn't human thinking" by the way that their ability "understand" something shifts and fades as the discussion continues. And these can be things that it previously explained to me.
When I look at a person moving in front of me, I don't have to recheck if their bone structure is going shift as they more - it just won't. So, as a human, I don't naturally notice things of that sort. So I don't notice all the ways an AI video is wrong unless I concentrate. Which is saying that LLMs are wrong in ways we aren't accustomed to. They drift in their interactions and seeming understanding. Perhaps "random walk parrot" is better than "stochastic parrot" but I still say a word that communicates that they're "things that aren't quite doing what you think they're doing" is legitimate.
> It's safe to say that argument has collapsed utterly. Modern LLMs virtually never collapse into endless babbling loops. Far more importantly, they solve objectively hard problems.
And yet, they still output gibberish. They also still fail to solve objectively easy problems on a regular basis. That is because they are, indeed, still stochastic parrots without any actual understanding of anything or ability to reason about stuff. The argument hasn't collapsed at all.
What is exactly "actual understanding" . If it tells 1+1 = 2 it can perfectly explain why the result is 2. Or do you mean the actual transformer work during the 1+1 question?
LLMs are certainly stochastic, but they no longer meet any sane person's definition of "parrot." I can't imagine Gebru disagreeing. Has anyone bothered to ask her?
(Consider responding with an actual counterpoint, so we can all see what reasoning might give rise to such a delusion. Does your pet parrot do calculus and write stories and reverse-engineer Adobe applications?)
Okay, but as I just put it in another thread (which was in "new" and never made it anywhere near the front page):
All we have to do is assume that things progress in a linear way.
Do you remember GPT-2? It was a bizarre curiosity, but was nevertheless considered a huge advance in late 2019. Practically useless, but interesting.
It grew, and three and a half years later we got GPT-4, which was exceptionally capable on release.
It grew still further, and three and a half years hence we now have GPT-6. This is already more intelligent, more imaginative, more disciplined, and far more erudite than the vast majority of individual humans. If you have an intellectual skill -- if you are, say, a mathematician or a chemist -- GPT-6 also has that skill and is functionally your peer. And then it has many other skills that you lack; if nothing else, it is superhuman in breadth.
Assume a steady rate of progress. In three and a half years, we're going to have something that is genuinely a general superintelligence, and we're probably going to see recursive self-improvement begin to kick in (though subject to physical constraints on compute, etc.)
Now is the time for an RSI start-up, I guess!
Anyway, all you have to do is assume that things don't plateau. They have not plateaued to date; if anything, progress appears to be accelerating, and people are starting to panic that there are no brakes on this thing.
> This is already more intelligent, more imaginative, more disciplined, and far more erudite than the vast majority of individual humans. If you have an intellectual skill -- if you are, say, a mathematician or a chemist -- GPT-6 also has that skill and is functionally your peer. And then it has many other skills that you lack; if nothing else, it is superhuman in breadth.
Kinda chilling to see one of your kind in the wild.
<< people are starting to panic that there are no brakes on this thing.
There are.. physical limits, as it were, which effectively restrict upper limit, but I hear what you are saying. A lot of the problems thrown at it and issues found in the process only seem to have helped. I feels ages ago when we were arguing 'it is merely a stochastic parrot'.
This is going to sound crazy, but, at the time, they were spooked by Neal Stephenson's June 2019 book The Fall -- where the internet was broken by the open-source release of something akin to a language model that generated endless quantities of spam. They thought that GPT-2 might be that invention, so it worried them.
Then, a few years later, they learned that Google and Anthropic were planning on releasing LLM chatbots to the public, so they hastily launched GPT-3.5-based ChatGPT as a "dev preview" to get ahead of them. Their original plan was to finish training GPT-4 before launching anything.
Commercial considerations -- and public glory -- triumphed in the end.
I would disagree regarding the opening paragraphs about intelligence, but then we don’t have a generally agreed definition of intelligence. There is a scientific view that a lot of what the human brain does and has biologically evolved to is predicting what will happen next, and next-token-prediction is reasonably related to that.
However, what seems to still be lacking in LLMs — aside from plasticity (continuous learning), perception of time, and anything related to emotions and desires (which underly motivations) — is various types of awareness, and sound judgement. We are still far from a good understanding of these aspects.
LLMs are like a static snapshot of a brain. Inactive till you run one tick with it. I expect AI models becoming much more dynamic. The model needs to be live updating itself. Then it will become a living thing. But hardware wont allow for that yet. Maybe first small dynamic LLMs are possible.
>I mean, technically speaking, a video game from 1980 was likely to contain more logical branching points than a modern LLM's kernel, and was likely to be a lot more interesting to read.
It's fascinating the hills people will die on. Let's even set aside the fact that LLM internals are mostly opaque but logical branching points ? Look at the Watson test. Most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters.
Where does this idea of general intelligence as logic automatons actually exist ? Because it's sure as hell not real life. Humans are not like this, Apes are not like this, Birds are not like this, Cetaceans are not like this. Fiction. Fiction is the only realm this reality of GI exists, so it's patently absurd when people hide behind it like anything that doesn't present as doing it must be obviously wrong.
It has been generally agreed since at least Plato that it’s up to people making a positive assertion to present evidence, not up to skeptics to prove it’s not possible. N.B. Russell’s Teapot Analogy.
You have to prove it is/can be conscious, not the other way around.
The author is trying to positively assert that it’s impossible that AIs have consciousness.
I think you’d be hard pressed to find even the most ardent of AI proponents claiming AIs are _definitely_ conscious. They are almost always claiming something weaker, which is that it is possible.
The original article said that not having a consciousness makes the AI model uncapable of intelligent judgment, so in this case it's fair to ask those who are asserting such connection to prove their point.
That is not the argument. The argument isn't "AI is impossible", it's "the thing you call AI doesn't actually have intelligence." Of course we can't explain what intelligence is, but neither can you, so the impasse is where the advocates say "intelligence is what this weird annoying thing we made does" and everybody else says "no it isn't, that's no kind of explanation".
I am going to say it with only a trace of mild bemusement in no specific order:
- careers, nay, nearly whole industries may ride on that being the case
- classifying it as conscious intelligence opens a range of ethical questions no thinking, conscious, intelligent ( and likely non-evil ) life-form would want to comprehend
- being able to ascribe judgment to it without intelligence undermines a lot of what we do on a daily basis and puts into question what it is that makes us human
- I am saying this part in the nicest possible way: hardcore cope of biblical proportions, where seemingly conscious, intelligent individual, who makes an argument for lack of consciousness manages to ( in the same sentence! ) to anthropomorphize the parrot and imbue it with hope
- average human tendency to fear the unknown
Consciousness is a first person experience, that's why this debate always degrades.
On the other hand, they have no biology or hormones, no real experience with birth, life, death, joy, fear, empathy... that is all simulated emotion through language. They will never know what it's like to be horny or hungry and the satisfaction that comes with fulfilling those.
There seems to be two common fallacies in SV technocrats' AI discourse:
(1) A far-reaching tendency to overextrapolate from the low-hanging fruit of the last few years of pretraining progress. GPT-2 to GPT-3 may have been a quantum leap, but GPT-3 to GPT-4 was not, and GPT-4 to 5 even less so.
The party has been kept going by RL and agents, but still, there is indeed a point of diminishing returns, not just relative to available compute but to how much training is possible when the entire intellectual output of humanity, plus a raft of synthetic data, has already been inhaled by the training process.
If one is to internalise the things that are said here on HN with regularity about model progress, and sentences ending with "yet" or "for now", then it would be easy to conclude that my 10 year-old son, who gained 3 inches of height last year, will be tallest structure on the planet by age 17.
(2) Inability to distinguish between technological, computational, and energetic limits of LLM capabilities vs. ontological / conceptual ones. There are some things LLMs cannot do, or at least do well, at any size, at infinite size and with infinite compute, simply due to the very nature of what LLMs are to begin with.
This latter topic receives almost no attention, except maybe from Gary Marcus and Yann LeCun. In that respect, this article is a breath of fresh air, insofar as it highlights that LLMs aren't "AI" at all, as we have traditionally understood the concept.
They really _are_ stochastic parrots. The relevant questions are about how much that matters for some domain or set of applications, not whether they are an emerging alien intelligence with civilisation-threatening capabilities.
> There are some things LLMs cannot do, or at least do well, at any size, at infinite size and with infinite compute, simply due to the very nature of what LLMs are to begin with.
How can we be so confident that there is anything LLMs fundamentally cannot do? Proving that impossibility seems hard.
Drawing trend lines far into the future is foolish, but the recent trend is clear and its not obvious how much more progress is needed before they start having a meaningful impact on more aspects of life.
> not whether they are an emerging alien intelligence with civilisation-threatening capabilities.
The people building them are explicitly attempting to do this. They may not succeed, but what if they do? Seems worth considering that scenario.
> How can we be so confident that there is anything LLMs fundamentally cannot do? Proving that impossibility seems hard.
This is where applied business programmers part ways with philosophers of mind and cognitive scientists. However, the lack of an inner model of the world is a formidable limitation, while the reliability of purely statistical-inferential processes will never be adequate for some basic building blocks of modernity.
LLM utility is exponential with increased LLM capability. Even if improvements to LLMs slow down their impact will continue to increase. And there is no evidence LLMs are slowing down. In fact improvements still seem to be accelerating.
You like these evocative words that imbue one with a feeling of "vroom" and "whoosh", like "exponential" and "accelerating", don't you?
But arithmetically speaking, is any of that true? Is model progress truly "accelerating"? How can you compare the delta from GPT-2 to GPT-3 with the delta from GPT-5 to GPT-6 with no sense of irony? And, at the risk of being quite blunt, do you know the meaning of the term "exponential"?
I didn't say that. However, the utility delta in the "early days" was infinitely more palpable. By the time GPT-5 or GPT-6 came out, these models were just doing things they did before more accurately, with more context integrity across long-horizon tasks, etc., not doing things they couldn't do before.
This is true of every frontier model. The real agentic coding breakthrough in Claude land was around Opus 4.5 or so, though it's hard to say if that might have just been coeval with the release of the Claude Code harness.
On the Chinese front (my preferred landscape because of the open weight aspect, not because I'm Chinese), Kimi has been perfectly viable since K2.5, DeepSeek since however many major releases ago, and GLM and MiMo and the rest of them too, although they all achieved breakout moments of publicity at different times. There's no denying that every release brings improvements, but if we're going to be honest, they're fairly marginal.
They seem more marginal because the models are already capable of so much. But there was a turning point about November of last year if I recall where software developers I know went from "lol look at how this thing codes, my job is safe" to "oh no". And the models back then were actually pretty bad at many things compared to Astra or Opus 5.5.
You're right that many of the less capable / open weights models are very good themselves, but that's part of showing how much more capable the frontier models are becoming. Cheaper (smaller) models are able to do the same things, in many cases better, as the way larger models were able to do a year ago. The efficiency is getting way better alongside the 'intelligence'. That'll also cause the models to become much faster over time which is the real improvement I'm waiting for.
I think a part of why many people aren't seeing big deltas if they're using it for code and they're actually coders is we actually sit down and reason out the project with the LLM. Someone who knows nothing about coding ~1 year ago wouldn't come even close to the kind of app they'd get now with the same instructions. If you just give it a task like "help me decompile this game" without knowing a whole lot about decompiling suddenly in the last 3 or so months you start to get some half-decent results.
The sea-change I see lately is about lifting up the people who couldn't do these things before more than the people who could. And I actually see problems coming from that already as people I've met think they can make enterprise grade software if they just tell their LLM to make it safe.
Necessary pedantry: if he's 4 foot 6 inches now, 3 inches is 5.555 (recurring) percent of that. If he grows at 105.555% per year then by 17 he is a fairly plausible 6 foot 6. (I think he then passes the forty foot mark soon after age 50.)
Ah, but he only grew about half an inch from 9 to 10, so the "rate" of "runaway progress" is "accelerating" "exponentially" (wait, how many derivatives is that?), and he will soon EsCapE CoNTaiNmEnT and hack HuggingFace...
That's a great narrative with a lot of truth in there. However, the big question is, who is the audience for this long essay? Practically no one. How long before this is all washed away from the memories and and vanished under the pile of slop that pours in everyday? Even if some soul has read it all, what are they supposed do about it? Would they really do anything? Even if they did something, what effect would it have? For all those to be true, let's say it would have a 0.001% probability.
That proves the fact that the audience is not really agentic (not the ones that can cause or influence things). The agentic nature in people was killed long back with all education, rules and pervasive reach of the state into the lives of people. Common people are just subjects without an intention and will at this point. They are like those leaves that float in a stream, while the stream finds its way.
At best, all that a today's average person can do is, post sone interesting post or comment on HN, or the other social media and count their likes, like I do now. There ends their agentic effort.
It's a great narrative with a lot of truth when the author argues billions are going to die from climate change in the next decades, and that Musk and Altman selfishly hope AI can make up for the loss of human labor?
Yeah, no. This is extremist nonsense that borders on delusions. Someone has to say it, apparently.
I've taken to calling it Human Unintelligence, in response to people arguing "it's not artificial, it's 'alien' intelligence". It's not intelligent. It is missing key aspects of intelligence, one of them being creativity.
> This brings us to a big underlying problem with the way AI is being sold to us: it's being exaggerated to sound like way more than it is, and it's being sold to people/governments as something that will be capable of exceeding human "intelligence" in the near future when it's not capable of intelligence at all.
How can you possibly hold this belief after Tuesday? An LLM solved a significant number of major open problems in many subfields of mathematics, including a major breakthrough on the Riemann Hypothesis. Get your head out of the sand.
I suspect people are afraid of change and hide behind truths like “it’s not conscious”. It’s not. But that doesn’t mean it’s not a force to be reckoned with.
People have been making this stochastic parrot argument for almost 5 years now and it's less and less accurate each year.
Everyone who rejected the stochastic parrot argument has been right about AI progress and the increasing abilities of LLM's in math, coding, writing, etc. You would be laughed at in 2020 for claiming an LLM could write coherently or could possibly pass the Turing test.
Those who claim that AI aren't capable of "genuine judgement" or whatever rhetorical flourish they feel is most effective at dismissing AI capabilities would tell you in 2022 that AI would never be capable of coding, or in 2023 it would never be capable of mathematics beyond grade-school level. Those same people now will claim that AI could never come up with a genuine breakthrough in math, science, etc. Detractors of AI have a full time job of moving goalposts.
That being said, AI, once sufficiently powerful, will grant whoever controls it the ability to mold the world as they see fit, for better or worse. They will have access to superhuman mathematicians, coders, strategists, PR, financial engineering, legal skill, etc. Essentially an army of superhuman intellects which can afford them guaranteed success in any domain they wish to pursue. Anyone who doesn't see this as potentially the most destabilizing event in human history likely isn't thinking deeply enough about it, or is still knee deep in denial.
Do you really believe that, or is that just an ego-preserving reflexive response? LLM's have been capable of near human-level writing for over a year now.
What LLMs lack is consistency. They may build a well researched article about an interesting article, and the next minute write a rambling, disjointed stream of thoughts in a totally different style.
A human mind keeps a personality that keeps generating thoughts in the same direction, unless some mind-altering force derails it. A LLM has no guiding stable personality of that kind, rather has millions of different personalities and which one you get is dictated by whatever information you put in its context window.
This sort of nihilism/defeatism always irks me. It's so selfish, so privileged.
Tell my starved, murdered ancestors "let us enjoy the end" of their era of stability's denoument.
And, truly, it is not the end. Inconveniently, many—if not most—of us will survive. Will continue on. And it's for the betterment of those who come after us that we ought to toil a little, and not indulge in this childish, self-interested to nihilism.
marginalized populations have repeatedly survived past "apocalypses" not through aesthetic surrender, but through the deliberate, daily reproduction of the commons, ie through pooling resources, sustaining life for its own sake, and preserving collective memory.
Yielding to fashionable doomism merely validates the oppressor's narrative that the future is already decided. A superstition of doom is no different from a superstition of inevitable progress; both models rely on an abstract determinism that strips humans of agency, that encourages the us to accept suffering as unalterable destiny or "the will of God" rather than an artificial, historical barrier.
So, I agree with a lot of what OP is saying, but I think they're a jerk.
If you want to describe the limitations of today’s LLMs, "frozen brains" might be a better metaphor than "stochastic parrots". They can’t learn continuously as we do, which makes it hard for them to invent entirely new concepts - like calculus.
You'd get better judgment from training an actual parrot, because it operates on a spectrum of data that the artificial parrot has no hope of operating on unless someone figures out how to make light move faster.
This statement belongs to the same epistemic class as Moon-landing denial. Flagged for kookery.
> This was crudely written in under 2 hours without the assistance, analysis, or hardening of an LLM, and with no editorial treatment whatsoever. I think this is how everyone should be writing in an age of shitty, sterilized parrot regurgitation. Now, let us enjoy the end, however it may come.
LLMs have genuinely modeled systems purely through training on natural language descriptions of those systems. It’s a massive lesson and an answer to the Chinese room thought experiment. I think that’s reasonable to label as intelligence and goes far beyond a stochastic parrot.
The stochastic parrots thing is already old. It might have made sense at one point, but now that the parrots are solving millennium problems, I think people have to admit things have changed.
He is right that the AI giants would like a bit of regulatory capture.
It can still be stochastic over more complex token graph traversals, but it is still stochastic at the core. Anyone who uses them daily knows how often they make the most absurd assumptions and false statements, and show the limit of engineered intelligence
If the parrot randomly yaps out a solution to an open math problem, it's a useful parrot.
The fact that it sometimes says things that aren't useful, well, old intelligence does that all the time.
If you've used AI recently, you can't avoid having found it useful. Just general day-to-day intelligence like finding things in your email, writing code for you, that kind of stuff, it does well. People have some weird hangups about it being stochastic, when it doesn't really matter.
yes, the spaghetti throwers (current training regime) sometimes hit the mark quite well, but they also do some of the dumbest things, reinforcing to the stochastic nature of their construction.
Eh, author has a bone to pick with AI companies and his person vision of what intelligence is that they can goalpost move because it doesn't have any stated testable positions. But don't worry anybody, it's just a parrot, not dangerous. Oh, and might want to check your bank because someone else's agentic loop just stole everything from your bank account.
Not anything interesting here that hasn't already been stated 20 other times.
The connection made to climate change in the latter part of the article is new to me, and would be interesting if true. However, I wish the author would cite supporting evidence for their conclusions.
> So why are many of the very same sociopaths who are driving us toward climate destruction now suddenly so concerned about the "danger" posed by AI, and seemingly acting in defiance of their fiduciary duties to warn us that we must slow down development and seek regulation?
> interesting if true.
Not sure what you mean by this. This is obviously about MAGA and other allied political forces in the US, at the very least. I wouldn't call it interesting to document the recent observed behavior. It's been evident through journalistic analysis of public statements, policy, and investment for most of the year.
> The hope of the Elons and Sams of the world is that this will come to fruition well enough that we only have to bother sustaining the "cream of the crop" in the labor force […]. So basically the plan here is to let things go to shit, and when billions of people die as a natural consequence of that, things will keep running as desired.
For one, I don’t think that Sam Altman can be characterized as MAGA (https://en.wikipedia.org/wiki/Sam_Altman#Political_engagemen...). Secondly, I’d be interested in sources substantiating a plan that explicitly includes billions of people dying as a necessary factor for the plan to work out.
> I don’t think that Sam Altman can be characterized as MAGA
He is not usually associated with MAGA. The reference was not directed at the heads of the most well know tech companies, despite the quote about "Elons and Sams". This wouldn't make sense in context. It was a reference to the ruling political regime.
> So why are many of the very same sociopaths who are driving us toward climate destruction now suddenly so concerned about the "danger" posed by AI, and seemingly acting in defiance of their fiduciary duties to warn us that we must slow down development and seek regulation? Of course the reasons are entirely self-serving, which you should almost exclusively be able to determine from their history of narratives! They're combining this "threat to the species" narrative with a "we must stay ahead of China for the sake of our babies and all of Western Civilization" (this is incredibly ironic, I'll explain why in a bit) narrative, while milking the shit out of every perceived accomplishment (especially their highly publicized and highly distorted mathematical "victories"); it's important to understand that these narratives are directly linked to each other, in order to understand the reasons behind them.
This is the paragraph in the entirety. While previous sections referred to “the Elons and Sams of the world”, this section that was specifically called out:
> The connection made to climate change in the latter part of the article is new to me, and would be interesting if true. However, I wish the author would cite supporting evidence for their conclusions.
Does not refer to them explicitly (many of the very same), although it could be believed that the second clause of the first sentence refers to Sam, et al. Elon doesn't fit as he has no fiduciary duty nor has he called for slowdown, nor are there "many" in this demographic, which is a sign that this is more stream of thought. I believe the narrative only fits if you consider this to be a tangent about political sociopaths that was muddled by a very human leading thought intruding (from previous paragraphs), on an otherwise straightforward observation.
It makes no sense in the context of billionaires, as they were not generally driving "us" toward climate destruction prior to the AI boom.
> The dangers posed by this technology on its present trajectory are best understood as simple accelerants for the existing dangers humans pose to themselves.
Is very true. All the doomspeak "aI EsCaPeD AnD HaCkEd aNoThEr sYsTeM" was a human error all along, and the tech was doing what it is programmed to do, retry until a satisifiable outcome is met.
The particular problem here is quantified the inherent danger of a technology. That is, once you remove the active management of humans, what dangers can happen.
With dangerous chemicals and radioactive materials these are generally pretty easy to classify. This said, after the physical behaviors, the above technologies can induce behavior changes in humans because they exist. After nukes, a lot more people had issues with extental dread. It also causes n-order interactions that get very hard to predict. For example MAD or laws against using chemical weapons.
And these are weapons with a very easy to map out causal chain in their own behavior. You can't prompt a chemical weapon to make a decision.
We come up with agentic AI and all of a sudden a lot of rules on causality break. In a harness they are a causal agent. They can interact with a digital reality, and with the right tools and/or human contacts they can causally effect analog reality.
Anyone that says 'programmed' about an LLM almost certainly does not know what they are talking about and can be summarily ignored.
How is that comforting? Humans pose almost infinite danger to themselves. We can (and have) done pretty much everything you can do. We've had the capability to glass the entire earth for 70 years. We've had bioweapons, chemical weapons, mass murder, you name it.
Accelerating the ability for any given person to have access to some amount of these capabilities is terrifying, is it not?
I'm getting so many replies like the one above yours recently. It's like the average person has not really incorporated the fact that we've made something different than all other things we made before now, we'll, except maybe babies. It is comforting and easy to think this AI technology is the same as everything before therefore I don't have to do anything and the problem will solve itself as other problems did in the past.
Maybe many were too young for the cold war. But it really does feel like we're setting a new one up.
Funny how these articles always use the same aggressive language and bombastic style, to the point where they are more similar than the output of said "sterilized parrot regurgitation".
When Timnit Gebru described LLMs as "stochastic parrots", she was making a deep point about what she believed to be the limitations of the technology. Her argument claimed that while we perceived legible text coming from ChatGPT, we were actually being fooled by our innate pattern-matching instincts. Notably, at the time she wrote this, it wasn't uncommon to see LLMs collapse into piles of gibberish, lending credence the the idea that there was never any actual meaning in the output other than what we readers brought to it.
It's safe to say that argument has collapsed utterly. Modern LLMs virtually never collapse into endless babbling loops. Far more importantly, they solve objectively hard problems. An LLM didn't merely inspire a mathematician to spot a counterexample to the Jacobian Conjecture that had been in front of his nose the whole time; it found the counterexample. These things are unquestionably generating meaningful outputs.
(That obviously doesn't mean that you can simply trust LLM outputs!)
At this point, trying to dunk on frontier models by calling them "stochastic parrots" is saying more about you than about the models. Moreover, it suggests you're not only out on a limb about the capabilities of LLMs, but also on what even the skeptic literature about it was saying.
You should know what "stochastic parrot" actually meant; you shouldn't be using the term just because you think it sounds snazzy.
> is saying more about you than about the models
It's interesting that you'd be choosing this canned phrase of all canned phrases, given just how hard that is linked to emotion-driven facebook comment fights that is.
This doesn't read like something that belongs into an engineering-driven discussion.
I see them collapse when faced unusual problems. They’ll go down a weird rabbit hole and then just keep going, even if their idea is completely irrational and not working.
They’re incapable of changing their mind, because they’re a Markov chain. They’re cool and they’re good at doing things, but they’re not sentient in the slightest.
Not at all the same idea. People also rathole into doomed approaches. What we're talking about here is literally LLMs producing line noise, or English words that don't fit into sentences.
(I don't think LLMs are "sentient".)
I've seen this reflex quite a few times now. To diminish the impressive capabilities of the latest LLMs, one resorts to the assertion that they're not sentient, or not truely intelligent, or various other semantic diversions. No one claimed that. GP was all about usefulness.
Then GP misrepresented the "On the Dangers of Stochastic Parrots" paper, which spoke about meaning. But GP also said "meaning" several times, so no.
There's some haziness here about the difference between comprehensible output and comprehending. To put it mildly. I mean that's been the whole question for five or six years.
LLMs collapsing into piles of gibberish is a misrepresentation, too. The paper doesn't mention it and the argument doesn't hinge on it, or even benefit from it, really. But the other bit:
> Her argument claimed that while we perceived legible text coming from ChatGPT, we were actually being fooled by our innate pattern-matching instincts
This is an accurate summary, I think, and the point the authors are making sits somewhere in the coordinate space of a simplified reframing of the Chinese Room, Grice's account of intention in meaning, Wittgenstein's "Say 'It's cold here' and mean 'It's warm here'" challenge, and a whole bunch of the hermeneutic tradition.
Very roughly, you might say that an LLM doesn't mean anything when it emits text, but that the text still has meaning (which I think is all tptacek wants from "meaningful outputs", but you could take that further as culturally or scientifically 'meaningful', i.e. 'significant').
Philosophically, there are people who hold the position that something has to have been meant to have meaning, or that something has to have meaning in the first place to be meant, and so on. There were always interesting edge cases to these accounts (actors delivering lines, or the Dada/Burroughs cut-up technique), but LLMs are challenging in a way that e.g. Racter wasn't.
"Stochastic parrot" is a fitting term for this because the parrot's words have meaning, but the parrot itself doesn't mean anything by them. Not literally, anyway. And that's it. That's the whole analogy.
Have some people watered the term down to "LLM dumb"? Yes. Is that what Gebru meant in the first place? Not in my reading. Is it incorrect to use the term today? Depends, uh, on what you mean by it.
I really wish the article in question hadn’t used that phrase, because you’re right. But the rest of the article is also correct in the dangers facing us from this technology as an accelerant regardless, and I think it’s much more interesting to talk about that: but this entire comment thread will dunk on the stochastic parrot paragraphs instead.
The funny thing is though, I think it’s an in group signalling mechanism. You need the anti-AI crowd to know that you’re anti-AI, and this is a rapid way of doing so, to get it upvoted here.
I couldn't read past the first paragraph
> I mean, technically speaking, a video game from 1980 was likely to contain more logical branching points than a modern LLM's kernel, and was likely to be a lot more interesting to read.
Are you really saying that the "AI" in 1980 is _Really_ much more interesting than today's LLM? The author is either dishonest or ignorant.
The article is extremist nonsense where the author argues billions are going to die due to climate change in the next decades, and that Altman and Musk are hoping AI can until then replace human labor in order to keep their living standard in such a future.
It's not like the "stochastic parrot" thing was the only problem here.
Sure, if you think so, but discussing or arguing for/against those points here is far more interesting than the done-to-death stochastic parrot stuff.
I’m glad the article used the phrase because it was a useful indicator of the author’s underlying biases
An LLM did not do those things! An agentic harness around an LLM did those things.
This is a distinction that only matters if you're having a philosophical argument, but the claim I'm addressing from this article isn't philosophical.
It is a practical distinction. An engine is not a car.
the most widely used API surfaces are already shaped around chat, it's different and most analogies will fall short in some way
Neither a chat or an API are an LLM.
LLM's have chat templates that sit at the lowest levels of how we feed tokens to them. It's quite integral to how they are trained to process tokens and rises up through the tech stack.
Go to a huggingface repo, look at the files, they have a literal chat_template.jinja
By that standard Gebru was also not writing about LLMs in the Stochastic Parrot paper.
For a mechanic, that's a practical distinction. For most people, it isn't.
I don’t see people driving engines around, since that is impossible. You actually need the rest of the car around the engine to have it function as a drivable vehicle.
It is a pretty practical distinction.
Of course. But most people don't buy an engine, ever. They don't have one laying around. They buy a whole car, every time.
So, since apparently I may need to clarify my actual point: For most people, an LLM-plus-agent-harness is just "LLM" or "AI". If you're trying to build one, you care about the distinction. Ditto if you're a pedant. But most people are not either of those things.
Well, that may be what it was coined to mean, what does it mean now? Stochastic = statistical, parrot = using training data.
https://en.wikipedia.org/wiki/Stochastic_parrot
"a metaphor that frames large language models as systems that statistically mimic text without real understanding". I suppose you're defending them as having understanding. It's not "safe to say that argument has collapsed utterly", you're part of a campaign to reject the meme. On the other side, the meme is popular as a way to be rude about AI, because it's a cutting insult.
> what does it mean now?
It means whatever you want it to mean, that's the best part. To me, it's best used as it was intended, but primarily in the contexts where the stochastic parrots are plowing through the ivory towers we built mostly for idiot savants.
I don't even think the claim (about what she was saying) is true. It's still used with the original meaning, which was not about being fooled into perceiving legible text, but fooled into perceiving understanding.
Sherry Turkle is far more articulate on these things in her latest book Artificial Intimacy (which is about much more than the relationship chatbots)
LLMs get better and better and do more and more things. But I still get the impression "their thinking isn't human thinking" by the way that their ability "understand" something shifts and fades as the discussion continues. And these can be things that it previously explained to me.
When I look at a person moving in front of me, I don't have to recheck if their bone structure is going shift as they more - it just won't. So, as a human, I don't naturally notice things of that sort. So I don't notice all the ways an AI video is wrong unless I concentrate. Which is saying that LLMs are wrong in ways we aren't accustomed to. They drift in their interactions and seeming understanding. Perhaps "random walk parrot" is better than "stochastic parrot" but I still say a word that communicates that they're "things that aren't quite doing what you think they're doing" is legitimate.
> It's safe to say that argument has collapsed utterly. Modern LLMs virtually never collapse into endless babbling loops. Far more importantly, they solve objectively hard problems.
And yet, they still output gibberish. They also still fail to solve objectively easy problems on a regular basis. That is because they are, indeed, still stochastic parrots without any actual understanding of anything or ability to reason about stuff. The argument hasn't collapsed at all.
> They also still fail to solve objectively easy problems on a regular basis
Which?
What is exactly "actual understanding" . If it tells 1+1 = 2 it can perfectly explain why the result is 2. Or do you mean the actual transformer work during the 1+1 question?
How'd your parrot do at IMO this year?
LLMs are certainly stochastic, but they no longer meet any sane person's definition of "parrot." I can't imagine Gebru disagreeing. Has anyone bothered to ask her?
(Consider responding with an actual counterpoint, so we can all see what reasoning might give rise to such a delusion. Does your pet parrot do calculus and write stories and reverse-engineer Adobe applications?)
Okay, but as I just put it in another thread (which was in "new" and never made it anywhere near the front page):
All we have to do is assume that things progress in a linear way.
Do you remember GPT-2? It was a bizarre curiosity, but was nevertheless considered a huge advance in late 2019. Practically useless, but interesting.
It grew, and three and a half years later we got GPT-4, which was exceptionally capable on release.
It grew still further, and three and a half years hence we now have GPT-6. This is already more intelligent, more imaginative, more disciplined, and far more erudite than the vast majority of individual humans. If you have an intellectual skill -- if you are, say, a mathematician or a chemist -- GPT-6 also has that skill and is functionally your peer. And then it has many other skills that you lack; if nothing else, it is superhuman in breadth.
Assume a steady rate of progress. In three and a half years, we're going to have something that is genuinely a general superintelligence, and we're probably going to see recursive self-improvement begin to kick in (though subject to physical constraints on compute, etc.)
Now is the time for an RSI start-up, I guess!
Anyway, all you have to do is assume that things don't plateau. They have not plateaued to date; if anything, progress appears to be accelerating, and people are starting to panic that there are no brakes on this thing.
> This is already more intelligent, more imaginative, more disciplined, and far more erudite than the vast majority of individual humans. If you have an intellectual skill -- if you are, say, a mathematician or a chemist -- GPT-6 also has that skill and is functionally your peer. And then it has many other skills that you lack; if nothing else, it is superhuman in breadth.
Kinda chilling to see one of your kind in the wild.
"It's better than you, and if it isn't, it's bigger than you."
<< people are starting to panic that there are no brakes on this thing.
There are.. physical limits, as it were, which effectively restrict upper limit, but I hear what you are saying. A lot of the problems thrown at it and issues found in the process only seem to have helped. I feels ages ago when we were arguing 'it is merely a stochastic parrot'.
> Do you remember GPT-2?
I recall them saying it was too dangerous to release, funny how they are only saying that again as they look to IPO
This is going to sound crazy, but, at the time, they were spooked by Neal Stephenson's June 2019 book The Fall -- where the internet was broken by the open-source release of something akin to a language model that generated endless quantities of spam. They thought that GPT-2 might be that invention, so it worried them.
Then, a few years later, they learned that Google and Anthropic were planning on releasing LLM chatbots to the public, so they hastily launched GPT-3.5-based ChatGPT as a "dev preview" to get ahead of them. Their original plan was to finish training GPT-4 before launching anything.
Commercial considerations -- and public glory -- triumphed in the end.
I would disagree regarding the opening paragraphs about intelligence, but then we don’t have a generally agreed definition of intelligence. There is a scientific view that a lot of what the human brain does and has biologically evolved to is predicting what will happen next, and next-token-prediction is reasonably related to that.
However, what seems to still be lacking in LLMs — aside from plasticity (continuous learning), perception of time, and anything related to emotions and desires (which underly motivations) — is various types of awareness, and sound judgement. We are still far from a good understanding of these aspects.
LLMs are like a static snapshot of a brain. Inactive till you run one tick with it. I expect AI models becoming much more dynamic. The model needs to be live updating itself. Then it will become a living thing. But hardware wont allow for that yet. Maybe first small dynamic LLMs are possible.
>I mean, technically speaking, a video game from 1980 was likely to contain more logical branching points than a modern LLM's kernel, and was likely to be a lot more interesting to read.
It's fascinating the hills people will die on. Let's even set aside the fact that LLM internals are mostly opaque but logical branching points ? Look at the Watson test. Most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters.
Where does this idea of general intelligence as logic automatons actually exist ? Because it's sure as hell not real life. Humans are not like this, Apes are not like this, Birds are not like this, Cetaceans are not like this. Fiction. Fiction is the only realm this reality of GI exists, so it's patently absurd when people hide behind it like anything that doesn't present as doing it must be obviously wrong.
> However, this parrot can't hope to emulate any serious form of consciousness, and therefore can't ever be capable of intelligent judgment
I often hear this asserted, but never with any actual justification.
Why can it not?
It has been generally agreed since at least Plato that it’s up to people making a positive assertion to present evidence, not up to skeptics to prove it’s not possible. N.B. Russell’s Teapot Analogy.
You have to prove it is/can be conscious, not the other way around.
I don’t agree.
The author is trying to positively assert that it’s impossible that AIs have consciousness.
I think you’d be hard pressed to find even the most ardent of AI proponents claiming AIs are _definitely_ conscious. They are almost always claiming something weaker, which is that it is possible.
I understand this but what happens when you apply this to humans? Can you prove humans are conscious?
We don’t have a precise enough definition to do either.
The original article said that not having a consciousness makes the AI model uncapable of intelligent judgment, so in this case it's fair to ask those who are asserting such connection to prove their point.
Ok, prove that only meat can be conscious.
That is not the argument. The argument isn't "AI is impossible", it's "the thing you call AI doesn't actually have intelligence." Of course we can't explain what intelligence is, but neither can you, so the impasse is where the advocates say "intelligence is what this weird annoying thing we made does" and everybody else says "no it isn't, that's no kind of explanation".
It's the opposite isn't it? You assume consciousness in other things which may have it. Higher order animals, other humans etc.
Imagine if we actually demanded that you prove your consciousness before we believe you.
Because people who say this are always begging the question. They're already at the conclusion and work backwards to justify it.
I am going to say it with only a trace of mild bemusement in no specific order:
- careers, nay, nearly whole industries may ride on that being the case - classifying it as conscious intelligence opens a range of ethical questions no thinking, conscious, intelligent ( and likely non-evil ) life-form would want to comprehend - being able to ascribe judgment to it without intelligence undermines a lot of what we do on a daily basis and puts into question what it is that makes us human - I am saying this part in the nicest possible way: hardcore cope of biblical proportions, where seemingly conscious, intelligent individual, who makes an argument for lack of consciousness manages to ( in the same sentence! ) to anthropomorphize the parrot and imbue it with hope - average human tendency to fear the unknown
Consciousness is a first person experience, that's why this debate always degrades.
On the other hand, they have no biology or hormones, no real experience with birth, life, death, joy, fear, empathy... that is all simulated emotion through language. They will never know what it's like to be horny or hungry and the satisfaction that comes with fulfilling those.
There seems to be two common fallacies in SV technocrats' AI discourse:
(1) A far-reaching tendency to overextrapolate from the low-hanging fruit of the last few years of pretraining progress. GPT-2 to GPT-3 may have been a quantum leap, but GPT-3 to GPT-4 was not, and GPT-4 to 5 even less so.
The party has been kept going by RL and agents, but still, there is indeed a point of diminishing returns, not just relative to available compute but to how much training is possible when the entire intellectual output of humanity, plus a raft of synthetic data, has already been inhaled by the training process.
If one is to internalise the things that are said here on HN with regularity about model progress, and sentences ending with "yet" or "for now", then it would be easy to conclude that my 10 year-old son, who gained 3 inches of height last year, will be tallest structure on the planet by age 17.
(2) Inability to distinguish between technological, computational, and energetic limits of LLM capabilities vs. ontological / conceptual ones. There are some things LLMs cannot do, or at least do well, at any size, at infinite size and with infinite compute, simply due to the very nature of what LLMs are to begin with.
This latter topic receives almost no attention, except maybe from Gary Marcus and Yann LeCun. In that respect, this article is a breath of fresh air, insofar as it highlights that LLMs aren't "AI" at all, as we have traditionally understood the concept.
They really _are_ stochastic parrots. The relevant questions are about how much that matters for some domain or set of applications, not whether they are an emerging alien intelligence with civilisation-threatening capabilities.
> There are some things LLMs cannot do, or at least do well, at any size, at infinite size and with infinite compute, simply due to the very nature of what LLMs are to begin with.
How can we be so confident that there is anything LLMs fundamentally cannot do? Proving that impossibility seems hard.
Drawing trend lines far into the future is foolish, but the recent trend is clear and its not obvious how much more progress is needed before they start having a meaningful impact on more aspects of life.
> not whether they are an emerging alien intelligence with civilisation-threatening capabilities.
The people building them are explicitly attempting to do this. They may not succeed, but what if they do? Seems worth considering that scenario.
> How can we be so confident that there is anything LLMs fundamentally cannot do? Proving that impossibility seems hard.
This is where applied business programmers part ways with philosophers of mind and cognitive scientists. However, the lack of an inner model of the world is a formidable limitation, while the reliability of purely statistical-inferential processes will never be adequate for some basic building blocks of modernity.
LLM utility is exponential with increased LLM capability. Even if improvements to LLMs slow down their impact will continue to increase. And there is no evidence LLMs are slowing down. In fact improvements still seem to be accelerating.
You like these evocative words that imbue one with a feeling of "vroom" and "whoosh", like "exponential" and "accelerating", don't you?
But arithmetically speaking, is any of that true? Is model progress truly "accelerating"? How can you compare the delta from GPT-2 to GPT-3 with the delta from GPT-5 to GPT-6 with no sense of irony? And, at the risk of being quite blunt, do you know the meaning of the term "exponential"?
Let's get concrete about the GPT improvements you were talking about:
On the MMLU benchmark, GPT-2 had an accuracy of 32.4%. GPT-3 improved this to 43.9%.
GPT-4 scored 86.4%. On GPQA Diamond, it scored 31%, vs GPT-5 at 86%. GPT-6 scores 96%.
If we take ARG-AGI-2, it would be 9.9% for GPT-5 vs 95% with GPT-6.
The benchmarks do not corroborate the picture you were drawing about improvements slowing down between GPT major versions.
I don't much care about benchmarks. Talk obvious, commonsensical increases in utility to me.
There wasn't that much utility in old, unreliable AI.
I remember their capabilities like so:
GPT-3 could produce convincing looking texts, sometimes.
GPT-4 was somewhat smarter and would give more accurate answers. At that time image understanding was released, wasn't it?
Then with GPT-5 we have a reasoning model, another jump in capabilities.
Now, compare GPT-6 Astra with its ability to implement software, work on long horizon tasks, and visual understanding.
If you haven't seen the (massive) increases in utility I'd wager you haven't been using the tools much.
So I'm holding it wrong?
My guess is you're barely holding it at all if you haven't seen several more uses for it in the last year or so.
I didn't say that. However, the utility delta in the "early days" was infinitely more palpable. By the time GPT-5 or GPT-6 came out, these models were just doing things they did before more accurately, with more context integrity across long-horizon tasks, etc., not doing things they couldn't do before.
This is true of every frontier model. The real agentic coding breakthrough in Claude land was around Opus 4.5 or so, though it's hard to say if that might have just been coeval with the release of the Claude Code harness.
On the Chinese front (my preferred landscape because of the open weight aspect, not because I'm Chinese), Kimi has been perfectly viable since K2.5, DeepSeek since however many major releases ago, and GLM and MiMo and the rest of them too, although they all achieved breakout moments of publicity at different times. There's no denying that every release brings improvements, but if we're going to be honest, they're fairly marginal.
They seem more marginal because the models are already capable of so much. But there was a turning point about November of last year if I recall where software developers I know went from "lol look at how this thing codes, my job is safe" to "oh no". And the models back then were actually pretty bad at many things compared to Astra or Opus 5.5.
You're right that many of the less capable / open weights models are very good themselves, but that's part of showing how much more capable the frontier models are becoming. Cheaper (smaller) models are able to do the same things, in many cases better, as the way larger models were able to do a year ago. The efficiency is getting way better alongside the 'intelligence'. That'll also cause the models to become much faster over time which is the real improvement I'm waiting for.
I think a part of why many people aren't seeing big deltas if they're using it for code and they're actually coders is we actually sit down and reason out the project with the LLM. Someone who knows nothing about coding ~1 year ago wouldn't come even close to the kind of app they'd get now with the same instructions. If you just give it a task like "help me decompile this game" without knowing a whole lot about decompiling suddenly in the last 3 or so months you start to get some half-decent results.
The sea-change I see lately is about lifting up the people who couldn't do these things before more than the people who could. And I actually see problems coming from that already as people I've met think they can make enterprise grade software if they just tell their LLM to make it safe.
Necessary pedantry: if he's 4 foot 6 inches now, 3 inches is 5.555 (recurring) percent of that. If he grows at 105.555% per year then by 17 he is a fairly plausible 6 foot 6. (I think he then passes the forty foot mark soon after age 50.)
Ah, but he only grew about half an inch from 9 to 10, so the "rate" of "runaway progress" is "accelerating" "exponentially" (wait, how many derivatives is that?), and he will soon EsCapE CoNTaiNmEnT and hack HuggingFace...
Ha, of course, add derivatives until desired prediction is met.
That's a great narrative with a lot of truth in there. However, the big question is, who is the audience for this long essay? Practically no one. How long before this is all washed away from the memories and and vanished under the pile of slop that pours in everyday? Even if some soul has read it all, what are they supposed do about it? Would they really do anything? Even if they did something, what effect would it have? For all those to be true, let's say it would have a 0.001% probability.
That proves the fact that the audience is not really agentic (not the ones that can cause or influence things). The agentic nature in people was killed long back with all education, rules and pervasive reach of the state into the lives of people. Common people are just subjects without an intention and will at this point. They are like those leaves that float in a stream, while the stream finds its way.
At best, all that a today's average person can do is, post sone interesting post or comment on HN, or the other social media and count their likes, like I do now. There ends their agentic effort.
You read it, you let it tickle your braingibbles, you think about it for a few moments. That's it, that's the purpose.
It's a great narrative with a lot of truth when the author argues billions are going to die from climate change in the next decades, and that Musk and Altman selfishly hope AI can make up for the loss of human labor?
Yeah, no. This is extremist nonsense that borders on delusions. Someone has to say it, apparently.
I've taken to calling it Human Unintelligence, in response to people arguing "it's not artificial, it's 'alien' intelligence". It's not intelligent. It is missing key aspects of intelligence, one of them being creativity.
> This brings us to a big underlying problem with the way AI is being sold to us: it's being exaggerated to sound like way more than it is, and it's being sold to people/governments as something that will be capable of exceeding human "intelligence" in the near future when it's not capable of intelligence at all.
How can you possibly hold this belief after Tuesday? An LLM solved a significant number of major open problems in many subfields of mathematics, including a major breakthrough on the Riemann Hypothesis. Get your head out of the sand.
I suspect people are afraid of change and hide behind truths like “it’s not conscious”. It’s not. But that doesn’t mean it’s not a force to be reckoned with.
[flagged]
People have been making this stochastic parrot argument for almost 5 years now and it's less and less accurate each year.
Everyone who rejected the stochastic parrot argument has been right about AI progress and the increasing abilities of LLM's in math, coding, writing, etc. You would be laughed at in 2020 for claiming an LLM could write coherently or could possibly pass the Turing test.
Those who claim that AI aren't capable of "genuine judgement" or whatever rhetorical flourish they feel is most effective at dismissing AI capabilities would tell you in 2022 that AI would never be capable of coding, or in 2023 it would never be capable of mathematics beyond grade-school level. Those same people now will claim that AI could never come up with a genuine breakthrough in math, science, etc. Detractors of AI have a full time job of moving goalposts.
That being said, AI, once sufficiently powerful, will grant whoever controls it the ability to mold the world as they see fit, for better or worse. They will have access to superhuman mathematicians, coders, strategists, PR, financial engineering, legal skill, etc. Essentially an army of superhuman intellects which can afford them guaranteed success in any domain they wish to pursue. Anyone who doesn't see this as potentially the most destabilizing event in human history likely isn't thinking deeply enough about it, or is still knee deep in denial.
> You would be laughed at in 2020 for claiming an LLM could write coherently or could possibly pass the Turing test.
My dude, an LLM still can't write coherently.
Do you really believe that, or is that just an ego-preserving reflexive response? LLM's have been capable of near human-level writing for over a year now.
https://vulsar.ai/benchmarks/creative-writing-v1/
What LLMs lack is consistency. They may build a well researched article about an interesting article, and the next minute write a rambling, disjointed stream of thoughts in a totally different style.
A human mind keeps a personality that keeps generating thoughts in the same direction, unless some mind-altering force derails it. A LLM has no guiding stable personality of that kind, rather has millions of different personalities and which one you get is dictated by whatever information you put in its context window.
> Now, let us enjoy the end, however it may come.
This sort of nihilism/defeatism always irks me. It's so selfish, so privileged.
Tell my starved, murdered ancestors "let us enjoy the end" of their era of stability's denoument.
And, truly, it is not the end. Inconveniently, many—if not most—of us will survive. Will continue on. And it's for the betterment of those who come after us that we ought to toil a little, and not indulge in this childish, self-interested to nihilism.
marginalized populations have repeatedly survived past "apocalypses" not through aesthetic surrender, but through the deliberate, daily reproduction of the commons, ie through pooling resources, sustaining life for its own sake, and preserving collective memory.
Yielding to fashionable doomism merely validates the oppressor's narrative that the future is already decided. A superstition of doom is no different from a superstition of inevitable progress; both models rely on an abstract determinism that strips humans of agency, that encourages the us to accept suffering as unalterable destiny or "the will of God" rather than an artificial, historical barrier.
So, I agree with a lot of what OP is saying, but I think they're a jerk.
If you want to describe the limitations of today’s LLMs, "frozen brains" might be a better metaphor than "stochastic parrots". They can’t learn continuously as we do, which makes it hard for them to invent entirely new concepts - like calculus.
You'd get better judgment from training an actual parrot, because it operates on a spectrum of data that the artificial parrot has no hope of operating on unless someone figures out how to make light move faster.
This statement belongs to the same epistemic class as Moon-landing denial. Flagged for kookery.
> This was crudely written in under 2 hours without the assistance, analysis, or hardening of an LLM, and with no editorial treatment whatsoever. I think this is how everyone should be writing in an age of shitty, sterilized parrot regurgitation. Now, let us enjoy the end, however it may come.
<3
Ahem ... Super Insanity
LLMs have genuinely modeled systems purely through training on natural language descriptions of those systems. It’s a massive lesson and an answer to the Chinese room thought experiment. I think that’s reasonable to label as intelligence and goes far beyond a stochastic parrot.
[flagged]
The stochastic parrots thing is already old. It might have made sense at one point, but now that the parrots are solving millennium problems, I think people have to admit things have changed.
He is right that the AI giants would like a bit of regulatory capture.
It can still be stochastic over more complex token graph traversals, but it is still stochastic at the core. Anyone who uses them daily knows how often they make the most absurd assumptions and false statements, and show the limit of engineered intelligence
That doesn't address what I'm saying.
If the parrot randomly yaps out a solution to an open math problem, it's a useful parrot.
The fact that it sometimes says things that aren't useful, well, old intelligence does that all the time.
If you've used AI recently, you can't avoid having found it useful. Just general day-to-day intelligence like finding things in your email, writing code for you, that kind of stuff, it does well. People have some weird hangups about it being stochastic, when it doesn't really matter.
yes, the spaghetti throwers (current training regime) sometimes hit the mark quite well, but they also do some of the dumbest things, reinforcing to the stochastic nature of their construction.
Eh, author has a bone to pick with AI companies and his person vision of what intelligence is that they can goalpost move because it doesn't have any stated testable positions. But don't worry anybody, it's just a parrot, not dangerous. Oh, and might want to check your bank because someone else's agentic loop just stole everything from your bank account.
Not anything interesting here that hasn't already been stated 20 other times.
The connection made to climate change in the latter part of the article is new to me, and would be interesting if true. However, I wish the author would cite supporting evidence for their conclusions.
> So why are many of the very same sociopaths who are driving us toward climate destruction now suddenly so concerned about the "danger" posed by AI, and seemingly acting in defiance of their fiduciary duties to warn us that we must slow down development and seek regulation?
> interesting if true.
Not sure what you mean by this. This is obviously about MAGA and other allied political forces in the US, at the very least. I wouldn't call it interesting to document the recent observed behavior. It's been evident through journalistic analysis of public statements, policy, and investment for most of the year.
Regarding:
> The hope of the Elons and Sams of the world is that this will come to fruition well enough that we only have to bother sustaining the "cream of the crop" in the labor force […]. So basically the plan here is to let things go to shit, and when billions of people die as a natural consequence of that, things will keep running as desired.
For one, I don’t think that Sam Altman can be characterized as MAGA (https://en.wikipedia.org/wiki/Sam_Altman#Political_engagemen...). Secondly, I’d be interested in sources substantiating a plan that explicitly includes billions of people dying as a necessary factor for the plan to work out.
> I don’t think that Sam Altman can be characterized as MAGA
He is not usually associated with MAGA. The reference was not directed at the heads of the most well know tech companies, despite the quote about "Elons and Sams". This wouldn't make sense in context. It was a reference to the ruling political regime.
Why would “the Elons and Sams of the world” refer to a political regime? That doesn’t make sense.
> So why are many of the very same sociopaths who are driving us toward climate destruction now suddenly so concerned about the "danger" posed by AI, and seemingly acting in defiance of their fiduciary duties to warn us that we must slow down development and seek regulation? Of course the reasons are entirely self-serving, which you should almost exclusively be able to determine from their history of narratives! They're combining this "threat to the species" narrative with a "we must stay ahead of China for the sake of our babies and all of Western Civilization" (this is incredibly ironic, I'll explain why in a bit) narrative, while milking the shit out of every perceived accomplishment (especially their highly publicized and highly distorted mathematical "victories"); it's important to understand that these narratives are directly linked to each other, in order to understand the reasons behind them.
This is the paragraph in the entirety. While previous sections referred to “the Elons and Sams of the world”, this section that was specifically called out:
> The connection made to climate change in the latter part of the article is new to me, and would be interesting if true. However, I wish the author would cite supporting evidence for their conclusions.
Does not refer to them explicitly (many of the very same), although it could be believed that the second clause of the first sentence refers to Sam, et al. Elon doesn't fit as he has no fiduciary duty nor has he called for slowdown, nor are there "many" in this demographic, which is a sign that this is more stream of thought. I believe the narrative only fits if you consider this to be a tangent about political sociopaths that was muddled by a very human leading thought intruding (from previous paragraphs), on an otherwise straightforward observation.
It makes no sense in the context of billionaires, as they were not generally driving "us" toward climate destruction prior to the AI boom.
Still
> The dangers posed by this technology on its present trajectory are best understood as simple accelerants for the existing dangers humans pose to themselves.
Is very true. All the doomspeak "aI EsCaPeD AnD HaCkEd aNoThEr sYsTeM" was a human error all along, and the tech was doing what it is programmed to do, retry until a satisifiable outcome is met.
The particular problem here is quantified the inherent danger of a technology. That is, once you remove the active management of humans, what dangers can happen.
With dangerous chemicals and radioactive materials these are generally pretty easy to classify. This said, after the physical behaviors, the above technologies can induce behavior changes in humans because they exist. After nukes, a lot more people had issues with extental dread. It also causes n-order interactions that get very hard to predict. For example MAD or laws against using chemical weapons.
And these are weapons with a very easy to map out causal chain in their own behavior. You can't prompt a chemical weapon to make a decision.
We come up with agentic AI and all of a sudden a lot of rules on causality break. In a harness they are a causal agent. They can interact with a digital reality, and with the right tools and/or human contacts they can causally effect analog reality.
Anyone that says 'programmed' about an LLM almost certainly does not know what they are talking about and can be summarily ignored.
How is that comforting? Humans pose almost infinite danger to themselves. We can (and have) done pretty much everything you can do. We've had the capability to glass the entire earth for 70 years. We've had bioweapons, chemical weapons, mass murder, you name it.
Accelerating the ability for any given person to have access to some amount of these capabilities is terrifying, is it not?
I'm getting so many replies like the one above yours recently. It's like the average person has not really incorporated the fact that we've made something different than all other things we made before now, we'll, except maybe babies. It is comforting and easy to think this AI technology is the same as everything before therefore I don't have to do anything and the problem will solve itself as other problems did in the past.
Maybe many were too young for the cold war. But it really does feel like we're setting a new one up.
Funny how these articles always use the same aggressive language and bombastic style, to the point where they are more similar than the output of said "sterilized parrot regurgitation".
I have no love for the language of lies and fear used by Big Ai either