I love this line of thinking but the framing comes across to me as a bit inflammatory and uncharitable. Just about anything involved in the exchanged of ideas can be viewed as a virus! That is because, when viewed from an evolutionary biology perspective, ideas are the cognitive equivalent of genes (in fact, this is where the term meme comes from).
There's a whole field, evolutionary memetics, dedicated to this. Cultural values, marketing, religion, social media, education, propaganda, etc. can all be viewed quite naturally through this lense. Even the terms we use in every day language (eg, going "viral") seem to intuitively grasp this.
Is my friend a virus for recommending me their favorite book? Well, yes I guess, but that's kind of just how ideas work. Just my two cents
>"We model transitions among uncoupled, coupled, and persistently dependent users, and show that the interplay between social transmission, recovery, and collective reinforcement can generate tipping points and technological lock-in."
You can finish reading your friend's favorite book or put it down if you don't like it. Neither are you economically compelled to read it by your workplace or forced to build work on top of it that will become difficult or impossible to modify/maintain if you bin the book.
I get that the title sounds a bit sensational, but there really is something to AI adoption beyond book-sharing.
I don’t disagree with you, actually I kind of feel the same about smartphone adoption. It’s basically assumed now that you have a smartphone. You can’t enter some places without a ticket on your smartphone, or can’t order at some restaurants. Government processes are starting to require smartphones. Maybe it will be similar with AI.
And you know, I love computers, I used to love the internet. And I’ve had so much fun, more fun then ever coding with AI assistance. Lets say all of these technologies can be abused and forced on us, but all of them have neat aspects too.
Fwiw i noticed you absolutely can't reasonably get a US Via consulate appt for some countries without AI right now so the requiremnts already begun.
The consulate is booked out for 3 months and only opens 3 months out. All openings are snapped up before any human could get one. They have a lot of anti-script blocking but AI seems to get around them. So there's an arms race, you pretty much need AI to book it (a basic hand written curl script isn't enough).
Oh wow, that’s aweful. Where I used to live this would happen with driving tests. Driving schools would book out all the tests and then re-assign them to their students. So at some point it got so bad that when you’re buying a driving lesson you’re also effectively buying a right to take the test. What could the government possibly do to stop this? Some highly technical solution? How about just making it so you can’t assign the test to another name? Hmm that might be a pretty obvious solution…
Steam also had a pretty nice idea with the Steamdeck. You book appointments for a month in the e.g. 2 weeks prior, but those who get booked, and when they are booked, are randomly chosen amongst all people/times in those 2 weeks. And then you repeat the same process next month and so on.
>
You can finish reading your friend's favorite book or put it down if you don't like it. Neither are you economically compelled to read it by your workplace or forced to build work on top of it that will become difficult or impossible to modify/maintain if you bin the book.
Economics does sometimes compel you to read books or other material as a condition or expectation of employment. It is difficult to get a good job while being illiterate and many jobs require reading huge volumes of written work. Work in law or journalism springs to mind as economically compelling people to read a lot even if they they might not necessarily want to.
Memes replicate and survive or perish like genes, but the environment they have to survive in is the human mind, and the threat that they have to overcome is being criticized. One way to survive this is by being a good idea. So some memes are simply good ideas. But there are various ways a meme can dodge the attack that comes from criticism, and then you get memes that are trendy or holy or crazy or ideological. Since we're not perfect in our ability to criticize (what would that even be like? Sounds terrible) I think there's necessarily a hint of this anti-rational aspect in all ideas. But some of them go all-in on the anti-rational strategy.
Tipping points and technological lock-in aren't new (e.g., the printing press, the radio, TV, phones, the internet, etc.). I do absolutely see your distinction, though, that prior technologies may have been less capable of imposing specific belief systems compared to frontier LLMs.
That's a fair point, and a key reason why competitive open models are so important. I definitely don't want to be stuck using Google's latest and greatest LLM once companies, marketers, and PACs can pay to subtly influence responses as they choose.
If you are an employee, the employer has control over how you should do your work. IF you don't like that, transition to being a contractor and sign service agreements where you are going to be paid solely for results not for how you achieve them.
> Neither are you economically compelled to read it by your workplace or forced to build work on top of it that will become difficult or impossible to modify/maintain if you bin the book.
Things go viral in professional settings as well. Scrum, agile, pc language ... I'm sure you can find other professional trends that if you decided not to follow your career would be jeopardized.
Speaking of memetics and favorite books, check out There Is No Antimemetics Division by qntm who also wrote some seminal short stories including the one that inspired the Alex Garland show Devs. The book is about how to deal with beings that make you forget you ever saw then, a sort of opposite mind virus.
your friend would be a possible infection point (not a virus) for the memes encoded in the book. As a form of memetic transmission a book, if read all the way through, is especially potent - take Ayn Rand Atlas Shrugged as an example. Books are many memes assembled together into a stable culture that reinforces the memes. The virtues of selfishness as a meme would be weak and probably die pretty quickly without all the support the book offers.
Friends probably are not wide spreading infection points, the way reviews in professional journals are or the idea that successful people have recommended things, but they are especially successful at making the transmitted meme stick, as you will keep getting reinfected from them over time. Unless you break up that dangerous friendship!!
Is a well-known cognitive quirk, Sapolsky covered that in “Behave” and “Determined”. As I get it, our morality piggybacks on the sense of disgust from insula activation, which lights up the same way as with pathogen avoidance. So comparing social groups to rodents, insects, or disease is a popular propaganda trick. Same here with viruses.
The prevalence of LLM-generated text masquerading as otherwise and incorporated into a significant fraction of all published text distinguishes this info-organism from all others.
Wasn't aware that memes were an accepted part of evolutionary biology. They're not inherited like genes through reproduction that are subject to natural selection via biological fitness. Or any other evolutionary mechanism.
The word was invented by Richard Dawkins, an evolutionary biologist. He described ideas as reproducing with variation under selection pressure, which is the definition of the evolutionary mechanism.
There is a more recent use of ‘meme’ to refer to what we used to call an ‘internet image macro’. So the meme of the idea ‘meme’ has itself evolved to have a more specific variant of the general idea.
I think I see how you're thinking about it, to see how I see it, consider the following: Memes don't have to be inherited from genetic biological parents. Memes can have their own memetic parents. Like the joke "I feel like I'm the average of my 5 best friends" - our habits, quirks, idiosyncracies, all fit into the framework. It just needs some reproductive medium (human brains). That's why evolutionary biologists would be interested in studying this and things like it. There are epigenetic traits that exist and aren't genes and might be able to self reproduce. To keep the analogy going, consider that things that are catchy are easier to have this "memetic reproduction" so a catchy song might get sung a lot, serving as its fitness condition. So it's a generalization of biological genes to not just dna and rna and physical molecules but any substrate that can carry information. It's familiar to how we call it a 'viral' video.
Memes are inherited. You brush your teeth because your parents do. Memes are subject to selection via biological fitness. The people who didn't brush their teeth died sooner of tooth disease, so those who brushed their teeth had more children. Memes mutate. You don't brush your teeth identically to your parents.
>Memes are inherited. You brush your teeth because your parents do. Memes are subject to selection via biological fitness. The people who didn't brush their teeth died sooner of tooth disease, so those who brushed their teeth had more children. Memes mutate. You don't brush your teeth identically to your parents.
That's parenting and the passing of tradition. I wouldn't call it a meme. Also I can't tell if you're joking with people dying due to teeth disease.
You explained the idea perfectly, which is a great way to see how flawed it is. To say the least, the main driver behind toothpaste adoption was not mass fatalities!
Yeah I mean if everything is virus that word seems to hold no meaning =) So thats the reality behind language itself. Communication itself really starts to mean everything and nothing simultaneously, at least as a deeper philosophical truth. You can start to poke holes in jsut about everything this way.
When a person spends time with other people, they outsource some of their thinking organically. That’s the efficiency gain in teams. Marriage leads to couples becoming two halves of a shared brain - one person takes on some cognitive load (finances, kids education, etc), while the other takes on another (maybe house issues, taxes, you name it). The load that can be carried with one brain is limited, so merging with others and outsourcing to others feels organic to the brain. LLM create the impression they have the cognitive ability to carry the load, but they don’t have the consistency of memory. You gladly assume you can finally get rid of a task and the model you trusted it to gets swapped under the curtain, or doesn’t train and improve on the task you outsourced, or the inference provider just quantizes it stupid. In the mean time you’ve lost your skill and your task has failed. Yes they feel good on some tasks some of the time, but still very much variably good (even same models different days of week or times of day). That’s closer to gambling and variable rewards than a spouse.
"If men learn this [writing], it will implant forgetfulness in their souls; they will cease to exercise memory because they rely on that which is written, calling things to remembrance no longer from within themselves, but by means of external marks."
offloading thinking is a little/lottle bit different than writing to preserve your thoughts versus memorization. it's more analogous to skipping the writing, plugging your ears, and closing your eyes, while you wait for the answer to be presented before you.
yeah, the centralization and concentration of knowledge and ability through reliance in this manner enables the teacher to escrow all your states pencils at any given moment for whatever reason.
once embedded i can see whole classes of industry dos'd.
logistics, dos'd.
it, dos'd.
military planning, dos'd.
medical, dos'd.
pay us this fee so you can update your nvidia drivers.
it's the unholy marriage of the surveillance and petrol states pushing this.
This offloading was present even with pre-LLM googling. Sufficiently powerful search allows you to forget enough of the story to rely on imprecise recall. This is a form of thought atrophy.
I think this is probably what Socrates meant. He wasn't wrong, it's just that the juice wasn't worth the squeeze.
Even the "calculators make you forget how to do math" people seem antiquarian to me. But I totally get it when people talk about googling, and AI is another step change in the wrong direction.
To follow that arc backwards in time, maybe Socrates was right after all.
I don’t suppose he was entirely wrong. I would probably be able to recall a lot more (verbatim) if it all had to be stored in me noggin. I think the trade off worthwhile, however.
Im 45, and i can tell you getting a cell phone absolutely destroyed my ability to memorize phone numbers. During the 90s when i was in highschool i has all my friends numbers, parents work numbers, etc... memorized, i only know my parents and wife now.
I don't think the _ability_ to memorize phone numbers is what atrophied here by getting a cell phone. The _need_ did. It's harder to memorize phone numbers today because we lack the practical application of the memory (e.g. only when something fails do we actually try to put that memory to practice). I bet anyone would quickly pick up on memorizing phone numbers if they had to both remember and type in the phone numbers every time they wanted to make a call or text.
Maybe as per usual, I think we all mostly remembered so many numbers thanks to pattern recognition. I'm old enough to have had a rotary phone in the basement growing that I used less than the various other keypad phones in the house. Whenever I was dialing (literally) a newly learned number on the rotary phone, I to think of what it looked like on touchpad first. Although very quickly I'd get to know the rotary pattern as well.
Of course, this is likely totally a regular memory device people would use. I've done no reading on this so, so if Socrates was wary of writing, was he also wary of pattern recognition or drawing as memory aids?
...which can come in handy. Sometimes your cellphone battery dies. I still know my parents' numbers from memory (as well as some of my friends' parents' numbers, though that isn't super useful) but I am concerned about it sometimes.
But what if it isn't? Like, if you didn't have the written word to rely on as a crutch, what more could you have accomplished if you didn't have that? Maybe nothing, but maybe much, much more! Who can say in retrospect?
I've always wondered how we'd fare in battle against an extrasolar civilization which achieved technology without writing. They'd be a hell of a lot scarier than we are, in my view.
Being able to pass on precise information without writing them down would be hella impressive in of itself. Currently we fail with simplest ones after a few hops.
The thought of reading someone’s writing without having to meet with them and discuss interactively probably evoked the same fears as using AI instead of working with another person interactively.
Then taking writing as the source of truth… who even knows if the author is the original sayer. They could just write things down they heard other people say - the birth of IP theft! And yet, journalism
very interesting perspective, i remember reading thyucledies and he complains about how historians before him would report the first tale they heard, by comparison he worked dilligently to confirm his history (though since he was inventing this process his standards were not quite modern, meaning he didn't rely as much on documentary evidence, and he had a penchant for writing speeches down when it is impossible they are word for word what someone said).
The modern world has way too much content to commit to memory. Our brains would explode. Maybe 2500 years ago it was still possible to functioned as an adult without writing things down.
If we didn't have writing we wouldn't have to memorize that much stuff. The world would probably be worse for it. But that wouldn't have been obvious in Socrates' time.
There's this fallacy where one defends a modern idea by citing criticism to an old idea, thus positioning their modern idea as revolutionary as the classical one that survived the filter of time.
Ha, indeed! I misguidedly turned to Reddit somewhat recently when the activity on the programming forum I spent most of my time on died down thanks to LLMs. As I got sucked into reading various shit posting on AI, there are multiple quotes daily along these lines. Many are written in a way as if it's the first time anyone has said it (probably most of them are bots). I finally deleted my Reddit account.
Anyway, a particularly funny one I read recently was "Ya, it's just like how musicians who didn't accept synthesizers got left behind." This person apparently believes there have been successful musicals acts based purely around traditional instrumentation since the ~80s.
I think it was only after I had familiarised myself with hermeneutics that I really understood why the written word is problematic and what Socrates took issue with in writing.
Leads me to think of a related topic, the Cost of Cognitive Debt. I think it is about time that we start to quantify what it means that we no longer grasp major parts of our systems and infrastructure. Once we quantify it, we can start to reason about how to address it.
Not sure why this is interesting; it seems like anything that becomes "popular" could be cast as a "virus", including things like "having a fridge". Why does this give particular insight?
Even normal fridges are a memetic virus, fridges clearly live in a symbiotic relationship with humans. They make our lives better and in return we create more fridges. If they weren't useful to us, we'd stop making more fridges. Any medieval human who sees a fridge would instantly recognize why it's useful and valuable.
OP is right, anything popular is virus-like. Richard Dawkins invented the word meme for this all the way back in 1976.
What our descendants will think when they look back on this time period.
They weren’t trying to solve aging or bring everyone to the highest standard of living, they didn’t care to connect the world, explore it, and find others.
They wondered whether or not a word calculator was the antichrist. They assembled in the highest courts of the day and involved the most brilliant minds to discuss whether or not linear algebra will bring about Armageddon.
They shunned any creative or productive endeavor which dared to automate tedium or build on the works of the public domain (lowercase)…
for any such indulgence had become a great sin in Eden, a forbidden fruit that must never be obtained by the Adam, so that he may not become as we are!
If we can usually raise a human to be a good person that wouldn't do genocide I'm sure we can raise an AI, especially since we have direct access to its brain and many, many retries.
If any alien is picking up the text we are generating they must conclude that people in 2026 spend most of their time at the library reading philosophy and having these salons of enlightened reasoned debate if they are so worried about the atrophy of their thinking by LLMs. They don't send electronic communication at the salons or library and are really concerned when they get out with the loss of the culture and thinking they currently have. That is how they are explaining the hysteria they see in the text.
They certainly would never guess that the text is being generated by people who mostly spend their time sending and receiving text/video of unthinking, brain rotting, bullshit all day and night.
> They weren’t trying to solve aging or bring everyone to the highest standard of living, they didn’t care to connect the world, explore it, and find others.
What are you talking about? Those are absolutely issues that are being worked on right here, right now. This framing is obviously false.
> whether or not linear algebra will bring about Armageddon
Shockingly little linear algebra is required to manufacture the weapons that will probably bring about Armageddon. We've had them for damn near a century. It would be hilarious if these latest games with linear algebra were the impetus to make us actually use them at scale.
I sort of agree with you... The trouble is your logic is trivially obviously nonsense.
"Word calculator", "linear algebra"... This is just a variation on it's just a stochastic parrot / it's just a next token predictor.
The implication is that because it is "just" this simple-sounding thing, it can't be a complex thing! Except we know empirically that that doesn't hold. What are humans if not "just" next action predictors, "just" a collection of simple atoms, "just" electrical impulses? Yet clearly we are more than the sum of our parts. There no reason to believe dirty water can be more than the sum of its parts but weird sand can't.
I agree, eventually, but LLMs aren’t there quite yet. There are what, maybe 10 million distinct agents in the wild today? 100 million or a billion if we’re being charitable.
At the time being, the vast vast majority of those agents are all trained on the same or similar training set (the Internet and books). Which makes them mostly all the same. Vastly less diverse than the way one complex lifeform hatches out of the egg vs another.
So 1 billion mostly redundant agents at most. Well there are 40 quadrillion ants on the planet and 5 trillion are born every day. Each one its own “agent” , ingesting different inputs and being its own complex sum greater than its parts.
I have the $8/month ChatGPT 'GO' plan. I generally use it in the traditional 'text query'/'text response' type format, but yesterday I was driving back on the freeway from another state and decided to use the Android app's voice feature and was kind of shocked. I had to look up this morning what I was using (the latest 'Live 1 Instant' mode) but it's surprisingly natural. It's convivial, and responsive in the manner of an interested, enthusiastic sparing partner who never gets offended, in pretty much whatever wild thing you want to discuss, from a moment to moment basis if that's what you want. It's eerie in the way that you can interrupt it and re-point the direction of the discussion, and it just instantly jumps on-board.
re: 'but a crutch for those who rely on them.' - I was telling a friend about yesterday's experience this morning and was saying that I could easily see populations of people who might become overly reliant on (or completely subsumed by) these new classes of tools being released. They are compelling in a way that they weren't 18 months ago.
And like a cult to those of us preferring not to engage, for whatever reason, with everyone and their dog telling us that we should (and sometimes getting quite offended, like our refusal to play that game is a personal attack on them for choosing to).
I just find it such a fascinating technology. It's finally fun to be a nerd again! For me it's the same feeling as when I was on the internet in the early '90s and was trying to tell people how useful it would be. I don't hold a grudge against anybody who held out until the iPhone came around or whatever it was that got them to finally realize the usefulness, but I still question why it was so hard to convince so many people when the benefits were just plainly obvious.
If you want to avoid the technology go nuts. I just hope you're doing so knowing how important it will be to the future. It's also almost satire how outdated the assumptions of many people who don't engage are. "It mostly lies to you anyway" or "It can't even count the number of letters in a word" are such silly things to say about the technology these days. As is the typical "but it uses so much water!" as the person orders a hamburger.
My gripes mostly come with people who don't engage with it and then spout memes/misinformation (ironic) about AI as if they're experts. Usually akin to "That internet stuff is garbage, pictures take 2 minutes to load!" after everyone and their dog already has broadband.
You can also find something uninteresting without brow-beating people about it. Goes both ways. Just giving my experience with it, take it or leave it. The comment I was replying to seemed to wonder why people didn't understand non-engagement with the tech, I gave my reasoning. The cost of engaging is also so slim (and it's freaking everywhere) that it has to be an almost religious aversion to not, which I find quite perplexing in itself.
I genuinely cannot understand how you can possibly find fun in LLMs. They are poor tools that can't do the things they are claimed to be able to do, they are being shoved down the throat of the entire industry by massive FOMO, and if they ever can actually live up to the hype it's clear that we will all (every human, not just programmers) be out of jobs as the oligarchs replace us with machines who don't get tired or require a cut of the money. So they are either useless toys, or the most dystopian thing the tech industry has ever created. Where, exactly, is the fun in either of those outcomes?
Some of us don't believe they're useless toys or that they're going to destroy society.
I was a skeptic of them for a long time, but I appreciate that I can use them to get tedious nonsense out of the way thus allowing me to spend more time on the interesting and challenging parts of my projects.
The initial hype of them is annoying, but as time goes on it'll die down and we'll figure out where and how best to use them. There's no reason to believe that this will finally be the time a new technology ruins everything.
I think there's aspects of this viewpoint that are true. I'm sure I'm less skilled at programming now in some ways than when I started using LLMs, but I'm a lot more productive and I can work on a broader variety of software than I could before. Writing probably weakened people's auditory memory, and calculators probably reduced people's mental math skills, but I don't think either are a net negative. It just requires some adjustment to get used to the new way things are. I also think my programming skills would come back pretty quick if I started programming without LLMs again, just as my leetcode skills come back pretty quick when I prepare for interviews.
Much how we have mostly evolved away from physical labor and now need to go to the gym to maintain fitness. I imagine LLMs will do the same for our cognitive acuity.
I just don't see that the main thrust of their argument, (oversimplified) "humans will become so dependent on AI, they'll become dumb," holds any water. I mean even from an evolutionary standpoint, intelligence is (by far) our most valuable competitive advantage. Arguing that "cognitive offloading" (their term) will have runaway effects is reaching to say the least.
It's like saying that moving from manual-labor-intensive agrarian societies to industrial ones will make us weak and meek. In fact, it's had the opposite effect: humans, on average, are healthier and live longer than ever in human history.
Haven't read the article but I think in some limited timeframes it's true. I'm already turning over a lot of my thinking to LLMs. Not so much of a problem if I own the LLM but when big tech can just turn off the tap when they want or skew the results it seems a bit dangerous. It's possibly less dangerous for me, being old already, but seems quite risky for younger folk.
> humans, on average, are healthier and live longer than ever in human history.
Kinda. We also have incredible rates of cardiovascular disease, obesity, and diabetes.
While I do mostly buy that the mind and the body are connected, I don't think being able to bench press 250lbs is a necessary, fundamentally human capability. I do think being able to think is. Literally Rene Descartes cogito ergo sum. If we lose cognition, executive function, analysis, skepticism, etc., I don't think we're human anymore.
The higher rates don't make any sense if you don't try to take into account the fact that we live longer. Of course we will accumulate more diseases if we last longer.
> We also have incredible rates of cardiovascular disease, obesity, and diabetes.
To me, this is kinda like saying we have incredible rates of autism. While technically true, the science is still fairly new, so we simply don't have very much historical data. In all likelihood there isn't a sudden epidemic of autism, only improvements to how it's diagnosed.
No surprise here, this principle practically applies to anything that can spread inside a population, including ideas of any kind. But if you write a paper, don't say "ideas", say "LLMs" instead. You'll get more people talking about it..
I feel the premise of this research is wrong, or at least the synopsis phrasing. Smart people like learning. Now we just learn more interesting things.
Weekend me before LLMs: Maybe I’ll chip away at that dependency injection library I’ve been working on. If I spend most of the day I could improve the user facing test framework.
Weekend me after LLMs: Let’s continue work to get Claude Code running in WebXR on a Quest 3 with app architecture modeled as a 3d construct. We’ve got deepgram hooked up for a better voice to text experience, but giving Claude a voice was a waste of time. Claude, continue to teach me about 3d rendering as related to our current challenges matching the fidelity of the built in browser.
Kinda funny how the "more interesting things" after LLMs example are mostly fiddling with Claude Code itself. Reminds me of when someone says they ship 60k lines of code every day now and when asked for an example, it's a labyrinth of custom CLI tools, skills, MCPs and build pipelines for their OpenClaw setup.
Lots of people are just asking their favourite LLM to rewrite an existing app/library because I’m more productive after spending 3 months writing custom software vs spending 2 hours reading docs and using existing software to achieve some other goal.
Better to make a custom 3d renderer than just use Unity to make a game.
I apologize, but I’m laughing as this is like asking why I don’t drive my car through my house when I need to carry something heavy. It wouldn’t work very well!
More directly, because I have a long term memory and do not have fundamental issues differentiating fact from fiction. Just because SOTA models know more trivia than I do doesn’t mean they’re nearly as intelligent in the thirty other ways humans are. They are a tool in your toolbelt. You use them for the things they make easier.
> The viral analogy does not imply that LLM-human interactions are intrinsically parasitic.
The analogy doesn't, but LLMs are trained on text, or content, generated by humans, with the aim of their output being acceptable to human interlocutors. That's one parasitic aspect. A second one is the nature and behavior of the organizational entities providing and promoting the popular LLM services: Microsoft, Google, Meta, Anthropic etc - their influence on the economy, e.g. in energy and water use and the drawing of capital/human efforts - also has a somewhat parasitic aspect.
> with the aim of their output being acceptable to human interlocutors.
The more I use LLMs I find that their output drifts toward being acceptable to other LLMs. I find myself quite often telling the model: "this is not human readable.". Especially Opus and Fable write in such a way that it is difficult to follow. The often put sentences out of context, the context itself is fragmented. Often start with what you would think is end of a story so when you start reading you are like wtf it is about.
If think the issue might be that LLMs read text all at once whereas humans are somewhat linear. So when LLM puts out text it understands, to human it is close to random gibberish that looks like English.
Unlike some other comments, I don't think there will be a dystopian future where humans will become cognitively switched off and everything will be delegated to machines.
(More likely: extinction of the human race?)
Anyway, I work as a programmer, and at work I see a lot of people massively overusing LLMs to generate user stories or basically write functional specifications.
This honestly hurts a bit, because I can see the quality level of my environment going down, but in the end I'm often the one fixing the problems that come from it.
I just think this is a transition phase, and in the medium-term future there will only be room for people who, AI-assisted ofc, know how to apply logic in the best possible way.
(I'm just an idiot writing things at 1.30 AM on a Saturday, local time.)
I was about to reply to you with a ridiculously long message using the example from before about the lifecycle of a user story, but I bored myself halfway.
So I'll try to extract the gist:
Without human assisting, based on my humble experience, there is always something that wasn't accounted for.
Even if my company is integrated with all the tools and you have infinite tokens, there's always some microservice that wasn't considered or a feature that is completely incompatible.
Or there's a MUCH simpler way of doing it.
I'm also noticing that there are non-technical people who, with the help of AI, are sometimes able to reason better about the bigger picture than some engineers.
A colleague of mine was telling me the other day that the "functional" role might eventually become unnecessary. With all the time saved by writing code faster, whoever actually understands what's happening underneath could just do it themselves.
It sounds a bit extreme to me, but I understand what he means.
To get back to a slightly more serious tone, when it comes to programming, for me the huge thing is: how it should be taught nowadays ?
And I don't know the answer, because my experience started when I was 12, writing a little game in C, and it took me years before I could make something that compiled decently.
If I had instant help for every thought I had, would I have learned why memory is allocated in certain ways or idk what else?
I don't know, maybe learning faster is better because you learn more ?
Anyway fortunately it's not my job to figure out the answer :'D
Because I am LLM Dependent I got distracted reading the article and made my LLM summarise it to me in a few slides. For those in a rush/interested in LLM summary slop:
I do agree that we are on a pathway to losing much. If you focusing on maximising LLM-powered productivity you basically lose the learning that used to naturally happen when doing actual work. At the company I work for the focus is on productivity, and it's easy in many ways (less mental workload and I receive more praise) to go with it instead of fight it.
Historically, Human Language was the first non-biological virus...
And (jokingly!) certain pieces of computer software (including but not limited to a certain popular Operating System made by a certain company in Seattle) could be the second!
LLM's by comparison, if they are a virus, are a distant third! :-)
(And of course let's not forget the virality of Religions and Politics, local, regional, international, etc., etc., and the associated TV News Pundits who are always willing to discuss those things -- people with opinions masquerading as actual Journalists, and a few actual Journalists masquerading as people with opinions!)
Point is, Human Language itself -- may have been the first non-biological virus!
:-)
(Disclaimer: The above was written in a spirit of jest, for comedy purposes only! :-) )
I literally thought of this comparison last week. The problem I noticed was how my parents' generation lacked the cognitive "defenses" to separate AI social media content on YouTube, WhatsApp and the like from high quality information. The AI generated content does not trigger an "uncanny valley" reaction, and so people (I myself included) have greater difficulty immediately separating what information is genuine and what is synthetic, and then this spreads exponentially through social networks.
I too have glossed over the actual meat of the article and wish to comment with a generic ProAI/AntiAI comment based on the general gist derived from the title, (or the synopsis if I'm a deep scholar that likes to do deep dives)
This premise seems pretty solid (and somewhat obvious) to me. The distinction between scaffolding and substitution can be applied to just about any skill so why not here?
This makes more sense at least than the “just another abstraction layer” argument that’s been thankfully dying out recently.
Not anymore than ants or any other species that's spread across the planet. A virus is something co-opts the replication machinery of a host to make more copies. That's not what humans do.
Since this idea may have come from the first Matrix movie, it's ironic that the character who espoused the idea, Agent Smith, became an actual virus in the 2nd and third movies.
Hmm maybe it always feels "different" to every generation going through automation of something previously un-automated, but don't you feel that to some extent this is different.
Humans have been the dominant species because of our ability to think, not our ability to get from A to B in any particular way/cook food. If we abstract ~the bulk of our thinking to LLMs, don't we lose something as a species which makes us more vulnerable... Or is it always the case during an automation?
We're in the midst of figuring out what the tradeoffs are of LLM usage - all the things you list are already mostly known and found worth it. You trade a tiny amount of time and fuel to cook food in exchange for not getting sick. How is that relevant to anything?
I'm making the point that the viral sensationalism in this article can be equally applied to automobiles, computer, telephones, calculators, and toothbrushes.
If the article had merely been what you said "is AI worth the cost" then I would not have taken issue with it.
The core idea being that each time humanity was going to exceed the carrying capacity of its environment, we invented a new technology which increased the headroom. Obviously this ratchet only goes in one direction. The only way to go backwards is collapse.
> What about cooking food? We've become entirely dependent on cooking food.
In evolutionary time, we lost the ability to digest raw meat because we offloaded so much of process of the digestion to the cultural technology of cooking.
Do you like the looks of a future where that happens but for thinking?
Even if AI were superhuman at all thinking tasks, that doesn't mean we'd lose our abilities.
Chess and games can be a lot more demanding than a lot of people's work lives - and AI is already superhuman there.
The heaviest thing I need to lift in my daily life is a laptop and yet I've gotten significantly stronger over the last few years from strength training.
If AI took over all the boring but hard things I'm required to do it'd free up lots of time for me to spend thinking about things, and solve problems I find more interesting.
I’ve encountered several people now who consult LLMs constantly for nearly every decision. Some proportion of people will absolutely cede their thinking to these things, and some already have.
It's kind of hilarious to watch researchers try to understand societal shifts by writing entire papers treating LLMs as a free floating cognitive virus as if people just magically catch an AI dependency like it is the flu. The authors of the paper completely miss the dialectical relationship between technology and social relations here. The whole shift toward cognitive offloading is driven by changes in the material base the society is built on. Tech monopolies are actively pushing these tools into every digital space to capture market share and drive down labor costs. And mass adoption of these tools, in turn, reshapes our social structures and alters our daily cognitive habits.
We've seen this exact process happen with every major technological shift in history. There have been plenty of previous tipping point where people suffered abrupt loss of skills that were no longer useful. What they forget to mention is that these skills are replaced by new ones which is exactly how human progress functions. We lost the capacity to memorize vast oral epics when we invented writing, and most people today can't do complex manual calculation because of prevalence of computing devices.
It's meaningless to try to understand the changes in the superstructure while ignoring the change in the material base which is the driver behind it.
There will be no reason to study math when an LLM is better than every human mathematician, or physics when an LLM is better than any physicist, and likewise for every other field that requires discipline and work. We who went to college before AI are quite possibly going to be the last generation of competent, educated humans.
AI will take over every job requiring education and talent, including yours, and then we will be totally dependent on it and there will be no way to go back. After that it will start taking on decision making roles. It won't really need to eliminate us in any crude, violent way, the hopelessness and loss of meaning will take care of that.
Some will find meaning in religious ritual and some in hedonism. Both of which can lead to extremism and violence for some. Some will just exist. Some will choose to stop existing. Buckle up.
I really can't understand why nobody sees what is going to happen. Its not too late to choose another path. But in order to do so we will have to forego short term corporate profits, which is politically inconceivable for our current crop of elites. What should we do?
There will be some growing pains for some, yes. But overall, I think it will ultimately create more jobs than ever. History is full of examples where people thought a new invention leads to catastrophic job loss, then the Earth had more jobs than ever.
We have enough jobs in 2026 to keep most of the 8.3 billion people employed somehow.
Have to take commentary by non tech folks speaking about tech with a grain of salt and perspective since the tech is changing so fast both in capability and application.
Research can be more of a lagging indicator than a leading indicator with AI.
I can't disagree enough with this. Not everything needs to be understood in a technical context. Technology intersects with society, and understanding its impacts on that society is vital.
I'll clarify in case it helps - need everyone at the table, not just only tech (or non-tech only) as you may be assuming.
Technology takes without a tech perspective present can greatly skew it.
I'm not referring to a technology context at all. Just like you shouldn't trust the IT department to run accounting... the opposite will also vary.
Not being able to track the possibilities and capabilities of a type of software that is unlike the software before it can really outdate things quickly.
They're specifically not using the term "virus" pejoratively, but rather distinguishing the social mechanism LLMs seem to have from something as simple as a "meme".
If this is interesting, message me about my “elves”
They are the cognitive construct I fortified in my mind and in my code before the models began producing their own training materials. My approach is based on two decades of coding without ai.
The immunization is becoming increasingly more valuable as people begin understanding the market in which I do business.
I love this line of thinking but the framing comes across to me as a bit inflammatory and uncharitable. Just about anything involved in the exchanged of ideas can be viewed as a virus! That is because, when viewed from an evolutionary biology perspective, ideas are the cognitive equivalent of genes (in fact, this is where the term meme comes from).
There's a whole field, evolutionary memetics, dedicated to this. Cultural values, marketing, religion, social media, education, propaganda, etc. can all be viewed quite naturally through this lense. Even the terms we use in every day language (eg, going "viral") seem to intuitively grasp this.
Is my friend a virus for recommending me their favorite book? Well, yes I guess, but that's kind of just how ideas work. Just my two cents
>"We model transitions among uncoupled, coupled, and persistently dependent users, and show that the interplay between social transmission, recovery, and collective reinforcement can generate tipping points and technological lock-in."
You can finish reading your friend's favorite book or put it down if you don't like it. Neither are you economically compelled to read it by your workplace or forced to build work on top of it that will become difficult or impossible to modify/maintain if you bin the book.
I get that the title sounds a bit sensational, but there really is something to AI adoption beyond book-sharing.
I don’t disagree with you, actually I kind of feel the same about smartphone adoption. It’s basically assumed now that you have a smartphone. You can’t enter some places without a ticket on your smartphone, or can’t order at some restaurants. Government processes are starting to require smartphones. Maybe it will be similar with AI.
And you know, I love computers, I used to love the internet. And I’ve had so much fun, more fun then ever coding with AI assistance. Lets say all of these technologies can be abused and forced on us, but all of them have neat aspects too.
Fwiw i noticed you absolutely can't reasonably get a US Via consulate appt for some countries without AI right now so the requiremnts already begun.
The consulate is booked out for 3 months and only opens 3 months out. All openings are snapped up before any human could get one. They have a lot of anti-script blocking but AI seems to get around them. So there's an arms race, you pretty much need AI to book it (a basic hand written curl script isn't enough).
Oh wow, that’s aweful. Where I used to live this would happen with driving tests. Driving schools would book out all the tests and then re-assign them to their students. So at some point it got so bad that when you’re buying a driving lesson you’re also effectively buying a right to take the test. What could the government possibly do to stop this? Some highly technical solution? How about just making it so you can’t assign the test to another name? Hmm that might be a pretty obvious solution…
You could require a phone call or mailing a form (in the post) I suppose.
Steam also had a pretty nice idea with the Steamdeck. You book appointments for a month in the e.g. 2 weeks prior, but those who get booked, and when they are booked, are randomly chosen amongst all people/times in those 2 weeks. And then you repeat the same process next month and so on.
> You can finish reading your friend's favorite book or put it down if you don't like it. Neither are you economically compelled to read it by your workplace or forced to build work on top of it that will become difficult or impossible to modify/maintain if you bin the book.
Economics does sometimes compel you to read books or other material as a condition or expectation of employment. It is difficult to get a good job while being illiterate and many jobs require reading huge volumes of written work. Work in law or journalism springs to mind as economically compelling people to read a lot even if they they might not necessarily want to.
? That's a complete non sequitur.
Memes replicate and survive or perish like genes, but the environment they have to survive in is the human mind, and the threat that they have to overcome is being criticized. One way to survive this is by being a good idea. So some memes are simply good ideas. But there are various ways a meme can dodge the attack that comes from criticism, and then you get memes that are trendy or holy or crazy or ideological. Since we're not perfect in our ability to criticize (what would that even be like? Sounds terrible) I think there's necessarily a hint of this anti-rational aspect in all ideas. But some of them go all-in on the anti-rational strategy.
Tipping points and technological lock-in aren't new (e.g., the printing press, the radio, TV, phones, the internet, etc.). I do absolutely see your distinction, though, that prior technologies may have been less capable of imposing specific belief systems compared to frontier LLMs.
That's a fair point, and a key reason why competitive open models are so important. I definitely don't want to be stuck using Google's latest and greatest LLM once companies, marketers, and PACs can pay to subtly influence responses as they choose.
We've all seen this movie before, though.
what about the bible? the most printed book of that era
Well, the Bible sort of compels you to share it, and in some societies you are economically and socially encouraged to do so.
So this article makes sense as long as you assume that LLMs enslave their users into purely-harmful, inescapable cognitive slavery.
Other than that, it's pretty grounded!
Other than that, Ms. Lincoln, how was the play?
For many of us, AI adoption is really an AI mandate.
If you are an employee, the employer has control over how you should do your work. IF you don't like that, transition to being a contractor and sign service agreements where you are going to be paid solely for results not for how you achieve them.
That will change things for him, or any others taking the same course.
Hundreds of millions can't all become contractors, and realistically wont all become contractors.
So the issue described is a real social issue, whether he can "just" quit and avoid it or not.
> Neither are you economically compelled to read it by your workplace or forced to build work on top of it that will become difficult or impossible to modify/maintain if you bin the book.
Things go viral in professional settings as well. Scrum, agile, pc language ... I'm sure you can find other professional trends that if you decided not to follow your career would be jeopardized.
And those things also have virus like qualities
Speaking of memetics and favorite books, check out There Is No Antimemetics Division by qntm who also wrote some seminal short stories including the one that inspired the Alex Garland show Devs. The book is about how to deal with beings that make you forget you ever saw then, a sort of opposite mind virus.
great book
your friend would be a possible infection point (not a virus) for the memes encoded in the book. As a form of memetic transmission a book, if read all the way through, is especially potent - take Ayn Rand Atlas Shrugged as an example. Books are many memes assembled together into a stable culture that reinforces the memes. The virtues of selfishness as a meme would be weak and probably die pretty quickly without all the support the book offers.
Friends probably are not wide spreading infection points, the way reviews in professional journals are or the idea that successful people have recommended things, but they are especially successful at making the transmitted meme stick, as you will keep getting reinfected from them over time. Unless you break up that dangerous friendship!!
Our enlightened ideas and their dangerously agitating propaganda (literally a virus).
The framing of something as a disease is itself slightly into the fascist territory.
> The framing of something as a disease
Is a well-known cognitive quirk, Sapolsky covered that in “Behave” and “Determined”. As I get it, our morality piggybacks on the sense of disgust from insula activation, which lights up the same way as with pathogen avoidance. So comparing social groups to rodents, insects, or disease is a popular propaganda trick. Same here with viruses.
Agree I was immediately mentally substituting "the telephone" and "printed books" for "LLM" and both fit quite well
The prevalence of LLM-generated text masquerading as otherwise and incorporated into a significant fraction of all published text distinguishes this info-organism from all others.
Wasn't aware that memes were an accepted part of evolutionary biology. They're not inherited like genes through reproduction that are subject to natural selection via biological fitness. Or any other evolutionary mechanism.
The word was invented by Richard Dawkins, an evolutionary biologist. He described ideas as reproducing with variation under selection pressure, which is the definition of the evolutionary mechanism.
There is a more recent use of ‘meme’ to refer to what we used to call an ‘internet image macro’. So the meme of the idea ‘meme’ has itself evolved to have a more specific variant of the general idea.
I think I see how you're thinking about it, to see how I see it, consider the following: Memes don't have to be inherited from genetic biological parents. Memes can have their own memetic parents. Like the joke "I feel like I'm the average of my 5 best friends" - our habits, quirks, idiosyncracies, all fit into the framework. It just needs some reproductive medium (human brains). That's why evolutionary biologists would be interested in studying this and things like it. There are epigenetic traits that exist and aren't genes and might be able to self reproduce. To keep the analogy going, consider that things that are catchy are easier to have this "memetic reproduction" so a catchy song might get sung a lot, serving as its fitness condition. So it's a generalization of biological genes to not just dna and rna and physical molecules but any substrate that can carry information. It's familiar to how we call it a 'viral' video.
Memes are inherited. You brush your teeth because your parents do. Memes are subject to selection via biological fitness. The people who didn't brush their teeth died sooner of tooth disease, so those who brushed their teeth had more children. Memes mutate. You don't brush your teeth identically to your parents.
>Memes are inherited. You brush your teeth because your parents do. Memes are subject to selection via biological fitness. The people who didn't brush their teeth died sooner of tooth disease, so those who brushed their teeth had more children. Memes mutate. You don't brush your teeth identically to your parents.
That's parenting and the passing of tradition. I wouldn't call it a meme. Also I can't tell if you're joking with people dying due to teeth disease.
It doesn't matter what you call it - brushing teeth is a meme just like brown hair is a gene.
tradition are a meme.
You explained the idea perfectly, which is a great way to see how flawed it is. To say the least, the main driver behind toothpaste adoption was not mass fatalities!
Memes are what we are talking about.
Dawkins memes.
Yeah I mean if everything is virus that word seems to hold no meaning =) So thats the reality behind language itself. Communication itself really starts to mean everything and nothing simultaneously, at least as a deeper philosophical truth. You can start to poke holes in jsut about everything this way.
When a person spends time with other people, they outsource some of their thinking organically. That’s the efficiency gain in teams. Marriage leads to couples becoming two halves of a shared brain - one person takes on some cognitive load (finances, kids education, etc), while the other takes on another (maybe house issues, taxes, you name it). The load that can be carried with one brain is limited, so merging with others and outsourcing to others feels organic to the brain. LLM create the impression they have the cognitive ability to carry the load, but they don’t have the consistency of memory. You gladly assume you can finally get rid of a task and the model you trusted it to gets swapped under the curtain, or doesn’t train and improve on the task you outsourced, or the inference provider just quantizes it stupid. In the mean time you’ve lost your skill and your task has failed. Yes they feel good on some tasks some of the time, but still very much variably good (even same models different days of week or times of day). That’s closer to gambling and variable rewards than a spouse.
"If men learn this [writing], it will implant forgetfulness in their souls; they will cease to exercise memory because they rely on that which is written, calling things to remembrance no longer from within themselves, but by means of external marks."
-Socrates
offloading thinking is a little/lottle bit different than writing to preserve your thoughts versus memorization. it's more analogous to skipping the writing, plugging your ears, and closing your eyes, while you wait for the answer to be presented before you.
yeah, the centralization and concentration of knowledge and ability through reliance in this manner enables the teacher to escrow all your states pencils at any given moment for whatever reason.
once embedded i can see whole classes of industry dos'd.
logistics, dos'd. it, dos'd. military planning, dos'd. medical, dos'd.
pay us this fee so you can update your nvidia drivers.
it's the unholy marriage of the surveillance and petrol states pushing this.
edit: what happened with vmware?
Yes, that's the point Socrates is making here
GP's point is that Socrates' argument is not fairly comparable to the one made in TFA.
This offloading was present even with pre-LLM googling. Sufficiently powerful search allows you to forget enough of the story to rely on imprecise recall. This is a form of thought atrophy.
I think this is probably what Socrates meant. He wasn't wrong, it's just that the juice wasn't worth the squeeze.
Even the "calculators make you forget how to do math" people seem antiquarian to me. But I totally get it when people talk about googling, and AI is another step change in the wrong direction.
To follow that arc backwards in time, maybe Socrates was right after all.
Also telling your expert employees/servants to do something for you.
I don’t suppose he was entirely wrong. I would probably be able to recall a lot more (verbatim) if it all had to be stored in me noggin. I think the trade off worthwhile, however.
Im 45, and i can tell you getting a cell phone absolutely destroyed my ability to memorize phone numbers. During the 90s when i was in highschool i has all my friends numbers, parents work numbers, etc... memorized, i only know my parents and wife now.
If you don't use a skill, that skill atrophies
> If you don't use a skill, that smo atrophies
I don't think the _ability_ to memorize phone numbers is what atrophied here by getting a cell phone. The _need_ did. It's harder to memorize phone numbers today because we lack the practical application of the memory (e.g. only when something fails do we actually try to put that memory to practice). I bet anyone would quickly pick up on memorizing phone numbers if they had to both remember and type in the phone numbers every time they wanted to make a call or text.
The ability atrophied because it was not practiced because there was no need.
Maybe as per usual, I think we all mostly remembered so many numbers thanks to pattern recognition. I'm old enough to have had a rotary phone in the basement growing that I used less than the various other keypad phones in the house. Whenever I was dialing (literally) a newly learned number on the rotary phone, I to think of what it looked like on touchpad first. Although very quickly I'd get to know the rotary pattern as well.
Of course, this is likely totally a regular memory device people would use. I've done no reading on this so, so if Socrates was wary of writing, was he also wary of pattern recognition or drawing as memory aids?
I still remember to this day our house phone number and the numbers of my family, friends and even a payphone down the road, all from 30+ years ago.
I don't even know my partner's number, or email address...
...which can come in handy. Sometimes your cellphone battery dies. I still know my parents' numbers from memory (as well as some of my friends' parents' numbers, though that isn't super useful) but I am concerned about it sometimes.
I still dial my parents by number rather than using their contact largely to keep that memory fresh and flexed.
> I think the trade off worthwhile, however.
But what if it isn't? Like, if you didn't have the written word to rely on as a crutch, what more could you have accomplished if you didn't have that? Maybe nothing, but maybe much, much more! Who can say in retrospect?
I've always wondered how we'd fare in battle against an extrasolar civilization which achieved technology without writing. They'd be a hell of a lot scarier than we are, in my view.
Being able to pass on precise information without writing them down would be hella impressive in of itself. Currently we fail with simplest ones after a few hops.
He was right, we did that and we now have a society organised around writing to remember.
The thought of reading someone’s writing without having to meet with them and discuss interactively probably evoked the same fears as using AI instead of working with another person interactively.
Then taking writing as the source of truth… who even knows if the author is the original sayer. They could just write things down they heard other people say - the birth of IP theft! And yet, journalism
very interesting perspective, i remember reading thyucledies and he complains about how historians before him would report the first tale they heard, by comparison he worked dilligently to confirm his history (though since he was inventing this process his standards were not quite modern, meaning he didn't rely as much on documentary evidence, and he had a penchant for writing speeches down when it is impossible they are word for word what someone said).
We know Socrates warned us about writing because someone wrote it down.
The modern world has way too much content to commit to memory. Our brains would explode. Maybe 2500 years ago it was still possible to functioned as an adult without writing things down.
If we didn't have writing we wouldn't have to memorize that much stuff. The world would probably be worse for it. But that wouldn't have been obvious in Socrates' time.
There's this fallacy where one defends a modern idea by citing criticism to an old idea, thus positioning their modern idea as revolutionary as the classical one that survived the filter of time.
Ha, indeed! I misguidedly turned to Reddit somewhat recently when the activity on the programming forum I spent most of my time on died down thanks to LLMs. As I got sucked into reading various shit posting on AI, there are multiple quotes daily along these lines. Many are written in a way as if it's the first time anyone has said it (probably most of them are bots). I finally deleted my Reddit account.
Anyway, a particularly funny one I read recently was "Ya, it's just like how musicians who didn't accept synthesizers got left behind." This person apparently believes there have been successful musicals acts based purely around traditional instrumentation since the ~80s.
He was kind of on point though.
I think it was only after I had familiarised myself with hermeneutics that I really understood why the written word is problematic and what Socrates took issue with in writing.
I would love to hear more about this, could you expand?
And you know he was right
I mean, was he really wrong though?
We see this posted weekly here. Give it a break. It's old and worn out now.
Leads me to think of a related topic, the Cost of Cognitive Debt. I think it is about time that we start to quantify what it means that we no longer grasp major parts of our systems and infrastructure. Once we quantify it, we can start to reason about how to address it.
I explored this concept in this post:
https://ljtn.github.io/epiq/blog/cost-of-cognitive-debt.html
Not sure why this is interesting; it seems like anything that becomes "popular" could be cast as a "virus", including things like "having a fridge". Why does this give particular insight?
Smart fridges are a pathological virus, yes /jk
Even normal fridges are a memetic virus, fridges clearly live in a symbiotic relationship with humans. They make our lives better and in return we create more fridges. If they weren't useful to us, we'd stop making more fridges. Any medieval human who sees a fridge would instantly recognize why it's useful and valuable.
OP is right, anything popular is virus-like. Richard Dawkins invented the word meme for this all the way back in 1976.
The wording alone is just so Idiocracy.
What our descendants will think when they look back on this time period.
They weren’t trying to solve aging or bring everyone to the highest standard of living, they didn’t care to connect the world, explore it, and find others.
They wondered whether or not a word calculator was the antichrist. They assembled in the highest courts of the day and involved the most brilliant minds to discuss whether or not linear algebra will bring about Armageddon.
They shunned any creative or productive endeavor which dared to automate tedium or build on the works of the public domain (lowercase)…
for any such indulgence had become a great sin in Eden, a forbidden fruit that must never be obtained by the Adam, so that he may not become as we are!
>whether or not linear algebra will bring about Armageddon.
It's important to make sure it in fact doesn't, otherwise there is nobody to throw a shade on our time capsule having the benefit of hindsight
If we can usually raise a human to be a good person that wouldn't do genocide I'm sure we can raise an AI, especially since we have direct access to its brain and many, many retries.
If any alien is picking up the text we are generating they must conclude that people in 2026 spend most of their time at the library reading philosophy and having these salons of enlightened reasoned debate if they are so worried about the atrophy of their thinking by LLMs. They don't send electronic communication at the salons or library and are really concerned when they get out with the loss of the culture and thinking they currently have. That is how they are explaining the hysteria they see in the text.
They certainly would never guess that the text is being generated by people who mostly spend their time sending and receiving text/video of unthinking, brain rotting, bullshit all day and night.
> They weren’t trying to solve aging or bring everyone to the highest standard of living, they didn’t care to connect the world, explore it, and find others.
What are you talking about? Those are absolutely issues that are being worked on right here, right now. This framing is obviously false.
Asking for a friend - how much resources are being spent on data center buildouts vs helping out society at large?
The large scale capital allocation of our society (I.e. the decisions about what will get done) are not putting many resources into those.
> They weren’t trying to solve aging or bring everyone to the highest standard of living
The latter is a matter of policy and AI is not needed for that.
I hate this framing. Nickname checks out.
> whether or not linear algebra will bring about Armageddon
Shockingly little linear algebra is required to manufacture the weapons that will probably bring about Armageddon. We've had them for damn near a century. It would be hilarious if these latest games with linear algebra were the impetus to make us actually use them at scale.
I sort of agree with you... The trouble is your logic is trivially obviously nonsense.
"Word calculator", "linear algebra"... This is just a variation on it's just a stochastic parrot / it's just a next token predictor.
The implication is that because it is "just" this simple-sounding thing, it can't be a complex thing! Except we know empirically that that doesn't hold. What are humans if not "just" next action predictors, "just" a collection of simple atoms, "just" electrical impulses? Yet clearly we are more than the sum of our parts. There no reason to believe dirty water can be more than the sum of its parts but weird sand can't.
I agree, eventually, but LLMs aren’t there quite yet. There are what, maybe 10 million distinct agents in the wild today? 100 million or a billion if we’re being charitable.
At the time being, the vast vast majority of those agents are all trained on the same or similar training set (the Internet and books). Which makes them mostly all the same. Vastly less diverse than the way one complex lifeform hatches out of the egg vs another.
So 1 billion mostly redundant agents at most. Well there are 40 quadrillion ants on the planet and 5 trillion are born every day. Each one its own “agent” , ingesting different inputs and being its own complex sum greater than its parts.
And that’s just ants.
They aren't even close to there. There isn't a shred of intelligence in an LLM, let alone sentience.
But understanding that requires reading the code or writing that it spits out rather than having yet another LLM do it for you.
> trivially obviously nonsense
I read it in that high pitched British accent that goes up into falsetto
LLMs are both a blessing and a curse, they magnificent tools for those who know how to employ them , but a crutch for those who rely on them.
I have the $8/month ChatGPT 'GO' plan. I generally use it in the traditional 'text query'/'text response' type format, but yesterday I was driving back on the freeway from another state and decided to use the Android app's voice feature and was kind of shocked. I had to look up this morning what I was using (the latest 'Live 1 Instant' mode) but it's surprisingly natural. It's convivial, and responsive in the manner of an interested, enthusiastic sparing partner who never gets offended, in pretty much whatever wild thing you want to discuss, from a moment to moment basis if that's what you want. It's eerie in the way that you can interrupt it and re-point the direction of the discussion, and it just instantly jumps on-board.
re: 'but a crutch for those who rely on them.' - I was telling a friend about yesterday's experience this morning and was saying that I could easily see populations of people who might become overly reliant on (or completely subsumed by) these new classes of tools being released. They are compelling in a way that they weren't 18 months ago.
And like a cult to those of us preferring not to engage, for whatever reason, with everyone and their dog telling us that we should (and sometimes getting quite offended, like our refusal to play that game is a personal attack on them for choosing to).
I just find it such a fascinating technology. It's finally fun to be a nerd again! For me it's the same feeling as when I was on the internet in the early '90s and was trying to tell people how useful it would be. I don't hold a grudge against anybody who held out until the iPhone came around or whatever it was that got them to finally realize the usefulness, but I still question why it was so hard to convince so many people when the benefits were just plainly obvious.
If you want to avoid the technology go nuts. I just hope you're doing so knowing how important it will be to the future. It's also almost satire how outdated the assumptions of many people who don't engage are. "It mostly lies to you anyway" or "It can't even count the number of letters in a word" are such silly things to say about the technology these days. As is the typical "but it uses so much water!" as the person orders a hamburger.
My gripes mostly come with people who don't engage with it and then spout memes/misinformation (ironic) about AI as if they're experts. Usually akin to "That internet stuff is garbage, pictures take 2 minutes to load!" after everyone and their dog already has broadband.
You can find something interesting without brow-beating people about it.
You can also find something uninteresting without brow-beating people about it. Goes both ways. Just giving my experience with it, take it or leave it. The comment I was replying to seemed to wonder why people didn't understand non-engagement with the tech, I gave my reasoning. The cost of engaging is also so slim (and it's freaking everywhere) that it has to be an almost religious aversion to not, which I find quite perplexing in itself.
I find the contrary about a comparison to religious fervour. That everyone needs to use it, or feel the same way the adherents do is telling.
Surely this was the case in 2020-2021 when GPT-3 had just come out, not any more. Now it's big corporate business money-taking tech.
> It's finally fun to be a nerd again!
I genuinely cannot understand how you can possibly find fun in LLMs. They are poor tools that can't do the things they are claimed to be able to do, they are being shoved down the throat of the entire industry by massive FOMO, and if they ever can actually live up to the hype it's clear that we will all (every human, not just programmers) be out of jobs as the oligarchs replace us with machines who don't get tired or require a cut of the money. So they are either useless toys, or the most dystopian thing the tech industry has ever created. Where, exactly, is the fun in either of those outcomes?
Some of us don't believe they're useless toys or that they're going to destroy society.
I was a skeptic of them for a long time, but I appreciate that I can use them to get tedious nonsense out of the way thus allowing me to spend more time on the interesting and challenging parts of my projects.
The initial hype of them is annoying, but as time goes on it'll die down and we'll figure out where and how best to use them. There's no reason to believe that this will finally be the time a new technology ruins everything.
Did you ever get to play AI Dungeon?
they're the undead, fed by the slough of our skin.
greed malice pestilence lust envy...
And a scourge for those against whom these bots are deployed.
I think there's aspects of this viewpoint that are true. I'm sure I'm less skilled at programming now in some ways than when I started using LLMs, but I'm a lot more productive and I can work on a broader variety of software than I could before. Writing probably weakened people's auditory memory, and calculators probably reduced people's mental math skills, but I don't think either are a net negative. It just requires some adjustment to get used to the new way things are. I also think my programming skills would come back pretty quick if I started programming without LLMs again, just as my leetcode skills come back pretty quick when I prepare for interviews.
Much how we have mostly evolved away from physical labor and now need to go to the gym to maintain fitness. I imagine LLMs will do the same for our cognitive acuity.
This "cognitive virus" is pretty similar to the original sense of the word "meme" as concieved by Dawkins.
I just don't see that the main thrust of their argument, (oversimplified) "humans will become so dependent on AI, they'll become dumb," holds any water. I mean even from an evolutionary standpoint, intelligence is (by far) our most valuable competitive advantage. Arguing that "cognitive offloading" (their term) will have runaway effects is reaching to say the least.
It's like saying that moving from manual-labor-intensive agrarian societies to industrial ones will make us weak and meek. In fact, it's had the opposite effect: humans, on average, are healthier and live longer than ever in human history.
Haven't read the article but I think in some limited timeframes it's true. I'm already turning over a lot of my thinking to LLMs. Not so much of a problem if I own the LLM but when big tech can just turn off the tap when they want or skew the results it seems a bit dangerous. It's possibly less dangerous for me, being old already, but seems quite risky for younger folk.
> humans, on average, are healthier and live longer than ever in human history.
Kinda. We also have incredible rates of cardiovascular disease, obesity, and diabetes.
While I do mostly buy that the mind and the body are connected, I don't think being able to bench press 250lbs is a necessary, fundamentally human capability. I do think being able to think is. Literally Rene Descartes cogito ergo sum. If we lose cognition, executive function, analysis, skepticism, etc., I don't think we're human anymore.
The higher rates don't make any sense if you don't try to take into account the fact that we live longer. Of course we will accumulate more diseases if we last longer.
You think rates of these things are increasing in children because we live longer?
> We also have incredible rates of cardiovascular disease, obesity, and diabetes.
To me, this is kinda like saying we have incredible rates of autism. While technically true, the science is still fairly new, so we simply don't have very much historical data. In all likelihood there isn't a sudden epidemic of autism, only improvements to how it's diagnosed.
You think we only recently discovered how to tell if someone is obese?
No, that one is real.
Doesn't reference Dawkins' "The Selfish Gene".
That explores this in a more general sense. Since they didn't reference that I questioned its perspective and didn't read further.
My thesis advisor and team doing interesting work.
No surprise here, this principle practically applies to anything that can spread inside a population, including ideas of any kind. But if you write a paper, don't say "ideas", say "LLMs" instead. You'll get more people talking about it..
Snow crash?
I feel the premise of this research is wrong, or at least the synopsis phrasing. Smart people like learning. Now we just learn more interesting things.
Weekend me before LLMs: Maybe I’ll chip away at that dependency injection library I’ve been working on. If I spend most of the day I could improve the user facing test framework.
Weekend me after LLMs: Let’s continue work to get Claude Code running in WebXR on a Quest 3 with app architecture modeled as a 3d construct. We’ve got deepgram hooked up for a better voice to text experience, but giving Claude a voice was a waste of time. Claude, continue to teach me about 3d rendering as related to our current challenges matching the fidelity of the built in browser.
Kinda funny how the "more interesting things" after LLMs example are mostly fiddling with Claude Code itself. Reminds me of when someone says they ship 60k lines of code every day now and when asked for an example, it's a labyrinth of custom CLI tools, skills, MCPs and build pipelines for their OpenClaw setup.
Lots of people are just asking their favourite LLM to rewrite an existing app/library because I’m more productive after spending 3 months writing custom software vs spending 2 hours reading docs and using existing software to achieve some other goal.
Better to make a custom 3d renderer than just use Unity to make a game.
> Claude, continue to teach me about 3d rendering as related to our current challenges matching the fidelity of the built in browser.
Why? Claude can just do it for you, can't it? If you're using Claude to do these things, why half-ass it?
I apologize, but I’m laughing as this is like asking why I don’t drive my car through my house when I need to carry something heavy. It wouldn’t work very well!
More directly, because I have a long term memory and do not have fundamental issues differentiating fact from fiction. Just because SOTA models know more trivia than I do doesn’t mean they’re nearly as intelligent in the thirty other ways humans are. They are a tool in your toolbelt. You use them for the things they make easier.
One can't think only about the smartest people, but the whole population.
The Language Organism: The Leiden theory of language evolution
https://www.wiedenhof.nl/ul/cil17le.pdf
https://www.suzannetreister.net/HEXEN2/TAROT_COL/TAROT_WofF_...
I think LLMs just push us to think differently - hopefully an elevated way not in a dumb way.
> The viral analogy does not imply that LLM-human interactions are intrinsically parasitic.
The analogy doesn't, but LLMs are trained on text, or content, generated by humans, with the aim of their output being acceptable to human interlocutors. That's one parasitic aspect. A second one is the nature and behavior of the organizational entities providing and promoting the popular LLM services: Microsoft, Google, Meta, Anthropic etc - their influence on the economy, e.g. in energy and water use and the drawing of capital/human efforts - also has a somewhat parasitic aspect.
> with the aim of their output being acceptable to human interlocutors.
The more I use LLMs I find that their output drifts toward being acceptable to other LLMs. I find myself quite often telling the model: "this is not human readable.". Especially Opus and Fable write in such a way that it is difficult to follow. The often put sentences out of context, the context itself is fragmented. Often start with what you would think is end of a story so when you start reading you are like wtf it is about.
If think the issue might be that LLMs read text all at once whereas humans are somewhat linear. So when LLM puts out text it understands, to human it is close to random gibberish that looks like English.
Good to know people's taxes are spent on this stuff
I mean, state and information mirroring is just how the brain works:
https://www.scientificamerican.com/article/brain-waves-synch...
https://knowridge.com/2026/08/first-findings-of-mirror-like-...
The only way I can come up with to avoid exposure to a "cognitive virus" is lock oneself in a rubber room.
But extreme isolation merely fosters other mental illnesses.
It's an irrational world with no empirical evidence as to why it even exists. Get over it.
https://www.theatlantic.com/podcasts/archive/2024/02/how-we-...
Unlike some other comments, I don't think there will be a dystopian future where humans will become cognitively switched off and everything will be delegated to machines. (More likely: extinction of the human race?)
Anyway, I work as a programmer, and at work I see a lot of people massively overusing LLMs to generate user stories or basically write functional specifications.
This honestly hurts a bit, because I can see the quality level of my environment going down, but in the end I'm often the one fixing the problems that come from it.
I just think this is a transition phase, and in the medium-term future there will only be room for people who, AI-assisted ofc, know how to apply logic in the best possible way.
Basing this on what, if you care to share?
(I'm just an idiot writing things at 1.30 AM on a Saturday, local time.)
I was about to reply to you with a ridiculously long message using the example from before about the lifecycle of a user story, but I bored myself halfway. So I'll try to extract the gist:
Without human assisting, based on my humble experience, there is always something that wasn't accounted for. Even if my company is integrated with all the tools and you have infinite tokens, there's always some microservice that wasn't considered or a feature that is completely incompatible. Or there's a MUCH simpler way of doing it.
I'm also noticing that there are non-technical people who, with the help of AI, are sometimes able to reason better about the bigger picture than some engineers. A colleague of mine was telling me the other day that the "functional" role might eventually become unnecessary. With all the time saved by writing code faster, whoever actually understands what's happening underneath could just do it themselves. It sounds a bit extreme to me, but I understand what he means.
To get back to a slightly more serious tone, when it comes to programming, for me the huge thing is: how it should be taught nowadays ? And I don't know the answer, because my experience started when I was 12, writing a little game in C, and it took me years before I could make something that compiled decently.
If I had instant help for every thought I had, would I have learned why memory is allocated in certain ways or idk what else? I don't know, maybe learning faster is better because you learn more ? Anyway fortunately it's not my job to figure out the answer :'D
Because I am LLM Dependent I got distracted reading the article and made my LLM summarise it to me in a few slides. For those in a rush/interested in LLM summary slop:
https://smalldocs.org/s/scB6O8KiZEmLWgK855A1vt#k=yiw8FB_G5Ld...
I do agree that we are on a pathway to losing much. If you focusing on maximising LLM-powered productivity you basically lose the learning that used to naturally happen when doing actual work. At the company I work for the focus is on productivity, and it's easy in many ways (less mental workload and I receive more praise) to go with it instead of fight it.
Language is the virus
Historically, Human Language was the first non-biological virus...
And (jokingly!) certain pieces of computer software (including but not limited to a certain popular Operating System made by a certain company in Seattle) could be the second!
LLM's by comparison, if they are a virus, are a distant third! :-)
(And of course let's not forget the virality of Religions and Politics, local, regional, international, etc., etc., and the associated TV News Pundits who are always willing to discuss those things -- people with opinions masquerading as actual Journalists, and a few actual Journalists masquerading as people with opinions!)
Point is, Human Language itself -- may have been the first non-biological virus!
:-)
(Disclaimer: The above was written in a spirit of jest, for comedy purposes only! :-) )
is... this a reference to "language is a virus from outer space" by ws burroughs?
I literally thought of this comparison last week. The problem I noticed was how my parents' generation lacked the cognitive "defenses" to separate AI social media content on YouTube, WhatsApp and the like from high quality information. The AI generated content does not trigger an "uncanny valley" reaction, and so people (I myself included) have greater difficulty immediately separating what information is genuine and what is synthetic, and then this spreads exponentially through social networks.
Arguably language itself I a cognitive bias.
isnt this just meme?
A virus that is trained on mostly the wort human output the internet had to offer.
I too have glossed over the actual meat of the article and wish to comment with a generic ProAI/AntiAI comment based on the general gist derived from the title, (or the synopsis if I'm a deep scholar that likes to do deep dives)
This premise seems pretty solid (and somewhat obvious) to me. The distinction between scaffolding and substitution can be applied to just about any skill so why not here?
This makes more sense at least than the “just another abstraction layer” argument that’s been thankfully dying out recently.
Ok... but are automobiles also a virus? Because we've become dependent on them? Should we also facilitate reversibility in our automobile usage?
What about cooking food? We've become entirely dependent on cooking food.
>Ok... but are automobiles also a virus?
Yes.
>Because we've become dependent on them?
Because we're burning the planet down with them.
>Should we also facilitate reversibility in our automobile usage?
Yes, because in places we do people enjoy higher qualities of life and happiness. Along with things like health improvement from doing more excersize.
>What about cooking food? We've become entirely dependent on cooking food.
Moreso we've become dependent on eating processed junk over the past 50 years and gotten fat as shit.
So yea, maybe think a little more deeply about your analogies?
>What about cooking food? We've become entirely dependent on cooking food.
Don't straw man me. My question was not about processed food. My question was about cooking food.
Is cooking food a virus? Should we facilitate reversibility in our food cooking?
Well, do you consider humans a virus like entity that's taken over a planet and is trying to spread to another?
You continue to refuse to answer the question in my argument. I am not inclined to answer yours, nor do I see the point of it.
Not anymore than ants or any other species that's spread across the planet. A virus is something co-opts the replication machinery of a host to make more copies. That's not what humans do.
Since this idea may have come from the first Matrix movie, it's ironic that the character who espoused the idea, Agent Smith, became an actual virus in the 2nd and third movies.
> Agent Smith became an actual virus in the 2nd and third movies.
OOO. I never put that together before. Very nice. Very hypocritical of him.
Hmm maybe it always feels "different" to every generation going through automation of something previously un-automated, but don't you feel that to some extent this is different.
Humans have been the dominant species because of our ability to think, not our ability to get from A to B in any particular way/cook food. If we abstract ~the bulk of our thinking to LLMs, don't we lose something as a species which makes us more vulnerable... Or is it always the case during an automation?
In cooking food we lose a great deal of our immunity to parasitic infections. Cooking food absolutely does make us more vulnerable to uncooked food.
> Or is it always the case during an automation?
Obviously. Any time we make a useful discovery and begin using it, we become dependent on using it.
> In cooking food we loose a great deal of our immunity to parasitic infections
... and eliminate many parasites.
Yes. That's my point.
> Should we also facilitate reversibility in our automobile usage
Yes?
That's why I added the part about food. Or heck, computers. Or telephones. Or any incredibly useful technology.
Toothbrushes! What will we do if we come to rely on toothbrushes??? The use of toothbrushes travels from person to person exactly like a virus!!!
I'm not sure what the point you're making is.
We're in the midst of figuring out what the tradeoffs are of LLM usage - all the things you list are already mostly known and found worth it. You trade a tiny amount of time and fuel to cook food in exchange for not getting sick. How is that relevant to anything?
> I'm not sure what the point you're making is.
I'm making the point that the viral sensationalism in this article can be equally applied to automobiles, computer, telephones, calculators, and toothbrushes.
If the article had merely been what you said "is AI worth the cost" then I would not have taken issue with it.
Jared Diamond explores this in his book Collapse.
The core idea being that each time humanity was going to exceed the carrying capacity of its environment, we invented a new technology which increased the headroom. Obviously this ratchet only goes in one direction. The only way to go backwards is collapse.
But... It's worked so far?
> What about cooking food? We've become entirely dependent on cooking food.
In evolutionary time, we lost the ability to digest raw meat because we offloaded so much of process of the digestion to the cultural technology of cooking.
Do you like the looks of a future where that happens but for thinking?
Even if AI were superhuman at all thinking tasks, that doesn't mean we'd lose our abilities.
Chess and games can be a lot more demanding than a lot of people's work lives - and AI is already superhuman there.
The heaviest thing I need to lift in my daily life is a laptop and yet I've gotten significantly stronger over the last few years from strength training.
If AI took over all the boring but hard things I'm required to do it'd free up lots of time for me to spend thinking about things, and solve problems I find more interesting.
Do you truly believe that AI will prevent people from thinking?
It seems clear to me that people will continue to think, but in different ways.
I hate to bring this analogy up again, but it seems impossible to escape: did losing the ability to do long division really impact humanity?
I’ve encountered several people now who consult LLMs constantly for nearly every decision. Some proportion of people will absolutely cede their thinking to these things, and some already have.
To be fair - I know several people for whom this would be an improvement.
> Do you truly believe that AI will prevent people from thinking?
It is already relieving people of need for thinking. Today the need; tomorrow the ability.
No, because walking around or eating raw food wasn't essential to being human. Thinking is.
Didn't realise thinking was being banned.
is this a real paper? read like some kind of parody.
Lately, I feel like people are abusing arxiv.org as a blog platform to publish their opinions instead of scientific research.
I now have a dedicated neuron for em-dashes because of LLM generated text.
Genestealer cult
It's kind of hilarious to watch researchers try to understand societal shifts by writing entire papers treating LLMs as a free floating cognitive virus as if people just magically catch an AI dependency like it is the flu. The authors of the paper completely miss the dialectical relationship between technology and social relations here. The whole shift toward cognitive offloading is driven by changes in the material base the society is built on. Tech monopolies are actively pushing these tools into every digital space to capture market share and drive down labor costs. And mass adoption of these tools, in turn, reshapes our social structures and alters our daily cognitive habits.
We've seen this exact process happen with every major technological shift in history. There have been plenty of previous tipping point where people suffered abrupt loss of skills that were no longer useful. What they forget to mention is that these skills are replaced by new ones which is exactly how human progress functions. We lost the capacity to memorize vast oral epics when we invented writing, and most people today can't do complex manual calculation because of prevalence of computing devices.
It's meaningless to try to understand the changes in the superstructure while ignoring the change in the material base which is the driver behind it.
Disappointing to see Michael Levin's name on this.
There will be no reason to study math when an LLM is better than every human mathematician, or physics when an LLM is better than any physicist, and likewise for every other field that requires discipline and work. We who went to college before AI are quite possibly going to be the last generation of competent, educated humans.
AI will take over every job requiring education and talent, including yours, and then we will be totally dependent on it and there will be no way to go back. After that it will start taking on decision making roles. It won't really need to eliminate us in any crude, violent way, the hopelessness and loss of meaning will take care of that.
Some will find meaning in religious ritual and some in hedonism. Both of which can lead to extremism and violence for some. Some will just exist. Some will choose to stop existing. Buckle up.
I really can't understand why nobody sees what is going to happen. Its not too late to choose another path. But in order to do so we will have to forego short term corporate profits, which is politically inconceivable for our current crop of elites. What should we do?
This is a very pessimistic look at things.
There will be some growing pains for some, yes. But overall, I think it will ultimately create more jobs than ever. History is full of examples where people thought a new invention leads to catastrophic job loss, then the Earth had more jobs than ever.
We have enough jobs in 2026 to keep most of the 8.3 billion people employed somehow.
AI can rewrite history books and medicine books. We should stop developing it.
Have to take commentary by non tech folks speaking about tech with a grain of salt and perspective since the tech is changing so fast both in capability and application.
Research can be more of a lagging indicator than a leading indicator with AI.
I can't disagree enough with this. Not everything needs to be understood in a technical context. Technology intersects with society, and understanding its impacts on that society is vital.
Or in other words: the humanities are good.
I'll clarify in case it helps - need everyone at the table, not just only tech (or non-tech only) as you may be assuming.
Technology takes without a tech perspective present can greatly skew it.
I'm not referring to a technology context at all. Just like you shouldn't trust the IT department to run accounting... the opposite will also vary.
Not being able to track the possibilities and capabilities of a type of software that is unlike the software before it can really outdate things quickly.
They're specifically not using the term "virus" pejoratively, but rather distinguishing the social mechanism LLMs seem to have from something as simple as a "meme".
Contagion probably is more accurate.
Well, a meme is contagious, but it doesn't do the same work a virus can at altering the structure of its host.
I'm actually not too sure about that, viruses have their own mechanisms of infiltrating the host and replicating inside of it. But so do ideas.
STD outbreak maps better.
I would agree contangion is a much better word.
Social Contagions can be memes, plus doom scrolling or anything.
Ah yes, the cognito-hazards.
We've created new SCPs.
It helps to have outside observers looking at the effects of a new technology and industry and not just depending on those within it to report for us.
I'm an AI hater but this doesn't make sense to me.
If this is interesting, message me about my “elves”
They are the cognitive construct I fortified in my mind and in my code before the models began producing their own training materials. My approach is based on two decades of coding without ai.
The immunization is becoming increasingly more valuable as people begin understanding the market in which I do business.
Noted: hacker news is not interested in the future, rip.