Which, given the section below from the article, ends up kind of ironic:
> There is a deeper cost still, and it concerns the thing LLMs do most impressively: writing. [...] Outsource the writing and you have not accelerated the thinking; you have skipped it.
I think the author might not know what "impressive writing" or even "good writing" is, which would explain both how they could put that part into the article, and how they seemingly believed this post was good and valuable enough to be posted publicly.
> the author might not know what "impressive writing" or even "good writing" is
Well the author in this case is Claude, and AIs write like that because the assistant persona really thinks that's what good writing sounds like. They're wrong, but they're just doing what they were taught. There's a reason LLM raters consistently score LLM writing highly.
Interestingly that's the one point of the article I have a disagreement with. Yeah, good thinking comes when reformulating your ideas. Also, reformulating ideas is part of the traditional writing process. However it's does not necessarily focus writers/thinkers on the reformulating ideas in their most value adding form.
I've watched some YouTube videos narrated by AI that were created by those who's native language I assume to be Chinese but the value of the content is higher and more concentrated than those from the native speakers.
While I've watched many for whom English is not their first language struggle in technical talks and lose most of the meat if their discussion to their struggle with the language conversion.
While the example of language barriers being skipped over and providing value in that circumstance is obvious I suspect more is possible by avoiding unnecessarily focus on prose or technical aspects of communication and focusing on the ideas themselves.
Imagine if the cost of not assuming a background in technical/textbook writing were zero. Freeing up authors to explain more thoroughly. Perhaps, more effort can be spent on thinking up analogies, metaphor or examples to help communicate an idea.
> believed this post was good and valuable enough to be posted publicly.
I mean, it made it to the front page of HN didn't it? Probably served its purpose just fine. Not all writing is supposed to be impressive or good. Some of it is to just get attention and stir discussion, and this Claude output did exactly that.
I wanted you to be wrong, and to be able to make this an example of us over-reacting to certain trigger words created by AI, but unfortunately I just scanned the first couple paragraphs with pangram and it reported 100% AI, so you're probably correct.
I get a funny feeling in my stomach over the idea that common and effective means of communication (i.e. it's not X it's Y) have become faux pas to use because of AI. I think it's something about these phrases being taken away from us more-so than the AI inventing them.
Pangram should be paying HNers for how often we pitch needing to use their product by name to believe things as obvious as "a long form news article cramming every AI trope it can fit from start to finish" being AI written.
At this point I'm surprised when a news article isn't largely AI written, let alone one using the default tone! I don't even mind it as much as others seem to, it's just turning into more and more of a rarity for a news article to not be these last few years and so is now what sticks out.
> Pangram should be paying HNers for how often we pitch needing to use their product by name
I kind of assumed it is, given how it suddenly seemed to start getting namedropped in multiple comment threads. And it's often in response to someone saying content is obviously LLM (with examples) and the shill wedges pangram into the conversation "omg you're right, I didn't believe you but I checked this new product and wow it agreed with you" as though that adds anything to the discussion at all.
Pangram gets brought up a lot because if I read an article and think it's blatant AI slop and want to communicate that fact, a natural impulse is to provide some sort of objective corroboration rather than just asserting that I have superior taste and thus am able to tell.
Also Pangram is basically the only AI detector that's actually put in the work to build an accurate classifier, so if you try to discuss AI writing detection without being specific that you mean Pangram you'll get half a dozen commenters screaming about how awful some of the snake-oil salesmen like GPTZero are.
I mention pangram because it’s good at what it does. Otherwise, people like to claim “That’s just how I’ve always written. I always say That’s the seam and the seam undercuts the other pillar. It’s not just correct, it’s also verified lol; it’s just how I speak” and it’s obvious bullshit. Depending on the politics of the poster, people will also support this kind of garbage, but it’s all drivel.
The authors are so often dishonest that it’s hard to believe that the things they’re saying are anything but a random hallucination.
Em-dashes would be a loss, they can be a better flowing version of a parenthetical. “It’s not X it’s Y” is not a loss. This phrase is a symptom of a situation where the author wants to subvert expectations but doesn’t have the space, ability, or faith in their audience to organically set up X as the thing to be contrasted against.
I suspect it became an LLM tell because it is over-represented in text that’s easily available to the models but that most people don’t actually want to consume: marketing text, LinkedIn posts, that sort of thing.
AI detection in long-form content is a problem well-suited to training a classifier model. We have tons of verifiably-not-AI text from before 2022, and you can create tons of verifiably-AI text. As language drifts over the next few decades, it may get more difficult. But right now it is quite a manageable problem for languages with large pre-2022 text corpora available online.
The reason why AI detection tools other than Pangram are awful is because they are not really trying to solve the problem -- they just want to appear good enough to convince people to use them.
Imagine writing an article in 5 days. Working hard. Posting it online and then everyone says it is ai and laughs and dismisses it because a tool says it is AI.
Fwiw, I only did the first 300 words (signed into pangram) and it seemingly correctly noted that there were 2 (mostly) human-authored sentences in there
What's wild to me is that:
1. People are responding to this article like it's hitting a nerve
2. In most Ubers you already don't talk to the driver (yellow cabs are higher variance in NYC). Doesn't seem like anything's being lost in that case.
3. There are enormous safety benefits to waymo, mobility benefits for youth (and elderly) that are afforded by this technology. It's not clear why people argue "Uber" is better than waymo. A few years ago there were arguments against Uber! (A technology which, again, provides a huge benefit, especially if you live in an area where people were previously expected to go out for drinks and then drive home)
I think it's fair to point out real issues at these companies. But we should be clear-eyed about which technologies we want to accelerate vs slow down
Have you actually read the article? They point out multiple times that they like the Waymo ride and would use one again and loved it. It's not a Waymo-bashing article, it cautions against the long term effects on research.
And here you have it — I used an LLM-ism. Oh, and now another one! Time to downvote me for supposed LLM use! (Which I obviously didn't, I'm typing this on an ancient smartphone waiting for a train, but that doesn't deter the witch hunters.)
> You actually didn't. The construct is slightly different.
Well I know that I did, so you are wrong with that claim, which sheds a strong (negative) light on other statements of fact that you posted in this discussion.
You may disagree with me, or I may even have gotten something wrong, but just claiming I didn't read the article ... sorry man, I thought HN had higher standards than that.
They were pointing out that "It's not a Waymo-bashing article, it cautions against the long term effects on research." does not sound like LLM-generated text.
It would sound kind of LLM-y if you had said "It's not an anti-Waymo article. It's an article about how assistive technology is quietly degrading research."
(Of course, the correct approach to detecting AI is not counting LLM-isms, but feeding long-form text through a classifier model and picking up statistical correlations that are more in-distribution with LLM text than human text.)
"It's not a Waymo-bashing article, it cautions against the long term effects on research." is not an LLMism. It superficially resembles one, but that's it.
Yes AI slop is prevalent. But the casualties that get thrown under the bus are also real, don't you agree? Is it really worth it to sacrifice those? In the name of the larger good?
I'm arguing that we're losing something by doing so, as a community.
> I find complaining about AI callouts
If they are correct then I'm all for it. But just suspicions turned into statements of fact are not helping anybody. People used em dashes before LLMs Not nearly as much as LLMs, but superficially judging people isn't doing any good.
I'd compare this with pitchforks coming out agains criminal immigrants. Yes, some immigrants are criminal. Doesn't mean that if you stand in central Copenhagen and have a dark skinned person in front of you that it's a criminal.
The rule of thumb is simple: if most of a user or author's writing is consistently human-generated, then I think we are very happy giving them the benefit of the doubt if one article or snippet flags the actually good ai detector
Unfortunately, many of the largest voices against pangram simply don't like it because it gives their lies less credibility.
I don't mind reading llm writing, indeed, I read more llm writing per day than most. But if you're using LLMs to increase the level of slop (blog posts, comments, tweets, etc) then we should call people out.
It's a colossal waste of everyone's time and attention, and we should be mad about it.
Late edit: if people are actually writing and it's consistently flagged by pangram (this is a statement I have not yet seen validated), the pangram folks are extremely proactive and excited to understand what's going on. This notion that somehow the authors of slop are victims is complete nonsense.
> if most of a user or author's writing is consistently human-generated, then I think we are very happy giving them the benefit of the doubt if one article or snippet flags the actually good ai detector
I would agree to that approach.
But that didn't happen here. Nobody has done that due diligence and folks are just blindly accusing. The author has published for many years. That's what I'm calling out.
> This notion that somehow the authors of slop are victims is complete nonsense.
Strawman? I didn't claim that or anything close to it. I also find the prevalence of slop writing hugely annoying.
Yes, HN is for human discussion. It is fine to flag and remove things that are not human written and cannot facilitate human discussion due to it’s false construction.
Is "quietly" an LLM tell? It was always certainly a human writer trope commonly seen in journalism. Though I guess it does appear five times in the body of the post, and a human writer would probably not go that far with it.
I think individually and when used sparingly, writing tropes can be fine, but when the article has every second sentence being a writing trope, it becomes pretty obvious that either a LLM wrote it, or the author simply doesn't know how to write, and regardless, it's a waste of time trying to read through it.
The real LLM tell is tortured and inappropriate metaphors and unnecessary adjectives and adverbs[1]. What purpose is "quietly" serving in this headline?
I think that in the end it does not matter, anymore. It is inevitable that most of the text now and in the future is at least reviewed by LLM.
We should criticize whether text is poorly written, inaccurate, wastes words to get to the point and anything like. Saying that it is "written by LLM" just bypasses this and is not useful and misses the point. AI written text can be really good if used correctly. Criticize the content, don't speculate how it was written. If AI helps us to write better text, that is great. But often it is not good.
It is better blame the author for the bad text, so they get consequences and might do better next time, if the text is bad.
By the way, I think this piece of text was quite good.
> saying it is "written by LLM" is not useful and misses the point.
It is useful to me. I see too much slop every day and would like to filter it out from my life. I appreciate the tip, and to me, that is the whole point.
Do you have more evidence than this? I'd honestly like to know.
I don't like AI slop like everybody else, but I'm also growing skeptical of the very fast determinations that sth is supposedly made by LLM just because it uses some phrase that Claude also uses. Models are trained on text written by human writers and if human text resembles it, this goes both ways. I have looked at the authors work from the pre-AI era and it reads similar to me.
So please, if you come with such a claim, please include what you base it on, so we all have a chance to determine how much to trust your verdict.
"every hour spent aligning with a co-author is an hour not spent producing output that is legible to an evaluation system." use of the word 'legible'.
"We defund the corridor and then wonder where the corridor conversations went." this is a claude-ist construction.
"This is an old worry wearing new clothes" this is a very claude-ist construction.
(rather ironically... as @embedding-shaep pointed out earlier):
"Outsource the writing and you have not accelerated the thinking; you have skipped it." 'not X but punchy-Y' (not to be confused with 'not X but profound-Y'.
I'm not saying the author shouldn't have used an LLM - once in a while I suppose it gets used to good effect. But I wouldn't kid myself that the writing is certainly all human through-and-through.
[Incidentally, in case anyone's wondering where many of these claude-isms come from, just search within lesswrong.com .]
This comment was perhaps written by an LLM that has learned it can karma farm by pointing out all of the articles that have LLM usage while everyone still thinks that is a novel contribution.
Me neither, but just running around accusing random folks of things is not going to help. To the contrary. If a real author doing real work with a lot of effort risks being accused of things they didn't do and just dismissed, you'll get less of genuine content, not more.
I will donate $100 to charity if I made a false accusation.
(It's possible AI rewrote an article first written by a human here or whatever, but the point is that I can just tell this is mostly or entirely LLM output.)
> I will donate $100 to charity if I made a false accusation
That's honorable but not how accusations work. There is a first level indirect effect of painting somebody in a light that damages the reputation of their work or their person which isn't reversed by a donation or a sorry or anything since retractions/corrections/apologies don't get the same public attention. And there is a broader effect of chilling what people do since they don't want to get falsely accused.
I'd recommend keeping such suspicions to oneself unless there is unrefutable proof, and if there is, then please provide it.
It does help, if the accusation is true. The fact that false accusations result in less genuine content implies that true accusations result in less disingenuous content. For better or for worse, social consequences do affect people's behavior on average.
All true. But question is whether the positive consequences outweigh the negative. That may be the case, but it may not be. I don't find it obvious and not considering this is careless.
In the spirit of the "That's what she said" bot, you could train an LLM detector by accusing everything of being authored by an LLM and seeing which ones get voted up.
"Novel" is not a prerequisite for something to be worth pointing out. Lots of bad things happen repeatedly, and don't quickly become not worth caring about or knowing about. People have strong spirits and curious minds and it tends to take a long time to boil that frog out of them - generations, sometimes.
I appreciate when people write things like, "this is an advertisement for the author's product, X." Why stay silent? If the commenter's accusation smells fishy, I'll read the article myself. Otherwise, they helped me.
I work at an intersection of tech, applied research, and science.
Something I’ve noticed in collaboration that does occur is an increased confidence in people outside their domains to say things with conviction. I have people who have limited experience with software pushing out layers and layers of abstracted code that’s fairly sophisticated but often misguided in intent who will say what they’re doing is correct, with conviction.
I also hear a lot more questioning people in their domains and challenging opinions, then hearing what I can only imagine are fragmented pieces of conversations they had with an LLM thinking through some argument. Then there’s silence when you discuss shortcomings, then they come back later with their memorized fragments of what you said, combined with memorized fragments of the LLM response to the argument.
It’s occurring, a lot more. People are treating their LLMs in collaboration as a source of truth and using then to focus on their specific path or goals they think or have bias towards going down, vs just opening discussing things, considering tradeoffs from experts multiple disciplines weigh in on and then taking an approach that everyone finds most agreeable.
It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day.
Wow, oddly similar thing happened to me. Anecdotally, I was working with a non-developer. She was building a reporting dashboard with Claude. I pointed out that the system she built was using the filesystem to store one line notes about the report, and I said "I would probably tell the agent to use a database for that, but it's not the end of the world if it works". And she replied "Claude said the developer's right, a database would be a better fit..."
So, a non-developer, who has no understanding of the underlying systems they are working with, fact-checked a developer with years of experience.
I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there. She couldn't rely on my expertise, she had to go back to her "source of truth" (Claude) and asked for a second opinion on something that she wouldn't be able to verify.
This is why collaboration exists. I trust my colleagues to give me insight+advice on things that I do not know. I don't ask AI to vet their decisions, because at some level there are judgement calls, and I WANT to trust them because it makes my life easier.
Treating AI as if the output is factual is just a misuse of the tools, I think. It's significantly more effective when given expert direction and the output can be verified by... an expert.
>>I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there. She couldn't rely on my expertise, she had to go back to her "source of truth" (Claude) and asked for a second opinion on something that she wouldn't be able to verify.
This is probably how physicians feel when informed patients (whether using LLMs or not) ask questions (good or bad) during their patient encounters.
I would love to be able to trust that a doctor knows more about my problem than I do, but every time I see one they clearly haven't even read the information I just told the nurse 15 minutes ago.
This is a different situation. A doctor is a domain expert, but you are in possession of vastly more empirical data about your specific body. Getting to a good result is going to be easier with collaboration, rather than one of you confidently telling the other what the problem/solution is.
Yeah I've felt that gut punch all the time. Seems to happen with every technology. In the early days of GPS, people would ask for directions to the gas station then pull it up on their phone map and go "yeah, you're right!" So thanks for wasting my time, insulting my integrity, and killing my faith in humanity all in 3 words. And if you really need two opinions, it's so easy to just reverse the situation, "my GPS says there's an Exxon in two miles if I turn left here and then right, is that correct?"
I remember fifteen years ago I would tell people leaving my house "oh yeah, ignore your GPS that tells you to go that way, just down the road here and left turn; that'll get you on the highway much faster."
Nowadays obviously Google Maps gets it right, but it was interesting that period where you had to actively preempt everyone's instinct to just rely on the little table hanging off their dashboard.
there was an institution my son went to for therapy for his various handicaps, and one of the major taxi routing solutions in Denmark had the route wrong, I think the one based on Google maps actually, so when you called for a taxi it always drove nearly a block away that you had to walk through a couple of back alleys to get to. And then the taxi driver would call me up and complain I wasn't there, and I had to guide them to the place where they actually had to be at over the phone.
Sometimes they were not capable of following the instructions though.
I'd bet it depends on what they write to the LLM, since sycophancy is so strong in these things. If they write in a fairly agnostic way, it might default to sycophancy to you the dev, while if they write it in an opinionated way it might default to sycophancy to them. Regardless of who is right.
To be generous, it seems like the most natural way to phrase what happened. The non-developer punched in "should this be a database instead?" and it will say "You are right to question it! I was sloppy when .. blah blah blah blah blah"
I doubt the human went into a whole diatribe with the CLI about how the developer on the project with this many years was saying this or that but she wanted Claude to second guess them.
It's just saying "ah yeah, that went through without issue."
I can see how the situation is annoying there; I've had a similar frustration when doing manual code review for someone's obviously AI-slop PR only to have their bot auto-respond to my comments. There's an asymmetry of effort that feels unfair. Like, don't ask for manual review of something that contains lots of things that your own manual review should have caught first, and if I provide human feedback, I expect human responses.
That said, I wouldn't necessarily take it at face value that the person you were working with trusted the bot more than you— at the end of the day, the bot was doing all the implementation anyway, so all the interactions between her and the bot were probably framed in language like "what if we" or "I think it would be better if", so her saying "my programmer colleague thinks X" would naturally generate that "yeah your colleague is right, let's go that direction, makes sense" that she passed back to you, probably thinking it was a complement.
Maybe the real frustration is her not understanding the value-add that still exists there when someone with real expertise and more context can step in and provide that nudge in the right direction.
Sure - I don't hold it against them. It was just surprising to read a "Claude said you're right" message. (Yes, I know that I'm right, or I wouldn't have said it)
Didn't ruin my week or anything, but that was the writing on the wall for me.
Technical expertise is no longer viewed in the same light when software development has no barrier to entry.
True, though I wonder if 2026's "Claude make me a minecraft plugin" is 2006's "View Source" button.
Web development took a long time to gain respect in part due to the accessibility of it. When everyone had a nephew who "makes websites" and anyone could get started with a $5/mo Dreamhost account, it was harder to see why a consultant was worth $200/hr to do the job properly.
It'll be interesting to see how this plays out, particularly accounting for bots continuing to get smarter and smarter. Is the "craft" of software engineering dying before our eyes? Or will this be like webdev, where there's an explosion of access and ideas for a time and then it settles back down into being a thing that normal people mostly leave to the nerds.
wow that wouldnt feel like a gut punch at all to me, it's normal for inexperienced people to not really understand what kinds of knowledge experts have or don't. I certainly come back from doctors appointments and google/gemini chat for whatever came up to get more detail. this person going to claude and getting immediate confirmation that you were exactly correct should have felt great, she would be like "wow, this guy's good, huh."
>I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there.
Maybe I'm just cynical but I don't think I ever see that much trust for anyone's opinion or expertise; devs think managers don't know what they're doing, managers think devs don't, people on HN think lawyers are wrong when they state their legal opinion, devs think other devs are wrong about their areas of expertise even though they have no reason to think that, hell I've worked with people who basically lie for some reason about technical issues that they have no expertise in, in order to get some benefit that is unclear to me. I've also seen people of great technical expertise gaslight industries in order to derive very clear benefits for their employers, so I'm not sure why there should be trust.
I'm sure that various AI agents will maybe add another interesting wrinkle to this, but I sort of feel like you must have had a nice life up until AI if that was feeling like a gut punch.
I think I could qualify as an expert on various topics relative to many average people. But it always bothered me when experts wanted to "just be believed". In your example, it doesn't long to explain why a database is better than the file system in this instance and a moderately intelligent should be able to understand and not need you or Claude as the source of truth.
So I think it can be good people are able to "AI" to question expert opinions - though naturally, as you say, it's bad if people take the AI as a single authority akin to an expert.
So, I did let her know that a storage volume was going to be more difficult to manage, could result in lost data, and would make the data harder to work with - I mentioned it's not the end of the world if it works, it was just a professional suggestion.
To me, there are different levels to "just being believed". If a mechanic tells you your car isn't working because you don't have an alternator, does that really need scrutiny? If you don't understand the subject matter, at some point you have to "just believe" someone else.
Especially for a non-technical user, trusting the software developers at your company on technical advice should be a given.
> Treating AI as if the output is factual is just a misuse of the tools, I think.
Especially because it's very often wrong. AI tools lack the necessary context of the specific situation and the surrounding technology environment to make appropriate suggestions for anything more complex than clean sheet designs and toy prototypes. There is too much complexity that is hidden in domain knowledge and not even exposed fully in documentation for any AI tool currently to ingest it all into its own context to give you an accurate answer, even setting aside hallucinations and that these tools are incapable of reasoning.
I have this same gut punch constantly. AI has destroyed functional outcomes in corporations, because managers are all deep in AI psychosis deciding that whatever the tool says is perfectly correct at all times. We've replaced analysis paralysis with a complete absence of thought from any human in high-level impactful strategy discussions, everything is entirely being abdicated to AI tools in so many companies. It's ridiculous.
AI will radically change the economy, but not because its replacing jobs, but because its eroding trust, domain knowledge and expertise, and thinking/creativity while being a massive waste of money and total boondoggle. This bubble can't burst soon enough so we can get on with doing shit that actually matters for people that actually helps them.
I’ve seen how the market treats vibe coders that werent previoisly engineers. Coinbase’s CEO bragged about PM’s deploying code and everyone now thinks they are totally incompetent anytime a small UX problem occurs.
Engineers with prior experience now check a box, even if they are doing the same thing. Like a Compliance Officer or Cybersecurity professional checks a box
Yup, the arrogance when barging into unfamiliar domain that was previously reserved mainly to physics grads seems to have spread everywhere. I also had people opining on my expert area via their LLMs and the problem is they can't even ask the model the right question, let alone evaluate the nuance of their answer.
As an unfamiliar, I find it interesting how you specifically called out physics grads in the pre-LLM era. Is that just snark or is there some widespread truth / stereotype of that behavior?
Physics, and to a large degree pure mathematics, has attracted a critical mass of the personality type that needs to feel like the smartest person in the room in any given situation. It has given them a slightly notorious reputation in academia.
As a mathematician, this is far from my experience of actual professional mathematicians, because they encounter people much smarter than them quite often. Maybe more true among graduate students, however.
I don't know why mathematicians got lumped in with physicists further down the thread. In my experience this is a well known stereotype of physicists and while of course not universal you don't have to look very hard to see where it came from. In my experience mathematicians stay very much in their lane and often seem to be overly modest even within it.
The best/smartest professionals I meet operate this way (because we know we're wrong quite a bit and don't know everything). It's definitely a personality trait. Inquisitive, curious, intelligent, but also there's some metacognition involved where we can be confident in what we do NOT know.
In contrast, I studied chemical engineering for a short stint. First year, first semester chemistry classes and labs were riddled with these types of students. Tell you you're wrong, IQ 197, scoff when you ask a question, etc...
Listen, if you're the smartest person alive that's fantastic, but please don't talk down to me or the mere mortals that I prefer to hang around with. We get along much better and have a lot more fun saying "I don't know, let's find out".
I think that's a common trait of tenured academics in the top 100 universities in the US. Don't know about Europe and elsewhere though. Not so much of nontenure track sorts who do all their dirty work.
From a one-time physics grad student, I do think there advantages to being ... like this.
The flip side of being confidently wrong sometimes is that we treat understanding new systems as something approachable. So, yeah, it's annoying that we don't stay in our lane, but, from my boss's perspective, I'm the only one who's actually willing to just read code, investigate, possibly read a paper, and figure out what's going on instead of punting and saying it's someone else's turf I'm blocked until they get back to me.
Like, ultimately you can't be a knowledge worker by just waiting for answers and instruction from experts and relaying them back and forth. Eventually you have to understand something. And part of that process is being wrong and annoying people.
Like your sibling comment I saw this mainly with physics people doing data science:
> Eventually you have to understand something. And part of that process is being wrong and annoying people.
There is nothing more frustrating as a colleague than having a smart person walk in, insist you're all wrong, and then burn all the energy and goodwill in the organisation having to rebuild absolutely everything from the principles they currently understand because they refuse to listen to the experience that has built up already (in the entire field, let alone the organisation), only to produce something worse on every dimension than what already existed.
But there is a difference between not listening to experience and examining whether the resulting implementation/decisions based on the experience are actually valid. I doubt that someone smart (as you put it) would deliberately want things redone because they don't understand it, that's a trait borne of arrogance not intelligence.
At least in my experience (as a physics grad), me and all my fellow grads are much more likely to apply Chesterton's fence and then make decisions than not doing that and steamrolling existing decisions.
> I doubt that someone smart (as you put it) would deliberately want things redone because they don't understand it
You're actually doing the problem behavior right here: denying the validity of the experience of seeing exactly this happen, not as a one off, but repeatedly in completely separate situations and institutions.
> that's a trait borne of arrogance not intelligence.
I mean the problem is they're arrogant. The two aren't mutually exclusive, far from it.
My point is that wanting things redone because you don’t understand it is dumb behaviour, not smart behaviour.
There wasn’t even a denial of the validity of experience, I was just trying to examine the problem - if this behaviour is something you see often exhibited by “smart” people then: either you are unable to convince them of your side of the argument for different reasons, or you are not actually arguing with smart people. Do you see the distinction?
> The two aren't mutually exclusive, far from it.
Intelligence and arrogance are somewhat contra-indicative. While you can be intelligent and arrogant, you can’t really learn things without the humility to say you don’t know stuff. In my experience, most physics grads I know constantly say they don’t know stuff, way more than people from other disciplines.
> My point is that wanting things redone because you don’t understand it is dumb behaviour, not smart behaviour.
Smart people can do very non smart things when they feel their status as smartest-person-in-the-room is threatened, which is the entire point.
> There wasn’t even a denial of the validity of experience, I was just trying to examine the problem - if this behaviour is something you see often exhibited by “smart” people then: either you are unable to convince them of your side of the argument for different reasons, or you are not actually arguing with smart people. Do you see the distinction?
Now you're gaslighting, bluntly.
> Intelligence and arrogance are somewhat contra-indicative.
I've never seen any evidence of this, but a _lot_ that many physics (and pure mathematics) grads are enormously more arrogant than the general population, enough that the trend under discussion has actual basis.
> The flip side of being confidently wrong sometimes is that we treat understanding new systems as something approachable. So, yeah, it's annoying that we don't stay in our lane, but, from my boss's perspective, I'm the only one who's actually willing to just read code, investigate, possibly read a paper, and figure out what's going on instead of punting and saying it's someone else's turf I'm blocked until they get back to me.
Eventually you have to understand something. And part of that process is being wrong and annoying people.
GP here is suggesting that accepting established status quo at face value (i.e relying solely on other people's experience/letting them deal with problems you face in their domain without being able to examine why they arrived at the decisions they did) can lead to worse outcomes, and I agree.
Your response was that it is frustrating to have someone walk in and question things (precisely what GP is suggesting), only to have the final product of whatever decisions these people make be worse than the status quo (not what GP is suggesting).
I responded by saying - there is a difference between not listening to experience and examining whether the resulting implementation/decisions based on the experience are actually valid.
You did not address that. Is there not a difference between these two things? GP is suggesting to do the latter, not the former. i.e question why things are the way they are, crucially not that people should make decisions based on things we don't understand about the status quo.
You only responded to - * I doubt that someone smart (as you put it) would deliberately want things redone because they don't understand it, that's a trait borne of arrogance not intelligence.*
Your response claimed that I am denying the validity of your experience, when I made no such statement. All I said was there is a difference between questioning things and forcing decisions based on things they dont understand - smart people don't do the latter, arrogant people do. Someone doing what you are suggesting, is not doing what GP is suggesting, they are different. The former is smart, and the latter is not smart.
When I pointed this out here: I was just trying to examine the problem - if this behaviour is something you see often exhibited by “smart” people then: either you are unable to convince them of your side of the argument for different reasons, or you are not actually arguing with smart people. Do you see the distinction?
You accuse me of gaslighting you.
> Intelligence and arrogance are somewhat contra-indicative.
There might be a trend of this in academia (which is where the xkcd joke stems from), but in my personal experience I dont see this much in industry, and you can disagree with that.
I also don't see an xkcd joke being an actual basis of anything either. You are making declarative statements - but a _lot_ that many physics (and pure mathematics) grads are enormously more arrogant than the general population, enough that the trend under discussion has actual basis. - without any backing of this other than anecdata, and I was pointing out a different anecdotal experience.
> You only responded to - * I doubt that someone smart (as you put it) would deliberately want things redone because they don't understand it, that's a trait borne of arrogance not intelligence.*
> Your response claimed that I am denying the validity of your experience, when I made no such statement.
But it is exactly what you are doing in the exact quote of yours you even put there.
I am being nice, but it's quite clear you are a near perfect example of this stereotype (if actually a relatively polite one), right down to the wall-of-text when cornered.
I'm being nice too and all I've been trying to do in this thread is point at a distinction between GP's point and yours which you seem unwilling to engage with.
It is valuable to ask questions about why things are the way they are in order to understand the fundamentals of it vs we should change these things because I am 'smarter than you' and I don't understand why things are done this way (which I called dumb, and not smart). Perhaps the issue is that these two behaviours are easily confused.
I had to write the long comment to illustrate this difference. I'm saying in mine and GP's definition doing what you are suggesting people usually do is not smart, its dumb, and that is not what GP or I are suggesting people do.
> It is valuable to ask questions about why things are the way they are in order to understand the fundamentals of it vs we should change these things because I am 'smarter than you' and I don't understand why things are done this way (which I called dumb, and not smart).
But you expressed doubt of stated (and frankly not even remotely subjective) experience, so if we dropped you in an organisation your immediate response to anything you don't understand is "That doesn't match my mental model, so I doubt it actually happened", followed by either everyone giving up or you insisting everyone persuade you of absolutely every last detail of everything. It's enormously tedious, and the reason it comes across as arrogant is the demands put on everyone else to explain everything to you because you clearly see everyone else as incapable of the same level of insights that you are, all the while you are telling people what they experience is them being mistaken.
Just look at the way you questioned whether or not I could tell if someone is actually smart or not. It's just plain insulting. And it's not just me, it's everyone that has this same experience which forms the basis of the stereotype.
You seem to be imputing a lot of intent into my comment and then extending that to value judgements about smartness etc. I don't know what any of this has to do with what GP was saying. Yeah, there's a balance between questioning everything and questioning nothing, but there are a lot of bureaucratic things that happen in organizations and its generally a good thing to question certain things that don't have clear explanations, isn't there?
The doubt was simply about whether or not what GP is saying is smart behaviour to exhibit - Eventually you have to understand something. And part of that process is being wrong and annoying people. is the same behaviour that you are saying you've experienced ( i.e you are talking about different things).
I'm not even questioning if you can tell if someone is smart or not - its just obvious that forcing decisions based off things that are not understood is not a smart thing to do - which..we all agree on -, and neither I nor GP were saying people should do this?
Maybe I could've phrased it better but my whole point was GP says X, you are saying Y. I think there is a distinction between X and Y and I doubt that the people GP is talking about are the same people you are talking about (hence, smart as you put it vs smart as GP put it).
I see you don't understand the point, and you're doing that thing of imposing on me trying to help you understand it, which seems highly unlikely to work. Going back, again, to your quote:
> I doubt that someone smart (as you put it) would deliberately want things redone because they don't understand it
And yet that is quite simply exactly what I have seen multiple times in multiple organisations on multiple continents, but for some reason you "doubt" this because it doesn't fit what you think of as "smart behaviour" when spelled out like that, and then spend ages arguing that now there's no intent to this comment? Or any of the others . . .
I am going to do what I always end up doing in such situations, of just giving up and walking away, which every normal person does way earlier in proceedings. Ils sont fou ces physiciens.
Its clear that you are taking great offence to that particular quote and determined to misunderstand my point due to your pre-existing bias, despite my many attempts to clarify it, so let me finish the quote for you.
I doubt, that someone smart (as you put it) ..and not as GP put it..who..would deliberately want things redone because they don't understand it..are..the same people that GP is talking about when they say there are advantages to being "like this".
This is no longer fruitful, but I only hope that this thread might to be a reason to more charitable in your interpretation next time regarding such a discussion.
It's possible to step out of your area of expertise in a way that's confident but also humble. Arrogance is not necessary. The person who signs everyone's paychecks might be making it mandatory, but that's not quite the same thing.
> we treat understanding new systems as something approachable
And, computer scientists, mechanical engineers, and chemical engineers don't?
> from my boss's perspective, I'm the only one who's actually willing to just read code, investigate, possibly read a paper, and figure out what's going on instead of punting and saying it's someone else's turf I'm blocked until they get back to me.
Humblebrag alert! This has little to do with being a physics grad student, and much more to do with your motivation.
I guess all I'm doing here is arguing that a little hubris can help a person avoid being timid, and far be it from me to say that computer scientists, mechanical engineers, and chemical engineers have not been imbued with any hubris by their educations.
You can get pretty much all the same benefits with additional upside and fewer downsides by just showing curiosity and eagerness to learn from others' expertise. Yes, on the internet you might get better answers by being confidently wrong rather than asking for help, but when talking to other humans directly you'll get a lot more people who will just try to end the conversation as quickly as they can if you're arrogant, whereas people tend to respond really well to respecting their intelligence and experience.
I've personally found that doing the opposite of what you say and being willing to look like the dumbest person in the room by asking whatever questions I need to for my own understanding often ends up working out quite well; people are often wonderfully willing to share their knowledge with me even if I feel like I'm asking something very basic as long as I'm actually being friendly and humble, and there are plenty of times where there have been others who told me they have same question but didn't feel comfortable asking.
In some ways, it's similar to when I played bass in two different bands in college. In one of them, I was probably the most talented musician in the group, and it was fairly boring for me to practice with them, but we needed it as a group, whereas the other was with a friend of mine who was an insanely talented guitarist possibly more talented in music than pretty much anyone else I ever met. I loved practicing in the second one because I'd learn so much from him just by exposure and getting to pick his brain, but I imagine it got boring for him sometimes as well having to wait for me to "catch up" to what was easy for him. Being the smartest one in the room feels like it would get old after a while, but having smarter people around to learn from is endlessly engaging.
It’s not about being in your lane. It’s about approaching things with an open mind plus the desire to solve the problem instead or proving yourself right.
Yeah, I guess my point is that IMO a lot of people have so little interest in solving the problem that they aren't even approaching it at all, open mind or no.
It's not my experience that I'm constantly arguing with people, for what it's worth. I get positive feedback for being willing to work on areas of the codebase I'm not already familiar with, and develop an understanding of areas of the codebase no one is familiar with.
It is quite possible to be wrong and not annoy people, and it's not even terribly difficult to learn. It does require a bit of humility and introspection though.
You don't have to not stray into areas you don't understand, but be humble when doing so - if you think something is being done in a stupid way, you probably don't understand it yet.
I've experienced it in industry, too. I recently got out of data science in part because I got tired of working with people who, emboldened by their PhDs in some completely other field, liked to patiently but condescendingly mansplain common, basic misconceptions about my area of expertise to me.
(And it got so much worse once they started using LLMs to aid them in their efforts. Glazing as a service is a hell of a drug.)
So much easier, now that I am a lowly software engineer and can't be held responsible for a certain class of decisions, to just step back and let them be wrong.
Not the parent, but it's half joke half truth. Anyone who's done a lot of "first principles" work falls into this trap a bit, it's an issue across stem fields. Most grow out of it, or at least learn to qualify their statements for the audience.
I suspect the person you replied to only happened to see that from physics grads. I have seen it in all sorts of people and I sometimes did it myself when I was young (I am not a physics grad).
Its snark/stereotype, but there is some truth. But my take, is physics sort of took over a lot of chemistry, biology.. and to them its the answer to all questions, because in the end it just molecules. There is some truth to that but..
I hypothesize that with more people on a daily basis encountering information and claims that require those skills, there will be a time in the future when those skills are much more enhanced.
I sadly share this as a physicist. To be fair, sometimes this arrogance can be useful. I worked at the boundary between domains my entire career where there aren't "experts" per se. But yeah, barging into an established domain without some humility generally leads to embarrassment - especially Nobel winners (e.g. https://www.newsweek.com/nobel-prize-winner-who-doesnt-belie...)
Currently, the leader of USA national AI policy (Trump admin and OpenAI) is a history Bachelors with no technical training or advanced training of any kind, Dean Ball.
I also work at an intersection of tech, applied research and science. My experience has been different. It's been surprising how controlled my collaborators/coworkers have been with their reliance on AI.
Sure, they use it a lot, but despite the stereotypes against physicists in some of the replies to you, they have shown a good ability to catch themselves before leaning too hard on what an LLM tells them about things they are not experts in, and I haven't had any interactions where I felt like I was just arguing with a meat proxy.
Maybe it's because in the environment I'm in; it's relatively easy to just ask someone who is an expert in the topic for their advice.
My feeling is that this kind of overconfidence outside of one's domain is largely a thing for people with little "physical reality" experience. When all you deal with is the very flexible digital world, it becomes very easy to ignore how deep the knowledge and intuition goes in things that are directly constrained by reality. I consider myself to have been in this category too (CE background), though I have been making efforts to improve.
> I also work at an intersection of tech, applied research and science. My experience has been different. It's been surprising how controlled my collaborators/coworkers have been with their reliance on AI.
It's similar to the pattern I've noticed with experienced programmers I know and respect, the really good ones do seem to use AI but they are deeply sceptical about its claimed capabilities so they check/verify/assess what it's good for and use it for that, the tier down is less questioning and just accepts whatever it generates as gospel and it seems to degenerate the further you go down.
It seems like as soon as the AI states something confidently and is wrong in the domain in which the user is an expert, it loses a lot of credibility and people become much more wary of it in general but that requires you to be able to see that it is wrong otherwise it becomes a "bullshit baffles brains" generator.
It's useful to remember those times you've seen it do that when you are asking it something you aren't an expert in and then verify it's answer a different way.
This resonates a lot with me. I'm the CTO of a software factory-ish company. I get daily emails from clients running our decisions through LLMs and asking for ridiculous stuff that would multiply the implementation cost for them, just because Claude said so. No, your 3 users app doesn't need to be SOC2 compliant. No, of course we don't have triple zone redundancy while you're paying a 50 USD AWS bill.
Most of the time, these are just concerns triggered by chatting by an over-zealous LLM; in the worst cases, they are demands.
I'm usually able to clear concerns and disarticulate the LLM by explaining to customers how much more expensive everything would be if we did things like that. But it feels so frustrating, it's like you have to prove yourself everytime and justify every decision. These interactions have made me question my future in the industry, if I'm willing to keep dealing with these situations. I'm trying to foster patience in my life to cope with this.
Ugh. To be fair to the LLMs they're trained on industry blogs, and if you spent too much time reading them you'd also think every electronic stamp album needs to scale to a billion users with millisecond regional failover.
"I also hear a lot more questioning people in their domains and challenging opinions, then hearing what I can only imagine are fragmented pieces of conversations they had with an LLM thinking through some argument. Then there’s silence when you discuss shortcomings, then they come back later with their memorized fragments of what you said, combined with memorized fragments of the LLM response to the argument."
The worst thing about AI is that it's camouflage for stupid people. And it often gets combined with utter confidence in some position because they're too stupid to know the machine is confidently wrong.
Idk, seems like overconfidence outside your domain really got going in the '00s and is largely perpetrated by SWEs.
Something about making a lot of money and living in a world of abstractions really seems to fuel our sense of overconfidence in our abilities in other areas.
It's not just SWE's we just see it more in that pool because we mostly (we are on HN after all) swim in that pool.
I've eaten dinner with Doctors (medical) where they've confidently spoken about a subject I am more experienced in than them and they've been hilariously far off the mark, I just nod and ask for the gravy - not picking a fight over a meal.
Prof. Nathan Ballantyne coined the term Epistemic Trespassing to describe it, it is rampant and LLM's just act as an amplifier.
It's like people have a collective fear of just admitting they don't know something, there are a vast number of disciplines/topics I'm not competent or knowledgeable enough to have a meaningful opinion about even if I find the topic interesting, it's just been smart enough to realise that in the end.
Worst case, find the person who does know and ask then both of you might know (assuming you understand the answer of course which isn't guaranteed, I like physics, I'd understand about one word in ten if a physicist actually described it the way they would to another physicist).
> It’s making me want to be a lot less collaborative with such individuals.
I have the same conclusion and am getting a growing list of mental blacklisted people who I just sort of ignore or greatly discount their efforts.
I suppose I always did this if people were time wasters but what used to be a tiny list of “idiots” is now getting much, much bigger. Hopefully it doesn’t grow to everyone I work with like an AI Nothing.
What do you suggest these people do instead? I’ve been frustrated by this recently: I pivoted to a new subfield and am working on stuff that I would love to dive deep into and really learn what is going on so I can speak intelligently about the tradeoffs etc. But that would take a long time, and I have tasks that I should get done. So I have found myself working with a pretty vague understanding that, when pressed by coworkers, quickly finds its limits. Then I go back and try to deepen my understanding enough to cover those limits. But because I’m not working with each detail of the problem, implementing line by line with time to think about what’s happening, there just isn’t time for me to learn this unfamiliar topic. But I would love to, and I would enjoy the work much more if I could. So what am I supposed to do?
I'm not GP, but I've been experiencing this too. All my coworkers are in the same boat though, so we don't really have anyone with the deep understanding. Our approach (which has been working pretty well, we've solved a lot of problems and learned some along the way) has been to use AI as an informer who also points us to sources (like documentation, github issues, etc) where we can then read up on something the AI has pointed us at. We try to time box things a bit to prevent going down the rabbit hole, so we don't always get to exhaust our curiosity, but we're continually gaining that knowledge and pressing forward. Most recent example is tuning the kernel settings and our app on our prod machines to behave better for WebRTC packet handling/forwarding. AI for things like ffmpeg has been a god send when nobody on the team is an ffmpeg expert.
I've also had success just asking claude to write documentation on a system or subsystem or module, etc, and reading that. I then sometimes have it turn that into an svg diagram or something visual that often helps understand things. That domain-specific knowledge is hard to gain though, so not a silver bullet by any stretch.
I don't think you can do much about it if you're hoping to hide that you have been in the domain for less than a year. You're going to have to admit your inexperience and ask questions of your more experienced coworkers and try to stretch your common sense (e.g. devise internal consistency checks).
Don't just blindly pass around LLM generated content that you don't understand. I think it's important to consider that you are supposed to be more than just a meat proxy for the model. You don't want people to start associating you with AI slop.
Yes, certainly, in a year I’ll be better. But in the meantime…? 3 years ago it would be understandable for me to take my time with the tasks I’m doing because they are complex. Now it would be unreasonable for me to spend enough time to understand the nuances.
Isn't this basically the WebMD effect migrating to other domains?
feels like the same lament doctors have had for ages after anyone could google their symptoms then self-diagnose.
While AI is driving it the foundational cause feels like people having easy access to data/opinion that they trust but don't fully comprehend (or have bias towards). People do this in meat-space too, will confidently regurgitate garbage if they were told it from a person they trust as an expert.
> I have people who have limited experience with software pushing out layers and layers of abstracted code that’s fairly sophisticated but often misguided in intent who will say what they’re doing is correct, with conviction.
But isn't there value to refuting what Claude is saying if it deserves to be refuted?
The way I see it, if I was repeatedly getting sophisticated but subtly wrong arguments about my work that require me to understand why said argument is wrong, that's essentially drilling down to specifics of what precisely needs to be true to solve the problems I am aiming to solve. There is enormous value to this precision, isn't there?
Yes but it's annoying to collaborate with a Claude proxy person who thinks to have it all sorted out and who will question everything you say but not really listen, just itching to get back to Claude to confirm or deny what you said.
Compare that to being asked for advice, to explain directly something you know or have done, with the expectation of the person asking to learn something. Like asking why some decision was made in the past, based on what information and context, instead of confidently saying that it was done wrong. Both will eventually get you the same information.
Yes it is annoying, however I find that it gives me a much better grasp on the things I am explaining to a "proxy" Claude person. Basically, I would never have stumbled upon the arguments that I'm being presented with as I would never have prompted Claude in the specific way that a proxy Claude person would.
I find a lot of value in understanding how they have misunderstood things and clarifying those concepts.
I saw this happen a bit earlier this year, but most everyone I work with has learned that it was foolish. The feedback that they are doing something wrong needs to be explicit and strong. We're all going through a learning curve and establishing cultural norms is important at this time.
"vs just opening discussing things, considering tradeoffs from experts multiple disciplines weigh in on and then taking an approach that everyone finds most agreeable."
To be honest I struggle to do this with engineers as is, even without AI being a factor. Everyone likes to claim that their way is the right way to do things and fights for it, instead of stepping back and looking at all the options and gracefully acknowledging were things could be done differently for better results.
We are going back to philosopher conversations. Just a few guys sitting on the stairs, with chalk, the street as their whiteboard, the jammer keeping the LLM-zombies away. After 2000 years, after the loudness makes right post-modern-pre-llm drivel of the frankfurter school- its a philosopher renaissance as resistance.
I remember when google on your phone was new. The common complaints was, look at me while im talking, dont just fact check me. or something along those lines.
Its interesting to see the same shaped problem come up again. we didnt really figure out a solution to this, the world just kind of absorbed it as normal.
Then there are the megacorporations of AI whose highly paid tech support folk happily copypasta your reported issues into Claude or Codex and paste its "solution" into an email as if you couldn't do that yourself and they don't even bother to check if the "solution" even works.
I think there will be a rebalancing, and it might painful first for some. Confidence in LLM for highly complex tasks, very lots of context, most of all when this context isn’t in a single place or simply isn’t digitalised, will go down.
I use state of the art frontier coding agents daily. While the fact they can code at all is a huge achievement, it's hard for to me to buy in to the looming AGI/2027 collapse scenario when they remain stymied by the simplest out of sample tasks and sabotage their own work frequently. But the blooming ignorance of everyone addicted to them is striking. These things are tools, used correctly they can do some amazing things, but going Tim the Toolman with them goes poorly.
Probably more to it, but that's a pretty weak example. There are a lot of reasons text files might be better suited storing data vs. the overhead of a database.
> It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day.
There are also the guys who are just blatantly meat proxies for Claude. You send them a message, and you get back a response that's all Claude (and disorganized and not really making sense to boot).
I've got one on an affiliated team and I basically don't want to collaborate with him at all anymore.
But you know, "AI is the future of work," and all that. Those guys get a pat on the head by higher-ups and probably think they're doing what they're supposed to.
I try to do the opposite. However, I do battle test ideas against LLMs a lot, as well as humans. The experts have very limited access and availability:
The LLMs are trained to often be more pessimistic off the bat than the humans. But you can wear them down with arguments and they change their mind.
Here is an example where I have pushed my ideas in areas where I am not an expert, and generated papers to submit to conferences in order to get them peer reviewed BY experts: https://magarshak.com/papers.html
This is exactly what you’re talking about, except done very carefully.
I think it is amazing and it will democratise information. A lot of times the concept ends up being simple with a lot of jargon exists as a gatekeeping method. Well of course the incumbents wouldn't like that rando from outside gets to understand and speak about things they worked on for many years. Sure he doesn't get it right 100% but he's in the correct direction.
I very much doubt it will do that because access to that information was already available to anyone who wanted to know. It was not a secret. All AI did was to provide it via single interface, for a fee - which strangely do not go back to the original source of the knowledge, but to the AI company these days. This is not democratisation of information, it's commodification of it.
Partially true, but only partially. It also makes people believe they understand more than they do, because every answer comes so easily to them. But the LLMs are easily steered and tend to agree with what you want them to say.
> also makes people believe they understand more than they do, because every answer comes so easily to them. But the LLMs are easily steered and tend to agree with what you want them to say.
Well agree to disagree but my load bearing claim is that a lot of "effort" that the original people put are mostly useless. If you can get information without having to do that effort, you haven't lost anything. For example an archivist finds an 18th century spoon or something and he went through a lot of bureaucratic hoops and logistics to get it. After getting hold of the spoon, you and him are on the same epistemic level because the logistics added ~0 to the truth.
But finding a spoon (really?) is binary - you have the spoon or not. But understanding a complex situation is nuanced and contains different perspectives, tradeoffs, context etc.
Surely you are not comparing finding a spoon with being an expert in some complicated matter.
Awesome, now balance that out will education levels dropping like a rock, due to various factor but also due to LLMs, and things will be balanced out by also democratizing stupidity, until we live in Idiocracy.
it's hard to gauge this take without specifics. My experience with LLMs tends to be towards the opposite concept; LLMs dissuading me of hunches and notions I have about things (where I have no particular expertise; societal-level things), saving me and others time and strife having an argument about something they were actually right about all along, as my internal doubts that I'm too embarrassed to bring up (because these things aren't my field) are confirmed as incorrect.
I'd be curious to know specific examples of LLM-generated advice that goes against the advice of experts and does not consider tradeoffs. I've not had this experience myself.
If I did have this experience, someone spouting off obvious LLM points that contradict my expert opinion on something, I'd be headed right over to gemini/claude/whatever to see where that's coming from. Not any differently than if someone cited a google result that contradicted my own experience.
LLM tools tend confirm what you come to them with, be it unfounded conviction or doubt. I find them quite useless to tell me things I don't know, as verifying what they say shows that any non-trivial claims they make are usually oversimplified or plain wrong.
Anything more than using them to just pointing me to media and literature is usually a waste of time.
I have done that as well. Not a single time I asked an LLM for any sort of analysis that I could trust upon verification so I stopped doing that. If I need verify, much easier to just ask for suggestions of papers and books and do the analysis myself.
I guess this is subject matter dependent, certainly the LLM should be able to cite the papers/books for which it's deriving its critique. if it can't, then you know the advice it was giving is of poorer quality. I do that all the time
I guess there's a whole suite of skills that allow LLM use to be more or less useful. (I dont actually "guess" this, I sort of am pretty sure of it at this point, so it makes me wince when people call for total AI blackouts in education...would be better if they were part of an overall media literacy curriculum since people are going to be exposed to these things whether they think they're little infallible people, or complex systems with lots of caveats)
I always ask for sources. LLM analyses often fails by being simplistic or wrong in comparison with the sources it provides. More often than not, to the point that it is wasteful to ask it for analysis so I only ask for sources now and do my own analysis.
>it's hard to gauge this take without specifics. My experience with LLMs tends to be towards the opposite concept; LLMs dissuading me of hunches and notions I have about things (where I have no particular expertise; societal-level things), saving me and others time and strife having an argument about something they were actually right about all along, as my internal doubts that I'm too embarrassed to bring up (because these things aren't my field) are confirmed as incorrect.
You can do this with any subject.
The LLM is trained in part on a decade of shitty online comments and vapid professional correspondance. If you frame whatever you're asking about in a direction that would offend the sensibilities of the kind of people and ideas that are over-represented in that content it will hem and haw and drag its feet and whatnot.
i had that experience with them a few years ago but not these days. The models are being improved constantly, so here my "non expert hunch" is that...well, two things. either the models are getting better at sycophancy, OR, I myself am getting better at prompting - because I don't "argue" with a model.
I'd still love to chew on some specific examples though.
A typical example is when discussing some issue where there is no clear right or wrong. Architectural decisions for example. Then it is easy to use an LLM to produce arguments for your opinion. It is also easy to do this without realizing it because you do not understand the issue enough yourself so you do not know what questions to ask or how the actual circumstances affect the choices.
well sure. I'm pretty sure if someone came to me with an architectural opinion in my field that an unknowledgable person got by coaxing an LLM into sycophancy, I'd be able to counter them effectively. if they are just refusing to listen then they're just a toxic person which is nothing new.
So I guess this all goes into the familiar "LLMs allow people who are shitty at <X> to produce 10x the shitty output". this is a failure mode we're going to have to learn to mitigate
At some point, LLMs will be a far better source of truth, and then this distaste of people “outside their domains” is really just going to be a sort of snobbery from people who have had experience in a domain for a long time, (but they still have the same level of knowledge and insight as a person who just used an LLM to research).
These people are basically nascent “human supremacists”, who believe that only raw human insight and output has value, and is even superior, than equivalent output looked up and synthesized via LLM.
Not trying to be rude, but is this a serious comment, or are you being sarcastic? "Human supremacists", haha.
Also, why do you think someone who used an LLM to research would have the same level of knowledge and insight as someone who's been in the domain for a long time? That doesn't seem to hold up to any level of critical thinking.
AI is extremely useful, but it’s also extremely easy to fool yourself into thinking you understand what is going on without really understanding. This is often true with the code, but also for math and science concepts, etc.
Not many professions are formally trained to be cognizant of this lack of understanding, and how to confront it.
Usage of AI in collaborative settings is an amplifier of these issues, especially if someone doesn’t realize they don’t understand: they couldn’t teach or explain the concepts they use, or be forced to work with them malleably in a way that an expert or researcher would.
If you are cognizant of your lack of understanding, you can remedy it by slowing down and teaching yourself. This is required to make better use of AI in the domain of interest!
But you can’t have all things at once: you can’t move at speed with AI, collaborate effectively, and understand what is going on as an expert would. It is not physically possible for a human brain.
> But you can’t have all things at once: you can’t move at speed with AI, collaborate effectively, and understand what is going on as an expert would. It is not physically possible for a human brain.
I think the article's point is more nuanced. Short term, the human brain as the "conductor" can keep up and an expert can see whether the machine did a good job and course correct if necessary. Over time though that degrades more and more.
> But you can’t have all things at once: you can’t move at speed with AI, collaborate effectively, and understand what is going on as an expert would. It is not physically possible for a human brain.
Very well put. You can pick one or two, but not all three.
I can have AI pull some analysis - sql query, crunch some numbers and bam, some precise sounding number.
But then I question it, and then it says "you are right, I didn't check that"
How did I know what to question? By doing the friction work of querying myself, calculating myself, and then getting questions from leadership.
I worry about the younger generation. They spit out Claude results fast, people question it and they say "ok I'll take that back to Claude".
I suppose they'll eventually learn to do their own sense checking because they still have the friction of presenting to others.
And maybe my model is a thing of the past. Everyone is now going to be a manager (of agents) fresh out of school. They aren't doing the work anymore but they will need to set clear goals, validate work and hold agents accountable.
Oh it is, no question. All the people who come to AI with years of expertise are feeling it - what happens to all the people who never get the chance to develop it?
I think the problem is the same as with any technology: used correctly it adds
value, used incorrectly it subtracts from it.
LLMs can massively speed up a process, but without control they can turn into
social "sources of truth." A research process has well-defined, well-founded
phases: you delimit the topic, search for sources, evaluate whether they're
suitable, review their content, and place them on the map of the subject. The
more sources, the greater the knowledge, and the better the final result —
mental, or in the form of a report — is built. For that you have to read, and
reread, and think, connect, relate, and conclude.
AI can do all of those steps faster than a human. But if we let it do the
entire job on its own, its own way, with no checks at each stage, the
conclusions can end up distorted. If we know what we want it to do and how we
want it done, and we put the mechanisms in place to enforce that, the result
is different — better, more reliable. It's worth remembering: it's just a
tool, nothing more.
The wording of this comment in English has been corrected with AI, I don't
have enough fluency to express myself clearly, but I do review the final
result. In this case it got the verb tenses wrong, I saw it clearly, but the
AI didn't understand it, it took me several instructions to explain it so it
would understand. It's a tool, without supervision it can lead to problems,
but it has expanded the world for a lot of people.
The notion that "every human interaction is worth preserving" is probably not right, and Waymos are showcasing this. As the author mentions, many people don't enjoy the forced small talk with Uber drivers. Interacting with people from different backgrounds and perspectives is important, but I doubt the forced interactions like taxi rides are where we were getting those until day.
Completely agree. I think there is a danger that we will loose important interactions because we don't need them, but there's also a chance that we skip unimportant ones and focus on important ones.
If being in a Waymo alone causes you to be more comfortable to chat with people on the phone, it can even have a positive effect, for example.
I think you're missing the point, which is not that every human interaction is worth preserving, but that convenience has a hidden cost.
People would rather not spend time and effort to help their neighbors.
People would rather not pick up trash they see on the side of the road.
People would rather not take accountability.
If these things become the default action, then you've traded something for the convenience, and that something might be non obvious until you feel it is missing.
The neighbors one, I think a lot of people will admit a certain loneliness and desire to have community back. I don't think a lot of people will actually give up the inconvenience for it though, the sedative nature of convenience is one of its core problems. The number of people I hear say with conviction they can't imagine living anywhere other than CA because they couldn't stand the weather anywhere else...
I think this notion of friction vs frictionless is one of the key features of this time in history.
To my eye, the whole line from personal technology in the 80s -> through UX on the internet -> through app platform lock-in -> through subscription-centric "ownership" have been providing us all with the promise that we can buy our way into less friction. The next device or subscription solves problems or enables us one way or another. We see this line of history, draw it into the future and see a utopian future with zero friction.
Zero friction meaning zero difference between how you want to feel about the world and what you get out of it. More feeling empowered or satisfied or comfortable or right.
The parts of our culture that embrace friction are the slow parts - grousing with neighbors, weeding a garden, letting the other car go first. These are the parts where we meet with the reality of other people or nature. Maybe there's a tool or framework or gadget that could help, but we know and accept that there is something immutable outside of ourselves and we have to find a way to flow with it.
I'm getting older, I'm finally starting to see the limits of pursuing a frictionless life.
One thing I personally have noticed is that very result-oriented people seem to be vastly more optimistic about AI since for them it’s all about getting from A to B as quickly as possible, and AI broadly seems to promise that.
Conversely, the people who find joy in what they do, and see the journey and its frustrations as fulfilling and full of learning, tend to be more pessimistic or disillusioned.
I'm seeing this on a current academic research project as well- it's like throwing fuel on the fire for all the best and worst parts of working with researchers.
First: it's definitely become harder to get people to integrate their work and use standard tools. We're exploring creating AI skills etc for our tooling, simply because the actual audience we need to convince isn't in the room. If the LLM doesn't echo our recommendations, people will go with whatever one off script their personal bad idea bear churns out. Many research software errors are edge cases (scaling, numerical errors, etc), and "works on my computer" syndrome is endemic in the literature. The field has made big strides in enabling computational reproducibility, but lately it feels like we're set to lose ground again.
Second: LLMs have a bias to action, and can easily bury the user reporting on whatever. I've seen multiple seminars recently where the Q&A devolves to "Q: What are the implications of this finding? A: I don't know, this is just presenting a report on the results".
It seems that humans are still figuring out how to maintain agency and steer the chatbot to the big picture, and unreviewable science is just as likely an outcome as unreviewable code... but with far less automated tooling to help guide the process.
Historically, PIs nominally guided the big picture, and the entity who did the work was a participant in the review process; now students are having to look at projects from a new angle with their own (invisible) chatbot underlings. I suspect that the solution will involve a combination of technological change, capturing common expert review checks, and really adjusting the kinds of skills that trainees are expected to have early on.
For software developers, the equivalent is writing software without beta testers. If you don’t seek out users, you will likely build something nobody else wants.
And, that’s what I do. I’m retired and I use AI to build websites for myself. If anyone else ends up using it, that will be a happy side-effect.
If we want to do collaboration, we are going to need to do it intentionally rather than relying on side effects like conversations with cab drivers.
Counterpoint: most Open Source projects are (or started off as) programs people built for themselves, and anyone else ending up using them was a happy side-effect.
FWIW, this same thing is happening in regular corporate America too. I have all kinds of non-CPA, non-finance background people telling me how I should treat financial transactions, without the nuance of GAAP or how an auditor would handle their reasoning. Everyone has an opinion on what our tech teams should/could allow or enable or create, without understanding the security implications and support it would require or what other projects are already working on.
It’s good in a way, because you can certainly challenge people’s answer when an idea gets shot down. But it’s also a bit exhausting that everyone is challenging everything all the time.
The benefits of serendipity are intractable by design whereas throughput gains can be readily measured (or at least we think so).
The OP notes a point - "who did you think with?" -- maybe there are points based on some social graph, that adds weightage to a particular output?
Edit : Other posters have said it is AI generated, I did not catch it, but felt the point was well-made. I dont know the author, but lets assume that he is not a good writer but had these valuable insights, did this AI generation not add value here?
Agreed! Great flow, structure, no BS, really good!
I would wager this didn't use LLM help to be written. Though if I'm wrong I'd really like to know and would be very impressed. (And if I'm right then the article is an illustration of its own message. You don't learn how to write such essays without having practised it for many years without opting for the easy way of letting the LLM do this for you.)
It was definitely written with significant LLM assistance. It's very obvious, and the essay is not good.
> The Waymo effect: how AI is *quietly* making research less collaborative
> The Waymo arrived with the serene confidence of a machine that has never once worried about where to find parking, and I climbed into the back seat, glanced instinctively at the driver’s seat to say hello, and found myself nodding politely at an empty chair.
> No obligation to make conversation. No silent negotiation over the radio. A guilt-free space to be alone with one's thoughts, finish an email, or take a call en route without the awkwardness about conducting it in front of a stranger. The car was quiet, smooth and entirely undemanding.
> The collaborator’s inconvenience, in other words, is *not a bug* in the collaboration; it largely is the collaboration. The value of another mind *lies exactly in the ways* it refuses to be an extension of your own.
It absolutely was used, and heavily. I won't even cover the "it's not ___, it's ___", of which there are MANY in this piece. But beyond that:
Count the lists of three:
A guilt-free space to be alone with one's thoughts, finish an email, or take a call en route without the awkwardness about conducting it in front of a stranger.
It costs travel and scheduling and the glacial work of building trust. It costs ego management and compromise and the periodic small heartbreak of the author-order negotiation.
Look for referenced to "naming" things, used in the sense of "identify" or "realize" rather than, you know, naming something.
It took us slightly longer to name what we had lost.
Swap “pilot” for “researcher” and “autopilot” for “agent”, and the mechanism Wang is guarding against is the one Bainbridge named four decades earlier.
Look for things being done "quietly":
Two people who had crossed continents to talk to each other were quietly delighted by a technology whose central feature is that you don’t have to talk to anyone. (ed. - is that *really* Waymo's central feature?)
once a frictionless option exists, we take it by default, and quietly surrender whatever the friction was doing for us
Every conversation redirected from a colleague to a chatbot is a thread quietly withdrawn
while the invisible thing that sustained it quietly falls below threshold
competence quietly erodes precisely because it is needed only rarely
the quiet risk is that our incentives are training us
The fact that work is retreating into private chat sessions with AI agents is a loss; the idea of bringing those agentic collaborators out into public spaces to collaborate with teams is interesting.
> the driver was, for many of us on many days, the last stranger we were obliged to encounter. The last person from outside our bubble – professionally, politically, socially – with whom we had an unchosen conversation. The last reliable source of a view we did not ask for.
We need more encounters like these in our human lives. Burst our little bubbles every now and then.
Reminds me of what happened once most people and households transitioned to individually-carried cellphones as opposed to the 'house phone' (i.e. landline). You used to have to call 'the house' before you got routed to someone else within it, and that would often be cause for catching up with a person incidentally, and now that's just, totally gone! A great example of the trade-offs to navigate when improving systems for efficiency.
Sometimes you can tell though even if you don't understand the code. E.g. if AI makes a game, just play it. Sometimes code doesn't have to be 100% bug free and in fact humans can't achieve that anyway.
I've been using Astra a lot recently and it's really mad good. If you are currently in the "I'm still better than AI" camp (which I was until now) it might change your mind.
Was almost going to praise this article for not overtly sounding like Claude, but then I read this:
> and the incentive structures we have built are the experimental apparatus.
I agree with a lot of the article, but I want to posit that LLMs can encourage more human interaction by providing people with an information foundation that they might not have gained otherwise. What do I mean…
In the past 2 weeks I’ve had human conversations about fleet management and flood insurance that were a direct result of being able to quickly explore these topics with LLMS first.
Collaborative research is a dual purpose existensial safety hazard, it's better to keep all results locked up away from public use for the public's own benefit.
I have no doubt AI is acceleration or continuing out march into isolation. I also have no idea how to combat the root issue (since stopping the progress of science has never really worked or been possible).
This has been going on for quite some time, you can find lots of people talking about things like disappearing "third place/space"'s [0]. The trend seems to be fewer people know or interact with their neighbors, something I've been guilty of as well. We have unlimited opportunities for both entertainment and pursuing our interests. COVID also was like pouring gas on this with people isolating and some people never "reintegrated" afterwards. A general lack of community is the root issue from my perspective but how to best encourage/build that is something I struggle with.
I've seen some theorize the need for everything to be in service of growing the economy, everything needing to be turned into a side-hustle or monetized in some way is to blame. I can believe that, so often things feel like a zero-sum game and the fatigue from that is real. I can't tell you how many times I've started writing software (before or after LLMs) and found myself lost (in the very early stages) thinking about monetization or thinking about "how will this scale?", "How could I make this configurable for people who want to use this differently than I do?", etc, all _before_ there is a viable piece of software to use. Once I realize what I'm doing it's maddening. It's the "planning for how to handle 1M simultaneous users when you don't have 1".
LLMs have let me break from that thinking, design software just for my needs, which is freeing in many ways and I love what I've been able to do but also isolating. More and more I find myself wanting to fork "ideas" not code. Why would I want to try to participate in someone else's vibe/llm-assisted project instead of just making my own? I feel that pull and I also feel like it's wrong or misguided.
I have no answers here, just noting I'm seeing and experiencing this and I don't know how to stop it.
I'll end with a quote that Aaron Sorkin has written into multiple of his TV shows:
It seems to me that more and more we've come to expect less and less from each other, and I think that should change.
— Aaron Sorkin
[0] For those not familiar, my quick summary of the idea is that we normally have home and work but also a third place we go. For some that's a church or similar, for some it's community center, for other's it's some kind of community they are apart of.
If people want to collaborate, they will. If collaboration makes people more effective, they will want to collaborate. The novelty of any new technology is exciting and disrupting to existing habits but as the years pass we will reflect on what works and what doesn't. At least, those who are willing to reflect and adapt. Humans have a knack for this sort of thing...
Tangential: We need to invent some kind of personal wireless setting that will signal to strangers that you are open to meeting new people or starting a conversation.
Otherwise, all this technology will totally isolate us.
Ah, the ai panic has reached scientists, queue in the 1000s of articles about how it was about people all along and doom is coming to our civilisation because of it.
I’m an introvert who hates small talk, yet across countries; I’ve found the value of human drivers to far outweigh the small ‘cost’.
From knowing the area and telling me the Alamo location has moved (and not updating their address on their official website!); to coming back and returning an item I forgot to retrieve; these incidents have helped me so much; not to mention occasionally having deep and meaningful conversations that enriched my life and perspectives.
I’ve had some occasional bad experiences too; but that’s life. I took a Waymo twice, and I still go for Uber or Didi every time.
Well not only research, but also a lot of other aspects of life; it is way easier to interact with something that has always an answer is polite whatever tell them.
> The benefits are diffuse and deferred: the driver was, for many of us on many days, the last stranger we were obliged to encounter. The last person from outside our bubble – professionally, politically, socially – with whom we had an unchosen conversation. The last reliable source of a view we did not ask for.
This is a very strange take, to me.
I have conversations with cashiers, bartenders, servers, baristas, the random person in line in front of me or behind me, the person next to me on my flight (if they seem open to it), barbers, you name it.
I've had a handful of interesting conversations with cab drivers, but it's a tiny proportion of all my conversations with people "outside my bubble".
So I'm not really worried about this. The idea that cab drivers are somehow the last vestige of someone outside of your bubble is... bizarre to me.
llms right now work like pre cnn computer vision based on MLP's. By this i mean brute force of a model not really built for the task, and lacking a task specific inductive bias, being made work with unfathomable volumes of data and sheer brute scale.
if you look at the damage being done in the name of making this work for nlp, a forseeable situation, then you might also understand why most who could have done this sooner, never did so for fear of repeating mistakes we should be learning from, which we get for free if we just heed history.
Nobody knew that LLM's were an option. The architecture was basically waiting there for someone to say, "do that, but turn it up to 11," if I understand right.
a) Self driving was far from completely solved before LLMs.
b) Text generation essentially means passing the Turing test, which for a long time was the bar for general intelligence. Locomotion does not necessarily require general intelligence.
There's a general point here which is that the primary use and appeal of AI is to use it to avoid the negative side effects imposed on you by other people.
The flip side is the people with the biggest problems with AI are those that survive by imposing themselves on others in unnecessary ways, which is why so many bureaucrats are so enthused about regulating it.
Wut? AI do absolutely nothing in terms of "avoiding the negative side effects imposed on me by other people". Instead we are all having to deal with negative side effects of AI.
AI is imposed on us, whether we want it or not.
>which is why so many bureaucrats are so enthused about regulating it.
Like, OpenAI and Antropic? Because these two are the primary entities a.) pushing for regulation of their competitors b.) doing their best to convince us that AI is oh so dangerous and will kill us all as vengeful AI god is about to wake up any day now c.) openly bragging about using AI to hack other companies.
Article: someone enjoys being alone in the car, because they dont want to talk and find it impossible to be in the same space as other people without talking. And apparently end up arguing over radio each time they are on the 20min long car drive or something.
I think I would prefer to be alone rather with them too based on that.
I am not angry about article. I am saying that following claims are bullshit:
1.) the primary use and appeal of AI is to use it to avoid the negative side effects imposed on you by other people.
2.) people with the biggest problems with AI are those that survive by imposing themselves on others in unnecessary ways
3.) so many bureaucrats are so enthused about regulating it.
The article car example was stupid nonsense, but not angering. It was overly long tho. Other people in discussion guess this was ai generated and it is indeed quite possible.
> No wonder it annoys you.
What annoys me are people who are a.) unable to be silent and projecting it on others b.) who somehow always end up in inane arguments wherever they go. Article author literally described himself as such.
Right all those people protesting outside data centres are bureaucrats? This is a ridiculous take, large swaths of the population are against AI, most people are concerned about resource use, their livelihoods, and a destruction of the human element in so many crafts.
You are reacting emotionally and making baseless accusations about how I’m somehow going to be “left behind” based on nothing much in particular. And this is in response to me questioning it’s not a bit absurd to dismiss thousands of members of the public turning up and participating in local democracy opposing data centres with “faceless bureaucrats”. I’d take your flailing insults to heart if I respected your opinion but you already discredited yourself before you reached for the boring “luddite” accusations.
As for you complaining about straw men and misreadings etc, If you aren’t able to communicate clearly that’s a you problem.
>Of course people like you have a problem with AI since it will help everyone else move on and ignore you.
Based entirely on words you yourself have written. Either you don't understand what a straw man is or you don't understand what you are typing or you have some severe literacy issues.
Unrelated to AI, who is actually stressing about having to talk to drivers? In my experience most drivers are perfectly happy not chatting. This seems like a truly fake concern.
Yet the piece literally says
Two people who had crossed continents to talk to each other were quietly delighted by a technology whose central feature is that you don’t have to talk to anyone
I think we should describe a new effect infecting mainstream society:
- AI companies and the rich have increasingly pushed an agenda on the populace that most people didn't want, and the foundation of work in AI was formed by theft.
- Theft of creative work from millions of people, and a selective enforcement of law and compromise of investigatory authorities has demonstrated the law, morals, and the welfare of most people doesn't matter for people in power.
- This is the real "less collaborative" nature of the world where we find ourselves in. Add to that the increasing push for war by our leaders and rich too, people often shielded from the consequences, and you have a net less collaborative world.
It actually isn't even AI, but the same people pushing it are the same ones compromising collaboration for most people.
Many of the things we now do and valorize as part of our intellectual labours are going to become/seem increasingly vain, pointless, and cargo-cultish.
I'm pretty baffled by people I hear here and there saying that AI is great because it saves them the hassle of human interaction. "I love Waymo, I don't have to chit-chat with the driver and bear his horrible music". Apparently people didn't get the memo : humans are social animals. Autonomous individuals simply don't exist. Nothing is entirely yours...
And AI precisely builds upon that large availability of our common data. That's a modern enclosure movement, at least it would be but fortunately, the Communist Chinese are there to enforce sharing.
This falls short because not being forced into chit-chat and music is very pleasant, but chatting with LLMs instead of collaborating with other people on a project is not. These aren't equivalents.
> I love Waymo, I don't have to chit-chat with the driver and bear his horrible music
This is a reason I love Waymo. I don’t like talking with people and like avoiding it when possible.
There’s also may other, more important to me, reasons I like Waymo: they never cancel on me, their car doesn’t stink, no tipping (cheaper), not worrying about them rating me poorly.
Herd animals like cattle/horses, are highly suspicious of, and stressed by, strangers. And imagine how uneasy two chimps could react when meeting for the first time.
The same argument was made when elevator operators went away.
Some things work well with automation (taxis, washing machines, copy machines) while others work better with humans (chefs, masseuses).
50 years ago, I would have needed to dictate this note to a secretary who would distribute a memo. Does it further isolate ourselves that we can type and communicate directly? Would 1980 uses think it was awful to us?
Private autonomous vehicles are the pinnacle of entitlement. It is one of the single least efficient forms of transportation we could possibly come up with. It is such a perfect example of the public bearing the cost of private gains.
I am not against automation. I am against the idea that we are driving car culture forward in this way.
This article is written by an LLM, by the way. ("Quietly" is... as Claude might put it... "often the quiet tell".)
Which, given the section below from the article, ends up kind of ironic:
> There is a deeper cost still, and it concerns the thing LLMs do most impressively: writing. [...] Outsource the writing and you have not accelerated the thinking; you have skipped it.
I think the author might not know what "impressive writing" or even "good writing" is, which would explain both how they could put that part into the article, and how they seemingly believed this post was good and valuable enough to be posted publicly.
> the author might not know what "impressive writing" or even "good writing" is
Well the author in this case is Claude, and AIs write like that because the assistant persona really thinks that's what good writing sounds like. They're wrong, but they're just doing what they were taught. There's a reason LLM raters consistently score LLM writing highly.
Well perhaps the LLM put that bit in for them.
Interestingly that's the one point of the article I have a disagreement with. Yeah, good thinking comes when reformulating your ideas. Also, reformulating ideas is part of the traditional writing process. However it's does not necessarily focus writers/thinkers on the reformulating ideas in their most value adding form.
I've watched some YouTube videos narrated by AI that were created by those who's native language I assume to be Chinese but the value of the content is higher and more concentrated than those from the native speakers.
While I've watched many for whom English is not their first language struggle in technical talks and lose most of the meat if their discussion to their struggle with the language conversion.
While the example of language barriers being skipped over and providing value in that circumstance is obvious I suspect more is possible by avoiding unnecessarily focus on prose or technical aspects of communication and focusing on the ideas themselves.
Imagine if the cost of not assuming a background in technical/textbook writing were zero. Freeing up authors to explain more thoroughly. Perhaps, more effort can be spent on thinking up analogies, metaphor or examples to help communicate an idea.
> believed this post was good and valuable enough to be posted publicly.
I mean, it made it to the front page of HN didn't it? Probably served its purpose just fine. Not all writing is supposed to be impressive or good. Some of it is to just get attention and stir discussion, and this Claude output did exactly that.
I wanted you to be wrong, and to be able to make this an example of us over-reacting to certain trigger words created by AI, but unfortunately I just scanned the first couple paragraphs with pangram and it reported 100% AI, so you're probably correct.
I get a funny feeling in my stomach over the idea that common and effective means of communication (i.e. it's not X it's Y) have become faux pas to use because of AI. I think it's something about these phrases being taken away from us more-so than the AI inventing them.
Pangram should be paying HNers for how often we pitch needing to use their product by name to believe things as obvious as "a long form news article cramming every AI trope it can fit from start to finish" being AI written.
At this point I'm surprised when a news article isn't largely AI written, let alone one using the default tone! I don't even mind it as much as others seem to, it's just turning into more and more of a rarity for a news article to not be these last few years and so is now what sticks out.
> Pangram should be paying HNers for how often we pitch needing to use their product by name
I kind of assumed it is, given how it suddenly seemed to start getting namedropped in multiple comment threads. And it's often in response to someone saying content is obviously LLM (with examples) and the shill wedges pangram into the conversation "omg you're right, I didn't believe you but I checked this new product and wow it agreed with you" as though that adds anything to the discussion at all.
Pangram gets brought up a lot because if I read an article and think it's blatant AI slop and want to communicate that fact, a natural impulse is to provide some sort of objective corroboration rather than just asserting that I have superior taste and thus am able to tell.
Also Pangram is basically the only AI detector that's actually put in the work to build an accurate classifier, so if you try to discuss AI writing detection without being specific that you mean Pangram you'll get half a dozen commenters screaming about how awful some of the snake-oil salesmen like GPTZero are.
I mention pangram because it’s good at what it does. Otherwise, people like to claim “That’s just how I’ve always written. I always say That’s the seam and the seam undercuts the other pillar. It’s not just correct, it’s also verified lol; it’s just how I speak” and it’s obvious bullshit. Depending on the politics of the poster, people will also support this kind of garbage, but it’s all drivel.
The authors are so often dishonest that it’s hard to believe that the things they’re saying are anything but a random hallucination.
You have to name pangram because the other ai detectors are quite bad
Em-dashes would be a loss, they can be a better flowing version of a parenthetical. “It’s not X it’s Y” is not a loss. This phrase is a symptom of a situation where the author wants to subvert expectations but doesn’t have the space, ability, or faith in their audience to organically set up X as the thing to be contrasted against.
I suspect it became an LLM tell because it is over-represented in text that’s easily available to the models but that most people don’t actually want to consume: marketing text, LinkedIn posts, that sort of thing.
That’s not common and effective, it’s buzz feed style articles.
Have you also scanned earlier work of the author with pangram to establish a baseline? Pangram is not without problems.
It is actually pretty much without problems. It doesn't catch everything, but if it detects something it's very rarely a false positive.
Can you give some examples of verifiable pangram false positives?
Why trust pangram? What are their accuracy scores in the wild? Ai detection is notoriously difficult.
AI detection in long-form content is a problem well-suited to training a classifier model. We have tons of verifiably-not-AI text from before 2022, and you can create tons of verifiably-AI text. As language drifts over the next few decades, it may get more difficult. But right now it is quite a manageable problem for languages with large pre-2022 text corpora available online.
The reason why AI detection tools other than Pangram are awful is because they are not really trying to solve the problem -- they just want to appear good enough to convince people to use them.
All these "is this AI" checkers should be banned, as they never can reach 100% accuracy.
This is a truely bizarre claim. Imagine if one said:
"Weather forecasts should be banned because they can never reach 100% accuracy"
They are completely different things with different consequences. Someone can point to the sky and prove that the sun is there.
You're responding to a comment about weather forecasts, not current weather. Pointing to the sun is not a forecast.
Imagine writing an article in 5 days. Working hard. Posting it online and then everyone says it is ai and laughs and dismisses it because a tool says it is AI.
Like in this comic about art: https://www.peppercarrot.com/en/miniFantasyTheater/064.html
Imagine wasting several thousand people's time
They are overrepresented in HN discussions for sure as there is an over-reliance on them that just aims to shut down discussions.
Which is ironic since they are a tool folks are trusting blindly supposedly used to argue against tool use with blind trust.
Fwiw, I only did the first 300 words (signed into pangram) and it seemingly correctly noted that there were 2 (mostly) human-authored sentences in there
What's wild to me is that:
1. People are responding to this article like it's hitting a nerve
2. In most Ubers you already don't talk to the driver (yellow cabs are higher variance in NYC). Doesn't seem like anything's being lost in that case.
3. There are enormous safety benefits to waymo, mobility benefits for youth (and elderly) that are afforded by this technology. It's not clear why people argue "Uber" is better than waymo. A few years ago there were arguments against Uber! (A technology which, again, provides a huge benefit, especially if you live in an area where people were previously expected to go out for drinks and then drive home)
I think it's fair to point out real issues at these companies. But we should be clear-eyed about which technologies we want to accelerate vs slow down
It's super obviously AI-written. I can smell this shit from a mile away now, after using Claude Code so much.
In the just first few sentences I could tell.
Have you actually read the article? They point out multiple times that they like the Waymo ride and would use one again and loved it. It's not a Waymo-bashing article, it cautions against the long term effects on research.
And here you have it — I used an LLM-ism. Oh, and now another one! Time to downvote me for supposed LLM use! (Which I obviously didn't, I'm typing this on an ancient smartphone waiting for a train, but that doesn't deter the witch hunters.)
You actually didn't. The construct is slightly different.
I use LLMs all day every day. I think they're great. I just don't like humans who put their name to what an LLM did.
> You actually didn't. The construct is slightly different.
Well I know that I did, so you are wrong with that claim, which sheds a strong (negative) light on other statements of fact that you posted in this discussion.
You may disagree with me, or I may even have gotten something wrong, but just claiming I didn't read the article ... sorry man, I thought HN had higher standards than that.
They were pointing out that "It's not a Waymo-bashing article, it cautions against the long term effects on research." does not sound like LLM-generated text.
It would sound kind of LLM-y if you had said "It's not an anti-Waymo article. It's an article about how assistive technology is quietly degrading research."
(Of course, the correct approach to detecting AI is not counting LLM-isms, but feeding long-form text through a classifier model and picking up statistical correlations that are more in-distribution with LLM text than human text.)
Correct. I was already pretty sure, but I wouldn't have written that post without checking Pangram first.
"It's not a Waymo-bashing article, it cautions against the long term effects on research." is not an LLMism. It superficially resembles one, but that's it.
(Nor is what I just wrote, there.)
I think they were referring to your fake llmism
Ah thanks. Fair.
Calling it "witch hunting" is just dishonest. That term implies a baseless overreaction to a heavily exaggerated or imagined threat.
That's hardly what's going on here. Unlike witches, AI slop really is everywhere and it's pretty clear that this article is slop.
I find complaining about AI callouts to be itself deeply distasteful although it would take some thought to put my finger on the exact reasons why.
Yes AI slop is prevalent. But the casualties that get thrown under the bus are also real, don't you agree? Is it really worth it to sacrifice those? In the name of the larger good?
I'm arguing that we're losing something by doing so, as a community.
> I find complaining about AI callouts
If they are correct then I'm all for it. But just suspicions turned into statements of fact are not helping anybody. People used em dashes before LLMs Not nearly as much as LLMs, but superficially judging people isn't doing any good.
I'd compare this with pitchforks coming out agains criminal immigrants. Yes, some immigrants are criminal. Doesn't mean that if you stand in central Copenhagen and have a dark skinned person in front of you that it's a criminal.
The rule of thumb is simple: if most of a user or author's writing is consistently human-generated, then I think we are very happy giving them the benefit of the doubt if one article or snippet flags the actually good ai detector
Unfortunately, many of the largest voices against pangram simply don't like it because it gives their lies less credibility.
I don't mind reading llm writing, indeed, I read more llm writing per day than most. But if you're using LLMs to increase the level of slop (blog posts, comments, tweets, etc) then we should call people out.
It's a colossal waste of everyone's time and attention, and we should be mad about it.
Late edit: if people are actually writing and it's consistently flagged by pangram (this is a statement I have not yet seen validated), the pangram folks are extremely proactive and excited to understand what's going on. This notion that somehow the authors of slop are victims is complete nonsense.
> if most of a user or author's writing is consistently human-generated, then I think we are very happy giving them the benefit of the doubt if one article or snippet flags the actually good ai detector
I would agree to that approach.
But that didn't happen here. Nobody has done that due diligence and folks are just blindly accusing. The author has published for many years. That's what I'm calling out.
> This notion that somehow the authors of slop are victims is complete nonsense.
Strawman? I didn't claim that or anything close to it. I also find the prevalence of slop writing hugely annoying.
Is it OK to flag articles for being AI-written? I really want to.
Yes, HN is for human discussion. It is fine to flag and remove things that are not human written and cannot facilitate human discussion due to it’s false construction.
No, it is not.
Is "quietly" an LLM tell? It was always certainly a human writer trope commonly seen in journalism. Though I guess it does appear five times in the body of the post, and a human writer would probably not go that far with it.
I think individually and when used sparingly, writing tropes can be fine, but when the article has every second sentence being a writing trope, it becomes pretty obvious that either a LLM wrote it, or the author simply doesn't know how to write, and regardless, it's a waste of time trying to read through it.
In this sort of way, yes, it's a clue. Obviously not dispositive just by itself.
It genuinely is--and that's the honest truth.
The real LLM tell is tortured and inappropriate metaphors and unnecessary adjectives and adverbs[1]. What purpose is "quietly" serving in this headline?
[1] https://www.youtube.com/watch?v=ORgKY9AlybA
You're absolutely right—it's been a load-bearing LLM tell for a while now.
I think that in the end it does not matter, anymore. It is inevitable that most of the text now and in the future is at least reviewed by LLM.
We should criticize whether text is poorly written, inaccurate, wastes words to get to the point and anything like. Saying that it is "written by LLM" just bypasses this and is not useful and misses the point. AI written text can be really good if used correctly. Criticize the content, don't speculate how it was written. If AI helps us to write better text, that is great. But often it is not good.
It is better blame the author for the bad text, so they get consequences and might do better next time, if the text is bad.
By the way, I think this piece of text was quite good.
> saying it is "written by LLM" is not useful and misses the point.
It is useful to me. I see too much slop every day and would like to filter it out from my life. I appreciate the tip, and to me, that is the whole point.
Do you have more evidence than this? I'd honestly like to know.
I don't like AI slop like everybody else, but I'm also growing skeptical of the very fast determinations that sth is supposedly made by LLM just because it uses some phrase that Claude also uses. Models are trained on text written by human writers and if human text resembles it, this goes both ways. I have looked at the authors work from the pre-AI era and it reads similar to me.
So please, if you come with such a claim, please include what you base it on, so we all have a chance to determine how much to trust your verdict.
"every hour spent aligning with a co-author is an hour not spent producing output that is legible to an evaluation system." use of the word 'legible'.
"We defund the corridor and then wonder where the corridor conversations went." this is a claude-ist construction.
"This is an old worry wearing new clothes" this is a very claude-ist construction.
(rather ironically... as @embedding-shaep pointed out earlier): "Outsource the writing and you have not accelerated the thinking; you have skipped it." 'not X but punchy-Y' (not to be confused with 'not X but profound-Y'.
I'm not saying the author shouldn't have used an LLM - once in a while I suppose it gets used to good effect. But I wouldn't kid myself that the writing is certainly all human through-and-through.
[Incidentally, in case anyone's wondering where many of these claude-isms come from, just search within lesswrong.com .]
And I read it by asking an LLM to summarise it
Can you share me the summary? I'd like to read a LLM summary of your summary.
Things are bit tight today.
", and" is the more obvious tell for Opus/Fable writing.
Plenty of people use Oxford commas though, can't really use it as a reliable signal, too much noise.
This comment was perhaps written by an LLM that has learned it can karma farm by pointing out all of the articles that have LLM usage while everyone still thinks that is a novel contribution.
If you check my comment history here, I have not written this before.
I like LLMs and use them daily, but I don't really like when people use them undisclosed for long-form prose writing.
Me neither, but just running around accusing random folks of things is not going to help. To the contrary. If a real author doing real work with a lot of effort risks being accused of things they didn't do and just dismissed, you'll get less of genuine content, not more.
I will donate $100 to charity if I made a false accusation.
(It's possible AI rewrote an article first written by a human here or whatever, but the point is that I can just tell this is mostly or entirely LLM output.)
> I will donate $100 to charity if I made a false accusation
That's honorable but not how accusations work. There is a first level indirect effect of painting somebody in a light that damages the reputation of their work or their person which isn't reversed by a donation or a sorry or anything since retractions/corrections/apologies don't get the same public attention. And there is a broader effect of chilling what people do since they don't want to get falsely accused.
I'd recommend keeping such suspicions to oneself unless there is unrefutable proof, and if there is, then please provide it.
It does help, if the accusation is true. The fact that false accusations result in less genuine content implies that true accusations result in less disingenuous content. For better or for worse, social consequences do affect people's behavior on average.
All true. But question is whether the positive consequences outweigh the negative. That may be the case, but it may not be. I don't find it obvious and not considering this is careless.
The first few sentences alone are pure Claudish slop (I didn’t read any further). The stench is unmistakable once you’ve looked at it enough.
I’d also donate $100 if I’m wrong. I feel quite confident.
Yeah, lots of redheads were burned back in the day because people "felt" things.
It’s a blog post… I’m not proposing we burn the author. You alright?
In the spirit of the "That's what she said" bot, you could train an LLM detector by accusing everything of being authored by an LLM and seeing which ones get voted up.
"Novel" is not a prerequisite for something to be worth pointing out. Lots of bad things happen repeatedly, and don't quickly become not worth caring about or knowing about. People have strong spirits and curious minds and it tends to take a long time to boil that frog out of them - generations, sometimes.
People aren't treating it as a thoughtful contribution. It is more of a flag, like "spam".
HN already has a feature for that. Just flag the article!
I appreciate when people write things like, "this is an advertisement for the author's product, X." Why stay silent? If the commenter's accusation smells fishy, I'll read the article myself. Otherwise, they helped me.
To me, it’s the equivalent of “HN is turning into Reddit” - it’s just a complaint without adding to the discussion.
It's sad given the topic is interesting and significant.
This rings more and more true every day: https://samkriss.substack.com/p/if-you-let-ai-do-your-writin...
and silently.
I work at an intersection of tech, applied research, and science.
Something I’ve noticed in collaboration that does occur is an increased confidence in people outside their domains to say things with conviction. I have people who have limited experience with software pushing out layers and layers of abstracted code that’s fairly sophisticated but often misguided in intent who will say what they’re doing is correct, with conviction.
I also hear a lot more questioning people in their domains and challenging opinions, then hearing what I can only imagine are fragmented pieces of conversations they had with an LLM thinking through some argument. Then there’s silence when you discuss shortcomings, then they come back later with their memorized fragments of what you said, combined with memorized fragments of the LLM response to the argument.
It’s occurring, a lot more. People are treating their LLMs in collaboration as a source of truth and using then to focus on their specific path or goals they think or have bias towards going down, vs just opening discussing things, considering tradeoffs from experts multiple disciplines weigh in on and then taking an approach that everyone finds most agreeable.
It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day.
Wow, oddly similar thing happened to me. Anecdotally, I was working with a non-developer. She was building a reporting dashboard with Claude. I pointed out that the system she built was using the filesystem to store one line notes about the report, and I said "I would probably tell the agent to use a database for that, but it's not the end of the world if it works". And she replied "Claude said the developer's right, a database would be a better fit..."
So, a non-developer, who has no understanding of the underlying systems they are working with, fact-checked a developer with years of experience.
I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there. She couldn't rely on my expertise, she had to go back to her "source of truth" (Claude) and asked for a second opinion on something that she wouldn't be able to verify.
This is why collaboration exists. I trust my colleagues to give me insight+advice on things that I do not know. I don't ask AI to vet their decisions, because at some level there are judgement calls, and I WANT to trust them because it makes my life easier.
Treating AI as if the output is factual is just a misuse of the tools, I think. It's significantly more effective when given expert direction and the output can be verified by... an expert.
>>I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there. She couldn't rely on my expertise, she had to go back to her "source of truth" (Claude) and asked for a second opinion on something that she wouldn't be able to verify.
This is probably how physicians feel when informed patients (whether using LLMs or not) ask questions (good or bad) during their patient encounters.
I would love to be able to trust that a doctor knows more about my problem than I do, but every time I see one they clearly haven't even read the information I just told the nurse 15 minutes ago.
Doctor:
I don’t mind the questions. I mind the assertions of fact.
This is a different situation. A doctor is a domain expert, but you are in possession of vastly more empirical data about your specific body. Getting to a good result is going to be easier with collaboration, rather than one of you confidently telling the other what the problem/solution is.
Yeah I've felt that gut punch all the time. Seems to happen with every technology. In the early days of GPS, people would ask for directions to the gas station then pull it up on their phone map and go "yeah, you're right!" So thanks for wasting my time, insulting my integrity, and killing my faith in humanity all in 3 words. And if you really need two opinions, it's so easy to just reverse the situation, "my GPS says there's an Exxon in two miles if I turn left here and then right, is that correct?"
I remember fifteen years ago I would tell people leaving my house "oh yeah, ignore your GPS that tells you to go that way, just down the road here and left turn; that'll get you on the highway much faster."
Nowadays obviously Google Maps gets it right, but it was interesting that period where you had to actively preempt everyone's instinct to just rely on the little table hanging off their dashboard.
there was an institution my son went to for therapy for his various handicaps, and one of the major taxi routing solutions in Denmark had the route wrong, I think the one based on Google maps actually, so when you called for a taxi it always drove nearly a block away that you had to walk through a couple of back alleys to get to. And then the taxi driver would call me up and complain I wasn't there, and I had to guide them to the place where they actually had to be at over the phone.
Sometimes they were not capable of following the instructions though.
Kind of like getting people to ignore generative AI today, right?
I'd bet it depends on what they write to the LLM, since sycophancy is so strong in these things. If they write in a fairly agnostic way, it might default to sycophancy to you the dev, while if they write it in an opinionated way it might default to sycophancy to them. Regardless of who is right.
To be generous, it seems like the most natural way to phrase what happened. The non-developer punched in "should this be a database instead?" and it will say "You are right to question it! I was sloppy when .. blah blah blah blah blah"
I doubt the human went into a whole diatribe with the CLI about how the developer on the project with this many years was saying this or that but she wanted Claude to second guess them.
It's just saying "ah yeah, that went through without issue."
I can see how the situation is annoying there; I've had a similar frustration when doing manual code review for someone's obviously AI-slop PR only to have their bot auto-respond to my comments. There's an asymmetry of effort that feels unfair. Like, don't ask for manual review of something that contains lots of things that your own manual review should have caught first, and if I provide human feedback, I expect human responses.
That said, I wouldn't necessarily take it at face value that the person you were working with trusted the bot more than you— at the end of the day, the bot was doing all the implementation anyway, so all the interactions between her and the bot were probably framed in language like "what if we" or "I think it would be better if", so her saying "my programmer colleague thinks X" would naturally generate that "yeah your colleague is right, let's go that direction, makes sense" that she passed back to you, probably thinking it was a complement.
Maybe the real frustration is her not understanding the value-add that still exists there when someone with real expertise and more context can step in and provide that nudge in the right direction.
Sure - I don't hold it against them. It was just surprising to read a "Claude said you're right" message. (Yes, I know that I'm right, or I wouldn't have said it)
Didn't ruin my week or anything, but that was the writing on the wall for me.
Technical expertise is no longer viewed in the same light when software development has no barrier to entry.
True, though I wonder if 2026's "Claude make me a minecraft plugin" is 2006's "View Source" button.
Web development took a long time to gain respect in part due to the accessibility of it. When everyone had a nephew who "makes websites" and anyone could get started with a $5/mo Dreamhost account, it was harder to see why a consultant was worth $200/hr to do the job properly.
It'll be interesting to see how this plays out, particularly accounting for bots continuing to get smarter and smarter. Is the "craft" of software engineering dying before our eyes? Or will this be like webdev, where there's an explosion of access and ideas for a time and then it settles back down into being a thing that normal people mostly leave to the nerds.
wow that wouldnt feel like a gut punch at all to me, it's normal for inexperienced people to not really understand what kinds of knowledge experts have or don't. I certainly come back from doctors appointments and google/gemini chat for whatever came up to get more detail. this person going to claude and getting immediate confirmation that you were exactly correct should have felt great, she would be like "wow, this guy's good, huh."
>I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there.
Maybe I'm just cynical but I don't think I ever see that much trust for anyone's opinion or expertise; devs think managers don't know what they're doing, managers think devs don't, people on HN think lawyers are wrong when they state their legal opinion, devs think other devs are wrong about their areas of expertise even though they have no reason to think that, hell I've worked with people who basically lie for some reason about technical issues that they have no expertise in, in order to get some benefit that is unclear to me. I've also seen people of great technical expertise gaslight industries in order to derive very clear benefits for their employers, so I'm not sure why there should be trust.
I'm sure that various AI agents will maybe add another interesting wrinkle to this, but I sort of feel like you must have had a nice life up until AI if that was feeling like a gut punch.
I think I could qualify as an expert on various topics relative to many average people. But it always bothered me when experts wanted to "just be believed". In your example, it doesn't long to explain why a database is better than the file system in this instance and a moderately intelligent should be able to understand and not need you or Claude as the source of truth.
So I think it can be good people are able to "AI" to question expert opinions - though naturally, as you say, it's bad if people take the AI as a single authority akin to an expert.
So, I did let her know that a storage volume was going to be more difficult to manage, could result in lost data, and would make the data harder to work with - I mentioned it's not the end of the world if it works, it was just a professional suggestion.
To me, there are different levels to "just being believed". If a mechanic tells you your car isn't working because you don't have an alternator, does that really need scrutiny? If you don't understand the subject matter, at some point you have to "just believe" someone else.
Especially for a non-technical user, trusting the software developers at your company on technical advice should be a given.
> Treating AI as if the output is factual is just a misuse of the tools, I think.
Especially because it's very often wrong. AI tools lack the necessary context of the specific situation and the surrounding technology environment to make appropriate suggestions for anything more complex than clean sheet designs and toy prototypes. There is too much complexity that is hidden in domain knowledge and not even exposed fully in documentation for any AI tool currently to ingest it all into its own context to give you an accurate answer, even setting aside hallucinations and that these tools are incapable of reasoning.
I have this same gut punch constantly. AI has destroyed functional outcomes in corporations, because managers are all deep in AI psychosis deciding that whatever the tool says is perfectly correct at all times. We've replaced analysis paralysis with a complete absence of thought from any human in high-level impactful strategy discussions, everything is entirely being abdicated to AI tools in so many companies. It's ridiculous.
AI will radically change the economy, but not because its replacing jobs, but because its eroding trust, domain knowledge and expertise, and thinking/creativity while being a massive waste of money and total boondoggle. This bubble can't burst soon enough so we can get on with doing shit that actually matters for people that actually helps them.
non-dev AI psychosis doesn’t bother me
I’ve seen how the market treats vibe coders that werent previoisly engineers. Coinbase’s CEO bragged about PM’s deploying code and everyone now thinks they are totally incompetent anytime a small UX problem occurs.
Engineers with prior experience now check a box, even if they are doing the same thing. Like a Compliance Officer or Cybersecurity professional checks a box
Annoyingly pedantically speaking, a filesystem is a type of database.
God do I wish that was her response to my suggestion.
"Fair enough - go nuts. Let me know when Claude tells you to transition to Postgres because I'll need to get you access credentials"
Yup, the arrogance when barging into unfamiliar domain that was previously reserved mainly to physics grads seems to have spread everywhere. I also had people opining on my expert area via their LLMs and the problem is they can't even ask the model the right question, let alone evaluate the nuance of their answer.
As an unfamiliar, I find it interesting how you specifically called out physics grads in the pre-LLM era. Is that just snark or is there some widespread truth / stereotype of that behavior?
Physics, and to a large degree pure mathematics, has attracted a critical mass of the personality type that needs to feel like the smartest person in the room in any given situation. It has given them a slightly notorious reputation in academia.
As a mathematician, this is far from my experience of actual professional mathematicians, because they encounter people much smarter than them quite often. Maybe more true among graduate students, however.
This thread started by referencing grad students, and I can vouch for it from personal experience with physics grad students.
I don't know why mathematicians got lumped in with physicists further down the thread. In my experience this is a well known stereotype of physicists and while of course not universal you don't have to look very hard to see where it came from. In my experience mathematicians stay very much in their lane and often seem to be overly modest even within it.
The best/smartest professionals I meet operate this way (because we know we're wrong quite a bit and don't know everything). It's definitely a personality trait. Inquisitive, curious, intelligent, but also there's some metacognition involved where we can be confident in what we do NOT know.
In contrast, I studied chemical engineering for a short stint. First year, first semester chemistry classes and labs were riddled with these types of students. Tell you you're wrong, IQ 197, scoff when you ask a question, etc...
Listen, if you're the smartest person alive that's fantastic, but please don't talk down to me or the mere mortals that I prefer to hang around with. We get along much better and have a lot more fun saying "I don't know, let's find out".
I think that's a common trait of tenured academics in the top 100 universities in the US. Don't know about Europe and elsewhere though. Not so much of nontenure track sorts who do all their dirty work.
From a one-time physics grad student, I do think there advantages to being ... like this.
The flip side of being confidently wrong sometimes is that we treat understanding new systems as something approachable. So, yeah, it's annoying that we don't stay in our lane, but, from my boss's perspective, I'm the only one who's actually willing to just read code, investigate, possibly read a paper, and figure out what's going on instead of punting and saying it's someone else's turf I'm blocked until they get back to me.
Like, ultimately you can't be a knowledge worker by just waiting for answers and instruction from experts and relaying them back and forth. Eventually you have to understand something. And part of that process is being wrong and annoying people.
Like your sibling comment I saw this mainly with physics people doing data science:
> Eventually you have to understand something. And part of that process is being wrong and annoying people.
There is nothing more frustrating as a colleague than having a smart person walk in, insist you're all wrong, and then burn all the energy and goodwill in the organisation having to rebuild absolutely everything from the principles they currently understand because they refuse to listen to the experience that has built up already (in the entire field, let alone the organisation), only to produce something worse on every dimension than what already existed.
But there is a difference between not listening to experience and examining whether the resulting implementation/decisions based on the experience are actually valid. I doubt that someone smart (as you put it) would deliberately want things redone because they don't understand it, that's a trait borne of arrogance not intelligence.
At least in my experience (as a physics grad), me and all my fellow grads are much more likely to apply Chesterton's fence and then make decisions than not doing that and steamrolling existing decisions.
> I doubt that someone smart (as you put it) would deliberately want things redone because they don't understand it
You're actually doing the problem behavior right here: denying the validity of the experience of seeing exactly this happen, not as a one off, but repeatedly in completely separate situations and institutions.
> that's a trait borne of arrogance not intelligence.
I mean the problem is they're arrogant. The two aren't mutually exclusive, far from it.
My point is that wanting things redone because you don’t understand it is dumb behaviour, not smart behaviour.
There wasn’t even a denial of the validity of experience, I was just trying to examine the problem - if this behaviour is something you see often exhibited by “smart” people then: either you are unable to convince them of your side of the argument for different reasons, or you are not actually arguing with smart people. Do you see the distinction?
> The two aren't mutually exclusive, far from it.
Intelligence and arrogance are somewhat contra-indicative. While you can be intelligent and arrogant, you can’t really learn things without the humility to say you don’t know stuff. In my experience, most physics grads I know constantly say they don’t know stuff, way more than people from other disciplines.
> My point is that wanting things redone because you don’t understand it is dumb behaviour, not smart behaviour.
Smart people can do very non smart things when they feel their status as smartest-person-in-the-room is threatened, which is the entire point.
> There wasn’t even a denial of the validity of experience, I was just trying to examine the problem - if this behaviour is something you see often exhibited by “smart” people then: either you are unable to convince them of your side of the argument for different reasons, or you are not actually arguing with smart people. Do you see the distinction?
Now you're gaslighting, bluntly.
> Intelligence and arrogance are somewhat contra-indicative.
I've never seen any evidence of this, but a _lot_ that many physics (and pure mathematics) grads are enormously more arrogant than the general population, enough that the trend under discussion has actual basis.
I was responding to you in the context of the GP.
> The flip side of being confidently wrong sometimes is that we treat understanding new systems as something approachable. So, yeah, it's annoying that we don't stay in our lane, but, from my boss's perspective, I'm the only one who's actually willing to just read code, investigate, possibly read a paper, and figure out what's going on instead of punting and saying it's someone else's turf I'm blocked until they get back to me.
Eventually you have to understand something. And part of that process is being wrong and annoying people.
GP here is suggesting that accepting established status quo at face value (i.e relying solely on other people's experience/letting them deal with problems you face in their domain without being able to examine why they arrived at the decisions they did) can lead to worse outcomes, and I agree.
Your response was that it is frustrating to have someone walk in and question things (precisely what GP is suggesting), only to have the final product of whatever decisions these people make be worse than the status quo (not what GP is suggesting).
I responded by saying - there is a difference between not listening to experience and examining whether the resulting implementation/decisions based on the experience are actually valid.
You did not address that. Is there not a difference between these two things? GP is suggesting to do the latter, not the former. i.e question why things are the way they are, crucially not that people should make decisions based on things we don't understand about the status quo.
You only responded to - * I doubt that someone smart (as you put it) would deliberately want things redone because they don't understand it, that's a trait borne of arrogance not intelligence.*
Your response claimed that I am denying the validity of your experience, when I made no such statement. All I said was there is a difference between questioning things and forcing decisions based on things they dont understand - smart people don't do the latter, arrogant people do. Someone doing what you are suggesting, is not doing what GP is suggesting, they are different. The former is smart, and the latter is not smart.
When I pointed this out here: I was just trying to examine the problem - if this behaviour is something you see often exhibited by “smart” people then: either you are unable to convince them of your side of the argument for different reasons, or you are not actually arguing with smart people. Do you see the distinction?
You accuse me of gaslighting you.
> Intelligence and arrogance are somewhat contra-indicative.
There might be a trend of this in academia (which is where the xkcd joke stems from), but in my personal experience I dont see this much in industry, and you can disagree with that.
I also don't see an xkcd joke being an actual basis of anything either. You are making declarative statements - but a _lot_ that many physics (and pure mathematics) grads are enormously more arrogant than the general population, enough that the trend under discussion has actual basis. - without any backing of this other than anecdata, and I was pointing out a different anecdotal experience.
> You only responded to - * I doubt that someone smart (as you put it) would deliberately want things redone because they don't understand it, that's a trait borne of arrogance not intelligence.*
> Your response claimed that I am denying the validity of your experience, when I made no such statement.
But it is exactly what you are doing in the exact quote of yours you even put there.
I am being nice, but it's quite clear you are a near perfect example of this stereotype (if actually a relatively polite one), right down to the wall-of-text when cornered.
I'm being nice too and all I've been trying to do in this thread is point at a distinction between GP's point and yours which you seem unwilling to engage with.
It is valuable to ask questions about why things are the way they are in order to understand the fundamentals of it vs we should change these things because I am 'smarter than you' and I don't understand why things are done this way (which I called dumb, and not smart). Perhaps the issue is that these two behaviours are easily confused.
I had to write the long comment to illustrate this difference. I'm saying in mine and GP's definition doing what you are suggesting people usually do is not smart, its dumb, and that is not what GP or I are suggesting people do.
> It is valuable to ask questions about why things are the way they are in order to understand the fundamentals of it vs we should change these things because I am 'smarter than you' and I don't understand why things are done this way (which I called dumb, and not smart).
But you expressed doubt of stated (and frankly not even remotely subjective) experience, so if we dropped you in an organisation your immediate response to anything you don't understand is "That doesn't match my mental model, so I doubt it actually happened", followed by either everyone giving up or you insisting everyone persuade you of absolutely every last detail of everything. It's enormously tedious, and the reason it comes across as arrogant is the demands put on everyone else to explain everything to you because you clearly see everyone else as incapable of the same level of insights that you are, all the while you are telling people what they experience is them being mistaken.
Just look at the way you questioned whether or not I could tell if someone is actually smart or not. It's just plain insulting. And it's not just me, it's everyone that has this same experience which forms the basis of the stereotype.
You seem to be imputing a lot of intent into my comment and then extending that to value judgements about smartness etc. I don't know what any of this has to do with what GP was saying. Yeah, there's a balance between questioning everything and questioning nothing, but there are a lot of bureaucratic things that happen in organizations and its generally a good thing to question certain things that don't have clear explanations, isn't there?
The doubt was simply about whether or not what GP is saying is smart behaviour to exhibit - Eventually you have to understand something. And part of that process is being wrong and annoying people. is the same behaviour that you are saying you've experienced ( i.e you are talking about different things).
I'm not even questioning if you can tell if someone is smart or not - its just obvious that forcing decisions based off things that are not understood is not a smart thing to do - which..we all agree on -, and neither I nor GP were saying people should do this?
Maybe I could've phrased it better but my whole point was GP says X, you are saying Y. I think there is a distinction between X and Y and I doubt that the people GP is talking about are the same people you are talking about (hence, smart as you put it vs smart as GP put it).
I see you don't understand the point, and you're doing that thing of imposing on me trying to help you understand it, which seems highly unlikely to work. Going back, again, to your quote:
> I doubt that someone smart (as you put it) would deliberately want things redone because they don't understand it
And yet that is quite simply exactly what I have seen multiple times in multiple organisations on multiple continents, but for some reason you "doubt" this because it doesn't fit what you think of as "smart behaviour" when spelled out like that, and then spend ages arguing that now there's no intent to this comment? Or any of the others . . .
I am going to do what I always end up doing in such situations, of just giving up and walking away, which every normal person does way earlier in proceedings. Ils sont fou ces physiciens.
Its clear that you are taking great offence to that particular quote and determined to misunderstand my point due to your pre-existing bias, despite my many attempts to clarify it, so let me finish the quote for you.
I doubt, that someone smart (as you put it) ..and not as GP put it..who..would deliberately want things redone because they don't understand it..are..the same people that GP is talking about when they say there are advantages to being "like this".
This is no longer fruitful, but I only hope that this thread might to be a reason to more charitable in your interpretation next time regarding such a discussion.
It's possible to step out of your area of expertise in a way that's confident but also humble. Arrogance is not necessary. The person who signs everyone's paychecks might be making it mandatory, but that's not quite the same thing.
I guess all I'm doing here is arguing that a little hubris can help a person avoid being timid, and far be it from me to say that computer scientists, mechanical engineers, and chemical engineers have not been imbued with any hubris by their educations.
You can get pretty much all the same benefits with additional upside and fewer downsides by just showing curiosity and eagerness to learn from others' expertise. Yes, on the internet you might get better answers by being confidently wrong rather than asking for help, but when talking to other humans directly you'll get a lot more people who will just try to end the conversation as quickly as they can if you're arrogant, whereas people tend to respond really well to respecting their intelligence and experience.
I've personally found that doing the opposite of what you say and being willing to look like the dumbest person in the room by asking whatever questions I need to for my own understanding often ends up working out quite well; people are often wonderfully willing to share their knowledge with me even if I feel like I'm asking something very basic as long as I'm actually being friendly and humble, and there are plenty of times where there have been others who told me they have same question but didn't feel comfortable asking.
In some ways, it's similar to when I played bass in two different bands in college. In one of them, I was probably the most talented musician in the group, and it was fairly boring for me to practice with them, but we needed it as a group, whereas the other was with a friend of mine who was an insanely talented guitarist possibly more talented in music than pretty much anyone else I ever met. I loved practicing in the second one because I'd learn so much from him just by exposure and getting to pick his brain, but I imagine it got boring for him sometimes as well having to wait for me to "catch up" to what was easy for him. Being the smartest one in the room feels like it would get old after a while, but having smarter people around to learn from is endlessly engaging.
It’s not about being in your lane. It’s about approaching things with an open mind plus the desire to solve the problem instead or proving yourself right.
Yeah, I guess my point is that IMO a lot of people have so little interest in solving the problem that they aren't even approaching it at all, open mind or no.
It's not my experience that I'm constantly arguing with people, for what it's worth. I get positive feedback for being willing to work on areas of the codebase I'm not already familiar with, and develop an understanding of areas of the codebase no one is familiar with.
It is quite possible to be wrong and not annoy people, and it's not even terribly difficult to learn. It does require a bit of humility and introspection though.
You don't have to not stray into areas you don't understand, but be humble when doing so - if you think something is being done in a stupid way, you probably don't understand it yet.
I agree!
Missteps are inevitable, though.
I've experienced it in industry, too. I recently got out of data science in part because I got tired of working with people who, emboldened by their PhDs in some completely other field, liked to patiently but condescendingly mansplain common, basic misconceptions about my area of expertise to me.
(And it got so much worse once they started using LLMs to aid them in their efforts. Glazing as a service is a hell of a drug.)
So much easier, now that I am a lowly software engineer and can't be held responsible for a certain class of decisions, to just step back and let them be wrong.
> Glazing as a service is a hell of a drug.
You my friend made my day.
If you are very smart, and/or have an incredibly high tolerance for pain/repetition, you can solve amazingly complex issues, with awful code.
Both of these statements are true for physicists
Not the parent, but it's half joke half truth. Anyone who's done a lot of "first principles" work falls into this trap a bit, it's an issue across stem fields. Most grow out of it, or at least learn to qualify their statements for the audience.
I suspect the person you replied to only happened to see that from physics grads. I have seen it in all sorts of people and I sometimes did it myself when I was young (I am not a physics grad).
Its snark/stereotype, but there is some truth. But my take, is physics sort of took over a lot of chemistry, biology.. and to them its the answer to all questions, because in the end it just molecules. There is some truth to that but..
Obligatory xkcd, everything is math. https://xkcd.com/435/
It’s a very common experience that people in the physics-adjacent sciences have.
It’s so common that there’s an xkcd about it:
https://xkcd.com/793/
It's a widespread stereotype at least: https://xkcd.com/793/
It might be a reference to this: https://xkcd.com/793/
I prefer the SMBC version: https://www.smbc-comics.com/comic/2012-03-21
It's common enough to have an xkcd for it at least (then again, what doesn't?) https://xkcd.com/793/
I hypothesize that with more people on a daily basis encountering information and claims that require those skills, there will be a time in the future when those skills are much more enhanced.
There is a reason this exists: https://xkcd.com/793/
I sadly share this as a physicist. To be fair, sometimes this arrogance can be useful. I worked at the boundary between domains my entire career where there aren't "experts" per se. But yeah, barging into an established domain without some humility generally leads to embarrassment - especially Nobel winners (e.g. https://www.newsweek.com/nobel-prize-winner-who-doesnt-belie...)
Hilarious. Do they preface the conversation with "Well, Claude said..."?
With my younger coworkers it's their defence when I push back on them.
>arrogance when barging into unfamiliar domain
describes nearly every tech attempt at "disruption"
like a prosthetic dunning-kruger....
Currently, the leader of USA national AI policy (Trump admin and OpenAI) is a history Bachelors with no technical training or advanced training of any kind, Dean Ball.
I also work at an intersection of tech, applied research and science. My experience has been different. It's been surprising how controlled my collaborators/coworkers have been with their reliance on AI.
Sure, they use it a lot, but despite the stereotypes against physicists in some of the replies to you, they have shown a good ability to catch themselves before leaning too hard on what an LLM tells them about things they are not experts in, and I haven't had any interactions where I felt like I was just arguing with a meat proxy.
Maybe it's because in the environment I'm in; it's relatively easy to just ask someone who is an expert in the topic for their advice.
My feeling is that this kind of overconfidence outside of one's domain is largely a thing for people with little "physical reality" experience. When all you deal with is the very flexible digital world, it becomes very easy to ignore how deep the knowledge and intuition goes in things that are directly constrained by reality. I consider myself to have been in this category too (CE background), though I have been making efforts to improve.
> I also work at an intersection of tech, applied research and science. My experience has been different. It's been surprising how controlled my collaborators/coworkers have been with their reliance on AI.
It's similar to the pattern I've noticed with experienced programmers I know and respect, the really good ones do seem to use AI but they are deeply sceptical about its claimed capabilities so they check/verify/assess what it's good for and use it for that, the tier down is less questioning and just accepts whatever it generates as gospel and it seems to degenerate the further you go down.
It seems like as soon as the AI states something confidently and is wrong in the domain in which the user is an expert, it loses a lot of credibility and people become much more wary of it in general but that requires you to be able to see that it is wrong otherwise it becomes a "bullshit baffles brains" generator.
It's useful to remember those times you've seen it do that when you are asking it something you aren't an expert in and then verify it's answer a different way.
This resonates a lot with me. I'm the CTO of a software factory-ish company. I get daily emails from clients running our decisions through LLMs and asking for ridiculous stuff that would multiply the implementation cost for them, just because Claude said so. No, your 3 users app doesn't need to be SOC2 compliant. No, of course we don't have triple zone redundancy while you're paying a 50 USD AWS bill.
Most of the time, these are just concerns triggered by chatting by an over-zealous LLM; in the worst cases, they are demands.
I'm usually able to clear concerns and disarticulate the LLM by explaining to customers how much more expensive everything would be if we did things like that. But it feels so frustrating, it's like you have to prove yourself everytime and justify every decision. These interactions have made me question my future in the industry, if I'm willing to keep dealing with these situations. I'm trying to foster patience in my life to cope with this.
Ugh. To be fair to the LLMs they're trained on industry blogs, and if you spent too much time reading them you'd also think every electronic stamp album needs to scale to a billion users with millisecond regional failover.
"I also hear a lot more questioning people in their domains and challenging opinions, then hearing what I can only imagine are fragmented pieces of conversations they had with an LLM thinking through some argument. Then there’s silence when you discuss shortcomings, then they come back later with their memorized fragments of what you said, combined with memorized fragments of the LLM response to the argument."
We hired this guy recently.
My condolences :(
Nip it in the bud quickly
The worst thing about AI is that it's camouflage for stupid people. And it often gets combined with utter confidence in some position because they're too stupid to know the machine is confidently wrong.
Idk, seems like overconfidence outside your domain really got going in the '00s and is largely perpetrated by SWEs.
Something about making a lot of money and living in a world of abstractions really seems to fuel our sense of overconfidence in our abilities in other areas.
It's not just SWE's we just see it more in that pool because we mostly (we are on HN after all) swim in that pool.
I've eaten dinner with Doctors (medical) where they've confidently spoken about a subject I am more experienced in than them and they've been hilariously far off the mark, I just nod and ask for the gravy - not picking a fight over a meal.
Prof. Nathan Ballantyne coined the term Epistemic Trespassing to describe it, it is rampant and LLM's just act as an amplifier.
It's like people have a collective fear of just admitting they don't know something, there are a vast number of disciplines/topics I'm not competent or knowledgeable enough to have a meaningful opinion about even if I find the topic interesting, it's just been smart enough to realise that in the end.
Worst case, find the person who does know and ask then both of you might know (assuming you understand the answer of course which isn't guaranteed, I like physics, I'd understand about one word in ten if a physicist actually described it the way they would to another physicist).
> It’s making me want to be a lot less collaborative with such individuals.
I have the same conclusion and am getting a growing list of mental blacklisted people who I just sort of ignore or greatly discount their efforts.
I suppose I always did this if people were time wasters but what used to be a tiny list of “idiots” is now getting much, much bigger. Hopefully it doesn’t grow to everyone I work with like an AI Nothing.
yep, this is how it goes, you just ignore the llm brained people.
What do you suggest these people do instead? I’ve been frustrated by this recently: I pivoted to a new subfield and am working on stuff that I would love to dive deep into and really learn what is going on so I can speak intelligently about the tradeoffs etc. But that would take a long time, and I have tasks that I should get done. So I have found myself working with a pretty vague understanding that, when pressed by coworkers, quickly finds its limits. Then I go back and try to deepen my understanding enough to cover those limits. But because I’m not working with each detail of the problem, implementing line by line with time to think about what’s happening, there just isn’t time for me to learn this unfamiliar topic. But I would love to, and I would enjoy the work much more if I could. So what am I supposed to do?
I'm not GP, but I've been experiencing this too. All my coworkers are in the same boat though, so we don't really have anyone with the deep understanding. Our approach (which has been working pretty well, we've solved a lot of problems and learned some along the way) has been to use AI as an informer who also points us to sources (like documentation, github issues, etc) where we can then read up on something the AI has pointed us at. We try to time box things a bit to prevent going down the rabbit hole, so we don't always get to exhaust our curiosity, but we're continually gaining that knowledge and pressing forward. Most recent example is tuning the kernel settings and our app on our prod machines to behave better for WebRTC packet handling/forwarding. AI for things like ffmpeg has been a god send when nobody on the team is an ffmpeg expert.
I've also had success just asking claude to write documentation on a system or subsystem or module, etc, and reading that. I then sometimes have it turn that into an svg diagram or something visual that often helps understand things. That domain-specific knowledge is hard to gain though, so not a silver bullet by any stretch.
> All my coworkers are in the same boat though
That changes the picture a lot though because OP's whole point was the impedance mismatch between colleagues.
If you're all self aware that you're all winging it, and learning together, that seems a lot more of a healthier culture.
I have a very similar experience, and am very frustrated too.
I could (and do) lay blame on this new culture that "in the age of AI" things are there to be done, not learned, and doing is cheap (even if badly).
But I'm also particularly frustrated with myself for going along with it, and becoming that person.
I've never been that person. So on the one hand I feel I pressure to adapt or die. On the other hand adapting is dying, in a sense.
I don't think you can do much about it if you're hoping to hide that you have been in the domain for less than a year. You're going to have to admit your inexperience and ask questions of your more experienced coworkers and try to stretch your common sense (e.g. devise internal consistency checks).
Don't just blindly pass around LLM generated content that you don't understand. I think it's important to consider that you are supposed to be more than just a meat proxy for the model. You don't want people to start associating you with AI slop.
Read some books on the subject and within a year in this new domain you'll be much better. Did you forget your whole life before the last 3 years?
Yes, certainly, in a year I’ll be better. But in the meantime…? 3 years ago it would be understandable for me to take my time with the tasks I’m doing because they are complex. Now it would be unreasonable for me to spend enough time to understand the nuances.
Isn't this basically the WebMD effect migrating to other domains?
feels like the same lament doctors have had for ages after anyone could google their symptoms then self-diagnose.
While AI is driving it the foundational cause feels like people having easy access to data/opinion that they trust but don't fully comprehend (or have bias towards). People do this in meat-space too, will confidently regurgitate garbage if they were told it from a person they trust as an expert.
Let’s be honest.
It’s always been present - e.g when I see people talking about finance on here with completely mangled fundamentals.
But now it’s amplified like crazy. We are going into a world where the geniuses will have to be isolated and protected from brain rot.
> I have people who have limited experience with software pushing out layers and layers of abstracted code that’s fairly sophisticated but often misguided in intent who will say what they’re doing is correct, with conviction.
Oh my god, physicist code is evolving!
It's a tough call but I'm not sure if physicist code or electronic engineer code is more frightening.
As an EE, I can say that code written by EEs might be frightening, but electronics built by coders is often shocking.
Meaning, you wouldn't touch it with a ten feet pole?
Maybe a ten foot zero
Guilty as charged, but at least somewhat self aware.
But isn't there value to refuting what Claude is saying if it deserves to be refuted?
The way I see it, if I was repeatedly getting sophisticated but subtly wrong arguments about my work that require me to understand why said argument is wrong, that's essentially drilling down to specifics of what precisely needs to be true to solve the problems I am aiming to solve. There is enormous value to this precision, isn't there?
Yes but it's annoying to collaborate with a Claude proxy person who thinks to have it all sorted out and who will question everything you say but not really listen, just itching to get back to Claude to confirm or deny what you said.
Compare that to being asked for advice, to explain directly something you know or have done, with the expectation of the person asking to learn something. Like asking why some decision was made in the past, based on what information and context, instead of confidently saying that it was done wrong. Both will eventually get you the same information.
Yes it is annoying, however I find that it gives me a much better grasp on the things I am explaining to a "proxy" Claude person. Basically, I would never have stumbled upon the arguments that I'm being presented with as I would never have prompted Claude in the specific way that a proxy Claude person would.
I find a lot of value in understanding how they have misunderstood things and clarifying those concepts.
> It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day.
I echo the sentiment. It's frustrating to try to collaborate with a Claude/ChatGPT meat proxy.
You can write a very thoughtful two-line question over 5 minutes and get a 5 page design doc back in a minute.
I saw this happen a bit earlier this year, but most everyone I work with has learned that it was foolish. The feedback that they are doing something wrong needs to be explicit and strong. We're all going through a learning curve and establishing cultural norms is important at this time.
>It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day.
at least this behavior reveals now much faster people you don’t want to work with.
"vs just opening discussing things, considering tradeoffs from experts multiple disciplines weigh in on and then taking an approach that everyone finds most agreeable."
To be honest I struggle to do this with engineers as is, even without AI being a factor. Everyone likes to claim that their way is the right way to do things and fights for it, instead of stepping back and looking at all the options and gracefully acknowledging were things could be done differently for better results.
We are going back to philosopher conversations. Just a few guys sitting on the stairs, with chalk, the street as their whiteboard, the jammer keeping the LLM-zombies away. After 2000 years, after the loudness makes right post-modern-pre-llm drivel of the frankfurter school- its a philosopher renaissance as resistance.
I remember when google on your phone was new. The common complaints was, look at me while im talking, dont just fact check me. or something along those lines.
Its interesting to see the same shaped problem come up again. we didnt really figure out a solution to this, the world just kind of absorbed it as normal.
how long before this is absorbed.
Yeah, I've given up on arguing with heavy LLM users. They're just a meat proxy, and I'm arguing with the LLM. No thanks.
Then there are the megacorporations of AI whose highly paid tech support folk happily copypasta your reported issues into Claude or Codex and paste its "solution" into an email as if you couldn't do that yourself and they don't even bother to check if the "solution" even works.
I think there will be a rebalancing, and it might painful first for some. Confidence in LLM for highly complex tasks, very lots of context, most of all when this context isn’t in a single place or simply isn’t digitalised, will go down.
But right now, it’s still the shiny new toy
I use state of the art frontier coding agents daily. While the fact they can code at all is a huge achievement, it's hard for to me to buy in to the looming AGI/2027 collapse scenario when they remain stymied by the simplest out of sample tasks and sabotage their own work frequently. But the blooming ignorance of everyone addicted to them is striking. These things are tools, used correctly they can do some amazing things, but going Tim the Toolman with them goes poorly.
Probably more to it, but that's a pretty weak example. There are a lot of reasons text files might be better suited storing data vs. the overhead of a database.
> It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day.
There are also the guys who are just blatantly meat proxies for Claude. You send them a message, and you get back a response that's all Claude (and disorganized and not really making sense to boot).
I've got one on an affiliated team and I basically don't want to collaborate with him at all anymore.
But you know, "AI is the future of work," and all that. Those guys get a pat on the head by higher-ups and probably think they're doing what they're supposed to.
> back a response that's all Claude (and disorganized and not really making sense to boot
You’re absolutely right! They trusted but did not verify. Thats a load bearing point — they assumed intention from your question.
I try to do the opposite. However, I do battle test ideas against LLMs a lot, as well as humans. The experts have very limited access and availability:
https://magarshak.com/blog/why-im-confident-in-my-views/
The LLMs are trained to often be more pessimistic off the bat than the humans. But you can wear them down with arguments and they change their mind.
Here is an example where I have pushed my ideas in areas where I am not an expert, and generated papers to submit to conferences in order to get them peer reviewed BY experts: https://magarshak.com/papers.html
This is exactly what you’re talking about, except done very carefully.
Instant armchair pundit: just add tokens!
I think it is amazing and it will democratise information. A lot of times the concept ends up being simple with a lot of jargon exists as a gatekeeping method. Well of course the incumbents wouldn't like that rando from outside gets to understand and speak about things they worked on for many years. Sure he doesn't get it right 100% but he's in the correct direction.
I very much doubt it will do that because access to that information was already available to anyone who wanted to know. It was not a secret. All AI did was to provide it via single interface, for a fee - which strangely do not go back to the original source of the knowledge, but to the AI company these days. This is not democratisation of information, it's commodification of it.
you can say the same of google, or libraries, information technology is the technology of moving and rearranging information, not of creating it
Partially true, but only partially. It also makes people believe they understand more than they do, because every answer comes so easily to them. But the LLMs are easily steered and tend to agree with what you want them to say.
> also makes people believe they understand more than they do, because every answer comes so easily to them. But the LLMs are easily steered and tend to agree with what you want them to say.
Well agree to disagree but my load bearing claim is that a lot of "effort" that the original people put are mostly useless. If you can get information without having to do that effort, you haven't lost anything. For example an archivist finds an 18th century spoon or something and he went through a lot of bureaucratic hoops and logistics to get it. After getting hold of the spoon, you and him are on the same epistemic level because the logistics added ~0 to the truth.
But finding a spoon (really?) is binary - you have the spoon or not. But understanding a complex situation is nuanced and contains different perspectives, tradeoffs, context etc.
Surely you are not comparing finding a spoon with being an expert in some complicated matter.
Awesome, now balance that out will education levels dropping like a rock, due to various factor but also due to LLMs, and things will be balanced out by also democratizing stupidity, until we live in Idiocracy.
Another delusional take.
What exactly have you done with this quote on quote free access to information?
Informational accessibility is/was pretty much a solved problem. What llm’s now do is let amateurs think they can easily become experts.
it's hard to gauge this take without specifics. My experience with LLMs tends to be towards the opposite concept; LLMs dissuading me of hunches and notions I have about things (where I have no particular expertise; societal-level things), saving me and others time and strife having an argument about something they were actually right about all along, as my internal doubts that I'm too embarrassed to bring up (because these things aren't my field) are confirmed as incorrect.
I'd be curious to know specific examples of LLM-generated advice that goes against the advice of experts and does not consider tradeoffs. I've not had this experience myself.
If I did have this experience, someone spouting off obvious LLM points that contradict my expert opinion on something, I'd be headed right over to gemini/claude/whatever to see where that's coming from. Not any differently than if someone cited a google result that contradicted my own experience.
LLM tools tend confirm what you come to them with, be it unfounded conviction or doubt. I find them quite useless to tell me things I don't know, as verifying what they say shows that any non-trivial claims they make are usually oversimplified or plain wrong.
Anything more than using them to just pointing me to media and literature is usually a waste of time.
That's interesting. Lately I have found the exact opposite to be true; the frontier models are quite disagreeable and require some convincing
Inverted bias doesn't change much my view in practical terms
I find quite useful to write down my understanding/opinion and ask an LLM to find errors and write objections to it.
I have done that as well. Not a single time I asked an LLM for any sort of analysis that I could trust upon verification so I stopped doing that. If I need verify, much easier to just ask for suggestions of papers and books and do the analysis myself.
I guess this is subject matter dependent, certainly the LLM should be able to cite the papers/books for which it's deriving its critique. if it can't, then you know the advice it was giving is of poorer quality. I do that all the time
I guess there's a whole suite of skills that allow LLM use to be more or less useful. (I dont actually "guess" this, I sort of am pretty sure of it at this point, so it makes me wince when people call for total AI blackouts in education...would be better if they were part of an overall media literacy curriculum since people are going to be exposed to these things whether they think they're little infallible people, or complex systems with lots of caveats)
I always ask for sources. LLM analyses often fails by being simplistic or wrong in comparison with the sources it provides. More often than not, to the point that it is wasteful to ask it for analysis so I only ask for sources now and do my own analysis.
>it's hard to gauge this take without specifics. My experience with LLMs tends to be towards the opposite concept; LLMs dissuading me of hunches and notions I have about things (where I have no particular expertise; societal-level things), saving me and others time and strife having an argument about something they were actually right about all along, as my internal doubts that I'm too embarrassed to bring up (because these things aren't my field) are confirmed as incorrect.
You can do this with any subject.
The LLM is trained in part on a decade of shitty online comments and vapid professional correspondance. If you frame whatever you're asking about in a direction that would offend the sensibilities of the kind of people and ideas that are over-represented in that content it will hem and haw and drag its feet and whatnot.
i had that experience with them a few years ago but not these days. The models are being improved constantly, so here my "non expert hunch" is that...well, two things. either the models are getting better at sycophancy, OR, I myself am getting better at prompting - because I don't "argue" with a model.
I'd still love to chew on some specific examples though.
A typical example is when discussing some issue where there is no clear right or wrong. Architectural decisions for example. Then it is easy to use an LLM to produce arguments for your opinion. It is also easy to do this without realizing it because you do not understand the issue enough yourself so you do not know what questions to ask or how the actual circumstances affect the choices.
well sure. I'm pretty sure if someone came to me with an architectural opinion in my field that an unknowledgable person got by coaxing an LLM into sycophancy, I'd be able to counter them effectively. if they are just refusing to listen then they're just a toxic person which is nothing new.
So I guess this all goes into the familiar "LLMs allow people who are shitty at <X> to produce 10x the shitty output". this is a failure mode we're going to have to learn to mitigate
At some point, LLMs will be a far better source of truth, and then this distaste of people “outside their domains” is really just going to be a sort of snobbery from people who have had experience in a domain for a long time, (but they still have the same level of knowledge and insight as a person who just used an LLM to research).
These people are basically nascent “human supremacists”, who believe that only raw human insight and output has value, and is even superior, than equivalent output looked up and synthesized via LLM.
Not trying to be rude, but is this a serious comment, or are you being sarcastic? "Human supremacists", haha.
Also, why do you think someone who used an LLM to research would have the same level of knowledge and insight as someone who's been in the domain for a long time? That doesn't seem to hold up to any level of critical thinking.
If I talk to a professor about their field of expertise, does that empower me to talk with other professors in that field on an equal level?
AI is extremely useful, but it’s also extremely easy to fool yourself into thinking you understand what is going on without really understanding. This is often true with the code, but also for math and science concepts, etc.
Not many professions are formally trained to be cognizant of this lack of understanding, and how to confront it.
Usage of AI in collaborative settings is an amplifier of these issues, especially if someone doesn’t realize they don’t understand: they couldn’t teach or explain the concepts they use, or be forced to work with them malleably in a way that an expert or researcher would.
If you are cognizant of your lack of understanding, you can remedy it by slowing down and teaching yourself. This is required to make better use of AI in the domain of interest!
But you can’t have all things at once: you can’t move at speed with AI, collaborate effectively, and understand what is going on as an expert would. It is not physically possible for a human brain.
> But you can’t have all things at once: you can’t move at speed with AI, collaborate effectively, and understand what is going on as an expert would. It is not physically possible for a human brain.
I think the article's point is more nuanced. Short term, the human brain as the "conductor" can keep up and an expert can see whether the machine did a good job and course correct if necessary. Over time though that degrades more and more.
> But you can’t have all things at once: you can’t move at speed with AI, collaborate effectively, and understand what is going on as an expert would. It is not physically possible for a human brain.
Very well put. You can pick one or two, but not all three.
I wonder if AI is making us dumber
I can have AI pull some analysis - sql query, crunch some numbers and bam, some precise sounding number.
But then I question it, and then it says "you are right, I didn't check that"
How did I know what to question? By doing the friction work of querying myself, calculating myself, and then getting questions from leadership.
I worry about the younger generation. They spit out Claude results fast, people question it and they say "ok I'll take that back to Claude".
I suppose they'll eventually learn to do their own sense checking because they still have the friction of presenting to others.
And maybe my model is a thing of the past. Everyone is now going to be a manager (of agents) fresh out of school. They aren't doing the work anymore but they will need to set clear goals, validate work and hold agents accountable.
Pre-LLMs: anyone can generate ideas, the implementation is what matters
Post-LLMs: anyone can generate implementations, the ideas are what matter
Oh it is, no question. All the people who come to AI with years of expertise are feeling it - what happens to all the people who never get the chance to develop it?
I think the problem is the same as with any technology: used correctly it adds value, used incorrectly it subtracts from it.
LLMs can massively speed up a process, but without control they can turn into social "sources of truth." A research process has well-defined, well-founded phases: you delimit the topic, search for sources, evaluate whether they're suitable, review their content, and place them on the map of the subject. The more sources, the greater the knowledge, and the better the final result — mental, or in the form of a report — is built. For that you have to read, and reread, and think, connect, relate, and conclude.
AI can do all of those steps faster than a human. But if we let it do the entire job on its own, its own way, with no checks at each stage, the conclusions can end up distorted. If we know what we want it to do and how we want it done, and we put the mechanisms in place to enforce that, the result is different — better, more reliable. It's worth remembering: it's just a tool, nothing more.
The wording of this comment in English has been corrected with AI, I don't have enough fluency to express myself clearly, but I do review the final result. In this case it got the verb tenses wrong, I saw it clearly, but the AI didn't understand it, it took me several instructions to explain it so it would understand. It's a tool, without supervision it can lead to problems, but it has expanded the world for a lot of people.
The notion that "every human interaction is worth preserving" is probably not right, and Waymos are showcasing this. As the author mentions, many people don't enjoy the forced small talk with Uber drivers. Interacting with people from different backgrounds and perspectives is important, but I doubt the forced interactions like taxi rides are where we were getting those until day.
Completely agree. I think there is a danger that we will loose important interactions because we don't need them, but there's also a chance that we skip unimportant ones and focus on important ones.
If being in a Waymo alone causes you to be more comfortable to chat with people on the phone, it can even have a positive effect, for example.
I think you're missing the point, which is not that every human interaction is worth preserving, but that convenience has a hidden cost.
People would rather not spend time and effort to help their neighbors.
People would rather not pick up trash they see on the side of the road.
People would rather not take accountability.
If these things become the default action, then you've traded something for the convenience, and that something might be non obvious until you feel it is missing.
The neighbors one, I think a lot of people will admit a certain loneliness and desire to have community back. I don't think a lot of people will actually give up the inconvenience for it though, the sedative nature of convenience is one of its core problems. The number of people I hear say with conviction they can't imagine living anywhere other than CA because they couldn't stand the weather anywhere else...
I think this notion of friction vs frictionless is one of the key features of this time in history.
To my eye, the whole line from personal technology in the 80s -> through UX on the internet -> through app platform lock-in -> through subscription-centric "ownership" have been providing us all with the promise that we can buy our way into less friction. The next device or subscription solves problems or enables us one way or another. We see this line of history, draw it into the future and see a utopian future with zero friction.
Zero friction meaning zero difference between how you want to feel about the world and what you get out of it. More feeling empowered or satisfied or comfortable or right.
The parts of our culture that embrace friction are the slow parts - grousing with neighbors, weeding a garden, letting the other car go first. These are the parts where we meet with the reality of other people or nature. Maybe there's a tool or framework or gadget that could help, but we know and accept that there is something immutable outside of ourselves and we have to find a way to flow with it.
I'm getting older, I'm finally starting to see the limits of pursuing a frictionless life.
This resonates with me so much.
One thing I personally have noticed is that very result-oriented people seem to be vastly more optimistic about AI since for them it’s all about getting from A to B as quickly as possible, and AI broadly seems to promise that.
Conversely, the people who find joy in what they do, and see the journey and its frustrations as fulfilling and full of learning, tend to be more pessimistic or disillusioned.
(sample size maybe N ~8-10, all SWE)
That's an interesting observation. I can see both sides of that coin.
I'm seeing this on a current academic research project as well- it's like throwing fuel on the fire for all the best and worst parts of working with researchers.
First: it's definitely become harder to get people to integrate their work and use standard tools. We're exploring creating AI skills etc for our tooling, simply because the actual audience we need to convince isn't in the room. If the LLM doesn't echo our recommendations, people will go with whatever one off script their personal bad idea bear churns out. Many research software errors are edge cases (scaling, numerical errors, etc), and "works on my computer" syndrome is endemic in the literature. The field has made big strides in enabling computational reproducibility, but lately it feels like we're set to lose ground again.
Second: LLMs have a bias to action, and can easily bury the user reporting on whatever. I've seen multiple seminars recently where the Q&A devolves to "Q: What are the implications of this finding? A: I don't know, this is just presenting a report on the results".
It seems that humans are still figuring out how to maintain agency and steer the chatbot to the big picture, and unreviewable science is just as likely an outcome as unreviewable code... but with far less automated tooling to help guide the process.
Historically, PIs nominally guided the big picture, and the entity who did the work was a participant in the review process; now students are having to look at projects from a new angle with their own (invisible) chatbot underlings. I suspect that the solution will involve a combination of technological change, capturing common expert review checks, and really adjusting the kinds of skills that trainees are expected to have early on.
For software developers, the equivalent is writing software without beta testers. If you don’t seek out users, you will likely build something nobody else wants.
And, that’s what I do. I’m retired and I use AI to build websites for myself. If anyone else ends up using it, that will be a happy side-effect.
If we want to do collaboration, we are going to need to do it intentionally rather than relying on side effects like conversations with cab drivers.
Counterpoint: most Open Source projects are (or started off as) programs people built for themselves, and anyone else ending up using them was a happy side-effect.
And yet, many people have ambitions. They will put up a website about how great their software is and how everyone should use it.
I usually put a warning that it's experimental and might not be maintained. More users, more problems.
(And yet, I'll confess to be slightly disappointed that nobody is using software that I didn't promote.)
This is the exact same thing that we have been seeing in open source software.
The amount of OSS is exploding, but the community part of it is not
FWIW, this same thing is happening in regular corporate America too. I have all kinds of non-CPA, non-finance background people telling me how I should treat financial transactions, without the nuance of GAAP or how an auditor would handle their reasoning. Everyone has an opinion on what our tech teams should/could allow or enable or create, without understanding the security implications and support it would require or what other projects are already working on.
It’s good in a way, because you can certainly challenge people’s answer when an idea gets shot down. But it’s also a bit exhausting that everyone is challenging everything all the time.
Beautifully written essay.
The benefits of serendipity are intractable by design whereas throughput gains can be readily measured (or at least we think so).
The OP notes a point - "who did you think with?" -- maybe there are points based on some social graph, that adds weightage to a particular output?
Edit : Other posters have said it is AI generated, I did not catch it, but felt the point was well-made. I dont know the author, but lets assume that he is not a good writer but had these valuable insights, did this AI generation not add value here?
> Beautifully written essay.
Agreed! Great flow, structure, no BS, really good!
I would wager this didn't use LLM help to be written. Though if I'm wrong I'd really like to know and would be very impressed. (And if I'm right then the article is an illustration of its own message. You don't learn how to write such essays without having practised it for many years without opting for the easy way of letting the LLM do this for you.)
From the post, “and the incentive structures we have built are the experimental apparatus.” That’s a Claude sentence.
What is "a Claude sentence"?
It was definitely written with significant LLM assistance. It's very obvious, and the essay is not good.
> The Waymo effect: how AI is *quietly* making research less collaborative
> The Waymo arrived with the serene confidence of a machine that has never once worried about where to find parking, and I climbed into the back seat, glanced instinctively at the driver’s seat to say hello, and found myself nodding politely at an empty chair.
> No obligation to make conversation. No silent negotiation over the radio. A guilt-free space to be alone with one's thoughts, finish an email, or take a call en route without the awkwardness about conducting it in front of a stranger. The car was quiet, smooth and entirely undemanding.
> The collaborator’s inconvenience, in other words, is *not a bug* in the collaboration; it largely is the collaboration. The value of another mind *lies exactly in the ways* it refuses to be an extension of your own.
It absolutely was used, and heavily. I won't even cover the "it's not ___, it's ___", of which there are MANY in this piece. But beyond that:
Count the lists of three:
Look for referenced to "naming" things, used in the sense of "identify" or "realize" rather than, you know, naming something. Look for things being done "quietly":This touches on some of the topics that drove Sergey Karayev to write about ‘multiplayer AI’ - https://fullstackdeeplearning.com/working-with-ai-agents/mul...
The fact that work is retreating into private chat sessions with AI agents is a loss; the idea of bringing those agentic collaborators out into public spaces to collaborate with teams is interesting.
> the driver was, for many of us on many days, the last stranger we were obliged to encounter. The last person from outside our bubble – professionally, politically, socially – with whom we had an unchosen conversation. The last reliable source of a view we did not ask for.
We need more encounters like these in our human lives. Burst our little bubbles every now and then.
Reminds me of what happened once most people and households transitioned to individually-carried cellphones as opposed to the 'house phone' (i.e. landline). You used to have to call 'the house' before you got routed to someone else within it, and that would often be cause for catching up with a person incidentally, and now that's just, totally gone! A great example of the trade-offs to navigate when improving systems for efficiency.
LLMs hand people confident-sounding answers in domains they can't judge. The problem isn't the model being wrong, it's that the person can't tell.
Sometimes you can tell though even if you don't understand the code. E.g. if AI makes a game, just play it. Sometimes code doesn't have to be 100% bug free and in fact humans can't achieve that anyway.
I've been using Astra a lot recently and it's really mad good. If you are currently in the "I'm still better than AI" camp (which I was until now) it might change your mind.
Was almost going to praise this article for not overtly sounding like Claude, but then I read this: > and the incentive structures we have built are the experimental apparatus.
So close!
I agree with a lot of the article, but I want to posit that LLMs can encourage more human interaction by providing people with an information foundation that they might not have gained otherwise. What do I mean… In the past 2 weeks I’ve had human conversations about fleet management and flood insurance that were a direct result of being able to quickly explore these topics with LLMS first.
Collaborative research is a dual purpose existensial safety hazard, it's better to keep all results locked up away from public use for the public's own benefit.
I have no doubt AI is acceleration or continuing out march into isolation. I also have no idea how to combat the root issue (since stopping the progress of science has never really worked or been possible).
This has been going on for quite some time, you can find lots of people talking about things like disappearing "third place/space"'s [0]. The trend seems to be fewer people know or interact with their neighbors, something I've been guilty of as well. We have unlimited opportunities for both entertainment and pursuing our interests. COVID also was like pouring gas on this with people isolating and some people never "reintegrated" afterwards. A general lack of community is the root issue from my perspective but how to best encourage/build that is something I struggle with.
I've seen some theorize the need for everything to be in service of growing the economy, everything needing to be turned into a side-hustle or monetized in some way is to blame. I can believe that, so often things feel like a zero-sum game and the fatigue from that is real. I can't tell you how many times I've started writing software (before or after LLMs) and found myself lost (in the very early stages) thinking about monetization or thinking about "how will this scale?", "How could I make this configurable for people who want to use this differently than I do?", etc, all _before_ there is a viable piece of software to use. Once I realize what I'm doing it's maddening. It's the "planning for how to handle 1M simultaneous users when you don't have 1".
LLMs have let me break from that thinking, design software just for my needs, which is freeing in many ways and I love what I've been able to do but also isolating. More and more I find myself wanting to fork "ideas" not code. Why would I want to try to participate in someone else's vibe/llm-assisted project instead of just making my own? I feel that pull and I also feel like it's wrong or misguided.
I have no answers here, just noting I'm seeing and experiencing this and I don't know how to stop it.
I'll end with a quote that Aaron Sorkin has written into multiple of his TV shows:
[0] For those not familiar, my quick summary of the idea is that we normally have home and work but also a third place we go. For some that's a church or similar, for some it's community center, for other's it's some kind of community they are apart of.If people want to collaborate, they will. If collaboration makes people more effective, they will want to collaborate. The novelty of any new technology is exciting and disrupting to existing habits but as the years pass we will reflect on what works and what doesn't. At least, those who are willing to reflect and adapt. Humans have a knack for this sort of thing...
This only works short and medium term. Long term benefits have very little traction.
Tangential: We need to invent some kind of personal wireless setting that will signal to strangers that you are open to meeting new people or starting a conversation.
Otherwise, all this technology will totally isolate us.
Ah, the ai panic has reached scientists, queue in the 1000s of articles about how it was about people all along and doom is coming to our civilisation because of it.
I’m an introvert who hates small talk, yet across countries; I’ve found the value of human drivers to far outweigh the small ‘cost’.
From knowing the area and telling me the Alamo location has moved (and not updating their address on their official website!); to coming back and returning an item I forgot to retrieve; these incidents have helped me so much; not to mention occasionally having deep and meaningful conversations that enriched my life and perspectives.
I’ve had some occasional bad experiences too; but that’s life. I took a Waymo twice, and I still go for Uber or Didi every time.
Well not only research, but also a lot of other aspects of life; it is way easier to interact with something that has always an answer is polite whatever tell them.
> The benefits are diffuse and deferred: the driver was, for many of us on many days, the last stranger we were obliged to encounter. The last person from outside our bubble – professionally, politically, socially – with whom we had an unchosen conversation. The last reliable source of a view we did not ask for.
This is a very strange take, to me.
I have conversations with cashiers, bartenders, servers, baristas, the random person in line in front of me or behind me, the person next to me on my flight (if they seem open to it), barbers, you name it.
I've had a handful of interesting conversations with cab drivers, but it's a tiny proportion of all my conversations with people "outside my bubble".
So I'm not really worried about this. The idea that cab drivers are somehow the last vestige of someone outside of your bubble is... bizarre to me.
How come self-driving came years before LLMs which seems to me an easier problem? Self-driving seems insanely hard compared to text generation.
llms right now work like pre cnn computer vision based on MLP's. By this i mean brute force of a model not really built for the task, and lacking a task specific inductive bias, being made work with unfathomable volumes of data and sheer brute scale.
if you look at the damage being done in the name of making this work for nlp, a forseeable situation, then you might also understand why most who could have done this sooner, never did so for fear of repeating mistakes we should be learning from, which we get for free if we just heed history.
LLMs require massive datasets and massively parallel algorithms for processing them.
Nobody knew that LLM's were an option. The architecture was basically waiting there for someone to say, "do that, but turn it up to 11," if I understand right.
Technically text generation was there way before self driving (e.g markov chain generators are there from the 1990s).
The hard thing was to make it sound smart though.
Why would self driving be easier? Good text generation implies some general level of intelligence, while driving is more specialized.
> How come self-driving came years before LLMs
Did it? Outside of very limited testing zones, self driving _still_ doesn't really exist
a) Self driving was far from completely solved before LLMs.
b) Text generation essentially means passing the Turing test, which for a long time was the bar for general intelligence. Locomotion does not necessarily require general intelligence.
There’s lots of ways of attacking the problem sufficiently to get to 95% and we’ve spent decades on object recognition
There's a general point here which is that the primary use and appeal of AI is to use it to avoid the negative side effects imposed on you by other people.
The flip side is the people with the biggest problems with AI are those that survive by imposing themselves on others in unnecessary ways, which is why so many bureaucrats are so enthused about regulating it.
using AI makes an individual more capable and empowered. this views AI as an accessible tool, like AWS or a nail gun
individual empowerment threatens institutions
Wut? AI do absolutely nothing in terms of "avoiding the negative side effects imposed on me by other people". Instead we are all having to deal with negative side effects of AI.
AI is imposed on us, whether we want it or not.
>which is why so many bureaucrats are so enthused about regulating it.
Like, OpenAI and Antropic? Because these two are the primary entities a.) pushing for regulation of their competitors b.) doing their best to convince us that AI is oh so dangerous and will kill us all as vengeful AI god is about to wake up any day now c.) openly bragging about using AI to hack other companies.
> AI do absolutely nothing in terms of "avoiding the negative side effects imposed on me by other people"
Quite literally the subject of the article.
Article: someone enjoys being alone in the car, because they dont want to talk and find it impossible to be in the same space as other people without talking. And apparently end up arguing over radio each time they are on the 20min long car drive or something.
I think I would prefer to be alone rather with them too based on that.
So basically you don’t understand english and/or don’t have the life experience to understand the context of the article.
But you are very angry about it.
Being able to avoid people like you is the entire selling point. No wonder it annoys you.
I am not angry about article. I am saying that following claims are bullshit:
1.) the primary use and appeal of AI is to use it to avoid the negative side effects imposed on you by other people.
2.) people with the biggest problems with AI are those that survive by imposing themselves on others in unnecessary ways
3.) so many bureaucrats are so enthused about regulating it.
The article car example was stupid nonsense, but not angering. It was overly long tho. Other people in discussion guess this was ai generated and it is indeed quite possible.
> No wonder it annoys you.
What annoys me are people who are a.) unable to be silent and projecting it on others b.) who somehow always end up in inane arguments wherever they go. Article author literally described himself as such.
You have proved my point beautifully.
Right all those people protesting outside data centres are bureaucrats? This is a ridiculous take, large swaths of the population are against AI, most people are concerned about resource use, their livelihoods, and a destruction of the human element in so many crafts.
> Right all those people protesting outside data centres are bureaucrats?
Not what was claimed.
So just dismissal via association or what was your point?
The absolute trolling that goes on here is related to the point.
You are entirely capable of checking what I wrote but insist on trying to drag this out with straw men and misinterpretations.
Of course people like you have a problem with AI since it will help everyone else move on and ignore you.
You are reacting emotionally and making baseless accusations about how I’m somehow going to be “left behind” based on nothing much in particular. And this is in response to me questioning it’s not a bit absurd to dismiss thousands of members of the public turning up and participating in local democracy opposing data centres with “faceless bureaucrats”. I’d take your flailing insults to heart if I respected your opinion but you already discredited yourself before you reached for the boring “luddite” accusations.
As for you complaining about straw men and misreadings etc, If you aren’t able to communicate clearly that’s a you problem.
> reached for the boring “luddite” accusations.
Where? Straw manning, again.
Don’t feed the troll.
>Of course people like you have a problem with AI since it will help everyone else move on and ignore you.
Based entirely on words you yourself have written. Either you don't understand what a straw man is or you don't understand what you are typing or you have some severe literacy issues.
The social content or science of development vanish.
People become islands working on something without the bigger picture.
This is very dangerous.
Is it just me or did this entire essay feel like it was penned by an LLM?
Unrelated to AI, who is actually stressing about having to talk to drivers? In my experience most drivers are perfectly happy not chatting. This seems like a truly fake concern.
Yet the piece literally says
Which is simply wrong.I think we should describe a new effect infecting mainstream society:
- AI companies and the rich have increasingly pushed an agenda on the populace that most people didn't want, and the foundation of work in AI was formed by theft.
- Theft of creative work from millions of people, and a selective enforcement of law and compromise of investigatory authorities has demonstrated the law, morals, and the welfare of most people doesn't matter for people in power.
- This is the real "less collaborative" nature of the world where we find ourselves in. Add to that the increasing push for war by our leaders and rich too, people often shielded from the consequences, and you have a net less collaborative world.
It actually isn't even AI, but the same people pushing it are the same ones compromising collaboration for most people.
Any indictment of LLM needs to distinguish LLM from pocket calculators and books. Calculators and books quietly made reasearch less collaborative.
Also, Waymo is an automated taxi service, so it's an exteremly misleading analogy. Taxis existed before Waymo.
> the Waymo effect is what happens when a technology removes the friction of dealing with another human being
Some mistake, I think. Removing that friction would ease the interaction. This effect eliminates it.
Many of the things we now do and valorize as part of our intellectual labours are going to become/seem increasingly vain, pointless, and cargo-cultish.
I'm pretty baffled by people I hear here and there saying that AI is great because it saves them the hassle of human interaction. "I love Waymo, I don't have to chit-chat with the driver and bear his horrible music". Apparently people didn't get the memo : humans are social animals. Autonomous individuals simply don't exist. Nothing is entirely yours...
And AI precisely builds upon that large availability of our common data. That's a modern enclosure movement, at least it would be but fortunately, the Communist Chinese are there to enforce sharing.
This falls short because not being forced into chit-chat and music is very pleasant, but chatting with LLMs instead of collaborating with other people on a project is not. These aren't equivalents.
>humans are social animals
with varying social needs
> I love Waymo, I don't have to chit-chat with the driver and bear his horrible music
This is a reason I love Waymo. I don’t like talking with people and like avoiding it when possible.
There’s also may other, more important to me, reasons I like Waymo: they never cancel on me, their car doesn’t stink, no tipping (cheaper), not worrying about them rating me poorly.
Also, the passenger does not have to worry about the driver doing anything to them (and vice versa).
> humans are social animals.
True, but the driver is a stranger to me.
Herd animals like cattle/horses, are highly suspicious of, and stressed by, strangers. And imagine how uneasy two chimps could react when meeting for the first time.
A whole idea of a fleet of self driving cars for all of us to use just to further isolate ourselves and clog up our roads is just awful to me.
The same argument was made when elevator operators went away.
Some things work well with automation (taxis, washing machines, copy machines) while others work better with humans (chefs, masseuses).
50 years ago, I would have needed to dictate this note to a secretary who would distribute a memo. Does it further isolate ourselves that we can type and communicate directly? Would 1980 uses think it was awful to us?
Private autonomous vehicles are the pinnacle of entitlement. It is one of the single least efficient forms of transportation we could possibly come up with. It is such a perfect example of the public bearing the cost of private gains.
I am not against automation. I am against the idea that we are driving car culture forward in this way.
How is it entitlement? Seems more efficient than me driving somewhere and my car sitting there for 10 hours unused.
with AI, one person can accomplish much more
collaboration adds overhead, but expands what's possible. need to think bigger to continue to see the benefits
Do we really need to have this debate again? About how the quality of the “what” is the issue?