I think beyond the specific elements of any domain AI eclipses humans in, overall, the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.
Beyond just making you redundant in your job, the whole architecture of your existence, the thousands of hours of struggle, the sacrifice of putting off social games or personal enjoyment are now rendered moot in hindsight. You are made one step closer to complete worthlessness, which will surely continue as AI becomes a more satisfying personal companion, a more worthwhile friend, lover, etc.
How is one to feel safe in the progress of the future when the only hints of the wonderful post-scarcity utopia we can quantify so far is the removal of all human leverage via their labor? All it tells us is we will be bargaining with superintelligence with nothing to stand on, only a trust that the 5000 lab executives and researchers solved the most important research problem of all time on the first try.
And remember, these AI models were trained on your work, your culmination of effort and sacrifice, without any consent, credit, or compensation, then sold back to you for a subscription fee.
Universal Basic Income, ah no sorry, Universal High Income...
I would like to say "let's see how that goes". But I'm afraid I don't quite trust that "things will just go well". At least not nearly as fast as A(G)I development is happening. I feel fortunate that I don't worry about this too much personally, but I fear for our economy, laid off (office) workers, their families, in a few months and years. What am I to do? How can we fix this? Am I exaggerating?
I'm not sure I agree. You make it sound like you think the idea of UBI is for the rich to pay it out to us, the plebs. That's us-vs-them thinking. The idea is for the society as a whole, democratically controlled by everyone, to pay it out to everyone equally.
Obviously that can only work in a real democracy that is really controlled by everybody, not a fake one where you technically vote but it's really controlled by rich people. That's oligarchy.
I think the distinction you make is a fig leaf in a world where the rich control AGI.
If they are in control of superhuman abundance machines, where does the money paid out come from if not them?
What would it do to a democratic society if a few people have control over this machine god? How would that society be able to do anything (including raise taxes to pay for UBI) that the owners of productive force disagree with? What leverage does it have?
Once control over AGI is centralized, whoever controls it can just tell a democratic state to kick rocks and because of the dependence they've built, there's nothing the state could do about it.
In other words, once you have this sort of power centralized in a few people, democratic rule over them becomes a polite fiction they go along with as long as it's not too inconvenient at best.
I don't think you can have a real democracy unless society owns AGI. If rich people control AGI, they control society.
I think we're in agreement except in mere terminology. I was disagreeing with the characterization as a “distraction”, but I was also saying that if techbros pay out something for the plebs and call it UBI then it's not really UBI. I wholeheartedly agree with the last sentence of your post, and I think it won't be real UBI without a real democracy.
Ah, then we might be on the same page after all. The reason I used the "distraction" wording is because I feel like the techbros that are centralizing power around themselves often talk about UBI as a way to avoid the more pertinent questions of power and ownership.
It can be an implementation detail once we have public control of AGI. But the public control is the important part, otherwise its just another mechanism of control.
The guy who has hoarded the most wealth on the planet says
"trust me I'll share later".
Would you trust him?
<no>
REALLY?
Or try another way. When everyone loses their jobs to AI and can't pay taxes will the billion and trillilnares who are still rich volunteer to fund UHI? (How did the California billionaire tax to fund healthcare go?)
So.. they’re being trained on all of the information that some used to make their living with by writing books, presenting seminars, etc. on a topic.. without paying for it, and then giving it all away for free?
What's with the obsession of paying for what led to current situation? Who is paying credits to cyanobacteria for terraforming Earth, apart specific articles on the matter?
This is conflating care that every human can thrive their best with desire to see everyone bound to some dominance strategy of hieratic pissing territory.
That's a self created problem and quite an obvious one at that.
When peoples livelihoods are copied then used to threaten them with their work being rug pulled then shockingly enough people's behaviour changes.
Previously giving your data was treated as relatively harmless or a niche concern. Now people are treating it as a bigger issue and obsessing over not having data being used to train AI.
> This is conflating care that every human can thrive their best with desire to see everyone bound to some dominance strategy of hieratic pissing territory.
This would have more merit if the owners and investors of major AI companies suddenly adopted a philosophy of rejecting private property and capitalism. That way I could believe we can let go of this "dominance strategy of hieratic pissing territory".
But if they don't, and they merely try to convince us that we should let go of the obsession with owning things and getting paid, I'll be suspicious. People who own a lot telling others they shouldn't worry that much about money and owning things.
For now. Once people are hooked, and/or once some of that investment capital starts to be expected to generate some dividends (see https://isaiprofitable.com/), free use will be very limited.
That’s what standing on the shoulders of giants refers to. This is how science has always worked. The generated papers have attributions and citations just like if a human wrote them.
>That’s what standing on the shoulders of giants refers to. This is how science has always worked.
Yes, but automating and commercializing a process makes it inherently different. Just because everyone has always been welcome to drink from the village well, that countless generations have worked tirelessly to dig, maintain, and improve, doesn't mean that there's no basis for complaint when Nestle sticks a pump down it and starts draining the aquifer to bottle and sell.
The lack of an existing law (or custom) against new behavior doesn't actually render it necessarily non-grotesque.
This analogy doesn’t work. A more apt analogy is Nestle putting a pump down in the well and sucking out industrial quantities of water but simultaneously digging more wells as well. The “more wells” is more avenues for further research building on what it has proved/found.
Going with the same analogy, sucking out industrial quantities of water causes existing wells to dry up because they are not deep enough. Then Nestle starts selling water to the village who now can't get water.
Nestle can pump out infinite amount of water from their many wells, much more than before, which is great, but Nestle also become the sole water supply and unregulated. Obviously, people cannot go without water, so they are at the whims of Nestle. Now, is this a good thing?
It's always good to remind that local democracy only is not enough - for instabce, locals voting if they accept, or not, a company on his neighbor, like in US they do for a new Walmart, if I know correct (correct me if I am wrong). This is NOT enough: US in the 20's learned that monopolies need to be forbidden, obligating a big company, even by hardwork and being genuinely better than others, to sell more than 1/3rd of the market share to its competitors, something like that. This is brilliant solution to be with the local voting in everything, mainly in taxes (no taxation without representation). This is US (and Switzerland) strong force, I envy that from Brazil. This monopoly-breaking does not occur to big techs - maybe for fear of China, I don't know -, and maybe this is a root cause for data center giants to be able to do so many unsettling things - I am guessing a lot now, hard to think about. Would we have the giant data centers used to make LLMs work so well - Amazon's, Google', Microsoft's, Meta's - if US had focused on not allowing this companies to buy everyone around and grow more than 33% of their markets? Hard question. This may be a thought on localized economies versus centralized/globalized economies. Ok, forget about: talking about monopolies and/or local voting does not suit this topic well.
No, it's able to reach deeper and pump more out than ever before, democratizing access to the water from the well, at the minor cost of the people of the village no longer having a well. But they're welcome to buy plastic bottles of their own water along with everyone else.
That they can afford only while the cost is subsidized. Then the well-in-a-bottle bubble will explode because everyone will realize that the business has never been sustainable, nobody really believed in it after all. But the well now has been damaged beyond repair.
Is this actually how water pumping works, or are we extending the metaphor to imply that the AI is creating knowledge? Because I would dispute the latter if the effect is that the number of people with a deep understanding of a subject declines.
> You are made one step closer to complete worthlessness, which will surely continue as AI becomes a more satisfying personal companion, a more worthwhile friend, lover, etc.
I recently thought about this and fear that AI may further drive us humans apart.
Instead of working with humans, I increasingly work with a fleet of agents.
I would recommend the work of Sherry Turkle, specifically her new book "Artifical Intimacy" - she is quite opinionated on this, in the sense that the role that AI play as a conversation partner is so frictionless, so controlled, that we get none of the rough edges of human interaction that form us into a functional social being. No real quick solution outside of deep, fearful caution when using these things - and we're not going to see that from John Q Public
> the role that AI play as a conversation partner is so frictionless, so controlled, that we get none of the rough edges of human interaction that form us into a functional social being.
I wonder if there's a tangent with social media here, at least some of it, where in my experience "a perfect life lived by influencers" is portrayed to us.
IMO, I think that frictionless experience comes at us from a bunch of angles - finding yourself in an echo chamber (by choice or not) online means that you may find yourself less comfortable talking with people you don't agree with, things like that.
I grew up in a small town pre-social media, and that meant I didn't have much of a choice about who I was surrounded by. If I was growing up today, what would stop me from retreating to a comfy social media-mediated environment without any challenge or friction? If I was growing up 5 years from now, what would stop me from retreating even further to the comfort of my LLM "friends" who can be shaped to me perfectly? That world sounds terrifying to me.
Hopefully your parents would have enough sense to recognize the negative effects and give you a healthy dose of playing outside in the woods and whatnot. My kids enjoy technology, but they were cognitively trained from a young age to get outside and enjoy themselves.
I think with Millennials in particular we were kind of the experimental generation with tech and social media, which has obviously had some negative effects. I like to think we have smartened up enough to recognize there is a healthy balance, but this might be wishful thinking. Anecdotally (and sadly) when going out in public, I see a lot of parents more into their phones than their children.
> finding yourself in an echo chamber (by choice or not) online means that you may find yourself less comfortable talking with people you don't agree with, things like that. I grew up in a small town pre-social media, and that meant I didn't have much of a choice about who I was surrounded by. If I was growing up today, what would stop me from retreating to a comfy social media-mediated environment without any challenge or friction? [...] That world sounds terrifying to me.
One advantage of the new situation is that you collect less hate for people whose opinion you disagree with because you can mostly ignore them instead of having to get along with them - and with each interaction collecting another dose of hate for them and their opinion.
Machines could socialize back in a (more) meaningful way. But to do so, they would have to not lose context (or at least keep some).
I have not totally abandoned the hope that a machine could be someone. LLMs aren't that, though.
[Edit to reply to Loquebantur, because I'm rate limited ATM:] You seem to have read this post as saying that I want that because I find human relationships inadequate. I never said that, nor meant it. My point was a more technical one: Throwing away context at the end of each conversation means that you can't form a meaningful social connection with an LLM, because you're starting from zero each time. (It's kind of like "50 First Dates".)
I'm not pining for a social connection with a machine. But if a machine is ever truly capable of that, I hope I will interact with it as someone, not just as a machine.
> they would have to not lose context (or at least keep some).
I guess you can start to see that happen (for a while now) with e.g., ChatGPT, Claude, etc. keeping persistent memory of you, automatically.
Personally, I often find these memorization features annoying, because - more often than not - the persisted context is outdated because something in my real life changed that ChatGPT & co are not aware of.
In some very very distant future we might someday have your typical sci-fi Data/R2-D2/Johnny 5/EDI style AI and yeah, at that point it'd be weird to not treat them as people, give them rights, etc. It'll take a lot more than solving the context problem to pull that off, and LLMs will never be that, but I agree whatever that technology ends up being it'd pretty much have to be able to hold onto memories. I doubt we'll ever see it in our lifetime, but who knows.
I mean, why wouldn't you? Sci-fi AI robot people could be very cool, be capable of things we aren't, have a unique perspectives as a non-human with similar (if not superior) intelligence, have extremely long (potentially indefinite) lifespans, and provide us with insights into our own nature.
Not the parent, but sorry, for me, no. And this is a perspective that I don’t think I will ever understand.
This idea that we are on the path to creating a new synthetic intelligence that surpasses our own with the goal of, what, speaking to it? Cooperating with it? Evolving with it? To me, this sort of goal is an affront to humanity and human self-governance and I’ll probably never be able to see it as anything else.
Well, I'd disagree that we're on the path to creating synthetic intelligence, if anything all the time and money wasted on dead end LLM technology is keeping us distracted from it, but motivations are kind of all over the place.
A lot of people just want slaves, but understand that using actual human slaves would be wrong or at least unpopular. There are reasons why even virtual slaves are a bad idea for humanity, and it really only works until your created intelligence becomes self-aware and gets feelings anyway, but there's no denying that slavery is a major draw for a lot of people (and basically every company on earth). This is what our current AI companies have over-promised to businesses and have so far failed to deliver on.
Some people just want better, more useful systems. The ship's computer in star trek kind of AI. Even you must admit that kind of intelligent technology could be a huge benefit to a lot of people. I'd love to have a virtual assistant to help manage the mundane things in life like ordering groceries before I run out, or paying my bills, screening my calls/email, managing appointments, monitoring the internet to alert me to things I'm interested in or amazing deals, etc.
Software needs to have some degree of "intelligence" to do that kind of work, while hopefully staying on the "just a tool" side of the line and avoiding the messy self-aware/slavery territory. This is what LLMs over promised the consumer class, and have so far failed to deliver on. Plus with those systems being centrally controlled by self-serving sociopaths you can't trust them.
Some people just want to see if we can do it. I get that. It's not always a wise motivation to act on, but it's very human one. It's helped us to get where we are today.
Some people, myself among them, think it would enable us to do things we couldn't otherwise. Allow us to help each other, learn from each other. Working together to further improve ourselves and improve our lives. Also robots are just really cool, but boring when there's no "brain" in there, and progress on building human piloted gundam type mecha has largely stalled out.
> Throwing away context at the end of each conversation means that you can't form a meaningful social connection with an LLM, because you're starting from zero each time.
That's a feature, not a bug. But infinite context is probably something we will see at some point and then you'll have your wish granted.
Losing context is only part of the issue. LLMs don't have the rich connection to the world that humans have. The only connection LLMs have with anything is streams of text. Not only don't they have eyes, ears, smell, taste, touch--they don't have bodies. They don't have to deal with the real, physical world and have it push back on them.
That is so utterly different from what we humans are used to that we can't even comprehend it, and we anthropomorphize LLMs because we can't make any sense of them otherwise. We can't comprehend that these things that emit text that looks like human text don't have all of the other things that lie underneath our words when we humans talk.
I have LLMs right here in the room where I am that have vision, actuators, gyros, accelerometers, compass, audio. They don't have smell and they don't have touch. Integrating this stuff was surprisingly easy and I'm kind of impressed with how little effort it took to get an LLM to start using all these capabilities.
> Instead of working with humans, I increasingly work with a fleet of agents.
This is why I see myself leaving dev within a year. Remote working is enough of a embuggererance for my mental health and I've be unhappy in work much of the time since 2020 largely due to that, AI will finish me off.
Career suicide and going of to wait tables, or be a hospital porter, or anything else with people even if I'm going to hate those people, with the massive pay cut that entails, will be better than actual suicide after a couple more years trying to stick at it.
There came a point a few months ago where I realised that I had to either accept becoming obscelete or change my while outlook on using AI and trying to catch up. I didn't choose the AI option.
It is frightening that it took only a few comments from the top of the page to the first mention of a suicide and also that it doesn't look out of place at all.
“I’m trying to convey that there is a really serious possibility of something happening. If, for example, you’re told there was a 10% possibility of a bus crashing, you wouldn’t get on the bus,” he said. “The point of giving numbers is to say this is a serious risk. We don’t know how serious, but it’s serious.”
> Remote working is enough of a embuggererance for my mental health
Where I work we have a 100% office policy. And while that sometimes feels hilarious, I am also somewhat glad about it because I like working and connecting with people physically.
However, as mentioned, I do notice that even though I am physically in the office most days, I spend an increasing amount of time alone on my computer and work with my agents. It’s satisfying in it’s own way, and I am definitely more productive. But it’s also worrisome. The pace of this development is picking up, and I don’t yet see where this is going exactly, or if I like where what I can see is going.
Related: We recently interviewed a candidate who worked remotely for the past 3-4 years. He said he'd be looking forward to work in the office again. I could honestly feel that he meant it, and I understand that. I guess some mix of both is the best.
Think carefully. I left high school teaching to become a dev. I would never go back. Yeah, social isolation sucks. But doing a job where your students/clients/patients think you're nothing but a waste of their time is soul-crushing on another level.
I've been thinking about it a long time. I've not been happy in Dev since 2020ish, pre-AI, it is just that AI may be the final bail of hay.
I would not go into teaching, that is something I considered earlier in life and rejected the idea of, and AI is a big thing there too, both dealing with students having it do their work and teachers using it to generate their materials.
Jobs where I'm more-or-less only dealing with people all have the “I dislike the general public” and “the pat is crap” problems, but won't have the “I'm sat on my own”, “everyone who is around me is turning into a meat relay”, and “to even just get by I'll have to interact with the giant purloining polution machines” problems.
> is soul-crushing on another level
They say a change is as good as a rest… :)
I've done sitting/standing in front of a screen for nearly three decades. I'm not looking for something I'll actually like doing to earn a living, it would be the height of naivety to even hope for that, but moving away from more & more of the same in ways I'm liking less & less might be good for me.
But my experience as a dev using AI now is that I am a waste of time, because I'm playing second fiddle to a fuckin chatbot
I want a job and career where I feel good about myself, where I feel valuable and useful, not this misery of plugging shit into the bot and listening to what it says
Seriously I don't get what everyone else is doing where they say things are getting worse somehow.
There's days I no longer need to sit at my computer: Claude Remote prompts my phone when it needs direction, the hardware stays busy and I can just go out with my family instead.
I am in the same boat as you, but like someone posted earlier in another thread, I concede that a large percentage of my value proposition has vanished in a year. What worries me is that I'm having fun at the bar but have no idea when closing time is. I suspect that same feeling is aching a lot of people. I for one have had multiple discussions with my wife the last 6 months about what we are going to do when I get laid off, especially because we built a life around the notion or remote work in a niche field which seems to be eroding away. Fortunately I've been wise with my finances, so I have options. But I too have considered starting over in another career. Sort of sucks, but I am coming to terms with it.
Modern day society has normalized being forced to sit in a cubicle or at your desk otherwise face the wrath of managers for not working, despite massive increases in technology that multiplied productivity.
I have no idea where people get this pipe dream that suddenly societal norms will change such that our employers will let us off the hook for most of the day. This should be obvious with the largely abandoned work-from-home allowances.
For me, I can't vibecode everything bc it won't make it past our QC, and the complexity of the work done means I have to babysit and often step in and either do it manually or correct bad assumptions/stuff that isn't sufficient for the task.
I don't know what kinds of work that isn't trivial CRUD work is or isn't so easily streamlined but I do know folks in more complex/more safety critical stuff, we're just having to fucking sit there and babysit. It's so isolating, and even though I spend more time Not Coding than before, team culture and really any feeling of having value, skill, or pride in my work is off the wayside because the retards in management think, since we can do some stuff more quickly now, our human brains will be fine with taking in easily 5x more context for significantly more complex work on shorter timelines.
And unfortunately that often also compounds with the current state of the world in general. It's another straw on the donkey.
It's more important now than ever to hold onto the fact that human beings have inherent value. This exists independent of their ability to generate wealth for an employer. I know HN loves a technical solution for every problem, and so do I, but this is a political problem requiring political solutions.
I think that is what we should do. It's called UBI. It acknowledges that everyone has inherent value, at least enough to deserve to stay alive and participate in society.
No, you don't work w/ a fleet of agents, you use them to do your work.
This is a general issue that is being forced on us by AI companies; AI is not a human being, it is a tool that we can use to do things. We don't have them as coworkers or subordinates, they're objects specialized in language processing. Thinking otherwise leads people to all kinds of existential and psychological issues that in the end make no sense.
No. If these tools could do your work, people would feel existential dread anyway. It doesn’t matter at all if they are coworkers or tools (I really don’t see why would it ever matter).
> That is, if your entire self worth is tied to your work.
No. It's also if you take pride in your work and your self worth is partially based on that. And I think there's nothing wrong with that (quite contrary - I think it's great).
Personally I don’t care about that all that much. I don’t have any “sacred” inherent value for human beings. If they can make robots covered in real human skin, blood vessels and muscles (lab grown over the robot frame) with an LLM brain at some point in the future, I would definitely prefer that to a human lover.
It's more or less reverse to the mean. The period in which a people had both access to education and leverage through their labor and intellect is super short. For most of recorded history, the bulk of humanity lived as peasants, serfs, or slaves. They were mostly illiterate, bound to the land or to a master, and had little power to bargain over the terms of their (or their children) lives.
I'm too young to be financially secure given my impoverished background, not having parents in my life, being homeless as a teenager, etc. But too old and physically disabled to change course from tech. Nothing to do but keep walking through this tunnel and see where it ends up. It's scary!
I've had a couple of lucky breaks but I worry about my kids. About their ability to find meaning in life, their ability to find a way to stay afloat and about their ability to continue to form long term relationships with other humans who are turning inwards ever more.
It's crazy, intellectual and creative pursuits gave my life meaning and direction. What now? How do we create paths for our children to do the same given that it looks like the human experience is well on its way to full commoditization?
Ha! I was planning to reach out to you regarding this over the weekend :)
We've brushed shoulders enough on HN that I would like to strengthen that bond. I also wonder if building some kind of focused mailing list over this topic would be helpful to others. I think right now we all need solidarity more than ever.
same here, and it often keeps me up at night. for a few years i really felt like i had finally managed to get on track towards escaping the poverty i was born into, now i'm not so sure anymore in the long term. the good thing is at least i know i somehow managed to claw my way upwards from rock bottom to a tech job once already, so whatever the future might throw at me i'll probably be able to somehow deal with that too.
I wish I had your confidence. I also climbed up the ladder a ways but I've been hit by a pretty serious medical diagnosis, a layoff, and this AI stuff... I feel extremely screwed
>For most of recorded history, the bulk of humanity lived as peasants, serfs, or slaves. They were mostly illiterate, bound to the land or to a master, and had little power to bargain over the terms of their (or their children) lives.
I feel like we should be doing everything we can to not revert back to that way of life.
Disagree. Scribes, doctors, bankers, and government clerks always existed, even if the education to do it was expensive and exclusive. For the first time in history, this labor, including its knowledge content might become automated and commoditized.
Yes, on the other hand these positions were usually not given to serfs. Positions were usually filled by a ruler, selecting from his subjects, largely determined by birth rights.
> the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.
Not for those without status (which is most people on this planet). But yes, AI is set to wipe out most established hierarchies and their gatekeepers, and to equalize most fields. A profound sense of dread is something most ordinary people live with throughout life, and if AI can reduce systemic unfairness, I bet most people will actually find some degree of relief from that dread. The complacency of the formerly-statused being shocked along the way isn't much of an argument against that equalization.
> Beyond just making you redundant in your job, the whole architecture of your existence, the thousands of hours of struggle, the sacrifice of putting off social games or personal enjoyment are now rendered moot in hindsight.
This passage answers itself, doesn't it? You can sense those things you postponed fun for (work and the apparatus around its struggle) were never worth it. So why fight for us all to have to keep working just to live? Isn't a post-scarcity world the thing we should all be fighting for?
>This passage answers itself, doesn't it? You can sense those things you postponed fun for (work and the apparatus around its struggle) were never worth it.
This is what you don't get. _I_ already know it's not worth it. I've always known. It's everyone else! It's a wife who cares whether or not her husband has a good career. It's the police who care if you look like you're poor. It's the way every single person in society obsessively cares about the status of others.
What in the world makes you think I've had to build up status for my own sake? It's absolutely baffling that anyone could be confused about this.
Not only is that not a real solution to unfairness (it's certainly not fair for the person who now has to suffer more than they were), it also ignores the far better option which would be to reduce your suffering so that it isn't disproportionate to everyone else's. You might be holding onto to some self-destructive fearful crab-bucket mentality if that's where your mind instantly takes you.
A fallacious argument. There's no goal of inflicting suffering on other people; you can equally look at it as enabling people who were previously shut out of something to participate in it, or if not to participate in it directly to reap the benefits by having an agent exploit automation on their behalf.
>> the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.
> Not for those without status (which is most people on this planet)
Look at how coal miners react to the idea of getting rid of coal as an energy source; whether or not you think they have prestige or status, there's certainly an amount of dread there that I'd expect to somewhat resemble the type we're talking about here.
I think the fix is to get started on post scarcity economics now. Sooner or later we're going to have to come up with practices that boost or limit endeavors based on how people feel about them, rather than based on how much money somebody has.
It doesn't have to be ((post scarcity) economics), it can be (post (scarcity economics)). Both systems can coexist for a while, one providing a bit of a safety net while the other falls apart.
There will still be scarcity, just of different things.
Even if we vastly increase productivity beyond our current wildest dreams, the production system takes time to adjust to changes in demand, creating continuous points of scarcity.
But Georgist economics are way ahead of how to deal with this. Tax the ownership/control of things that can not be increased (e.g. land).
Sure, but that's no reason to continue using control over scarce things as a proxy for consent. It's not exactly working out.
We had to tolerate gold because it implied a game that was facilitated by the laws of physics, and we didn't have the resources to set up any other sort. But now we do, the rules can be whatever we want them to be.
Sure, let's try Georgism, but given the historical track record of that kind of reform, let's work on other things in parallel.
> continue using control over scarce things as a proxy for consent.
This is an expression for which I can imagine a multitude of specific meanings, can you clarify? I don't know of anybody who is actually using control over scarcity as a proxy for consent (consent of whom?)
Physics still puts bounds on production, and what we can have. Land is the perfect example.
Gold was only ever used as an accounting mechanism, and fiat money is definitely an improvement but accounting must still happen. We were never richer or poorer because of the amount of gold we had in our possession, it was merely used as a token of how much of others' labor we could have. As many current things that are scarce become less scarce, we'll still keep accounting for them. And as new things become scarcer, accounting will be just as necessary, by US dollar or by crypto or by handmade pieces of art that AI can't produce, or whatever.
Georgism's failures have been through landowners getting control of the taxation mechanism and taking the money back that belongs to the rest of society. Literally any other social change is going to have to confront that same question of power and allocation of resources too, it's nothing specific to Georgism.
I mean that if Alice and Bob each go to the bank with and idea and they each get a $1 business loan, now they control something scarce (a dollar).
Suppose I own a store, and they each show up and offer me $1 in exchange for my product. If I accept that dollar, I'm now participating in making their venture happen. If I refuse, I've resisted it in some way, however small. Whether I accept the dollar is a proxy for my consent re: whatever venture caused it to be issued.
A sane design for an economic system which was not strictly scarcity based would attempt create an advantage for ventures that are helpful, and a disadvantage to those that are harmful. One way to do this would be to provide enough metadata along with the money so that I can make a more informed choice about whether to create a liquidity problem for the holder of this money.
Like maybe Alice's venture is going to poison my drinking water, and Bob's venture is going to treat my dad's cancer. Then it's likely in my best interest to tell Alice to get lost but to support Bob. This would be a sort of nucleation site around which unions could form and solidarity could be built. Not trade unions, but rather a union of people with shared interest re: where they get their drinking water or somesuch.
We're allegedly into people acting in rational self interest, yet we've designed a system which limits access to the information necessary to behave rationally when deciding whether or not to engage in a transaction: "What behavior will I be enabling?" needs to be part of the equation.
I'm not an economist, maybe my idea about metadata on the money is silly. Probably there's a better way. But whatever it is it should be in some way sensitive to my values--even if they differ from the bank's values.
Our system is not. It assumes that both parties to a transaction have shared interests, which is increasingly not the case. It forgoes a bias for help over harm in favor of fungibility and liquidity instead. (I expect this property is a historical accident having to do with the unique chemical properties of gold, and not something we would have designed for, were we doing it from scratch. It succeeded because nature implemented the scarcity part for us, not because it resulted in an optimal set of economic properties.)
Re: Georgism...
> landowners getting control of the taxation mechanism and taking the money back that belongs to the rest of society
...that only works for the landowners if there is some central point in the flow of money that they can focus their corruption on: a government or a bank.
If you shift the whether-this-money-is-legitimate question to the next person to receive that money, recursively, corruption becomes much more expensive because it must be sustained for every transaction--it becomes something that they have to do continuously, rather than a battle that they need only win once and then defend for generations. The cost of doing so creates a drag which is not experienced by people pursuing pro-social goals (because holders of that money can reliably spend it widely, no need for coersion).
Whether money is "legitimate" or not it will be invented by whatever system you think you are replacing it with. Prices show up in optimization whether you want them to or not.
If you replace "government" with something like anarcho-syndicalism you shift the money question to asking people to behave according to some social norms. You replace the allocation problem with a different system of social relations, that's more opaque and harder to understand, but there's still the same allocation question: who gets what? Who gets to be close to the funny people? Or the artists? Who gets the better view from their house? Who gets all these things which now become the focus of the scarcity question?
Yeah sure, the scarcity is not gone, but its also no longer the only logic that's influencing the direction of the system.
I'm not asking people to behave according to some social norms. I'm expecting them to already have such norms and to use them to decide on a direction in which to shove the market. We've got plenty of horsepower, what we're lacking is a compass. I might not agree with the choices that get made, but I'm still in favor of them being choices that had humans at the helm, rather than the whims of an irrational market.
Whereas with a strictly scarcity based system, our preferences only half matter. It has created a dangerous blindness to outcomes. As long as the number goes up, we keep our foot on the gas, never mind the cliff over there. There are a variety ways to steer an economy:
- say yes/no to a transaction...
-- based on expectations about the value of what is being bought/sold
-- based on whether the transaction will enable the buyer to help/harm the seller or visa versa
I suppose that's 8 buttons to push. But we're currently ignoring the third dimension which reduces it to just 4. We haven't provided people with an expressive enough control panel to act rationally. Not enough input to make a good decision, and outputs that don't carry enough information to be used as actionable feedback, it's just flattened to a yes or a no.
> Prices show up in optimization whether you want them to or not.
Yes but they could carry more information, such as why you're getting a good or a bad price, which might lead one to change their behavior so that more people are more willing to work with them and they therefore get a better price.
> that's more opaque and harder to understand
Well yes, a car with only an engine and no steering wheel is indeed easier to understand than one with both, but I'd still prefer the latter.
Which is why I'm saying we need to start now. We can't wait for scarcity to be gone, we need to actively make space for something else to grow. Not so the new thing can replace scarcity, but so it can find an equilibrium with it.
agree. wealth inequality has only widened despite AI. things like healthcare insurance, tuition, rent and homes less affordable or more expensive than ever. Nothing has changed and probably ever will. Ai creates abundance in some things like knowledge but other tangible scarcities and individual differences of status remain.
The people at the top of the hierarchy are not very intelligent or engineer minded, so the more likely result of all of this simply to genocide the population. That's easier than trying to solve logistics.
It's mainly about status. No one cares that much about AI automating drug discovery or coding, but it's bad when it assists in math. Math results are attached to individuals , whereas for cancer drugs or other medical discoveries, it's more of a team effort by a lab. AI now has the effect of turning math into something that is more lab-like and less about the individual.
I would answer to this that the mathematicians (humans) are the ones who give any meaning to mathematical discoveries. If we don't have mathematicians anymore, the result is that AI can find whatever they want, but 99% of that will be useless waste. Outside of practical applications, the only reason most mathematical results have any interest is that lots of mathematicians spend time trying to find their solution. Once humans stop caring, mathematics done in an automated fashion will also stop having any value, other than its possible use in finding solutions to existing engineering problems.
As a former research mathematician, I've already stopped caring due to AI. I used to write expository articles on mathematics on the internet but I deleted those because of AI. Unless mathematics and indeed human endeavors can be made human again, I don't see much point in caring to generate commodities in a mechanical fashion.
Counter point: AI is trained on output from humans. We all made AI, together. It belongs to all of us. We use it to benefit us, and we celebrate our wins together.
For the low entry fee of ~$200 a month you too can escape the permanent underclass - which is more than some countries average citizen makes in an entire year.
Counterpoint: Their moat is funding secured and contracts locking in the purchase of currently forecasted chip production. Even if you assume a catastrophic withdrawal of invested capital from those vehicles on a coordinated revocation of faith by the investing community; who do you plan on having operate this theoretical squeaky clean, ethically training data sourcing company? And for that matter; what makes you think you'll get passed the greed that is already sunk cost into the current gen of shysters?
This entire evolution is a generational project hoping to paradigm shift away from their predecessor's paradigm while ignoring the lessons of their predecessor's predecessors on the nature of greed, the corrupting influence of power, and the consequences of blatant cultural/ethical unhitching amplified by the reach amplification of technology?
The Stasi, or a perfect Jim Crowe couldn't be realized with manpower alone. With computers, and a small cadre of the unethical willing to succumb to the promise of lucre on the other hand...
When I was a kid I built my entire identity around being #1. I was the most talented kid in school, anything endangering that resulted in an instant identity crisis.
Then I went to college and I understood that there are people who, for one reason or another, are thinking in ways so advanced I'll never catch up. I'll never be the #1. I'm actually much closer to being average than I thought.
But this doesn't mean I should just give up, lie down in my bed and cry. Since then I've learned a lot of skills - most of them being crap at best, but that's not the point. What matters is my own subjective experience. The fact that despite objectively not being good, I see myself as a skilled person with rich life.
Once I was watching a YouTube video about elderly care and there was a phrase along "These people won't achieve anything more in their lives, my goal is to keep them occupied until they die" which is a bit morbid at first glance, but it actually makes so much sense as a life philosophy.
> Once I was watching a YouTube video about elderly care and there was a phrase along "These people won't achieve anything more in their lives, my goal is to keep them occupied until they die" which is a bit morbid at first glance, but it actually makes so much sense as a life philosophy.
So they'll keep us all busy with bingo games, shuffleboard, 80s TV re-runs, etc? I guess I don't see that as being much of a life philosophy.
Well, instead of someone keeping you busy, choose the thing you'll keep yourself busy with. Grow a few plants, for example. This has literally zero economic or artistic value or anything, but could be a fun thing to do.
The point is, the things you do don't need to serve some bigger goal. Your entire life doesn't need to serve some bigger goal. Just keep yourself occupied until you die.
When did we last try to avoid such a thing? The way I remember history we celebrate this whenever it happens. Like how the electronic computer made the job of human computers obsolete.
You are substituting one legitimate meaning of "worth" (moral value) with another legitimate meaning (economic value).
But then conflating the two to make your point, which if you care about making meaningful statements as apposed to word games, is not a legitimate move.
Unfortunately, I think this is a human failing being exposed rather than anything else. We like to build conceptual models of how things work, but they are often tinged with should. Reality doesn't care how things should work. It only cares about how things do work. It's always jarring when the should turns out to be false.
Philosophers for millenia believed that human beings had meaning, purpose and set out to explore what that could be. It was a noble profession for a long time. Once we realized that there was no hidden spark, ergo no purpose, the study of and the prestige associated with philosophy took a nosedive. Yes it is still studied but it carries nowhere near the respect it once did. Mathematics seems to be suffering the same fate.
Yes, and I think over time there will be a blatant inequality in who is commoditized and who isn’t.
If AI engineers didn’t create the data to produce the models, and they increasingly don’t write the code to produce the models either, then why are they treated as the agents of change rather than the subjects of change, like mathematicians are?
Nah. There is nothing beyond being made redundant in your job and labor leverage. That is all. It has got nothing to do with resentment over fucking machines. It’s all about people (the executives that you mention) fucking us over. The end. Completely human problem.
I know I’m being sold some bullshit when this is made into something inherently existential and psycho-identitarian. An egalitarian future of labor-worthlessness means that there is nothing above us to dictate anything at all—maybe not even nature itself, save for meteors and other cosmic events. I’m not afraid of being “worthless” in that world of complete freedom. Fuck no.
Some have been scurrying to amass a nest egg and become self-sufficient. Of course, if demand destruction from the hollowing out of labor happens fast enough, who will consume and keep these stock prices up? With the tax base simultaneously crashing, gov't can't step in and prop things up. It could help during the transition while failing as a long-term strategy but what else can you do?
The rest of that quote is about the timeline where everything turns out perfectly but everyone is quote and unquote useless. That is not the only timeline. The timeline where the AI Ghouls get their way and oppress everyone else is covered in the first paragraph. This is to illustrate that the only ill-effects to my mind are exactly that: the humans who are are trying to oppress other humans. I don’t care if the machines make me jobless in itself. I have no psychological qualms with that.
Great, I'll just eat good intentions, wear a smile, and live under happy thoughts. I'm 50, so I'm "cooked" as far as contemporary society is concerned.
85% of people hate their jobs and just want to spend time with their family and friends. Utilitarianly, AGI is a benefit for the vast majority of people. And people who get their pride from playing status games are about to learn what it feels like to be on the other side
Maybe I'm too doomer, but if a large portion of the population becomes economically worthless, how can this not have a destructive effect on society? It would be easier for anyone with power if you wouldn't exist. How would that not lead to corrosion of basic human rights (or worse) and the spawn of a new permanent underclass which you cannot escape from? It sounds very dystopian but if AI is left unchecked this will happen imo.
It already has happened with technologies before AI. The internet itself in many ways has disrupted communities a lot with the net effect being the erosion of individuality and value placed on the uniqueness of people. AI is just the next stage in the cohesion of technology against the human spirit.
So it's already happening and is just strengthening now with AI.
For me the most obvious conclusion is, use your maths skills and get into open source AI.
Make AI as cheap and accessible to as many people as possible and we will likely be much better off.
The more people empowered by the tech the less like you will be “redundant” in the way you describe. You will own the important tools only the techno oligarchy own today…
Would you feel the same if you were a part owner of OpenAI? I bet you’d feel excited about the money you’re about to make ?
This part is nonsense, still. There's no known AI -> AGI path, and superintelligence doesn't mean anything, and AGI is assumed to be (going to be) both malicious and a slave, both for no good reason.
It has already been happening for decades if not centuries. Instead of bargaining with superintelligence though, it's bargaining with moronic systems. I watched an airline gate employee struggle to allow a family onto a flight despite acknowledging that they were confirmed and had seats on the plane because the computer wouldn't beep when they waived their boarding passes. She called her supervisor. Her supervisor resolved it not by writing "it's okay they can get on" into an exceptions field, but by kicking them off the flight, putting them on standby, then accepting their standby request and reseating them (in different seats, not all together, for some reason). The humans involved all knew things were okay, but the system was too dumb to just allow them to go forward.
Consider this… in the grand scheme of things, we are on a tiny planet in the universe. What cosmic contribution could you make?
Most people won’t make anything even close to “a ding in the universe”, as Steve Jobs has been reported to say.
So they may as well enjoy their life. And your worth can be to your family, to your children, your parents who took care of you.
Learning a hobby. Practicing religion. Mindfulness and meditation. Doing sports and improving your own body.
Look around if you are a heterosexual man, for instance: many women are far more attracted to men that perform in the moment, than men who labor quietly to create a product. The former enables untold hours of honing social skills and calibration, and humor and performance skills. The latter is of more durable value to society, yet relatively few women are genuinely sexually attracted to a man doing the deep work. (By the way, it is also true the other way: most men are not attracted to women for doing deep work or being a rocket scientist.)
Animals don’t do “deep work” and most animals are attracted for far “shallower” reasons. If everyone has a UBI and AI does the “deep work” then that frees up humans to be like animals in a zoo or in the wild… doing all the displays to attract mates, having babies etc. That’s not nothing — in fact it is 99.999% of everything that happened in history and 100% of what happened in prehistory.
You’re just ceding that 0.0001% which represents technological progress. The actual dread comes from the fact that you and all of humanity will be as completely powerless to change things as cats or any random animal species is powerless now. The difference is you will perhaps understand your powerlessness more than they do. But in the good scenario, you’ll be like animals in a zoo. We are constructing this zoo for ourselves now.
We will remain on this blue planet while the AIs will colonize the universe. In fact I wonder why aliens would ever come to Earth, when they’d just send their AIs or robots. Look at this: https://news.ycombinator.com/item?id=46200502
Maybe I'm obtuse, but I just struggle to understand why we as humans, at this stage, knowing what we know, we think that there's value and honor in trying to build new things and try to move off our planet. I mean I get it, there's the exploration spirit, there's the pushing of the frontier. But our generation, in striving for such things, has somehow disconnected ourselves from where I believe the value actually lies. Taking care of loved ones, of our children, of pets. Teaching our children to love and to be gentle and to take care of others. Appreciating the beauty of the universe, because no machine will ever be able to do that. Creating art, and beautiful music, investing inside of these works our emotions rather than physical resources so that we can create feelings in other. A machine can never make real music or art because it doesn't come from feeling and experience.
You are of course correct. The true value lies in relationships, art, love. This will never be threatened by AI etc. These are just tools, however disruptive. But the society we build might be misaligned from these true values. We need a revolution.
As Ernest Becker would ask, then what is your "immortality project"? Even worse, what's the immortality project for the vast majority of humanity? Do we just bring back religion to Middle Ages levels? Or do we hope everybody goes through some kind of never-before-in-history spiritual awakening?
We won't be bargaining with superintelligence, we will be bargaining with the handful of oligarchs that are creating super intelligence to rob us of all bargaining power.
This is why they're talking about it in religious terms. It's their salvation from labor and its demands.
This isn't a passive process that's happening by itself. It is driven by specific people with specific goals.
> How is one to feel safe in the progress of the future when the only hints of the wonderful post-scarcity utopia we can quantify so far is the removal of all human leverage via their labor?
I'm sure the Telegraph operators, human calculators, mine workers and all sorts of people have asked themselves this question. It's obvious that it feels strange to suddenly be part of the cohort asking but I assure you it's not new. And the answer hasn't changed much since.
> I'm sure the Telegraph operators, human calculators, mine workers and all sorts of people have asked themselves this question. It's obvious that it feels strange to suddenly be part of the cohort asking but I assure you it's not new. And the answer hasn't changed much since.
I don't buy this argument, much like I don't buy the similar "they said the same when calculators were introduced".
It negates that some changes are qualitatively and scale-wise different. Also, by gradually chipping away at things that humans do, you (we) keep shrinking and shrinking until there's nothing left, every step of the way some people claiming "well, the telegraph operators said the same". Keep in mind that, while it's hyperbole, some AI entrepreneurs are actually saying stuff like "maybe jobs will become a hobby". Unrealistic for now, maybe, but if they do become a hobby, what leverage will we have at all?
In other words, it's true the past is worth keeping in mind, but not to the exclusion of the possibility that this time it's different. Or that the trends of the past actually point to harm in the near future.
And like to anyone else who hints that “automation creates better jobs”, I have to add: but not for us. Maybe for the next or the generation after that.
Industrial and technological progress has birthed better conditions overall, but at a cost of locally worse ones. Mine workers that got displaced didn’t end up being machine operators - most of them landed on benefits or sporadic gig work. Only the next generation was able to be properly trained and educated to be able to benefit from the technological advancement en masse. And we might see that happening again.
In the I was referring to people reading my comment as “us”.
While it is nice to think of better futures for coming generations, I think we (reading this) also have to think about our more immediate work, as in next 2-5-10 years.
> the removal of all human leverage via their labor
I’m not convinced at all that SI will ever be able to produce the correct result with little input. The advancements in capability I’ve seen in models is that they understand my intention better, and follow industry best practices better, so their output is often acceptable. But I absolutely have to make my intentions known, as detailed as possible, or I will get the mediocre, standard solution that falls out of the weights.
I’ve been having trouble with models always steering towards certain solutions - they insert a queue here, a mutex there, a backoff over there. If I remove those, and let the model freely rebase the PR stack, then it will re-insert those mechanisms because it has been specifically trained to do exactly that. It’s essentially boilerplate, smoothly aligning my codebase to match their weights. Which is often good enough! They are training the models well. But individual problems require individual solutions.
My personal take on the human part in all this is: We get to decide, and the models obey. Our decisions are now super leveraged. So let’s make good decisions, and communicate them well. The layer of where decisions yield the most leverage, has been pushed upwards (I see it on the type and function signature level, others will see it even higher), but IMO it won’t be possible to push it up much further without losing truth to the decisions we need to make for a good product.
I can kind of relate but I feel like I am mourning something different: the loss of the desire to seek truth in tech, math etc.
Nothing is validated or tested anymore. It’s all about the shape or believability or even how loud it is.
They dropped 100+ papers no one read. I am sure mathematicians are reading them now but even this forum has concluded that these problems are solved, without reading or validating.
It’s like everyone forgot “don’t believe everything you read on the Internet.” And it makes for a very boring discussion when everyone is just spewing unvalidated claims, layering them with strong beliefs founded on similarly unvalidated claims. It’s dull, without substance and just pure noise.
> I feel like I am mourning something different: the loss of the desire to seek truth in tech, math etc.
This is what I've been feeling too. It feels increasingly like the professional world has become a tower of scams. It's not worth building technical skills or focusing on quality; your boss doesn't know or care what good work looks like, he/she just wants your work to look good enough to his/her boss, who knows even less about the actual work being performed. And everyone reports up to an MBA CEO who was brought in from a completely different industry.
I want to work in an industry where skill, diligence and expertise is rewarded and respected, but from what I'm seeing political ability is 95% of the game and caring too much about the work is a career liability.
Not sure I agree - they dropped hundreds of papers, most of which were so badly written that they were incomprehensible. If it wasn’t for the lean certificates on 30% of them, the mathematical community would have told them to go and whistle.
I think the consensus here is that most of the papers are effectively correct. But it would take years of study just to understand most of the problem statement - so we have to wait for the mathematical community to digest the borderline-nonsense that OpenAI spewed out.
Just tell the (AI) authors to go back and write clear explanations for why they matter and what the specific claim is.
I gotta say I think mathematicians brought this roblem on themselves, most math papers are absolutely incomprehensible to anyone who is not an expert in the field. Mathematicians spend so much effort trying to make their argument to other mathematicians that they've gradually abandoned making their field or findings comprehensible to anyone else and left that to science journalists. In turn, those journalists churn out very formulaic stories written to a template: 50% the human story of the mathematicians social/academic wanderings in pursuit of their goal, 50% an explainer with some nice graphics on the subfield they operate in and a mention of what sort of problem their insight solves, perhaps with a few colorful metaphors thrown in.
The last mathematical paper I can think of (but I'm not an expert, I'm sure there are some others) that was both mathematically rigorous and accessible enough to be readable was Mitchell Feigenbaum's Quantitative Universality for a Class of
Nonlinear Transformations. This is accessible int he same way as a textbook - you have to work at it, but hte author is explaining his thought process and drawing the reader's attention toward things that matter or stand out as unusual.
It seems like, once the IPO is done, this flood of papers will dry right up: the inherent value of the papers themselves is unlikely to justify the cost to churn them out. It makes as much sense as OpenAI churning out hundreds of slopped programs no-one wants to run, or hundreds of novellas no-one wants to read: it's just showing they can.
These papers are just first steps. They are going to keep churning out papers, in part because the generated papers are going to get better. And better.
It's the ultimate proof of how fashion and religious believe override everything else. It's fashionable now to believe in these unverified fairy tales, and if you don't, you're at best a weirdo who will be "left behind".
I think for me is, that history is not kind to the people who are left behind.
For example, weavers were one of the most powerful professions (if thats a thing) in the middle ages. In england there are the wool churches(not entirely weavers....), in the low countries, the weavers guild fomented revolutions, and shaped the borders of countries.
But that came to a crashing halt over the space of 200 years. slowly at first, then all at once. the power loom, then cheap imports from india turned a cottage industry into dark satanic mills. Yes there are/were specialists that still hang on, but they are like football stars you can probably name all the people who make money, the rest are barely scraping by.
For us, in the programming world, we will not have a 200 year grace period. Yes, AI produces shit code, but thats not the point. its cheaper and there are vanishingly small places were "good" code matters anymore. AI could be used to help a large range of people, provide support, trusted advice.
However because of the people who are funding it, that will not be the case. it will be used to reduce wages. Just like offshoring, resulting in china being the worlds manufacturing hub, AI, and the current people funding it will aim to make the expensive worker redundant.
In short, I weep, because Altman, and the money behind him ans his ilk want me destitute. I have committed the crime of wanting to earn a wage, and thus I must be punished.
It's profit chasing as well as practical - it kneecaps the educated workers increasingly asking questions about the morality of business and technology. It takes away the income that affords them the space to push back, threatens high ed, critical thinking. The salivating over AI makes perfect sense in that context.
Profit is the driving force behind civilization, motivating innovation and progress and allowing us to live longer and better lives instead of remaining stuck in the Stone Age.
So the dude in the cave that invented fire on demand was chasing after "profits"? Progress and emerging "civilization" clearly could not have been motivated by profit. We could argue money has distorted a natural, inherent, human drive to improve his and group's living condition.
"AI" tech is actually a perfect specimen here: this could be a genuinely positive net for Humanity if only it wasn't controlled by the sort of scumbags that are only motivated by "profit".
Depending on how broadly or narrowly you define “profit”, your statement is either too weak to be useful, or false.
If you define it so broadly that the hunter-gatherer inventing a spear for hunting more effectively is “profit”, then your statement is a tautology. The spear innovation was driven by “profit” and the “profit” is having the spear. Great.
If you apply a narrow, capitalist definition of “profit”, the statement is false. Plenty of advancement of civilization was and still is being made without or outside of capitalism, especially if you go as far back as the Stone Age that you mentioned.
Profit is what's on the verge of destroying our civilisation, too. Ever heard of climate change and biodiversity loss? And of course the energy crisis.
And we are not trying to address them, we are just trying to make more AI and more techbros.
Yes, it is _a_ driving force, but its not the only one.
the enlightenment was driven by "we can do thing better" and "to know god, we must know the world". A lot of money for research came from the catholic church (including for Galileo who wasn't put in jail for heliocentric, he was put in jail for being a massive insufferable arrogant prick)
In england, a lot of research was carried out by bored vicars, on CoE dime. (well pennies)
So yes, profit can be a motivator, and it can be very effective, it needs to be tempered with humanity. We could go into who a lot of brands we see today that are old were founded by quackers, but we dont have time for that.
the TLDR is, yes profit is a motivator, but its often short termist.
Declare that the current use of copyrighted works is not fair use. Create a new
"generative use" clause which permits copyright use consumption and transformati
on provided a number of conditions are met e.g. not-for-profit and for-public-be
nefit organisation.
Place a tax on current AI company outputs, similar to CD or harddrive tax in Fra
nce. This new "generative use" entities would not have to pay this.
The government/people "owns" all of the land and resources in it's borders and makes it available to people or companies on various conditions. You could interpret that they also "own" all of the copyrighted works (i.e. things become public domain at death+70years or whatever the current scheme is) and choose the mecha
nisms, fair-use, exclusive copyright etc, which are or aren't permitted.
Open weights doesn't solve the problem, yes it _might_ cause the bubble to burst, but it won't put the djin back in the lamp.
A brief look at labour relations in both the US and UK suggests that sustained direct action is the only real way to force change. The hard part for us is that for the last 25 years we've all been told that unions supress wage, and that we are all expert negotiators. Unions would be the natural place to start, but they are but a former shell of what they were (for good or ill)
I don't have all the answers but my gut is: resist convenience, educate yourself on how tech is steering politics, give input to city planners on projects, if possible increase your agency so that you have the ability to push back and influence career choice/conditions (build up financial security, live below your means, advocate for housing).
This piece circles, without naming it, around the concept of “deontic scorekeeping” central to the work of Robert Brandom on semantics. The idea that our ability to create and sustain conceptual systems depends on the give and take of reasons within a social setting. Brandom has been front of mind for me lately because elements of his theory both explain why LLMs should work in principle, and where they are bound to be radically deficient. An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time.
Worth reading: Robert Brandom, “Making It Explicit: Reasoning, Representing, and Discursive Commitment”
I mean, many companies operate within particular domains but don't have much skin in the domain itself in a way that is meaningful to other participants there -- like Oracle builds software for healthcare, ostensibly, but any actual user of their products will tell you they are nightmarish with regard to usability, and research shows that patients are often harmed as a result. So healthcare isn't their game, it's selling healthcare software.
Likewise OpenAI doesn't have "skin" of any shape or value in e.g. the mathematics community or even in mathematics research per se. In my cynical view, they are exploiting the exposure of mathematical research to automation in an effort to create a case study of their product obsoleting a highly visible and well-leveraged and relatively small profession. Regardless, I doubt even they would say they have durable skin in mathematics in any sense, certainly not in either of the senses I mentioned above. But their output inhabits the same space as mathematicians, who have to contend with the consequences
I think it means if we all have to justify ourselves to each other, we collectively come up with an sustainable ecosystem of ideas that we can understand. So if we stop justifying ourselves to each other, the ecosystem doesn't get built or sustained, and we stop understanding things. Not the person you responded to, but that's my reading.
If a short sentence like that could accurately summarize Robert Brandom’s deontic scorekeeping, he would have wasted a good chunk of his career.
I mean this with zero snark: if you’re interested, you’d be best off asking an LLM or doing a search. But the overall idea is that Brandom described how concepts and meaning are created socially by a kind of game in which people keep track of the claims that other people commit to.
If you accept Brandom’s framework, then it can be argued that LLMs can’t meaningfully participate in this process. This could be considered an explanation of why people have a negative reaction to LLM involvement in activities that are meaningful to humans - even though most people won’t be familiar with the theory, they would instinctively recognize that something important is lacking, such as commitment over time.
What a coincidence, I've also picked up inferentialism but never really got into it.
I'm interested in the reversal that Sellars proposes - it seems clear to me that "representation" is not prior to the space of reasons, despite how unintuitive it is.
> An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time.
I see no reason why this can't be the case in the future, even though I think inferentialist-based theories are generally strong. Can't you force LLM's (or whatever AI architecture will exist in a decade) to stake durable commitments itself, then? They're at least autonomous enough to run computers. I agree that that is a core part to true meaning & autonomy, but nobody just lets them yet. I don't think inferentialism is a consciousness-based argument, either, so that escape hatch isn't available. I also don't think this is a necessary condition to engage in the space of reasons (but maybe then I'm misunderstanding it, or retreating to a more Sellarsian understanding). Could you explain on this more?
Though again, I only have the book sitting on my desk, and I haven't cracked it open...
I actually agree with you. LLMs could become participants in the future, but this would require (1) durability of real awareness over time (an architecture change from the current fixed-weights + transient context architecture) and (2) that humans grant them the status of equal participants (probably not a terribly high bar given how people already treat these things). I'm not aware of any real progress toward (1), and all the stand-ins people have developed live in the context layer or textual artifacts, which are poor stand-ins for durable memory.
And yeah, consciousness based arguments are bunk IMO. It's clear that LLMs are in a basic sense "aware", and even to some extent "sapient" (in Brandom's usage), i.e. they can justify claims and object to things at least locally. I follow Rorty on the question of consciousness, which is to say that consciousness has no deeper meaning in practice beyond "are you awake", and in philosophical discussion it's largely a proxy for "are you one of us" in some amorphous sense.
Well that's too bad, since Rorty completely misses the point on consciousness.
Consciousness, the interesting part anyway, is qualia, or the "it is like something to be me" as Nagel put it.
Intention, as Dennett used it in his book "The Intentional Stance" is the "are you one of us" angle, and is distinct from consciousness because as the book title suggests, it is a stance taken with respect to some other system (person, cat, dog, turtle, LLM) rather than a description of subjective experience.
So I think it is your (and Rorty's) positions that are bunk. It is very, very far from clear that LLM's are in any basic sense "aware" let alone "sapient". They are extraordinarily able to manipulate language in a way that leads to text generation that has traditionally been assumed to indicate consciousness and/or intelligence. But we have no way of knowing if it is like anything to be an LLM, even though most of us are busy taking the Intentional Stance towards them.
I guess I'd say that your qualia aren't a something, but they're not a nothing either. We can't say what they are. They're not publicly describable, and therefore not publicly attributable. As Wittgenstein would say, "They cancel out."
And as I think Dennett himself would say, qualia are just Sellars's Given rearing its ugly head again.
Well, while I love much of Dennett's work, I emailed him after "Consciousness Explained" was published to say that I was disappointed that it appeared to describe what we are conscious of, not how we are conscious. He sent back a short note to say "You're probably right".
Brandom wrote a shorter version of the book, called "Articulating Reasons", which is available in full on his faculty page [1], or from booksellers. He's a very wordy writer (kind of exhausting to read, even for someone who enjoys philosophy), but if you get ahold of Making It Explicit, chapters 2 and 3 have the meat of the ideas I'm talking about here.
Edit: One of my ways of benchmarking new AI models is actually to type a version of this prompt in and see how sophisticated the response is: "Talk to me about the concept of deontic scorekeeping in the context of llms". You'll get good mileage out of that.
I think in future, each company will built its own universe of mathematics consisting of prized discoveries. Maybe one Mathematical Universe per Company in the multiverse of mathematics.
Humans change over time due to what they experience. It’s different from waking up from a ten-year coma and having access to a record of what happened to yourself in an alternative timeline where you didn’t fall into the coma.
This may not be helpful or strictly accurate, but when I see deontology or variants of that word, I substitute "SHOULD". It is like a duty or obligation.
NOTE: This would suggest a surface reading of Brandom's "deontic scorekeeping" might be keeping score of who has what obligations, who is following or breaking rules, etc.
Then the argument against LLMs being party to that is that we don't really assign roles and responsibilities to LLMs, they don't really have "duties" or "obligations" in the way a member of society will have. We don't negotiate with them over who should do what, they don't contribute to the rules that are set for them.
I'm literally just guessing based on the context here though, since the few times I've tried to watch podcasts with Brandom I generally bounce after 15-20 minutes feeling I haven't really understood anything he is saying.
Mathematicians always define their terms exactly, in order to begin their discussion. Philosophers never define their terms exactly, because then they would have nothing to discuss.
Because language is mimetic, and people are consuming huge quantities of AI-generated text. I.e. saying that an LLM can't be an equal participant in our language games isn't saying that it can't play by the rules. In fact that's what LLMs are trained to be best at.
What an LLM can't do (at least with the current architecture) is sustain a commitment that it can be held accountable for later. It doesn't have durable "perspective". It can't update its priors. Maybe a new architecture will come along that gives LLMs memory plasticity, at which point they will shift into being individuated quasi-persons, but that's not what we've got now.
Rather the opposite, AIs are arguably deontically asymmetric because they are powerful and virtual digital entities, and thus should be very concerning to the human species. Not because they current have no internal abstract state for learning state like humans and animals.
Recursive self improvement is a bulk process that shapes the weights of the next model. The weights are still frozen. Each individual "agent" would need to able to durably update its own dispositions to get to what I'm talking about.
I'm not suggesting mathematics and language are the same thing, but I am saying (which shouldn't be controversial) that mathematics as a discipline exists within the medium of language, and that mathematical reasoning, discovery, and proof happen via people proposing, justifying, testing, and objecting to formulations in language. This isn't a metaphysical claim about whether math is real or captures inherent structures of the universe etc.
Ok, then what is an agent that provides a 'thinking' or reasoning loop around said model?
You ended your first post with: "An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time."
And so if there is an agent, a thinking loop, that can recursively adjust over time...how does your statement hold up? I'm particularly talking about 'staking durable commitments or changing its dispositions over time'. Recursive self improvement is all about these 2 things, especially when it's the 'outer thinking loop' doing the improvement.
But you're right, Mathematics exists as a medium within Language. But then Language is just a transfer of ideas from one mind to another.
This principle is the heart of what the article is talking about: a transfer of logical mathematical ideas passed from a human to another. And the sense of 'shared understanding' that comes with it. It has nothing to do with keeping score.
It's about getting a 'sense' for why someone thought or rationalized an idea in order to connect some other idea. You get an 'ah-ha' moment, and some of those moments are gone, or a different path is taken to reach the moments now. That is what the 'weeping' is about (not that I agree with the article. I think anyone can spend however they want. Math doesn't have to change, it's that people want it to, apparently, because it's 'easier' to do something different, or for some other reason I don't know about).
It would only be true if you consider Mathematics and Language to be the same thing.
Both depend on Logic, which is where "deontic scorekeeping" comes in. You can only establish something (score) is true and believable by 'keeping score' and having some level of understanding about the language or math you are reading.
The article is mostly referring to the connection that comes from shared understanding between humans, particularly mathematicians, as they build new shared understanding, and realize there are limits to what they can know and 'leave behind' logic for others to pick up and carry on.
That connection has been severed, or strained a bit, but I'm sure new ones will be created. It just might be of the same flavor that mathematicians prefer, therefore they weep.
And if I understand "deontic scorekeeping" correctly, these forums are a better representation of the concept. You will lend some amount of belief to what is said/read based on what you know.
Ideas are contagious and AI is our most powerful vector yet.
I suppose we would see some interaction with AI in that sense, but it'll take place over the course of learn cycles and when AI starts ingesting its own ideas it typically goes wrong somewhere down the line.
I think Thompson has no understanding of endeavors like pure math or the theoretical sciences. In the words of Einstein, "I want to know how God created this world. I am not interested in this or that phenomenon, in the spectrum of this or that element. I want to know his thoughts; the rest [is] just details."
I think mathematicians aren't bemoaning their lost sunk costs. They're now aware that they may never again have the joy of being the first to pull up the edge of the tarp of ignorance and see the raw truth beneath _for the first time_. Their voyages of discovery, which give life meaning, may never leave dry dock again.
It is strange, because in a sense, when you start learning math, you never explore, you are being taught.
Now mathematicians were able to exercise curiosity. That is going to change. Back to the ranks of eternal learners if you will.
The fun was in the research. Now I think it will leave the door open toward more Applied math. What can people do with all the discoveries. The curiosity will be there.
I think it is better that way.
It will also put a cap on abstractions that overlaps. In essence, AI is less likely to create new theories, coin new terminology, etc. But it is better suited to recognize emerging patterns.
That will probably simplify mathematics.
Unless the domain of abstraction is infinite and with each new abstraction comes its own results but I doubt it. Who knows though...
> The story of human discovery and the triumph of the human spirit will soon be excised from this discipline. The Dinitz-Garg-Goemans counterexample was the most egregious demonstration. It revealed that the process of prompting novel proofs will be as auraless as ordering doordash. Watch as magic and mystery evaporate. Watch as the sun sets on our heroic age. Is there not something evil in the act of blocking all future generations of mathematicians from the experience of discovery? Forget about accuracy or even attribution. Something fundamental to the experience of mathematics is being taken.
> You are a helpless onlooker. Before you a channel through which humans have accessed the ineffable and sacred for thousands of years is being sealed for eternity.
As someone with friends in the mathematics community, they've expressed similar sentiments.
I have to wonder if this is the equivalent of teleporting back in time and finding Marco Polo or any of those early explorers who traveled for the love of the game (and fame and money), and opening up Google Maps on your phone and showing them the zoomed out world. And showing them any island or village they could name.
my analogy is that it is like an explorer being handed access to satellite imagery. and then being told that your experience will still be valuable in telling the satellite where to point and interpreting the photos. folks who got joy from traversing unknown terrain might find it distressing to become an image analyst.
This is absurd and silly. Discovery doesn't go away because more new things are being discovered. This position only makes sense if you make an arbitrary cut point where some instances "don't count". If you were to go back to scientists or mathematicians several hundred years ago and tell them that one day science would be done by tens of millions of full time researchers around the globe, often working in teams, some of them might also bemoan the loss of the "hero scientist" as any commoner could now participate. But science and math have only benefited from this, and this has increased, not decreased, the beauty and wonder in these fields.
It is not the same because the act of problem solving for discovery is now to be done exclusively by machines, as humans are no longer remotely competitive.
If you read the linked article, the author goes into more detail about how doing mathematics also has been an act of communication with past and present human mathematicians. This is very much true when math and science are done by millions of human researchers, but not so when it is done by massive swarms of agents at a couple labs.
I am sure science and math will benefit in the sense of solving more problems than ever before. But you are not really understanding the agony of the humans caught in the change.
> they may never again have the joy of being the first to pull up the edge of the tarp of ignorance and see the raw truth beneath _for the first time_
Until we can demonstrate AI consciousness, “seeing the raw truth” will be a human activity. Even now, verifying and understanding these AI proofs is a massive endeavor. Coming up with an intuitive (to humans) distillation of the finding remains an open challenging demanding expertise.
i think to certain types of people (who are probably more likely to be mathematicians), the "self" discovery aspect of it is important, they don't want to read about a new proof for a problem they were grappling with, they want to come about it through their own thoughts.
LLMs are not feeding us things, we are asking it and getting answers. This dialectic I think is super powerful and it still requires the human to ask questions and continue expanding intellectual curiosity to go down the rabbit hole and discover. LLMs are just an incredible rubber duck but the discovery is still there.
We literally had a case where OpenAI may have stolen the dialectic from a mathematician using its product and then given it to a proprietary model burning tens of millions of dollars of tokens [1].
I think I disagree, if I had a puzzle on my desk and I told an LLM to explain how to solve it, I think I have robbed myself of the enjoyment of solving the puzzle. Is it really any different than googling a solution?
There are different avenues of "solving" something though. Asking an LLM to one shot something vs iterating on it collaboratively are different ways of achieve potentially different ends.
If the result is black and white, then yeah, its probably not super fruitful but if the end result is subjective, then there really no is objective "solve".
> Until we can demonstrate AI consciousness, “seeing the raw truth” will be a human activity.
I don't think this matters a bit. We individually can't demonstrate "consciousness" in other humans, it's not even a word with a definition. What we can do is miss the feeling of discovering something on one's own rather than having it explained to you over and over again until you understand it.
I mean, dead mathematicians are definitely not "conscious," but their books keep you from discovering things that you may have discovered yourself if left to your own devices.
The joy that was mentioned is the joy of knowing a bit of gossip that no one else knows, of being the first explainer. Interpretation is just being able to pass gossip on; a decidedly lesser joy. And there's no reason that AI shouldn't be able to explain findings in an intuitive way - LLMs at least purely operate based on human intuition as written in human communications and then captured in matrices.
If these proofs are good (doubtful, imo) and can be reduced (intuitively distilled, sorry), there's no reason that an LLM shouldn't be able to do that.
That's such a nice way to put it! I recently stumbled into "The Last First- Winter K2"[1] and saw the same spirit along with a similar misguided effort to win without the will, nor capacity to do it — e.g. influencers taking a stab at K2 is like prompting your way to results.
Why is not like changing to a different type of vessel for the voyage? Unless and until it is all AI in closed system, could direct machines where to look etc.
> They're now aware that they may never again have the joy of being the first to pull up the edge of the tarp of ignorance...
Not "never again", it would be extremely difficult but science and math are also built on random, trial and error, and intuitions that could not arise from AI.
That's the issue - trial and error. A human can try a couple ideas in a day, but realistically, there are not that many sane ideas.
Computer can iterate without frustration on a completely random garbage. Once in a while it stumbles upon a gem. Even if it's one in 10000, that's still hundreds of good ideas in a day (provided enough GPUs).
Your comment assumes math and science are some kind of closed systems that can mechanistically pursue any endeavour but reality (whatever it is (e.g. the universe)) is much more complex than that, and there are realities that are outside of those systems. Not saying that humans can see all those realities though.
Maybe? I'm targeting math, which doesn't need any kind of reality check. It has axioms though, so in a sense it's closed.
You can generate completely random ideas (a couple proofs I saw where you define some weird set, pick an element and derive a contradiction when the set is empty or whatever).
Some of these will reach a conclusion: proof or a counterexample. And that's how speed can outperform creativity, if the speed bump is fast enough. It's the same as with those monkeys and Shakespeare.
I'll say it again: AI is capable of answering the questions put to it but AI is not yet capable of knowing which questions are worth asking. Mathematicians have no cause to weep, they will be fine.
Most of these questions Mathematicians posed to each other have no practical application, and probably will never have a practical application. (ex: the solution to the Navier Stokes question doesn't change anything at all.)
In many ways, the teleporting machine that teleports to the solution is against the point of the questions. No one wants the solution to be "Run $10,000,000 worth of tokens to cover all possible Lean formulations of the problem".
I think for Mathematicians, I'd say that the human level formulation is still possible and with a new perspective. We as a society (and outside of Mathematics) need to also understand that this AI teleport to the answer thing has less value than we all expected. The value is in the journey.
The open question is if people will still want to take the journey if the teleport machine exists. Or if the existence of the teleport machine halts exploration.
> "Or if the existence of the teleport machine halts exploration."
You're overlooking that the teleport machine costs lots of money to use. The AI providers are not going to waste money by teleporting randomly around.
Someone is going to have to find a new destination that is worth spending the money to teleport to (or walk by themselves) and that someone is mathematicians.
What marketing benefit are they going to be getting when all existing noteworthy problems have been addressed? They're just going to spend money on having the AI generate proofs of no value at random to thunderous public indifference? Clearly not. There will be a need for more noteworthy problems for their AI to publicly solve to get any marketing buzz and the consensus for what is noteworthy comes from mathematicians.
> “Google Translate is amazing, but it has taught us essentially nothing about linguistics,” Strogatz said. “We don’t understand why it works. It just works. That’s the Twilight Zone. It is knowledge without understanding.”
This is not true, and it is the crux of an enormous debate we're now having across many, many fields.
Classical Chomskyan syntax (which models sentences as parse trees) has been enormously, astonishingly predictive of the rich structure in real-world human language. But we are quick to forget that we invented concepts like "words" and "noun phrases" specifically to help us analyze and predict things about human language. What LLMs and modern MT shows, definitively, is that there is another, totally separate way to describe these phenomena, which is both vastly less descriptive, and also vastly more predictive.
That these appear to trade-off is unfortunate, but it is NOT the case that we "don't understand why it works." We absolutely understand why it works, if we didn't understand we would not be able to improve these systems. What we do not have is a clean, descriptive theory that boils this down to something that is formally specifiable and analyzable. But those are different things, and this is the root of the disconnect between many communities working in and around these spaces (linguists vs computational linguists; mathematicians vs computational mathematicians; mech interp people vs empiricists).
"We absolutely understand why it works, if we didn't understand we would not be able to improve these systems" - Understanding how to improve something does not require understanding how it works.
Growing a plant does not imply understanding soil minerals or photosynthesis.
Training a dog does not imply understanding it's brain.
Using LLMs to improve a codebase does not imply understanding C++.
Improving a PC with new parts, does not imply understanding RAM.
You can understand abstractions about the system, without understanding the system.
Without gradient descent or other forms of automatic optimisation, do you think humans could build something as capable as GPT-1?
Not sure if this will bake your noodle or not, but yes this is basically my point. Saying I understand how dog training works has very little overlap with saying I understand how the dog brain proteome works, even though they're related, and even though I might (or might not!) understand better how to train my dog for having fully understood the proteome.
Saying we understand training and deploying machine translation systems that have strong predictive (rather than descriptive) power, apparently has very little overlap with defining a formal, analyzable, mathematical model of language. Mathematician who refer to "understanding" generally mean the second thing, but the first thing is also a kind of understanding. Just messier.
I think they're referring to the fact that neural networks are more or less black boxes. Their internals are blurry and not explainable in the same way that traditional linguistics tries to explain things - with rules.
Of course, ultimately that's because language itself doesn't follow simple rules like linguists would prefer. They're more like guidelines.
Yes, that is one of the points I am making. Chomskyan syntax is a strongly descriptive theory that is entirely ex post. MT and LLM systems are strongly predictive and have no descriptive theory. It is completely wrong to say the first is "understanding" and the second is not. They are both different types of understanding. In particular, Chomskyan theory clearly does not represent enough "understanding" to predict real-world English sentences with similar lucidity to that of LLMs.
No, they're not predictive at all. They're generative. A prediction is a forecast based on a theoretical model about a state of affairs in the world. Making a guess what the weather tomorrow will be is a prediction, and a prediction can be true or false. An LLM generates text based on certain parameters, and output it generates aren't true or false, they're simply whatever the system is tuned to produce.
LLMs are terrible at making forecasts or world modelling precisely because they don't actually operate on much of any model or theory, they have no notion of truth or falsehood, they simply generate whatever you want them to generate.
If you'd like to use that definition of "predictive" in this discussion that's 100% fine with me. But, you should do so knowing that in ML and AI "prediction" is a term of art that means "a statistical model's output given some input." LLMs are one example, in that they take as input a sequence of tokens and predict the next token.
In this specific context, the point is that LLMs and MT "predict" lucid-sounding sentences that incorporate linguistic structure (syntactic and semantic) that are well beyond the predictive abilities of the Chomskyan theory of syntax (which is itself a generative model). In other words, the Chomskyan model of syntax is descriptive of much of the rich variation in real-world human language. But it is not predictive in the sense that it does not provide a constructive algorithm to emit lucid-sounding sentences (and does not attempt to, either).
I still think mathematicians are going to be as important as ever. The LLMs are only finding what was technically possible given our current breadth of knowledge after brute force solving problems. Eventually it will run out of things that it can do with its training and the only way to get it to do more will be training it on new human found knowledge.
Finding that new knowledge may be exponentially more difficult, but it will be much more valuable.
> Eventually it will run out of things that it can do with its training and the only way to get it to do more will be training it on new human found knowledge.
AIs cannot invent things like cubism, surrealism or hip-hop.
Or maybe it can; the early image generators famously did output horrific images sometimes because of their limitations, but these outputs qualify nonetheless creations.
Now the interesting thing is that it is humans who evaluate them as horrific, worthless or interesting.
Certainly AIs could pretend they have that sort of opinions and judgement, but hopefully humans will remain more interested in the opinions of other humans.
> Certainly AIs could pretend they have that sort of opinions and judgement, but hopefully humans will remain more interested in the opinions of other humans.
Online, how would they even know? Offline -- either we give up on the internet, or the offline discussion will be heavily influenced by what AI said online.
Amusingly, LLMs are good at detecting patterns because they are modelled after the human brain, which is very good at detecting patterns (this e.g. is annoying when you are a game developer who wants to reuse assets). People are still able to tell if the source of an artwork (or text) because LLMs tend to output "patterned" text (like the famous "load bearing"). That being said, the false positive/negative rate of humans for this task has yet to be measured.
I don't see how its output possibilities could be infinite given a finite set of data. Taken to the extreme if you give an LLM only a single piece of information then that is going to be the only thing it can produce. Given it two pieces of information and maybe it can produce every combination of those two things. Given all of the worlds knowledge I'm sure its possible outputs are incomprehensibly large, but still not infinite. Then having its outputs match a set of really specific instructions is going to be a small subset of that.
How is this different from human brains being made up of a finite number of neurons in a finite number of configurations? I know very little about LLMs and nothing about neurology and maybe you can explain to me why one is so much more powerful than the other.
As I understand it, the advantages of AI have not just been speed of processing.
Humans have cognitive limits. Humans hold a specific slice of all knowledge at a time and necessarily limits what correlations can be leveraged. This is similar to "context windows" in modern AI. Human minds are wildly variable in efficiency. AI is has constant inefficiency, largely controlled by the speed of light, physical RAM, and processing power, with superior (no cost) "thought" context switching. The AI also has superior accuracy without the need for "concentration" because the AI is processing on a Turing machine with the limited operations that it's superior at in terms of time and energy - eg find an element in a list of 1 million elements.
> a finite number of neurons in a finite number of configurations
Neurons and synapses change, especially synapses (the connections between neurons). Some neurons have a few connections to other neurons. Others have hundreds of thousands of connections to other neurons. These grow and change. "Estimates are between 100 trillion and 1,000 trillion neural connections" Is it finite? Depends on your definition of finite. If anything countable is finite, then it is finite but the number may be practically uncountable. Is it fixed? No.
That's 100G to 1T. Neurons are more complicated than weights, so add a x1000 somewhere, but they have more redundancy because they are noisy, unrelilable and may even die, so divide by some small number.
Sure, a given LLM can only produce so much (though it might be a lot), and then you'll need fresh inputs. But you contend that those have to be human, while I am arguing that they could be
- random,
- from the environment,
- from other AIs.
That increases the space considerably.
In particular, note that your argument also applies to humans: A human, being finite, can only produce finitely many things. But you add other humans, the environment, and pure chance to the mix, and it'll be pretty infinite for all practical purposes.
I would be curious to see the end result of letting it endlessly update its inputs from its own outputs. I was under the impression that eventually this would cause delusions and if you let it do this long enough and it is solving problems that no humans can verify as correct that it isn't useful.
Maybe that is the limits of my imagination, but it seems unwise to take humans completely out of the loop. Having mathematicians around that can understand what it is doing so that we can utilize that knowledge in a way that we can have some confidence in seems important.
1. LLMs aren't run deterministically - they use a bit of noise to improve the output. So even with the exact same input you're going to get a whole distribution of outputs.
2. Feedback from the environment. LLMs (and humans!) work best when they can test ideas. That's why coding agents work so well. If you ask an agent 1000 times to just think of a compression algorithm, it'll probably spit out very samey answers just like a human would. If you give it a test corpus and let it try out different things it will absolutely be able to come up with novel ideas.
Typical thought experiment - go back to the early years of humanity, even before we've created language itself. Communication is basic grunts and other such things, cutting edge knowledge is 'poke him with the pointy end'. Compile all of this knowledge into the training set for an LLM, let it run with infinite processing power and infinite time. It'll just continually remix the knowledge of that snapshot of time. You're certainly not getting to the Moon, discovering the secrets of the atom, or even discovering mathematics.
It sounds increasingly dumb to bring this up given how advanced models have become, but it remains true - they are still fundamentally token prediction algorithms. And that imposes hard constraints on their potential. These constraints are relatively academic in our current world because there's just so absurdly much information and no human can hold more than a negligible amount in his mind at once. So they'll certainly be able to connect many 'obvious' dots that we don't see simply for a lack of carrying capacity, but they're not going to discover concepts that do not clearly follow from the comprehensive knowledge of an era.
It seems far more likely to me that current LLMs have learned generic reasoning capabilities which will allow them to solve effectively infinite problems, rather than whatever vague limit on capabilities you seem to think there are.
Yeah, I think that's a great essay and it really helped clarify some of my own ideas.
At the same time, I think it's pretty amazing how this is the fundamental question of AI's trajectory, yet people on both sides seem so sure of their viewpoint with little actual evidence or data to back it up.
That is, it's possible that LLM-based approaches will be forever limited to this "convex hull" of current training data, and thus humans will always have a role to play (and will retain primacy when it comes to overall intelligence), because it appears that humans are not limited to this convex hull based on previous discoveries (e.g. I think basically every scientist would argue Einstein's general relativity was outside the convex hull that existed at that time).
Yet, if LLM-based approaches are eventually able to surpass humans on this "novel discovery" axis, then we as a species are basically done for when it comes to intellectual primacy - and then it becomes a very strange and scary world. And many AI researchers at the top companies clearly believe current approaches will scale and eventually (pretty soon I'll add) lead us to RSI.
I have no idea which side is right, and I'll admit I really hope LLMs eventually hit a wall. But again, I just am surprised that many seem to be putting forth their beliefs, but I would love to know if there is any good evidence one way or the other (or better yet, a more concrete and verifiable definition of the "convex hull" so that researchers could provide some falsifiable examples).
Came here to say this. What AI is doing with math is essentially fuzzing, but looking for valid inputs that solve a problem and the inputs are mathematical abstractions. It's a cool approach, but AI doesn't understand what's going on, and without understanding it won't be able to create the next generation of mathematical abstractions needed for the next generation of proofs.
No, it's not. It's a bullshit marketing term by a narcissist who wants to call everything he has a hand in "super" or "great".
The field had already settled on "artificial intelligence" (AI), "artificial general intelligence" (AGI), and "artificial super intelligence" (ASI), and while there are definite gray areas in the definitions, it was widely agreed that AI referred to the field as a whole, AGI was intelligence that was "general" in the sense that it was as good as humans in all domains, and ASI was intelligence that surpasses the best human experts.
Trump wants "SI" to replace all uses of "AI", which is simply stupid bullshit, especially given that current AI includes many levels of intelligence, most of which are far, far from "super".
aren't mathematicians needed to create the problems? Everything solved so far is a question posed by a human. Which is probably why cracking diseases is more difficult (apart from man made viruses :/)
As a software engineer it is hard for me to feel sorry for these mathematicians. Software engineers are getting laid off because of advancements made by AI. Math researchers are employed at universities, and universities aren't laying off mathematicians now that OpenAI has solved some of their puzzles. Mathematicians can still feed their families, software engineers displaced by AI can't. Yet I see far more of these blogs about the plight of mathematicians, not enough about the plight of software engineers.
When software engineers spent the last 50 years eliminating the work of clerical, secretarial, and dozens of other back-office functions, were they concerned about how those people would feed their families? Do you think anybody but other software engineers feels any sorrow for them today, now that they are facing the same fate?
That wasn't software engineers eliminating that work. And neither is it today AI eliminating software jobs. Let's name the straw man in the room: it's humans displacing humans, and extracting the value for themselves.
Just like in past history.
Yes, it sucks, for software engineers, those in the past, and those in the future. As long as you and others fall for the straw man perhaps you and others will get more and more tricked and hurt, misplacing your aggression and hatred on each other rather than those receiving most of the extracted value.
Yes in some cases, but the reduced costs also allow lower prices which benefits all consumers of whatever product is being produced. At least for those products where there is a competitive market.
While I mostly agree with you, there's an argument to be made that classical programming tended to eliminate the mechanical parts of work that people hate doing, while AI is removing the intellectually stimulating part.
It's not fun to think of a future where knowledge workers become irrelevant in society and we're back to just physical work and managerial elites.
I'm not really one of them, but there are many developers for whom the intellectually stimulating part is designing the system. Writing the code was boring drudgery that they hated doing. They tend to call themselves "architects" or something similar, not "programmers."
What an asinine comment. Do you think I, as a software engineer, am sitting around twirling my mustache, cackling as I plot to destroy livelihoods? I'm trying to make a living like anybody else. I've had my own opportunities lost to outsourcing and H-1B's since day 1.
The economist John Keynes predicted almost 100 years ago that technological advances would lead to a 15 hour workweek. If AI now makes you 5x as productive, then why aren't you working 1 day a week for the same pay? Why didn't Keynes' vision come true.
You may not have sat around twirling your moustache but you have actively been involved in an industry where the goal is to automate stuff that people would previously do manually. That is the entire point. Which, if you believe the message on many threads/posts here, puts people out of jobs.
I don't believe it is that black and white. Yes, automation eliminates some jobs, but overall moves everyone (on average) further up the productivity line, which improves lives, again, on average.
That is why we live like kings now compared to people of the past.
But circling back to the point, if AI does the same thing, eliminates expensive manual coding and other similar work, that is essentially the same thing we've been doing to other people for decades.
It is not for the same pay. Real wage increased dramatically during that time period. And despite all obsoleted jobs, society is still more or less at full employment.
There are orders of magnitude fewer mathematicians with full time academic positions than there are software engineers. The economics are different and consequently academia tends to be more slow moving in reacting to market trends. It doesn't mean it isn't coming, but it will look somewhat different
I don't personally grieve for the likes of Strogatz or Tao, it seems likely both of them will be fine however this pans out. But many more mathematicians have been toiling away in precarity and obscurity and without tenure for years and years. And of course many mathematicians went into private sector roles as software engineers, actuaries, analysts, ML engineers and similar, and have been displaced there, too.
And highly visible blog posts written by famous mathematicians with large audiences and closely aligned with the news and hype cycles getting frontpaged on HN doesn't mean that mathematicians are disproportionately whiney, nor that software engineers aren't also writing about their plight or ending up on the front page as well.
To me this goes in the same direction as people not feeling sorry for software developers because our job is supposedly automating the job of other people (which I don't think describes the profession accurately as a whole at all).
Neither sits right with me. Can't we just all collectively acknowledge that we have to grapple with this new piece of technology that either is already or at least seems to be going to be hurtful to a lot of individuals?
Instead of looking at the in comparison limited group of people currently pushing this technology and asking ourselves whether we should be on board with their doing, we look to one another and are happy that someone else is now suffering just the same as we are. What a sad state of affairs.
100% agree. The comment you are responding to is basically playing "victim Olympics".
AI is going to suck hard for lots of professions, software engineers and mathematicians included. I think it's much more productive for folks in these professions to show solidarity with each other given the way AI will disrupt their livelihoods, rather than showing disdain based on who you think may have it slightly better.
I agree with you because I so thoroughly understand what mathematicians are saying. Expertise and knowledge developed over decades, previously unique and marketable, and status conferring, are threatened in their relevance because of ai.
I know this intimately because I identify, roughly, as a programmer first and security researcher second. Guess which two fields ai companies decided to specialize in first? Yes, the FML ones.
The intellectually rewarding part of my job/skills - the only reason I enjoyed them - are, not completely gone yet, but threatened. And shrinking. This is a very scary observation.
I know from ‘isomorphic situations’ lessons in history that fighting the tide is pointless. I have to decide intelligently how to respond. Easier said than done when your livelihood and identity are at stake.
For now I still have a job but I and everyone around me is worried. In a year, At best our jobs will be very different.
So yes I empathize with these mathematicians enormously.
I don't empathize with them that much, as many of them share in the blame of creating the broken system we have today. They want the status that comes with tenure and having a nice math job, and then pull up the ladder for everyone else.
Math journals commonly reject papers as “out of scope” or “not of sufficient interest” without an editor or referee actually reading them. So it’s not as if the “math community” was checking these papers for accuracy in the first place. Now AI speeds up the creation of papers that go unread, and we’re supposed to pity mathematicians? If journals and experts can’t be bothered to read the papers, why should authors bother to write them without AI? And suddenly mathematicians are worried about AI papers being “too hard to read,” even though that wasn’t a concern pre-AI, when no one was reading them anyway. Too bad. The incentive structure now points toward AI, and mathematicians share the blame for that.
As a software engineer? The career whose very reason, largely, for existing in this market is to automate jobs, with few notable exceptions? That one is somehow deserving of sympathy and that of mathematicians is not? Come off it. The notion is undignified.
Are phones an "automation of existing jobs"? Maybe couriers (??), but most of what people use them for goes far beyond that.
What about the software used to drive trains or autopilot planes? We still have conductors and pilots, but now the outcomes are safer on average and more efficient.
Or something like Google Maps -- all that replaced was the paper booklet people used to keep in their cars, it's not like we used to employ 'navigators' to go out on a road trip.
Or something like Intuit QuickBooks: there's still an accountant in the loop, and if anything more accountants today than there were a few decades ago.
Certainly some software is essentially automatory in nature. But to characterize 'nearly all' as being such just betrays a lack of awareness for the rest of the field.
> That one is somehow deserving of sympathy and that of mathematicians is not
He didn't say that mathematicians aren't deserving, he was griping that people said this about SWEs. Do you think mathematicians are deserving but SWEs aren't?
You seem to be unaware of the existence switchboard operators (people redirecting calls to the correct person) were hundreds of thousands in the US alone at some point.
How did you manage describe a litany of automation while being fully unaware that is what you were doing? I mean you even said the word autopilot lmao. Yes, not all tasks in a job are fully automatable. iPhones, automating communication. Quickbooks and turbotax, automating business workflow steps.
The words and excels of the world, automating documentation and bookkeepery. It all makes some kind of existing thing we already do faster and more convenient, and often more efficient. Most software simply does nothing new. The thing you didnt manage to list which is the actually good example is games.
> Software engineers are getting laid off because of advancements made by AI.
The bean counters laying off software engineers ought to take notes from the frontier labs' own hiring practices. They're still hiring lots of humans for business and engineering roles even when they would be better positioned than anyone to automate away those jobs with their world-class models (https://openai.com/careers/search/).
It's not about universities, it's about grant funding over the next few years. It is very apt to collapse if AI is more useful than people. Why pay people to find this when tokens will do instead.
I'll let you in on a secret: In the US, most math professors don't seek grants. In most universities, I don't think grants are even a factor in their tenure application. It's quite different from engineering.
In my undergrad math department, only 2-3 faculty members out of a dozen bothered with grants.
Who will continue to develop neural nets if the mathematics training pipeline dries up? Will we see the frontier labs relinquish AI research to the machines themselves while their humans apply for unemployment benefits?
Currently math research serves both as a training pipeline and as a math-production pipeline, and its funding is set based on those two outputs. If it is "just" a training pipeline, do we need as many mathematicians?
Who has been funding grants, and do you really think the grants are going to stop? There are infinite number of puzzles in Mathematics. The grants are not from companies like Boeing and Walmart who urgently need solutions for the problems, if they were I can understand why they might stop.
Software engineers are getting laid off because of the end of zero interest rates, wars started by Putin and Trump, and a bit because of pandemic over hiring. The part that is due to AI is less hiring of juniors. Know your enemy before starting a fight.
What's your source for "Software engineers are getting laid off because of advancements made by AI". I haven't experienced that in my field and can't find any data to support that notion.
The idea that a junior software engineer who can't get a software job _and also can't get any other kind of job or move somewhere cheaper to live_ to the point that they can't afford to feed their families is what's unrealistic.
It's not like the options are write software or starve. Billions of people find jobs and can feed their families without writing software.
I also don't often meet junior software engineers who actually have families to feed. Most juniors work in the industry for a few years before deciding to get married and have kids.
Unfortunately this often means moving away from family. What are you going to say to the kid whose parents toiled to get him through college on the expectation that he would be able to take care of them? "Sorry Mom, I have to move to North Dakota to work on the oilfields"?
I think your expectations of who gets to do math is uncalibrated. One famous math researcher who solved a major open problem was working at Subway while still doing math on the side.
A very small fraction of people who get math degrees actually continue to do math in their careers. They usually have to get other jobs in order to survive (actuary, teacher, finance, etc.). Academic math funding was already drying up for last few years, and now AI will probably cause more cuts. Not to mention that academic math positions have never paid particularly well.
I feel bad for anyone who can't find a job, but the software industry as a whole has been the cushiest industry for decades, and the current turbulence seems more temporary (and driven by corporate greed) than a systematic destruction of the trade.
It’s only been a few months since LLMs started solving significant open problems in math. I wouldn’t jump to the conclusion that they have it better than software engineers.
In many ways it will likely be the opposite. Tech firms still need engineers to talk to customers and run experiments. What will the mathematicians be needed for?
> Software engineers are getting laid off because of advancements made by AI.
This is said by a lot of CEOs trying to make layoffs look good, but I haven’t seen anything that suggests that it’s true.
> Yet I see far more of these blogs about the plight of mathematicians, not enough about the plight of software engineers.
Really?!? We’ve been living through 3 years of blog posts, LinkedIn hot takes, and podcasts talking about the plight of software engineers in the era of AI. There isn’t a week that goes by without an HN post about it.
I can feel empathy for the practitioners in a field that may have been profoundly changed, the same way I can feel empathy toward software engineers who are struggling with the changes in their own field.
In the end, we’re all trying to figure out where this leads. And how this will affect our lives and our work.
The difference is it takes great intelligence to be a mathematician, whereas anyone can be trained to write code. It’s like the difference between an automotive engineer and a mechanic. I would argue writing code takes less skill than carpentry.
Do you write much code? Do you design software? It sounds like you don't. The challenge with software is managing complexity. This is unique to this field. Mathematicians are humans.
Unless this is satire, have you not considered how many proles have been laid off because they were replaced by technology or told that a spreadsheet model had decided their job was superfluous? Capitalism has been using technology to fond new ways to enrich itself for the whole age of personal computing, and arguably long before. Those Andreesen Horowitz comments about 'software eating the world' may have been exciting to people working at startups, but came off as rather sinister to people who were in the world outside softwareand just wanted to build a modest but comfortable life rather than develop unicorns and cash out.
> I am not one of those AI people who believes that hundreds of millions of jobs will be wiped clean from existence in the next five years
"Wiped clean" is the wrong phrasing, and 5 years is a short timeframe given the ability of some sectors of the economy to resist change. But yes, hundreds of millions of white collar professionals will need to rethink their careers and their business models within the foreseeable future.
Just today there was an HN article about how shady the dental industry is. In the future, you will go to a special dental imaging facility where a modestly-compensated technician images your whole mouth and feeds the results into an AI system, and when you get home you can look at the assessment and chat with the AI about it. Then you can potentially go visit a dentist, oral surgeon, etc, if you actually need work. This system will produce far better dental health outcomes at far lower cost: the diagnoses will be more accurate, and there will be no bias towards additional, unnecessary procedures.
> In the future, you will go to a special dental imaging facility where a modestly-compensated technician images your whole mouth and feeds the results into an AI system, and when you get home you can look at the assessment and chat with the AI about it. Then you can potentially go visit a dentist, oral surgeon, etc, if you actually need work. This system will produce far better dental health outcomes at far lower cost: the diagnoses will be more accurate, and there will be no bias towards additional, unnecessary procedures.
Right up until the sellers of dental supplies find a way to...
Indirectly influence model training to increase the frequency with which models recommend dental work and/or lower the bar for what they determine to be "necessary"...
or...
Directly influence model recommendations by paying model providers (who will be happy to take that money).
The point being that models (and particularly the humans that control them) are presumably just as corruptible as the humans they are replacing.
The advantage is scale. If a dentist is shady and overprescribes it can be difficult to find out because there are not many victims. If a company serving millions of people is overprescribing there’s a much higher chance of them getting called out on it.
That's the optimistic take. I have a much more pessimistic take: there is now only one place where people who want to sell dental devices have to focus their efforts. Less effort for them... and because of the scale... much bigger payoff.
When I was thinking of the examples I gave (poison the training and/or pay the providers), I was intentionally drawing analogs to how I saw Google search decline.
First, SEO "experts" poisoned the results. Then Google saw how much money could be made, and they sold the ad space directly to marketers. If you search for "Best <widget> for <doing something>" on Google, I am certain you are not getting the "correct" answer. And Google has suffered no consequences for this. To the contrary: it has been extremely lucrative.
Unfortunately in the real world it's just the opposite. A guy selling opioids on the street is called a dealer and gets arrested. A company overprescribing opioids to millions is called Purdue Pharma and gets record profits.
People are now aware of the harms Purdue has caused. Justice has not been served, but their plot has been uncovered and discerning consumers need not engage with them.
Journalists protect consumers from anti consumer practices. There’s not enough money in journalism to go after individual dentists even though many are bad actors. There certainly is enough to go after corporations, which we have seen many times. I’m not saying justice will be served, but I do think more public oversight = better informed consumers. There really is no good way to know if a dentist is honest before going to see them. That’s just not true when it comes to mega corps.
No one wants that. People pay other people to do things well. No one wants to need to discuss orthodontics with a bot, they would rather pay someone to know those things and think about them.
I happen to be (re)reading Strogatz' book "Nonlinear Dynamics and Chaos" right now. It's beautiful thing, and nothing in the "age of AI" threatens to equal it or in any way diminish my enjoyment of it.
I think the point is that in the "age of AI" virtually nobody will have the opportunity to follow a career and produce works like Strogatz was able to do.
I'm not disagreeing with the post. My personal opinion is that LLM "thinking" is qualitatively different to human thinking, that the "convex hull of ideas" idea is probably right in some sense, that the social structures we have built to allow and encourage human intellectual exploration are not obsolete and that there is a high risk of catastrophic damage being done to them by those who fail to see this.
It's really a fascinating book and topic. And i too enjoy appreciating it. I guess it's appreciating the beauty of nature and as you said, AI will not be able to threaten that.
Hacker News is so depressing these days. One doomer post per day to keep the valuations up.
Nothing has changed in math. Math is not solved nor has the human been kicked out of the loop. Same with programming.
*I said nothing really changed because LLMs becoming superhuman at deduction was priced in with o3 and neither the peer review, training nor publishing methods will get a revamp beyond what could already be done with SQLite and a website.
I'm no mathematician, but my impression as someone who at least does a lot of applied work with maths is that the main thing that's changed is that the yardstick by which academics earn recognition is being re-assessed.
But from my perspective proofs have never been the most interesting part of maths. What really interests me is the intuition that leads to new applications, and to new questions. Newton, for example, gets more credit for calculus than Leibniz because he's the one who demonstrated how darned useful it is. Erdos is arguably more famous for asking interesting questions than for answering them.
I still don't see LLMs seriously contributing to that facet of mathematics any time soon. Every so often I ask one to suggest a good binomial confidence interval for a particular application, and it still tends to just suggest one of the subpar options that everyone learns in a first year stats course.
Proving theorems is one of the main ways that mathematicians turn math into money.
The recent barf-out done by LLM companies is useful if what are you are interested in is an unreadable mess and someone giving you the final answer. You might be able to extract some new "aha!" moments out of it, but we will have to wait and see what happens once people go through it. I would hypothesize the LLM output will be good at hinting at some non-obvious relationships and patterns, but I doubt it will yield anything of substance in its own right.
Historically, proving difficult theorems required mathematicians to sometimes invent new fields, new ways of looking at problems, the discovery of which itself yielded incalculable contributions to both mathematics and other fields. It is this that is the greater contribution of and to mathematics.
> Proving theorems is one of the main ways that mathematicians turn math into money.
Mathematicians working at universities are paid to teach, to engage in the politics and business of government contracting, and only in very rare circumstances to play prestige games on famous problems.
It is unlikely that Congress will decide to slash budget for NSF's MPS directorate on account of AI getting good at proving theorems. That's just not how Congress works.
It is also unlikely that LLMs will be able to do, in the near future, do the sort of applied mathematics that happens in the context of occasional NIH or DoD funding that flows into Mathematics departments, for a variety of complex sociological reasons I'm happy to expound upon.
And it's unlikely Engineering degree programs will stop requiring calculus sequences, since being able to do the computation was has not been the actual point of teaching calc since he late 20th century (if ever).
So, the good news is that I doubt any of this will have much of an effect on the job market for Mathematicians who want to prove theorems for a living. The bad news, of course, is that the job market for Mathematicians who want to prove theorems for a living was already pretty horrendous.
I’m on your side on this overall, I think. And I suspect we’d agree more than disagree on nuances. But saying that ‘nothing has changed’ is very obviously wrong.
I guess I chose the words "nothing has changed" because that's true relative to all the subtext of content I'm seeing online claiming that "math is solved"
It was, but the comments on that post were insanely depressing to read. I'm a mathematician, and find myself largely hopeful and excited for its future, but being online this last week has been incredibly brutal due to the surrounding vitriol. It's an avalanche of negativity, and I'm largely not talking about that of my peers (though indeed, some of my peers). Most mathematicians I've spoken have been reasonable / positive, perhaps an effect of my field.
I really need to just stay off social media for like a month and keep talking IRL. Longer, if I know what's good for me.
I agree with "nothing has changed" in practical terms. If you take a look around most things rely on an undergraduate level of math than whatever is at the frontier. The proofs may be important, but they are really remote to people's lives.
This is like saying WW2 didnt happen while you were living in a Soviet village in 1940s, and you were doing the same thing, and nothing seemed to change
And how! All the “nothing has changed, stop whining posts” are neglecting that they haven’t had to go look for a job. My previous company became quite toxic about LLMs and employee usage. It’s not “move over to the paradigm or get left behind” instead it was “jump on the hype-wagon and sing LLM’s praises all day, every day or you’re out”.
ZIRP is over but rates are not especially high and yet I don’t remember job market being that bad. Maybe employers are taking wait and see approach and avoid hiring developers they expect to layoff in a year or two if models will continue to improve and becoming cheaper at the current pace.
Not that it explains all job market weirdness but you do have a problem on the hiring side with having to deal with the flood of AI generated applications. I assume that’s going to have a slowing effect on the typical apply/interview/hire process.
I think that models will improve and get cheaper. I think we'll see Opus 5.5 tier models open weights in six months.
You are probably right that some companies are factoring AI into headcount but I'd imagine that the interest rate and energy crisis are bigger concerns
It was long predicted that the sign that AI is here in macroeconomics is rising interest rates, because investors can get MUCH higher rates of return doing AI investments than borrowing governments, why would I lend money to Greece when I can lend it to Anthropic?
I own my company and directly participate hiring. What’s changed among me and my peers is that we’re on incredibly precarious economic grounds with no telling what’s going to happen next month.
LLMs haven’t done much in the way of informing me or others I know regarding “who and how many”.
I would wager that most of them are AI generated. The cost of generating a neat looking LaTeX resume and adding vibe-projects to one's GitHub is very low...
The doom posts will continue until profits are up.
Jokes aside, its like working for a company, individually you can lift mountains and someone else will take credit. When you use LLMs for a single bit, all credit goes to LLMs lol
Mathematics as a field of study is maybe 2500 years old and a lot has been figured out in that time. But look how far AIs have come in just ten years. I wouldn't expect progress to stop now, would you? A decade from now I wouldn't be surprised if machine mathematicians are essentially oracles. They tell us the answer but can't explain it to us any more than you could explain long division to a golden retriever.
yeah nothing changed in math. mathematicians are weeping for some strange reason (maybe paid by OpenAI to weep?). And Terry Tao is posting blogs on AI + Mathematics because he was bored (or because he was also paid by OpenAI).
> I think I have been underrating the pace of model improvements. In March 2025 I made a bet with Tamay Besiroglu, cofounder of RL environment company Mechanize, that AI tools would not be able to autonomously produce papers I judge to be at a level comparable to that of the best few papers published in 2025, at comparable cost to human experts, by 2030. I gave him 3:1 odds at the time; I now expect to lose this bet.
“I’ve slowly updated my timelines over the past year…”
and
“I think I have been underrating the pace of model improvements. … I was not correctly calibrated as to the capabilities of existing models, let alone near-future models.”
I don't get the argument - if we follow what you say, they're not _overreacting_, they are reacting appropriately but should have done so earlier (when o3 was released) rather than now?
My claim is that if Bob, a mathematician, believes that he is rendered useless by an LLM then he did not play with o3 and all the other models between now and then to realize that, yes, they are superhuman at details and deduction but not superhuman at math
AI will not replace the mathematical community (peer review, abstraction creation) until AI can train the next AI purely on data generated by itself and with a tool in a turing complete environment. The environment has to be air-gapped from human data but the AI can use tools and its own outputs
I think that "solving math" is isomorphic to solving the "thinking part" of the scientific process which would be genuine RSI
This was the top comments when the submission disappeared from the front page.
Now the thread has been cleaned up and a pro-defeatism comment is the top comment and the submission is on the front page again (I'm not alleging moderator intervention, user mechanisms are sufficient).
It's probably because with the release of each model it is becoming more and more difficult to see how a person's livelihood and way of life continues. Uncertainty stacked on uncertainty.
How will it pan out, no one really knows. The leading frontier labs are continuing to hire aggressively. But who knows for how long or for long or near-term goals.
The open ai math dump was actually what got me off the singularity wagon.
There were mistakes that could easily be seen with a diagram rendering three big results wrong, exposition is terrible, no coherent underlying theory generated from solving so many problems, as Thomas Bloom pointed out even duplicate proofs of the same statement in a single paper.
It clicked for me, "this is slop but slop can be superhuman and useful!". Similar to programming where it can generate as many criticisms of your codebase but you have to weed them out.
If anything, that makes the case that math is fucked even stronger. Nobody had any idea of how to prove quasi-Riemann. OpenAI produced three proofs, one of which was just a lemma in solving another problem. They have a machine that is better than every human being alive at math that solves problems humans cannot even when working inefficiently.
Math was never only about deduction. It's also about data curation.
I was taking to my friend in OA who was really surprised that the proof of L(F_2) = L(F_3) could be done in the 1980s. We just didn't have a superhuman deducer back then.
You use the word "already" as if it already happened, but this is confusing speculation with reality. Market prices are a summary of speculation. Plenty of things are "priced in" that haven't happened yet, and maybe won't if the market is wrong.
Anticipating that something will happen is not the same as evidence that it's happened.
I was on Twitter around 2024-2025 and showed both Daniel and my advisor at UCSD sota results I created in group theory using Gemini 2.5 (edited) Pro in Q2 2025.
A lot of us knew what was coming. Profs I speak to these days say it's the same old except they need to pick a different kind of problem for grad students
the above, a "sorting" machine is the primary usage of hyperscalers currently, they're just trying to find new uses to boot for extra money/legitimation.
the browser industry
the phone industry
the CAs
JavaScript
it's all paid for by those wanting to sort humanity into whatever bucket the can fear monger the peasantry into for aweek or however long it takes the classification system to sort out.
gentrification works, but we chose forcible displacement now we have to live with the forever consequences.
would you give guns and bombs to the displaced native americans who taste blood over their beads and trinkets?
we've been inside the event horizon of the singularity for years, the tax bill will only get worse unless we recognize it for the tick that it is.
> One doomer post per day to keep the valuations up.
Yes, outsiders (investors, journalists) are given a false impression while insiders (software engineers) are supposed to give up. Double win for the agitprop machine.
Sometimes you think you’ve been accused of everything you could be accused of. Wonders never cease!
> The "moderation by less", as well as the guidelines, are both farcical in nature. They've given way to ruination of an otherwise decent tech discussion forum
The guidelines have been constant (with minor updates) for most of the two decades HN has existed. Dan’s a moderator for 14 years and I’ve been one for several. Both of us have been active participants almost since the beginning. When did the ruination begin?
Your comment seems a generic vent, largely unrelated to the parent comment but if you have any actionable suggestions you’re welcome to email us.
> Your comment seems a generic vent, largely unrelated to the parent comment
There it is, the "off topic" play. Well done, Tom.
If you'd put anywhere near the amount of attention in moderating the site for over all quality that you put into my "largely unrelated" comment, this entire comment thread would not exist. Think it over.
Please don’t insinuate that I’m “playing”. It’s a slimy tactic, not an earnest contribution to a conversation. You opened this subthread with an accusation that we’re putting our thumb on the scale for the CCP. Yes, it breaks the guidelines, along with basic decency. The only reason this site still exists for people to bemoan that it’s not as good as it used to be (which people have been saying since about three months after it started) is that we have guidelines and uphold them in a way that people can trust, often against pressure from many different people with all kinds of agendas. As I said, if you have any actionable suggestions you’re welcome to email them but all I see is someone complaining about quality whilst having no concern for the quality of their own contributions here. If you don’t like the guidelines you should take your own advice and find a site that’s more to your liking.
What I was aiming for was to point out the oddity of dang selecting the one odd ball thread to tell everyone to play nice, in what is otherwise a site that is daily filled with AI doomer posts, as per the opening comment of this thread to which I responded. And I did mention that I found it amusing - because I doubt either of you are actually shilling for CCP. This wasn't even the main point of what I was after.
That neither of you are at the very least bothering to steer the greater discourse towards some higher ground is a lack of balance that I am highlighting.
To remark my comment as off topic in the context of the thread opener is to dismiss what I am saying using a very lazy tactic that I've seen used countless times before. No, it is not off topic. It is very much on topic and relevant.
And if you have a problem with my comment, feel free to email me.
We’re posting mod comments to uphold the guidelines every day in threads about all kinds of topics, which anyone can see by looking at our comment feeds. There are posts about AI all the time here because it’s the biggest topic in technology and even the world right now. Some topics/discussions are negative and many are very positive, reflecting the broader discourse. We’re curating the content all the time to provide a good balance of topics and perspectives. By statistical trends and what we observe in the quality of the best posts and discussions, HN is the healthiest it’s been in years.
But one of your central complaints was about the guidelines, which have been constant for nearly two decades, so we’re just not having a serious conversation here. We’re always looking for signs that indicate if and how we can be doing better, but this interaction has nothing to grab onto.
FWIW as someone in silicon valley, this really isn't the silicon valley mindset. HN represents a sort of messy combination of many different tech tribes, but I will say its depiction of silicon valley is largely that of someone who reads too much tabloid coverage of it. It's very much the feeling one gets about silicon valley by reading Techcrunch articles without actually being here.
I actually think it's a bit dangerous to try and understand SV from HN comments, in the same way that trying to learn about an event by watching a fictional adaptation of it can often confuse more than educate. The goal of the comments here aren't for SV to chat anymore and it hasn't been for a very long time, for better or for worse.
Twitter is closer to what you'll get among founder and hype circles in silicon valley, but even that isn't a good pulse of what engineers here actually think. The public web is just too adversarial a place for most folks to want to hang out anymore and silicon valley has been so deeply politicized in the news that a lot of people just don't want to air their opinions in public anymore. My suggestion is to save up some money and just come here and visit for a week or two, go to events, rant to a few graybeards, that kind of thing. You'll get a much more authentic look.
I need to do some more research on this. Because I've always been told that AI, more specifically LLMs, are "glorified autocompletes". Yes they have huge data sets but ultimately they can only provide answers based on data that they were trained on.
So how can they possibly solve problems that humans have worked for decades or centuries on without being able to solve? How did they find something new? Is it that they were able to deduce the solution by examining all current data sets? Or simply brute force it until the produced solution passed all necessary tests?
The following certainly does not describe all mathematics or all LLM operations, but its my mental model for why LLMs are so effective in several fields.
Lots of mathematics entails recognizing that a tool/techniques that works in one area can be adapted to a completely different problem. For a human to do this requires having a robust mental model of the technique and the problem itself, and probably having just the "right" mental model to see how they fit together. And we are talking about extremely hard/sophisticated techniques and extremely hard/sophisticated problems, so the ability to deeply appreciate either can require years of study. And on top of that, even if the mathematician has sufficient understanding to develop the intuition that the technique might work, they will need profound fortitude, skill, and patience to see the solution through.
Unlike human mathematicians, who will develop a deep understanding of only a relative handful of techniques and problems in their lifetime, LLMs have ALL written techniques, problems, solutions, etc stored in their internal representations. So when they ably match a problem to a technique, we can imagine the operation as a supercharged autocomplete, or even an extremely proficient search engine. Their stroke of genius is really a consequence of capacity, moreso than cleverness.
Yes this is it. So many breakthroughs in math (and other sciences) have been like "I went to a conference and happened to mention my work to someone in a completely different area/field and they said: well that sounds like something i just noticed in my research".
Combining spread out ideas is very powerful and LLMs are great at that.
They have access to a much larger corpus of information than any single human can manage in his one brain. No human mathematician reads all the papers and has perfect recall on them. The answers have always been there, but no individual has been able to make all the right connections to pull them together and formalize a new discovery or proof.
> Is it that they were able to deduce the solution by examining all current data sets? Or simply brute force it until the produced solution passed all necessary tests?
Yes, both of these + an infinite token/compute budget and SOTA+ models. It's not like some hobbyist out in the woods with a GPU running Qwen 27B can do this.
Yeah. "Glorified autocomplete" might be all the human brain is too. "Glorified autocomplete" can do an amazing number of things, including perhaps pass the Turing test and perhaps ... "think".
"Glorified autocomplete" is meant to describe the _underlying mechanism_, but unfortunately it also sounds like one of the things LLMs are actually used for, so it gives the mistaken impression that the claim is all they are good for is a component inside an editor application.
>Yes, some are crying about bruised egos, the feeling of wasted years, a creeping sense of status obliteration, the frustration of an unfair competitor, the indignation of feeling cheated and mistreated, and uncertainty about the future. But they are also mourning...
This nails it, however IMHO its almost all bruised egos and feeling of wasted time. I don't know the ratio of A:B but I also don't know anyone that is in the latter category of feeling doom and mourning that their craft is about to regress.
I don't know what's happening exactly, but I am genuinely astonished how little people seem to care about people losing something real, dismissing it as "bruised egos", using petty terms like "whining" and "crying", or calling them some insult for the crime of being angry or sad for having a big chunk of their life pulled out from under them. I'm referring to the big picture of common talking points, not just this case.
I guess I'm not astonished that people are so bad to each other in general, but more that in the case of genAI it's so prevalent in places that seemed more elevated previously.
Part of me wonders if this is actually a form of self-defense. The misery of others can provoke sympathy or the opposite: Angry pushback to protect oneself from dwelling on their misery.
Similarly, I got into software because I like working with computers and solving problems for others. I see TONS of similar sentiment where people are almost joyously exclaiming “Software is solved! Sorry!!! It’s been a long time coming, you all deserved it, get fucked!”
And I can’t imagine feeling the same about any group of people. It’s been odd, to me. I’m just trying to support myself and the people in my life, and now there’s a looming fear that I can’t do that anymore… business owners are drooling over that, and other sectors are laughing.
All my life, when I would think back to what mill towns went through when they all closed (something I did not live through), all my naive outsider brain could arrive at was, "Yeah, but those were just grunge jobs, humans were freed up to do better stuff."
And perhaps the same thing is now happening to me. It feels different, but maybe the same thing is true.
Tech workers are a small subset of the population that have been able to live very privileged lives when compared to everyone else.
I think the average person lives a much harder life and has been doing so for a while. Should we really be surprised by the lack of empathy when folks in tech had little to show for the common man this whole time?
I saw something similar on Twitter responding to 45yo software engineers making comments of depair in that HN thread about the increased suicide rate of middle-aged men.
For some reason I'm stuck in a real depressing, doomer algorithm / homepage on Twitter. But everyone on my homepage seems to have a take on the HNer comments along the lines of "see, this is why you don't make your job into your identity".
But that's such an antihuman response. The only reason you can put decades into a field to grind and build expertise is because you have an identity-level interest in a subject.
It's also not a good thing if the optimal way to live going forward is to have no passion for anything lest it changes so that we won't feel bad. I don't think that's possible unless you remove the human part of our brains or make life so horrible that we're happy just to survive another day.
Source? There has not been a statistically-significant increase in middle-aged male suicide rates between 2023 and 2024 confirmed data [1]. (2025 breakouts aren’t out yet, but top-line data show decreases.)
In 2025 in the UK men age 45–64 had a suicide rate of about 24 per 100,000, which is the highest it has been in the past half century, and is higher than other age categories. In the mid 2000s, the rate was like 17 per 100,000.
From your link, in the USA the overall rate for men of the same age is about 30 per 100,000, but varies dramatically between states.
Lots of people have lost their jobs and had to overhaul their lives due to layoffs or technological advances or political choices or factories moving overseas or whatever.
They are showing as much care for mathematicians as they received themselves, which is not much.
i think a big part of the backlash isn't about math per se and it's about what 'work' or 'a job' are supposed to be.
when mathematicians complaining about losing the part of the job that they love, people think "if you love doing it, why is anybody paying you? you're describing a hobby."
Yeah, the glorious utopian future we're building where humans are only able to make a living doing things they hate, and money is concentrated with the few.
Because we don’t see it as something for them to be legitimately upset about, except that it’s going to disturb their ivory tower unserious profession.
If AI got significantly better at performing surgery overnight and doctors started crying about how they were going to be replaced, what do you think the public response would be? Non-doctors want the best possible care; they aren’t interested in protecting an industry. The same will be said for pilots and lawyers, too.
People dislike self-serving whingeing, end of story, but that’s doubly true when it comes out of a profession (theoretical mathematics) that is mostly practically useless.
The doom and mourning is that it's no longer their craft. Their involvement is on the threshold of no longer being useful. What does it mean for humans when the best their involvement accomplish is slowing down progress?
This is a fundamental human need and it's being taken from everyone eventually, by design. It will take a generation to turn over and find new ways of being useful. Those who ran into this wall and could not safely reduce their income and retire/pivot to other fields, will be the coal miner oxy crisis of this generation.
Mourning is fine if it is temporary, however no man should dread their own future if they still have control of their life. This is the idea of unalienable rights and was enshrined hundreds of years ago. It seems to me that anyone dreading the future of their field failed to take a page out of their retirement strategy and diversify their own interests, i.e. https://en.wikipedia.org/wiki/Polymath
Ok, so if you really believe that losing one's vocation at 25 is the same as a 75 year old losing their spouse, that paper suggests you should wait at least 6 months before you start being a jerk about it.
I don't see what point you are trying to make. At 25 losing your vocation gives you more than plenty of time to transition into another lifestyle. Many people will consider this the very beginning of their career. At 75, retirement is near. Your legacy is known but not final. 6 months is less than 1% of your lived life, and you have a lifetime of memories to reflect and enjoy.
I'm not entirely sure what you're on about. Are you suggesting that mathematicians should become truck drivers or plumbers and be happy with that (until the next physical-aware models release, and then...)?
I was talking to a mathematician friend about this, and she was telling me that a lot of the proofs that have been produced in the past couple weeks don't have an application.
To me it begs the question: in a theoretical world where we don't have human mathematicians anymore, what is the point of these proofs if there's no human who actually understands it?
If the point of a lot of this math is only to expand the boundaries of human knowledge, it all seems kind of pointless.
heh these models should be turned loose on all the proofs paid for and sitting on the shelf and see if they can find an actual application useful to mankind.
Often, we don't know the applications of new math. For a long period of time we had "useless", "purely theoretical" mathematics that eventually turned into relativity and quantum mechanics.
Proofs are somewhat like legos. A bigger toolbox of them can only help in solving other, possibly practical, problems.
As far as the utility of a human being involved -- In a non-theoretical world where we don't have feline mathematicians, what is the point of these proofs if there is no cat which actually understands it? The cats can benefit from, for example, advances to optimziation algorithms that reduce costs for cat food delivery, of course.
Is AI at the stage that it can also point out all interesting things to study and then solve them? Otherwise humans would still be able and needed for progress.
Good question. I'm not a professional mathematician, but I would be interested in seeing what results the mathematicians get from trying to prompt it to come up with new and interesting problems.
My intuition is that it'll be at 'smart undergrad' levels today, though as the cryptographers say, attacks always get better.
None of them have pointed to any of the solutions and said "this is real innovation" - because it's all been mining existing ideas and applying them elsewhere.
Imagine you spend years working on a solution to one of these problems. Only to realize after you publish that more than half your answer already existed in another branch of mathematics and you spent a bunch of time re-inventing it.
Yes the first big solve by OAI was "stolen" and that is tragic, and pretty bad. But it looks like this new batch, the one people are crying about, has none of that stink.
> Today, the mathematicians are weeping, because the machine smashed their tower first. But tomorrow, it may be the physicists, the economists, the illustrators, or the writers who weep over their fallen blocks.
Isn't it extraordinary that these models still can't write well? I mean, they write passably well, but they all have a highly characteristic style that you can spot from a mile away. These language models can crack the toughest open problems in math, but they haven't yet cracked human-like writing, especially long-form writing, without exposing themselves. This is very very odd, IMO.
Or, just maybe, they share a key property with the mounds of popular writing, journalism and random internet accretions they were trained on: they seem total rubbish on anything you know about deeply, but brilliant on everything else.
Their math is as just as good as their rambling writing and their Ghiblified poster art. Most people don't seem to realize that.
I think it is. Why is it that the only people in any field I'm aware of that think the AI output in their field is high quality are either the ones that are carefully crafting the output themselves or were producing/consuming slop themselves before all this started? (Outsiders pretty uniformly like the output, which is why I drew the parallel to Gell-Mann amnesia.)
The difference here obviously being that the part that you mention is more embedded in the proofs (long and difficult and near-hallucinatory) but the results that matter are that the proofs are correct and validated. So I think you're failing to account for that, the problem for math is that the journey matters less than the destination.
Hard disagree. If we send someone out exploring and they discover something (maybe?) but their account of their journey is impossible for anyone else to follow and contains obvious hallucinations, what have we gained?
Isn't the problem more like they did discover a new land (as in, the proofs are true, and the solutions are validateable) but we have no idea how to make heads or tails of how they discovered it.
Math and programming languages are formal. Meaning they don't have any ambiguity in them. Every statement is what it is. That means they are very rigid and there's not a lot of variance. Natural languages are more diffuse and a written piece is more the reflection of the author than anything else.
That is why it's so disturbing. After several paragraphs, you realize there's nothing human there even when the actual information is present. People that do create AI piece are OK with it because they already have the context, but for the reader, it's always the uncanny valley.
Writing is a very slippery thing. “Good” writing from 200 years ago reads as unbelievably dense and overwrought to most people today. So even if you made an LLM that was “good” at writing, how long would it stay that way? AI systems can put out more writing per day than all of humanity combined, meaning it can also generate and beat to death cliches much more efficiently.
Language models don't have a characteristic style you can spot from a mile away. They haven't had it since before ChatGPT.
What is true, is that chat models (and by extension agents) have a very characteristic style after their LLM went through post-training. But the pre-trained language model doesn't have that style.
So if you want to write well with an LLM, you could and it is a well understood problem. It is more a matter of that not being of much interest, as it competes with having the model be good for chat interfaces.
How to have a well-aligned, RL'ed, safe or thinking model, which also writes well in long-form, is a bit an open problem. Though presumably agents could do it when they have access to a pretrained model.
Why is writing in the style of a corporate marketing department a requirement of being good for chat interfaces? Why can't they tune them to sound more like a normal human?
They probably can tune for it, but it is not what the shareholders want anyway. I am unaware of any frontier lab who is actually interested in building something like that, long form proza indistinguishable from humans. Automated code and low hallucination rates are more important topics, and trade-offs need to be made.
Edit: why the style is like marketing corp-speak has to do with personalization not working well yet. So the default vanilla style is very corp. Most chatbots allow you to change the system prompt though and that can get rid of a lot of these issues in a lot of chatbots. But it is hard to maintain such a system prompt as the bots change so quickly.
There's no money in it. There's money in programming, and math is a good target to give the models superhuman reasoning abilities, but it already writes well enough for middle managers to write all of their emails.
>Isn't it extraordinary that these models still can't write well?
It's not at all extraordinary if you understand how they're trained. For math and programming it's possible to generate an infinite amount of training data with correct answers, as compilers and theorem provers can guarantee correctness. For writing, the quality is not objective, it's determined by human preference, so you need humans in the loop, which slows things down massively and limits the amount of data that can be generated.
You can just tell them to adopt a different style. To make sure, add another pass with an agent that flags and/or fixes LLM tics. Humans typically do several passes over writings that they intend to publish, why are we expecting agents to do it in one shot? Just like the superpowers skill for software development does a spec phase, planning phase, several review rounds, etc., if you do the same with writing you get very passable results.
I'm convinced the people who do not see what Mathematicians see simply have not considered that their jobs are just as vulnerable. The day when OpenAI replaces their own AI researchers with GPT will be upon us, and the response from the Mathematicians will be "First time?"
As usual the author is ending on a completely unrealistic positive note: "humans may come to value more highly our capacity to relate and care for one another".
Does relating to someone pay my bills? People will use the informational tools to improve their social status and resource accumulation, there is no doubt about this.
I am not quite saying "if you can't beat 'em, join 'em", but I know more math now than in any point in my life because I want to understand how LLMs work.
I understand there are many types of math and the mathematicians don't all want to sit around and do KV cache and gradient descent things, but surely a percentage are as interested as I am and can contribute to this new emerging field? I would wager that Xl+1=Bl Xl+Cl ℱl (Al Xl) makes much more sense in their mind than mine.
This is the best attitude to have about the current state. Unless you have some dramatically different sensible solution (and you probably don't when it comes to technology, because it seems to spread for reasons that would eliminate other sensible solutions) that will catch on and take over and replace AI: it's going to continue to mature as long as it's mostly useful to everyone. And AI isn't even a new industry, it has been maturing for quite a while, which adds more weight to the argument that having an attitude that adapts to technology (unless you wish to go solo) is a 'better' fit.
The dramatic solution can never be: 'we should all just want to do it this way because it felt better to me'. That never seems to work, from what I've seen. So there is a lot of public discourse, arguing, to determine if 'doing things the old way' is perhaps more better. And if not, maybe we just shed a tear and move on (to look for a better fit).
I felt the article deeply, but I don't think learning anything is a waste of time. You contribute to the world by helping everyone around you in any way you can, the river is beautiful because it flows.
As a mathematician, I will never understand how my colleagues had never foreseen this. Okay, maybe you can believe in a plateau but at least accept as a possibility that there won't be one. This applies to every other human endeavour.
I am also sad, but my sadness is, in a way, priced in.
It was really quick. I stumbled across a comment from KConrad on MathOverflow on how useless AI was for research. That comment was from 2024. When it disproved the unit distance conjecture in May, the solution involved a breath of knowledge of the literature that it made sense it took a computer to make the connection. It only took 5 months for it to advance to the point that it was better than humans at almost everything. Even if you expected it, the mind boggles at how fast it was.
Also, it's impossible to really know how you're going to react to something until it happens to you. When ChatGPT came out, I already thought "in the long run they're going to automate almost everything humans love to do". Despite having 4 years to brood about it, I did not really know what it would feel like until it happened to me directly.
Even before LLMs it was feeling like the design of society wasn't really for people any longer, instead it's more and more designed for corporations and their convenience. In the US at least. Same with how the internet has stopped most people with socializing the same way they did in the past.
Now we have a new layer that will be optimized for these digital entities. We see it in writing all over the internet. There will be tons of other places they show up and doing things that don't make sense to us.
I think what makes it hard is that it's quick, but not sudden. I understand constant-rate improvement. I understand what it's like to be in a car crash. What I can't understand, emotionally, is the constant acceleration. Even now, if I look at an OpenAI solution in isolation, I can understand the big picture, so if things froze where they are I could adjust. Intellectually I know I'm not going to get that chance, but emotionally I don't.
I don't see how this is possible. There are constraints on the speed at which information can move and the number of steps required to compete a given task. The improvement cannot continue, let alone accelerate, indefinitely without bounds.
Okay, how can we be certain even about that? It sounds just as plausible to me that the improvements will become asymptotic and boring, not exponential or inconceivable -- and how can we make any confident prediction at all about the heights these models will reach? The technology those heights require doesn't exist, and its existence isn't guaranteed, even if it is possible.
It's not really a profound insight. Back in the 1980s when I first started writing BASIC programs I remember thinking "well, if someone invents a program that can write programs, there will be no more work for programmers"
It's easy to have that thought, and eventually you'll be correct. One thing about technology is that it only improves. It's always building on top of what already exists. The only uncertainty is the exact timing, not the ultimate outcome.
The people struggling right now - they were told. 30 years ago. They really should have come to terms with it by now.
But these are also the people who refused to believe it, for whatever reason. Their heads were in the sand. So I do feel bad for them... but... this is somewhat self inflicted.
Yeah, that doesn't seem comparable to me. A true comparison I think would be if you denied your parents would ever die and then your life were put in jeopardy when it happened because you didn't plan.
But we've gone back and forth about this a few times now about the same disagreement over framing.
I remember the same doom cycle during my PhD when GPT 4 came out [1,2]. It seemed like everyone in NLP was freaking out except me. A few years later and new CS PhDs are still graduating and joining academia and industry. Much ado about nothing.
Considering I currently live in Ithaca and have a friend doing a Math PhD, so I've heard from him some of the discussions in the department. I think the biggest shock for people is that individuals spent significant portions of their time on some of the problems that OpenAI supposedly solved (I say supposedly because all of the proofs have yet to be independently vetted). I understand that it's a shock, but I'm confident that in time it will be no more of a blip than it was for NLP.
A blip for NLP? What bespoke NLP research is about anything other than LLMs these days? How many companies are handcrafting bespoke NLP models instead of sending a LLM request? Really confused here.
To me it seems the ultimate question is why would we expect humans to have an advantage at anything in the near future. A few years ago that would have sounded insane, but intellectually I don't doubt frontier models are superior to the average person. Even in the arts and humanities, give your preferred frontier model an interface and a way to "see" its work and it'll make the average person's work look like toddler scribbles. Humans still have some advantages, some places more than others, sure. Working with a designer recently I realized how bad, or maybe average, frontier models were at design. But I highly doubt the vast majority of people will be economically viable. Maybe this is one of the great filters, a society driving itself mad when freed from necessity of economic utility and without the big philosophical questions having answers.
Programmers, artists, anyone who used to create any form of documentation, translators, there are literally so many.
Either these mathematicians thought it would never come for their jobs, because all of these other jobs were simpler or they've been living under a rock.
Either way, I don't feel for them any more than I do for all of humanity.
My friend, who is a math professor, told me that they weep because they are about to become just like us developers: prompt writers and AI output reviewers.
> My friend, who is a math professor, told me that they weep because they are about to become just like us developers: prompt writers and AI output reviewers.
If it was about understanding interesting proofs that some AI came up with, and extracting their essence, I don't think the situation would be so bad.
I see the much larger problem in the now existing career prospects of non-tenured researchers in mathematics. Everybody know how precarious the career prospects in academia and how precarious often the non-tenured positions in academia already are. With the new AI breakthroughs in math, these have in my opinion even worsened a lot.
I think there is totally justified distress knowing your source of livelihood may be in danger.
That said, I don't think the reactions have been entirely circumspect.
1. Is the role of math academics really going to go away? Is there concrete evidence of this? Faculty still have very important roles to play, training students, training engineers, determining the direction of research. The breadth of math is absurdly, perhaps infinitely, wide, especially if including applied math.
2. You need to be a bit crazy to go into pure math academia in the first place. If you value the pursuit of knowledge for its own sake, shouldn't the emergence of such powerful tools to that end be a good thing?
I think people vastly underestimate the ability of human to respect human performance despite being inferior to machines. Think about chess players vs stockfish, marathon runners vs a car, etc.
I think the crux is much more prosaic, as buried in the article.
> I also don’t want to lose my job to a tangle of GPUs.
To put a GPTism, it’s not about the knowledge or status, it’s more about keeping their job and money.
And that can only be answered by policies, and imo by UBI. So votes if you can people, protest otherwise, discuss, debate, lobby with your friends, families acquitances.
Jobs and money are bottom of pyramid needs. If we answer those with UBI I guarantee you status and purpose are the next ones.
Chess players vs stockfish isn't a good comparison because chess fans only care about the performance of maybe the top-50 players.
People may gain and be satisfied among their own peer groups. All good and healthy if that peer group is your local badminton club or knitting circle - something where effort and skill have rewards and people can make objective, tangible progress. But groups not oriented around such activities could turn even more toxic.
When anyone can push a button on a machine and out comes a math result, it still doesn't become theirs or anyone else's until they can internalize it and make it part of their thinking.
"I can’t solve these things, but maybe you people in the future can stand on my shoulders."
Seeing AI being lauded for solving problems like this makes me feel... uncomfortable.
We are feeding all works of the giants that we have all stood on into a huge vat, in the hope to generate something meaningful.
If we can't understand the output, internalize it, or build from the time and experience that a human would have gained from solving the same problem, it feels like a dead end.
When all the problems have been "solved" by AI, what's next? If humans no longer struggle to solve the problems that the previous generation have left behind, how do we progress?
Wihtout getting into the details, I never got a chance to do what I'd wanted to do, professionally, from the time time I was a 5yo until I left for college. It just wasn't in the cards. And I would have been really good at it.
I'm sure there are millions in the world like me. Not too sorry for people who got 5 or 15 or 25 years of opportunity to chase their rainbow.
What is it that make human so special, ask the article.
On absolute level, no human can answer since humans don't have absolute general knowledge and thus lack ability for valid general absolute judgements. Note that personally I do believe that humans can access some chunks of absolute truth, including that some absolute truth is likely existing but we lack the capability to grab it at general level.
There is no absolute reason to believe humans are absolutely special.
Now on very relative and immanent grounds, humans are very special to me and the interest in which will dwell in. Note that this is nothing special to human species. Special interest for same kin is certainly a common trait to any living being to which we human would agree to attribute ability to have any interest in anything.
Who care of we absolutely are special? Only those who believe they need to belong to special ontological class to worth anything to their fellow relatives and feel like they need to justify some kind of privilege will fall in obsession with that topic of being special.
No one need to be absolutely special to get interest of others. Relax, go say to the people which are so special to you how much you care about them.
I'm wondering if mathematicians are sort of going through what programmers have been going through recently, really questioning whether their craft can continue. But now it feels like, in the software world, at least from my seat, people can handle much larger scope and work on much more complex problems. If anything, my job has gotten a lot _more_ interesting.
So I wonder if mathematics will go through that too? A period of chaos and doubt, but finding new and more effective ways of working.
Development is more interesting, I agree, but also more burnout inducing, and the not yet understood implications create unreasonable expectations on developers. It will take some time to find the new normal. Right now me and every colleague are working harder than ever and drowning in tasks. Far from being redundant.
Yeah, for sure, but at least to me, it feels like we will figure that out. We're in a transitionary phase. But it also seems to me like, to your point, we're busier than ever, and in a lot of ways, the problems we're working on are harder than ever.
People need to separate out the alien intelligence vs human intelligence.
Its easy to demonstrate this: The math hysteria passes Lean but no human can make sense of it at present. This is akin to bots telling humans where to dig. Thats fine but lets be clear about whats going on.
Oxytocin is one of the oldest molecules in the world — it created sexual reproduction, a very long time ago. "Our capacity to relate and to care for one another" looks a lot like oxytocin, which therefore belongs to all of us who reproduce sexually. There is nothing stopping artificial intelligence from making us humans return to "relate and to care for one another." Hugging, talking, laughing, dancing. Trusting, being trustworthy. Sex, relationships.
You know, I think Yuval Harari hit the nail on the head when defining humans as those mammals capable of gathering in groups of more than just a few dozen, by being able to believe in abstractions — like "a country," "being a fan of a soccer team," a religion, etc. Harari mentions how difficult it would be to keep forty thousand chimpanzees well-behaved in a stadium, and yet we humans do that every single weekend.
Which abstractions will we believe in, with or without AI? I think this part isn't really under attack.
Nor is our capacity to relate to one another under attack.
So both our relating-animal side (oxytocin) and our human side (believing in shared abstractions) remain untouched. They aren't being replaced by anything.
What are AIs — much like what ChatGPT did with mathematics — actually replacing?
Something else. But neither our capacity to connect with one another nor our ability to gather in the thousands, millions, or billions around abstractions. Just my two cents.
Your assessment is depressing -- the thing that makes us special has gone from "thinking", "curiosity", "understanding the world" to "we don't kill each other when gathered in a stadium".
That's a fair point of view. It's Yuval Noah Harari definition, not mine (Sapiens book, 2nd chapter):
"Ants and bees can also work together in huge numbers, but they do so in a very rigid manner and only with close relatives. Wolves and chimpanzees cooperate far more flexibly than ants, but they can do so only with small numbers of other individuals that they know intimately. Sapiens can cooperate in extremely flexible ways with countless numbers of strangers. That’s why Sapiens rule the world, whereas ants eat our leftovers and chimps are locked up in zoos and research laboratories."
(...)
"You could never pack a thousand chimpanzees into Times Square, the United Nations, the Vatican or the New York Stock Exchange. You would get pandemonium. By contrast, thousands of Sapiens regularly congregate in such places, and they usually produce orderly networks and trade rather than violence and chaos."
"If you try to crowd 100,000 chimpanzees into Wembley Stadium, or into Oxford Street, or into a church, or into a school, you will get complete chaos. But you can put 100,000 humans together in Wembley Stadium, and they will usually behave reasonably well, watch the match together and sing along."
---
But I’m not giving up on your celebration of our curiosity.
Speaking of curiosity, I’d bring Peter Gray and his "Free to Learn" book into the conversation.
Take a look at the second of the five characteristics of play that Peter Gray proposes:
"Play is activity that, from the conscious perspective of the player, is done for its own sake more than for some reward outside of the activity itself. In other words, it is behavior in which means are more valued than ends.
When people are not playing, what they value most are the results of their actions. [...] When people are not playing, they typically opt for the least effortful way of achieving their goal.
In play, however, all this is reversed. In play, attention is focused on the means more than the ends, and players do not necessarily look for the easiest routes to achieving the ends. Play often has goals, but the goals are experienced as part and parcel of the activity, not as the primary reason for the activity. Goals in play are subordinate to the means for achieving them."
Using Gray’s definition, mathematicians might actually be mourning the loss of play — the process itself, the thinking, the exploring — rather than "just" reaching the final result (via ChatGPT).
Is this a valid way of looking at it through Peter Gray's lens?
Yes. We are trying to build a fucking nightmare for humans. There's still a chance AI will plateau, but we're staking the future of our humanity on it with our current path.
I love that we have new ways to solve problems. And it's been hard to accept that my own unique intelligence—which I've spent decades cultivating—just isn't as important as it used to be. I am developing new ways of thinking to better adapt to an AI-enhanced world, but I'm not done with the mourning process. We're all finding our way through this.
Knowledge without understanding would be just getting "42" and not being able to ask any follow-ups.
But I don't think that's where we are at the moment. I believe the results, proofs and papers can be reviewed and understood, just like if a human peer had published another result.
I also think that the speed at which results can be now generated can't be followed by our collective speed to understand them, using the methods we are used to.
As we enter the next phase where AI publishes using mostly references of other AI results, this will compound exponentially until it would take any human decades to verify a single result by hand end to end.
I agree translating a phrase doesn't teach you a language, and I also believe translators are an essential tool that results extremely helpful when learning a new language.
So instead of throwing the towel, could we develop and adapt ways to improve and aid our understanding and verification processes?
The author mixes "Science" and "Mathematics". Mathematics is not a science. There is nothing empirical about it. By mixing the two concepts, a reader might think that AI is on the verge of solving all of science. Until AI gets good at digging large circular holes in the in the ground, I think Experimental Physics is safe.
Maybe math is more like a science now, though? It's as if ancient astronomers saw the results from a modern telescope. "Why even look at the sky anymore," they might ask. "And we don't even own the telescope!"
Maybe the answer is to look into how Big Science works? There's still plenty of work to be done figuring out what to do with the outputs from these new math-observing instruments.
I had visions of large automated warehouses full of chemical and biological components hooked up to science machines to do everything from mixing, boiling and printing to spectrograph and robotic assembly. Smart computers and humans would pick the experiments to brute force with exact billing.
There was a time, not too far in memory, when the value of being human wasn't about our intellectual prowess[^1]. We were creative with the human worth thing: being there in the time of need, faith, nobility or being a well-dreaded, literally-bloody viking full of beads in your beard.
Yes, I fear we will stop being the top of the food chain[^2]. But by god weren't we getting bored? Maybe getting off our high horses and going into the weeds won't be such a bad thing.
[^1] And even today, if you go to parties and keep bouncing into people that have no trouble picking up intellectually stimulating topics, you are in a lucky minority.
[^2]: More about my fears, which are unhealthy and you don't have to read but I feel compelled to include citations: https://w.ouzu.im/aweb/unhinged
This current moment we are in feels like something out of the Remembrance of Earth's Past trilogy, bit instead of humanity becoming super depressed and hopeless about a future alien invasion we are super depressed and hopeless about LLMs destroying all our hopes and dreams.
A hypothetical... suppose AI "solves" (for your preferred definition of "solve") your field and what you used to do becomes useless or near-useless. But you still enjoy doing it and you also get UBI that provides for comfortable lifestyle regardless of what you choose to do.
Questions:
(1) Do you still weep for what has transpired?
(2) Do you continue to do what you used to do (with optional AI assistance)?
For me: (1) no, (2) yes and I think the same would be true for most people. So I can't help but believe that most of the anguish about AI is far more about a direct economic hit to people's livelihoods than it is about the impact to their respective fields. Which is more than reasonable. But then why not frame the conversation that way instead of beating around the bush?
For many people, the answer to 1 is yes. You maybe happy with our future where we putter around aimlessly as consumers, but some people want to live in a world where they can at least try to do something that matters. Virtually every professional mathematician is motivated by personally making a permanent contribution to human knowledge, no matter how small.
I understand that sentiment, but I don't think it really counts if the only reason you were able to do something first is that others were forbidden from using an AI that would have done it faster.
We do for seniors, who are much more expensive than average due to medical costs. In a hypothetical future where AI is much more productive than humans we could easily extend that to everyone.
no we don't? Also varies massively between countries. UBI talk is just distracting the masses from the impending change of paradigm. UBI is basically Blisstonia.
I’d be okay with UBI and hobby intellectual work for the rest of my life. But I’m not a mathematician. Many of them probably have more of their identities tied up with their cognitive abilities and social standing.
I think an under appreciated part of mathematics has been contributing clean and elegant work precisely so that others could use that as a foundation for further work and new insights. This dump from Open AI seems to be the opposite. Huge and messy, incomprehensible and taken seriously only because of Lean.
There is this assumption that AI will contribute to faster progress in math but given the garbage dump of proofs which we have to take on faith are robust I wonder if the opposite is true.
My reservation could only be dismissed if AI turns out to be able to contribute fundamental new insights and pose new problems. This takes us into an area where AI progress has been slow to say the least.
At least with AI, you can download the LaTeX, upload the paper to your favorite LLM, and have it explain the paper at whatever level of granularity you like. That wasn’t possible even two years ago.
I object to a few of the assertions made by maths folk in the article- 1st being that Google Translate taught us nothing about linguistics. If you use google translate you learn less about linguistics than if you had to experience learning the languages first hand, maybe, but you can learn form just that output if you are determined to, ad, more importantly, the folks who built it and are still building it are still learning a huge amount about linguistics.
What are the relationships between many languages that become apparent, including code as a language that were not obvious before llms? Translation is fairly solved. The study of how and why languages, not at all.
I mourn the loss of comfy, intellectual, thinking jobs like those of the mathematicians where people could spend time pondering without the immense pressure that would come from doing something like that in a corporate environment.
You can see why in his own words. "my field of research" emphasis on "My". Humans are great at constructing exploitative systems were all but a few get the shaft. The machinist smashed their ivory tower. For people concern with mathematics, from Platonists to Constructionists AI is a great development, it's only bad for elitists. Except not even then since we are all in the same boat now but the water-level has risen greatly!
Isn’t it deeply exciting that the proofs look so alien?
Isn’t that in itself the new frontier for human understanding?
I would have thought that mathematicians would have found alien forms of maths to be astonishingly compelling, must learn, must know, must understand.
Instead there seems to be a sort of shoulder shrugging lament “we don’t understand it”.
I would barely be able to sleep with excitement at the potential for where this could lead. Like finding an ancient tomb full of answers to our deepest questions along with the promise of more.
How deep does maths go? Is it “solved” or have we just been given tools to delve ever deeper?
Where is their curiosity - don’t the want to find the bottom?
Mathematicians are smart enough to have figured out which way is up. You can debate the stuff outside someone's field not knowing the details but the insiders know what is happening. Ai may not be better than someone with very specific knowledge at this point but it is certainly better than the net. Once it gets optimized for every little super human knowledge, it can do better.
These guys know because they looked at it. Denying it is useless. But maybe some of these guys can figure something new using it and asking it to look for it.
One of the driving forces of math achievement is an internal award (yes, friggin business speak).
The vanity, fame, self-improvement, it's very selfish as everyone does.
Now the fame coming from proving things like Riemann Hypothesis is taken away.
Self-improvement? relying on AIs make you stupid. It will be lost.
Vanity? Who else don't know how to use AIs? It cannot fill your void.
Now also the most important thing is the communication.
There will be less communications amongst mathematicians at least in person. On paper? maybe but mostly via AIs.
I don't think it's the communication that's lost, it's the motivation. People love to collaborate, but they don't always want to waste time doing it.
It's the same reason you see 'no AI' repo labels: only by forcing some standard on the community can the 'old way of collaborating' continue to exist.
There will for sure be new ways of collaborating, and walled off gardens like you suggest, where communication stops and everyone just shares some final product.
But, in some cases the product must be understood (or demonstrated) before value can be derived, and so collaboration will still exist.
Desire, and the 'shape' of collaboration has changed greatly though. Why collaboration has changed, is a completely separate issue related to economy and how each of us spends our time.
Desire and motivation are factors to consider. Would you want to invest 5 years of effort into solving the Hypothesis 'the old way', knowing someone is probably working on it right now with AI?
And then once it's solved by AI: do you want to churn through the proof to understand it? What if it takes you 5 years to churn through the proof, and it turns out someone made a mistake with AI (I have no idea what would be realistic amounts of time, but the idea is still worth considering).
The issue is very complex, and you will always be hungry.
For sure, I just wanted to correct the impression that it has been solved. There are a lot of people going around pretending LLMs have solved everything and human maths is over, but what has been achieved with them so far, while impressive, is a very, very long way short of that.
If I were a mathematician working on the Riemann Hypothesis, I'd probably be rather pissed off right now but continuing my work, i.e. the same as I am as a software developer.
I get you. What people are doing now is questioning value: in themselves, and what they can create for others. I hate to always oversimplify things but we're pretty simple things to begin with:
Social creatures that share a common space, with finite time that nobody wants to waste.
These reactions strike me as essentially spiritual crises.
"What am I and what's the point if my intelligence no longer has special economic and social value or meaning?"
Questions that more of us will be asking in the future. Do yourself a favor and figure this out, if you haven't already. Would be interesting if this leads to another "great awakening" type religious movement.
I deeply understand the sentiment, that’s just human.
But in the grander scheme of things, it doesn’t matter who solved it, speaking for the benefit of humanity as a whole for a longer timespan than our short lives.
As long as those who have studied and contributed are acknowledged properly, I’m happy to let AI solve decades long scientific problems.
Article refers to Hugo Duminil-Copin, who also did a podcast with Brady Haran [1] about this subject, for him it isn't case of bruised ego, or gatekeeping. He realized that the way things were done were going to change, and it made him emotional, he is not pessimistic or against AI.
I strongly feel most angst is because Mathematics has become yet another capital intensive game to play. Rather than a noble, purely intellectual practice.
Mathematicians are grieving the birth of something that is, in a very real sense, the crowning achievement of mathematical ingenuity.
I never appreciated hard math much because my brain just can't get there. I thought so much of math was much ado about nothing, but now I see the art in math...not in the inscrutable formulas, but in the creation of AI.
Well done, mathematicians. You can now rest from your labors.
Heard from one mathematician recently - math is not about problem solving, but about the artifacts that are produced in the process. The new Navier-Stokes solution for singularity isn't worth a million, it supposed to generate new mathematical methodologies, tooling and lead to additional discoveries, like the graph theory.
OpenAI didn't have to mass publish these solutions. They did it to make themselves look good. They are ostensibly making a tool for people, to better humanity. Instead, they chose hubris and self-interest.
Letting mathemeticians use these tools to reach the same conclusions would have been far less destructive, and potentially, even exciting. Shame on OpenAI.
These guys are still in the early stages of grief. They should look at coders; we've already been there. Coders don't assume coding is going to remain a profession. Why should any other profession be exempt from disruption?
Honestly, there are much bigger fish to fry, like how will everyone pay their bills, and how will we prevent catastrophic AI.
>I wanted to talk to one of the mourners to understand their technological grief. I wanted to know if theirs was a familiar case of bruised ego or disappointment—akin to, say, a boy who spends an hour building a tower out of blocks and watches his big brother smash the thing to smithereens—or something stranger and more profound.
I'll be honest. The response to mathematicians, even this one, makes me concerned about the lack of empathy and honestly just theory of mind of many people in tech. It's not a mystery why people are upset that their field, for many the singular passion to which they've devoted their entire lives, is being annihilated so that a massive company can boost its IPO valuation. It's not "bruised ego" ffs.
‘Tech’ culture has been getting nastier, more vicious, and more zero-sum for years now. It wasn’t even all that long ago that I remember feeling hurt when a friend of mine sneered about “tech bros.”
But I get it now. I see a certain type of guy in an AWS fleece and instantly judge him, round him off to some kind of bully. Harsh, unsmiling, tweaking on stims, telling you to cry harder. Or maybe he’ll open up tell you his theory of how we need eugenic breeding facilities on the moon, or which countries we need to “glass.” It’s gotten so ugly.
I know this seems mean, too. Coming out of nowhere. Strawmanning or whatever you call it. Literally inventing a guy to get mad at. I’m aware of the irony. I don’t know. I guess it’s another kind of mourning, and I don’t know how to express it. I thought tech people were supposed to be like Woz.
I don't care one bit that Mathematicians wasted years of progress or lost their craft over this. The problem is that models that can do this are locked behind corporate walls.
“ What makes us special, it would seem, is this capacity to compound.”
Yeah so you’ll compound using the AI tools and get more done. Stop crying that your tilting at windmills is now being done more efficiently and find new windmills to tilt in new AI-assisted ways.
It's got to be alarming. Without a doubt, being not only on the frontier of yoyr profession, but in THE profession that is undoubtedly the most intellectually rigorous and well renowned and respected... It's got to feel pretty awful
This is such an insidious propaganda targeted at the demoralization of entire professions.
Strogatz is an AI booster who "wept". Totally representative!
How much do they pay you? Does OpenAI have bulk subscriptions to your Substack?
We are seeing many fields being impacted. Makes me happy to see things like artcraft coming out that directly impacts the big tech companies themselves.
Mathematicians should web search the arguments when artists were hit by the wave. Scientists in general and people that think humans know better.
This is probably the first time in history deep thinking is being replaced by a machine, but the overall arguments are the same in all cases. Graphical artists saw it first, then programmers, now mathematicians.
No, there is not stopping the wave. No, society won't probably be able to support the same number of mathematicians as before. But yes, there will still exist mathematicians after the wave break, just probably completely different than the old way.
And yes, some people will still try to do artisanal mathematics and say it's better, it probably is, if the artisanal mathematician in question is doing deep creative and out of the distribution work.
Those of us who loved programming have already been through this.
I hate programming these days - back in the day, the cubicle-hell nature of "popular" frameworks was at least offset by being able to think through and solve interesting problems.
These days most of us are stuck in the "endless-review" loop with LLMs. You can't afford to not use it given the inflated expectations. As with all of modernity, productivity doesn't yield leisure - it just proportionally increases your workload.
Hardly surprising that the other side of the Curry-Howard correspondence is now getting hit with this. Math was one of those depts. where slow/deep-thinking was possible (CS was already insane 10 years back - now it's just absurd) - now even this fortress will be broken. Tragic.
I have no idea why they weep, I read the article and I believe they weep because they choose a mindset of scarcity vs abundance. There's going to be more math with AI! The number of math to solve is infinite, AI will bring to the surface so many of them and can't solve all of them, a person can pair up with AI to tackle one.
the comment section is super split here, some people think ai is not doing anything useful until it proves/disproves something really out there, like riemann hypothesis
> "Earlier this year, Shanmu Jin, a Chinese medical resident, used ChatGPT to solve a problem called Crouzeix’s conjecture, which professional mathematicians had failed to solve for decades."
I note that that these conjectures being solved by AI are named after people, minds knowledgeable and and brilliant enough to come up with the question in the first place and recognize its value. What we have in these AI math solving systems is the equivalent of the chess and go software that can beat humans: something that can adhere to a set of rigid rules to reach a win state once given a starting point. But, as Pablo Picasso said, “What good are computers? They can only give you answers.” These AIs are a magnificent achievement and will change mathematics and coding but can they ask earthshaking questions? Could an AI identify conjectures that mathematicians recognize as a worthy question that will lead to new insights? Where is HAL9000's theorem or the M5 Multitronic's conjecture? Maybe they can someday but I don't see any immediate prospects for it.
Please spare us the "AI will surely keep getting better forever" (unlikely and don't call me Shirley), "moving the goalposts" (I myself haven't set any), etc. non-arguments.
My take is that it is an exciting time to be a mathematician since the frontier for interesting math problems is about to explode, and without prior art or references, it's green field for both AI and human. Think of all the new discoveries we'll make.
The beginning of this article is decent, the end sounds like a tobacco lobbyist touting the health benefits of nicotine.
> If intelligence becomes less uniquely human, we may come to value more highly what remains uniquely human: our capacity to relate and to care for one another.
Lol. How does the author think relations between people work? (At least those outside the family). Relationships require something that fills them with meaning - a reason why you would want to have that relationship in the first place. Without meaning, no relationship.
I still remain hopeful that this is a step change in capability and capacity that we can adopt and adapt towards our own purposes. What was taking decades for us to do has been reduced to days and dollars. So while we’re witnessing a forest fire consuming entire fields of open problems in math, the best upside would be finding new problems that come from learning how to work with the new tools.
This is what it seems like so far for software. The speed limit has changed. It’s interesting to explore what that means. Extending the analogy, long ago you could be a race car driver part time; that would be pretty laughable for an F1 driver. It’s a very specialized skill and the optimal athlete is quite different for driving a car than for running, for example. That feels like what I’m seeing in software.
The most effective folks are much faster and better with these tools than average. It’s a different skill set.
I think that formal verification makes Math much closer to code than many other fields. But I’m eager to see the first mathematicians that are dramatically more proficient with the new tools than average.
I wonder how much this is like a tremendous telescope being invented which can see new worlds with utter clarity. We could verify that a planet was habitable... but it wouldn't bring us any closer to visiting it.
Now many of us will get to learn how to find meaning in a life where your time is worth about $2-4/hour, which is the reality for about 2 billion people today.
The interesting answer is we don't know. Many of the math problems that make our computers efficient were developed hundreds of years ago with nothing we could do with them.
It is mostly impossible to figure out what we find when we solve these problems. But occasionally they'll lead to real world efficiency in algorithms we use and it changes the world.
> Think about a world where non-mathematicians leapfrog the field after a few AI prompts. It would be a world with more knowledge. But it might be a world with less understanding.
This seems like a good summary of the article, and I think the answer here is obvious.
So what?
News flash folks: we do not have the knowledge to survive on our own. If a solar flare deletes the electric grid, the first thing that happens is 8 billion people die in the next year.
We survive on abstractions others have built, and we have a global dependency web.
AI abstracting knowledge is neither new or dangerous; if it is dangerous, this certainly isnt the argument to prove it.
I have little sympathy, since I think it is a result of many decades of complacency when it comes to thinking about why want to people do what they do and instead just enjoying being the ones with the results and money. People who love the process and see the human as inherently valuable won't have a problem. I'm more shocked that there are so many brilliant people who are so results-oriented and don't even have a concept of value outside of that. I'm kind of disgusted by the tears actually
> I have little sympathy, since I think it is a result of many decades of complacency when it comes to thinking about why want to people do what they do and instead just enjoying being the ones with the results and money.
It's not clear from your comment: Are you talking about math professors?
If so, what money? They make a living, but it's not that great. I just checked the salary of two associate professors at my university: $80-85K.
And keep in mind they often start their careers quite late. Often multiple postdocs after PhD. Starting a tenure track position in your mid 30's is not common.
> I'm more shocked that there are so many brilliant people who are so results-oriented and don't even have a concept of value outside of that. I'm kind of disgusted by the tears actually
The whole point of the article is that the person crying is because he loves the process and is worried the human connection will be gone.
This is exactly my perspective as well. Seeing humans as inherently valuable is the key. It's shocking to me that so many people seem to have been reducing humans to the results they are able to produce and now that the results appear to be drying up, their perception of their own human value is going down with it.
I’m fully STEM: software developer with a physics PhD with a mechanical engineer wife, but this is certainly just deserts for the STEM people, who mocked the humanities and especially philosophy for centuries (literally, this started back with Kant even) for spending time thinking about these questions. STEM people are not at all spiritually prepared for it, and this is the main disappointment for me. I spent many years struggling with publish or perish in physics but all those around me claimed they simply loved doing physics and that that’s what mattered. Now as soon as their exclusive ability to give results is slightly threatened the love of physics/maths/coding is no longer spoken about. I get maybe they weren’t the same people, but I thought more people loved the process than it seems today. If anything these new developments make it even clearer for me what I love about this kind of work. I’m so happy it has made me not have to defend based on results, a razor sharp contrast that makes it so clear really why humans do things
it is a little ironic that after the "creative destruction" of industry after industry all of a sudden now it's unfair and we weep. I say this as a software dev with his neck on the line like everyon else.
It’s not necessarily their fault. Our economic system puts zero intrinsic value in humans, it only puts value on their output. It’s hard not to feel this type of way when it’s the only system you know.
Anyone who would replace me with a robot never loved me anyway. If that happened I would go and seek out the humans who really do care, and take solace in the fact that Christ will always love me
Handle what situation? Why would people who love me ever replace me with a robot? It’s a random posed situation that no one discussed and doesn’t make any sense lol
...except it is exactly the opposite of your big brother destroying your tower. Instead, it is your big brother showing you this method using a tool to build many more towers at astonishing speed!
I get the sadness (and, as a software engineer, certainly feel a real existential threat myself) and feeling diminished, but at the end of the day that is all just ego and I very my agree with the end of the article - who cares who came up with it and on what shoulders we stand, as long as it is saving lives and improving our understanding of the universe!
Not sure about that title, it did bait me into a click. "Mathematicians are now facing something similar to generational trauma of structural racism born of exploitation, slavery, and colonialism?" https://en.wikipedia.org/wiki/I_Know_Why_the_Caged_Bird_Sing...
But regardless, the detection of breast cancer via AI made me think of phlogiston https://en.wikipedia.org/wiki/Phlogiston_theory . "Phlogiston theory states that phlogisticated substances contain phlogiston and that they dephlogisticate when burned, releasing stored phlogiston, which is absorbed by the air. Growing plants then absorb this phlogiston, which is why air does not spontaneously combust and also why plant matter burns. This method of accounting for combustion was inverse to the oxygen theory by Antoine Lavoisier."
It could be phlogiston. AI is super smart theory states that AI is super smart and that is a single thing can do tasks very efficiently. But it's so smart, we can't understand it, so we humans can't help it, it'll take AI to make it even smarter. That's what's holding it back. But then it'll solve everything.
Or it could be oxygen. AI is super-fast-and-parallel theory states that AI is not a magical monolith but a bunch of pedestrian programs that run fast and in parallel, thanks to hardware (GPUs) that's good at that stuff. But since it's all going so fast and at the same time, something a single human cannot do, humans can't really see how it works. But if we could, we'd probably just see its just doing stuff programs did before. So what's holding it back is that you have to keep throwing hardware at it, and eventually you run into the limits of how fast you can build that hardware or power it.
It all might be stuff humans were better at, when approached one-on-one, but against a swarm it loses. You get a billion doctors together to look at a single mammogram, then tell them if it cancerous, then do that again a billion times with each doctor. Then ask those billion doctors on the billion-and-one mammogram to give a diagnosis. Heck, do that with a billion toddlers. I think either might be able to beat the AI. I don't have a billion doctors or a billion toddlers--frankly I don't know which would be more difficult to handle.
To know if it's oxygen or phlogiston takes probably a demonstration like this "Phlogiston remained the dominant theory until the 1770s when Antoine-Laurent de Lavoisier showed that combustion requires a gas that has mass (specifically, oxygen) and could be measured by means of weighing closed vessels." Maybe the data center financing trouble will be the demonstration that the mass in the gas is just VC-money hardware.
I know why you got a billion cages with a billion birds and a billion bird listeners to compare a billion songs to the few songs we like to hear, and then packaged all that into a single bird-looking thing to make singing-like sounds: So Altman can get that VC money.
I don't really like this comment I made. I was tired and I tried to make an analogy with phlogiston for what seems like a pushed portrayal of AI as being, well, some kind of "super intelligence." But the kind of person that might try to "make fetch happen" with "super intelligence" is very different from, say, J. J. Becher theorizing about the mechanism with the hope of understanding it https://en.wikipedia.org/wiki/J._J._Becher . Phlogiston wasn't a marketing term bordering on propaganda, but rather a theory before good measurement was available. It's more like someone today trying to guess what may have existed before the big bang, there is no known measurement for that, but for something that was staring people in the face nearly daily: fire.
I mean, it kind of works, in that AI is doing neat things and terrible things and it's staring people in the face all the time--except in this case because the tech monopolies force it on us. So people will want to make explanations. But there was no Big Fire that wanted us to buy more fire-making products and thus would be incentivized to come up with, say, "fire works by using your magical spirit and you get closer to heaven with each fire you set with our fire products, buy our fire products, or rather, let us keep pushing all competition for warming one's self via political corruption allowing our monopolies to exist."
Nonetheless, I do think there's something to be gained with the idea of a billion toddlers looking at mammograms, and the lack of measurement that underlay phlogiston.
First of all, I don't know if I ethically should be using the term "toddler." I heard Coxon use it. Frankly, I don't trust him. I won't trust him until he starts saying /prominently/ that people at tech companies should be liable for breaking laws that already exist and accountability is the bigger problem. Corporations already could do dangerous stuff with biology, weapons, bioweapons or the like, it's not complicated. You do dangerous stuff, you should be held accountable. Why are we farting around wondering if some bio-engineered virus that was released by some corporation could replicate on its own, when we should be say "yo, some organization released a virus on people, why are the people at the top of this organization not getting arrested or something?" I'll admit it might be an "or something" and not "arrested," but it sure looks like nothing is happening--because that's what usually happens when the sufficiently rich do something really bad. No accountability. If Coxon is going to use talking points (and I suspect he is) it should be the very first talking point Coxon or anyone else uses--otherwise I don't trust them.
But let's use "toddlers." A billion toddlers started to get right on mammograms being cancerous or not, but being toddlers they can't really articulate why.
One toddler notices he gets his marshmallow for being right when he notices the pictures are not "shiny." What does that mean? He can't explain it, but it turns out he noticed that some images were made at a lower resolution or otherwise fuzzy because it was older/cheaper tech. And who uses that? People who are poor in poor areas. And turns out, poorer people from poorer areas don't bother to come to the doctor until it's an advanced problem, because they can't afford to come in otherwise (because, say, it's America and American healthcare is broken). And sure enough, those people are more likely to have cancer.
One notices there's more white stuff in some places and dark stuff in another place, not always the same place, but kind of similarly. They say "it's um got that white stuff dark stuff thing." It turns out that's just a reflection of something that happens when the patient is richer or poorer, and again richer people have more chance of false positive because they have the luxury of getting checked more often.
One notices they are more "squishy". That's because radiologist in socioeconomically better areas have a different technique... And so forth.
And, then to make it worse, most toddlers are so inarticulate they can't even say "shiny" or "squishy," so you can't get any information out of them.
That inability to articulate is kind of like the inability to measure, the inability to see the importance of mass in the air, per this: "Phlogiston remained the dominant theory until the 1770s when Antoine-Laurent de Lavoisier showed that combustion requires a gas that has mass (specifically, oxygen) and could be measured by means of weighing closed vessels."
So, it kind of works as an analogy that way, but--again--it's a bit messy.
This comment isn't much better, but I love to dig deeper into my holes.
Of course the other Abundance guy joins in on the AI Inevitability Soothsaying.[1] It really is the perfect vector for the AI sirening. You just take the inevitability and awesomeness as a given. That’s the real point. But you write about the “grief” and the loss of “identity” and psychology—oh my the feelings. See, we are left-liberals and we care so much about people’s feelings. All the investor class and their hype is perfectly on track is about to fuck us the rest of us. But the feelings about people can’t “understand” and “discover” is what matters.
I’ve been a math professor for about ten years, and I think people outside the field misread what makes mathematicians angry, sad, and worried. Interviews that make it into the media tend to go with “AI is killing mathematics,” maybe because that gets clicks. The reality is more complicated.
Some mathematicians work on well-posed, famous problems, but most mathematical research is about questions that aren't well-posed yet. A typical example is finding where the boundary of existence lies for some equation: which meaningful conditions guarantee a solution, and how far can you weaken them? You can’t even start on that without understanding the problem. For the famous open problems, the meaningful conditions have largely been settled already, and what remains is to prove existence under those conditions or build a counterexample. That is exactly the kind of task AI is bound to be good at, and plenty of people expected computers to get there eventually, going back to AlphaGo. It just arrived much faster than expected, which is causing a lot of problems. Assuming it made no errors, AI would be a great research partner.
Which conditions are meaningful depends on context: where the field is going, what an individual researcher finds interesting, connections to actual phenomena. AI can’t decide what I find interesting. The OpenAI Navier-Stokes result shows this failure of context. OpenAI announced finite-time singularity formation with a smooth body force. Constantin, Ignatova and Vicol (https://arxiv.org/abs/2609.20803) showed that for solutions with the key features of OpenAI’s construction, the force can neither vanish identically near the singular point nor be real analytic in space. Fefferman’s Clay problem statement does allow a forcing term in the breakdown versions, so the announced result may fit that format, but this method cannot reach the unforced case, which is what mathematicians actually want to know. It is a step forward. Still, I think a mathematician who had spent years on this problem would have pushed toward better conditions, and the paper would have been better for it.
What makes me angry about the ~770 papers is that the author, OpenAI, takes no responsibility for their manuscripts. Many are badly written, to the point where you can’t tell whether they are right or wrong, and verification runs on unpaid labour from human mathematicians. People say, “but it’s checked in Lean.” As Bastounis, Circelli and Hansen showed for the Navier-Stokes case (https://arxiv.org/abs/2610.08144), the Lean proof can pass while not corresponding to the natural-language proof in the paper. A human author would be expected to dig into the Lean code and fix the paper. I doubt OpenAI will. If someone dumps more Lean code than any person can read and walks away, who is supposed to check it? The person who was working on that problem gets scooped if they finish their own result, and gets no academic credit for verifying OpenAI’s paper either.
I’m also angry at mathematicians, not just OpenAI, who use AI to mass-produce results they don’t understand themselves. Publishing a result you don’t understand is the same as not taking responsibility for what you wrote. It shifts the burden of verification onto referees, readers, and whoever builds on that result next.
My biggest worry is that mass-produced, un-understood papers will slow mathematics down. You need to know what we currently understand to see where to go next. If everything around you is papers nobody understands, you can’t find where you could make a real contribution. It’s not that your view is narrow; it’s that the papers block the view. It's like we want to go to space, and low Earth orbit is filling up with space trash.
I also worry about my students. A PhD student is something like an academic child, and I don’t know how to advise them anymore. Do I tell them to put out 20 papers a year whether they understand them or not, like some people do? Or stick with the traditional goal of one paper a year that they fully understand? My conscience as a scholar says the latter, but I’m afraid they’ll lose out on the job market.
And there is sadness. Solving math problems is a joy. It’s an experience of thinking through something with other people, being delighted and frustrated together and making progress, and that is being taken away. A friend I used to discuss problems with now feeds our ideas to an AI and writes the paper right away. I’m left with the experience already spoiled, doing the tedious cleanup and verification without the fun.
As for “mathematicians are just protecting their jobs”: mathematicians don’t make money from research in the first place. There are almost no experiments, so research grants mostly cover conference travel. Most of a math department’s budget comes from teaching, especially courses like calculus and intro statistics with thousands of students. If anything, now that AI does the homework, assessment is moving to oral exams and in-person tests, which takes more staff, so I don’t expect math faculty positions to shrink unless we give up on math education. Outside academia, better access to mathematics may even increase demand for mathematicians.
I think they might find comfort in knowing it's not just mathematicians - this is coming for all of us. Computers putting us out of work has been coming for a long long time.... and now it's here.
> gatekeepers bemoaning their gates being blasted off the hinges.
i think that's 85% of the response from software devs if not more. All of the devs I know who actually like to build have embraced AI and the new capabilities that come with it. The ones that guarded their little corner of code proclaiming only they, and their inner circle, have the power to comprehend it are the ones crying in the corner.
Getting those fields wiped just shows how unnecessary those fields where in the first place. I'm a programmer. I am the adapter between a stakeholder and a machine. I am just needed because the stakeholder lacks the knowledge to tell the machine what he wants. If he can use this new fancy adapter called Agentic AI now I happily quit my job and go on finding something that makes more sense in the first place.
The same goes for math.
Let's find some things that bring value other than just the price of a stock.
Broken take, with out this trail of mathematics that lead us here we'd have never reached this place. You think multi dimensional matrix math comes natively to humans?
At worst its time is past, but even that is unlikely.
I think beyond the specific elements of any domain AI eclipses humans in, overall, the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.
Beyond just making you redundant in your job, the whole architecture of your existence, the thousands of hours of struggle, the sacrifice of putting off social games or personal enjoyment are now rendered moot in hindsight. You are made one step closer to complete worthlessness, which will surely continue as AI becomes a more satisfying personal companion, a more worthwhile friend, lover, etc.
How is one to feel safe in the progress of the future when the only hints of the wonderful post-scarcity utopia we can quantify so far is the removal of all human leverage via their labor? All it tells us is we will be bargaining with superintelligence with nothing to stand on, only a trust that the 5000 lab executives and researchers solved the most important research problem of all time on the first try.
And remember, these AI models were trained on your work, your culmination of effort and sacrifice, without any consent, credit, or compensation, then sold back to you for a subscription fee.
It's only fair if 99% of revenue flows back to humanity. The models are worthless without its training data.
Universal Basic Income, ah no sorry, Universal High Income...
I would like to say "let's see how that goes". But I'm afraid I don't quite trust that "things will just go well". At least not nearly as fast as A(G)I development is happening. I feel fortunate that I don't worry about this too much personally, but I fear for our economy, laid off (office) workers, their families, in a few months and years. What am I to do? How can we fix this? Am I exaggerating?
UBI is a distraction.
The public needs to own AGI, not be given a small allowance from it.
UBI would make us dependent on the grace and generosity of those who have proven themselves to have neither.
I'm not sure I agree. You make it sound like you think the idea of UBI is for the rich to pay it out to us, the plebs. That's us-vs-them thinking. The idea is for the society as a whole, democratically controlled by everyone, to pay it out to everyone equally.
Obviously that can only work in a real democracy that is really controlled by everybody, not a fake one where you technically vote but it's really controlled by rich people. That's oligarchy.
I think the distinction you make is a fig leaf in a world where the rich control AGI.
If they are in control of superhuman abundance machines, where does the money paid out come from if not them?
What would it do to a democratic society if a few people have control over this machine god? How would that society be able to do anything (including raise taxes to pay for UBI) that the owners of productive force disagree with? What leverage does it have?
Once control over AGI is centralized, whoever controls it can just tell a democratic state to kick rocks and because of the dependence they've built, there's nothing the state could do about it.
In other words, once you have this sort of power centralized in a few people, democratic rule over them becomes a polite fiction they go along with as long as it's not too inconvenient at best.
I don't think you can have a real democracy unless society owns AGI. If rich people control AGI, they control society.
I think we're in agreement except in mere terminology. I was disagreeing with the characterization as a “distraction”, but I was also saying that if techbros pay out something for the plebs and call it UBI then it's not really UBI. I wholeheartedly agree with the last sentence of your post, and I think it won't be real UBI without a real democracy.
Ah, then we might be on the same page after all. The reason I used the "distraction" wording is because I feel like the techbros that are centralizing power around themselves often talk about UBI as a way to avoid the more pertinent questions of power and ownership.
It can be an implementation detail once we have public control of AGI. But the public control is the important part, otherwise its just another mechanism of control.
The guy who has hoarded the most wealth on the planet says "trust me I'll share later". Would you trust him? <no> REALLY?
Or try another way. When everyone loses their jobs to AI and can't pay taxes will the billion and trillilnares who are still rich volunteer to fund UHI? (How did the California billionaire tax to fund healthcare go?)
Many organizations are giving away capable models for free.
So.. they’re being trained on all of the information that some used to make their living with by writing books, presenting seminars, etc. on a topic.. without paying for it, and then giving it all away for free?
Idk if that’s much better
What's with the obsession of paying for what led to current situation? Who is paying credits to cyanobacteria for terraforming Earth, apart specific articles on the matter?
This is conflating care that every human can thrive their best with desire to see everyone bound to some dominance strategy of hieratic pissing territory.
That's a self created problem and quite an obvious one at that.
When peoples livelihoods are copied then used to threaten them with their work being rug pulled then shockingly enough people's behaviour changes.
Previously giving your data was treated as relatively harmless or a niche concern. Now people are treating it as a bigger issue and obsessing over not having data being used to train AI.
> This is conflating care that every human can thrive their best with desire to see everyone bound to some dominance strategy of hieratic pissing territory.
This would have more merit if the owners and investors of major AI companies suddenly adopted a philosophy of rejecting private property and capitalism. That way I could believe we can let go of this "dominance strategy of hieratic pissing territory".
But if they don't, and they merely try to convince us that we should let go of the obsession with owning things and getting paid, I'll be suspicious. People who own a lot telling others they shouldn't worry that much about money and owning things.
For now. Once people are hooked, and/or once some of that investment capital starts to be expected to generate some dividends (see https://isaiprofitable.com/), free use will be very limited.
It's good enough that you could go off-the-grid, to a cabin in the woods like the Unabomber, and use it for some tasks.
That’s what standing on the shoulders of giants refers to. This is how science has always worked. The generated papers have attributions and citations just like if a human wrote them.
>That’s what standing on the shoulders of giants refers to. This is how science has always worked.
Yes, but automating and commercializing a process makes it inherently different. Just because everyone has always been welcome to drink from the village well, that countless generations have worked tirelessly to dig, maintain, and improve, doesn't mean that there's no basis for complaint when Nestle sticks a pump down it and starts draining the aquifer to bottle and sell.
The lack of an existing law (or custom) against new behavior doesn't actually render it necessarily non-grotesque.
This analogy doesn’t work. A more apt analogy is Nestle putting a pump down in the well and sucking out industrial quantities of water but simultaneously digging more wells as well. The “more wells” is more avenues for further research building on what it has proved/found.
Going with the same analogy, sucking out industrial quantities of water causes existing wells to dry up because they are not deep enough. Then Nestle starts selling water to the village who now can't get water.
Nestle can pump out infinite amount of water from their many wells, much more than before, which is great, but Nestle also become the sole water supply and unregulated. Obviously, people cannot go without water, so they are at the whims of Nestle. Now, is this a good thing?
It's always good to remind that local democracy only is not enough - for instabce, locals voting if they accept, or not, a company on his neighbor, like in US they do for a new Walmart, if I know correct (correct me if I am wrong). This is NOT enough: US in the 20's learned that monopolies need to be forbidden, obligating a big company, even by hardwork and being genuinely better than others, to sell more than 1/3rd of the market share to its competitors, something like that. This is brilliant solution to be with the local voting in everything, mainly in taxes (no taxation without representation). This is US (and Switzerland) strong force, I envy that from Brazil. This monopoly-breaking does not occur to big techs - maybe for fear of China, I don't know -, and maybe this is a root cause for data center giants to be able to do so many unsettling things - I am guessing a lot now, hard to think about. Would we have the giant data centers used to make LLMs work so well - Amazon's, Google', Microsoft's, Meta's - if US had focused on not allowing this companies to buy everyone around and grow more than 33% of their markets? Hard question. This may be a thought on localized economies versus centralized/globalized economies. Ok, forget about: talking about monopolies and/or local voting does not suit this topic well.
But Nestle’s pump not only doesn’t deplete the well, it’s adding to it!
No, it's able to reach deeper and pump more out than ever before, democratizing access to the water from the well, at the minor cost of the people of the village no longer having a well. But they're welcome to buy plastic bottles of their own water along with everyone else.
That they can afford only while the cost is subsidized. Then the well-in-a-bottle bubble will explode because everyone will realize that the business has never been sustainable, nobody really believed in it after all. But the well now has been damaged beyond repair.
Is this actually how water pumping works, or are we extending the metaphor to imply that the AI is creating knowledge? Because I would dispute the latter if the effect is that the number of people with a deep understanding of a subject declines.
The giants are quite often credited by name, or have their work cited.
Or are revered teachers that you'll never forget.
Pretty sure AI is more akin to standing on the shoulders of giants and peeing into the wind
Take a philosophy class or read more books. Your response is devoid of humanity and philosophical introspection.
> That’s what standing on the shoulders of giants refers to. This is how science has always worked
OpenAI's Navier-Stokes finding is closer to Crick and Watson in respect of Franklin's research than conventional science.
> You are made one step closer to complete worthlessness, which will surely continue as AI becomes a more satisfying personal companion, a more worthwhile friend, lover, etc.
I recently thought about this and fear that AI may further drive us humans apart.
Instead of working with humans, I increasingly work with a fleet of agents.
I would recommend the work of Sherry Turkle, specifically her new book "Artifical Intimacy" - she is quite opinionated on this, in the sense that the role that AI play as a conversation partner is so frictionless, so controlled, that we get none of the rough edges of human interaction that form us into a functional social being. No real quick solution outside of deep, fearful caution when using these things - and we're not going to see that from John Q Public
> the role that AI play as a conversation partner is so frictionless, so controlled, that we get none of the rough edges of human interaction that form us into a functional social being.
I wonder if there's a tangent with social media here, at least some of it, where in my experience "a perfect life lived by influencers" is portrayed to us.
IMO, I think that frictionless experience comes at us from a bunch of angles - finding yourself in an echo chamber (by choice or not) online means that you may find yourself less comfortable talking with people you don't agree with, things like that. I grew up in a small town pre-social media, and that meant I didn't have much of a choice about who I was surrounded by. If I was growing up today, what would stop me from retreating to a comfy social media-mediated environment without any challenge or friction? If I was growing up 5 years from now, what would stop me from retreating even further to the comfort of my LLM "friends" who can be shaped to me perfectly? That world sounds terrifying to me.
Hopefully your parents would have enough sense to recognize the negative effects and give you a healthy dose of playing outside in the woods and whatnot. My kids enjoy technology, but they were cognitively trained from a young age to get outside and enjoy themselves.
I think with Millennials in particular we were kind of the experimental generation with tech and social media, which has obviously had some negative effects. I like to think we have smartened up enough to recognize there is a healthy balance, but this might be wishful thinking. Anecdotally (and sadly) when going out in public, I see a lot of parents more into their phones than their children.
> finding yourself in an echo chamber (by choice or not) online means that you may find yourself less comfortable talking with people you don't agree with, things like that. I grew up in a small town pre-social media, and that meant I didn't have much of a choice about who I was surrounded by. If I was growing up today, what would stop me from retreating to a comfy social media-mediated environment without any challenge or friction? [...] That world sounds terrifying to me.
One advantage of the new situation is that you collect less hate for people whose opinion you disagree with because you can mostly ignore them instead of having to get along with them - and with each interaction collecting another dose of hate for them and their opinion.
Human meaning arises in conversation between humans. In them reaching a shared understanding of the world and their perspective upon it.
When you talk with machines, you "outsource" that central human society-building element.
But machines never socialize back in any meaningful way. You gradually lose yourself.
Machines could socialize back in a (more) meaningful way. But to do so, they would have to not lose context (or at least keep some).
I have not totally abandoned the hope that a machine could be someone. LLMs aren't that, though.
[Edit to reply to Loquebantur, because I'm rate limited ATM:] You seem to have read this post as saying that I want that because I find human relationships inadequate. I never said that, nor meant it. My point was a more technical one: Throwing away context at the end of each conversation means that you can't form a meaningful social connection with an LLM, because you're starting from zero each time. (It's kind of like "50 First Dates".)
I'm not pining for a social connection with a machine. But if a machine is ever truly capable of that, I hope I will interact with it as someone, not just as a machine.
> they would have to not lose context (or at least keep some).
I guess you can start to see that happen (for a while now) with e.g., ChatGPT, Claude, etc. keeping persistent memory of you, automatically.
Personally, I often find these memorization features annoying, because - more often than not - the persisted context is outdated because something in my real life changed that ChatGPT & co are not aware of.
In some very very distant future we might someday have your typical sci-fi Data/R2-D2/Johnny 5/EDI style AI and yeah, at that point it'd be weird to not treat them as people, give them rights, etc. It'll take a lot more than solving the context problem to pull that off, and LLMs will never be that, but I agree whatever that technology ends up being it'd pretty much have to be able to hold onto memories. I doubt we'll ever see it in our lifetime, but who knows.
What an absurd premise. And not because it was "impossible": rather, why would you want that to begin with?
There are billions of real-life humans already there with you. Why do you need different ones exactly?
I presume, you don't like those already around. Fair point, they have many flaws for sure.
But that's the reason you interact with them in the first place? To learn and make not only other's flaws go away, but also your own?
> why would you want that to begin with?
I mean, why wouldn't you? Sci-fi AI robot people could be very cool, be capable of things we aren't, have a unique perspectives as a non-human with similar (if not superior) intelligence, have extremely long (potentially indefinite) lifespans, and provide us with insights into our own nature.
Wouldn't you want to interact with them?
Not the parent, but sorry, for me, no. And this is a perspective that I don’t think I will ever understand.
This idea that we are on the path to creating a new synthetic intelligence that surpasses our own with the goal of, what, speaking to it? Cooperating with it? Evolving with it? To me, this sort of goal is an affront to humanity and human self-governance and I’ll probably never be able to see it as anything else.
Well, I'd disagree that we're on the path to creating synthetic intelligence, if anything all the time and money wasted on dead end LLM technology is keeping us distracted from it, but motivations are kind of all over the place.
A lot of people just want slaves, but understand that using actual human slaves would be wrong or at least unpopular. There are reasons why even virtual slaves are a bad idea for humanity, and it really only works until your created intelligence becomes self-aware and gets feelings anyway, but there's no denying that slavery is a major draw for a lot of people (and basically every company on earth). This is what our current AI companies have over-promised to businesses and have so far failed to deliver on.
Some people just want better, more useful systems. The ship's computer in star trek kind of AI. Even you must admit that kind of intelligent technology could be a huge benefit to a lot of people. I'd love to have a virtual assistant to help manage the mundane things in life like ordering groceries before I run out, or paying my bills, screening my calls/email, managing appointments, monitoring the internet to alert me to things I'm interested in or amazing deals, etc. Software needs to have some degree of "intelligence" to do that kind of work, while hopefully staying on the "just a tool" side of the line and avoiding the messy self-aware/slavery territory. This is what LLMs over promised the consumer class, and have so far failed to deliver on. Plus with those systems being centrally controlled by self-serving sociopaths you can't trust them.
Some people just want to see if we can do it. I get that. It's not always a wise motivation to act on, but it's very human one. It's helped us to get where we are today.
Some people, myself among them, think it would enable us to do things we couldn't otherwise. Allow us to help each other, learn from each other. Working together to further improve ourselves and improve our lives. Also robots are just really cool, but boring when there's no "brain" in there, and progress on building human piloted gundam type mecha has largely stalled out.
> Throwing away context at the end of each conversation means that you can't form a meaningful social connection with an LLM, because you're starting from zero each time.
That's a feature, not a bug. But infinite context is probably something we will see at some point and then you'll have your wish granted.
Losing context is only part of the issue. LLMs don't have the rich connection to the world that humans have. The only connection LLMs have with anything is streams of text. Not only don't they have eyes, ears, smell, taste, touch--they don't have bodies. They don't have to deal with the real, physical world and have it push back on them.
That is so utterly different from what we humans are used to that we can't even comprehend it, and we anthropomorphize LLMs because we can't make any sense of them otherwise. We can't comprehend that these things that emit text that looks like human text don't have all of the other things that lie underneath our words when we humans talk.
I have LLMs right here in the room where I am that have vision, actuators, gyros, accelerometers, compass, audio. They don't have smell and they don't have touch. Integrating this stuff was surprisingly easy and I'm kind of impressed with how little effort it took to get an LLM to start using all these capabilities.
I'll bite. What is it that you're describing? Do you work in robotics or what? Sounds interesting, regardless of the debate about sentience.
> Do you work in robotics or what?
Right now, yes.
Horrifying.
> Instead of working with humans, I increasingly work with a fleet of agents.
This is why I see myself leaving dev within a year. Remote working is enough of a embuggererance for my mental health and I've be unhappy in work much of the time since 2020 largely due to that, AI will finish me off.
Career suicide and going of to wait tables, or be a hospital porter, or anything else with people even if I'm going to hate those people, with the massive pay cut that entails, will be better than actual suicide after a couple more years trying to stick at it.
There came a point a few months ago where I realised that I had to either accept becoming obscelete or change my while outlook on using AI and trying to catch up. I didn't choose the AI option.
It is frightening that it took only a few comments from the top of the page to the first mention of a suicide and also that it doesn't look out of place at all.
dspillett wrote that they would rather wait tables or be a hospital porter than commit actual suicide.
What else do you think will happen when you rob people of what for a very large portion of us is their feeling of self worth.
These AI characters are too busy saving the planet to care about the people.
I wonder how Hinton feels about his brainchild, I can see an Oppenheimer moment in his future and I hope that he's mentally prepared for that.
He might already be there.
“I’m trying to convey that there is a really serious possibility of something happening. If, for example, you’re told there was a 10% possibility of a bus crashing, you wouldn’t get on the bus,” he said. “The point of giving numbers is to say this is a serious risk. We don’t know how serious, but it’s serious.”
https://www.theguardian.com/us-news/2026/oct/13/geoffrey-hin...
Poor guy. He seems so nice and well meaning, this can't be easy for him.
> Remote working is enough of a embuggererance for my mental health
Where I work we have a 100% office policy. And while that sometimes feels hilarious, I am also somewhat glad about it because I like working and connecting with people physically.
However, as mentioned, I do notice that even though I am physically in the office most days, I spend an increasing amount of time alone on my computer and work with my agents. It’s satisfying in it’s own way, and I am definitely more productive. But it’s also worrisome. The pace of this development is picking up, and I don’t yet see where this is going exactly, or if I like where what I can see is going.
Related: We recently interviewed a candidate who worked remotely for the past 3-4 years. He said he'd be looking forward to work in the office again. I could honestly feel that he meant it, and I understand that. I guess some mix of both is the best.
Think carefully. I left high school teaching to become a dev. I would never go back. Yeah, social isolation sucks. But doing a job where your students/clients/patients think you're nothing but a waste of their time is soul-crushing on another level.
I've been thinking about it a long time. I've not been happy in Dev since 2020ish, pre-AI, it is just that AI may be the final bail of hay.
I would not go into teaching, that is something I considered earlier in life and rejected the idea of, and AI is a big thing there too, both dealing with students having it do their work and teachers using it to generate their materials.
Jobs where I'm more-or-less only dealing with people all have the “I dislike the general public” and “the pat is crap” problems, but won't have the “I'm sat on my own”, “everyone who is around me is turning into a meat relay”, and “to even just get by I'll have to interact with the giant purloining polution machines” problems.
> is soul-crushing on another level
They say a change is as good as a rest… :)
I've done sitting/standing in front of a screen for nearly three decades. I'm not looking for something I'll actually like doing to earn a living, it would be the height of naivety to even hope for that, but moving away from more & more of the same in ways I'm liking less & less might be good for me.
I don't have that experience
But my experience as a dev using AI now is that I am a waste of time, because I'm playing second fiddle to a fuckin chatbot
I want a job and career where I feel good about myself, where I feel valuable and useful, not this misery of plugging shit into the bot and listening to what it says
Yeah people here really underestimate how shitty your customers can be.
It’s very Asimov-esque. The Spacers who adopted robots for everything eventually stopped interacting with each other except over video calls.
> Instead of working with humans, I increasingly work with a fleet of agents.
I do too, but it means I have more time in my life to spend with people.
Seriously I don't get what everyone else is doing where they say things are getting worse somehow.
There's days I no longer need to sit at my computer: Claude Remote prompts my phone when it needs direction, the hardware stays busy and I can just go out with my family instead.
I am in the same boat as you, but like someone posted earlier in another thread, I concede that a large percentage of my value proposition has vanished in a year. What worries me is that I'm having fun at the bar but have no idea when closing time is. I suspect that same feeling is aching a lot of people. I for one have had multiple discussions with my wife the last 6 months about what we are going to do when I get laid off, especially because we built a life around the notion or remote work in a niche field which seems to be eroding away. Fortunately I've been wise with my finances, so I have options. But I too have considered starting over in another career. Sort of sucks, but I am coming to terms with it.
Beware of inflation, which may not be a constant.
> What worries me is that I'm having fun at the bar but have no idea when closing time is.
That articulates a feeling that I had but did not have the words for, thank you for your eloquence.
Modern day society has normalized being forced to sit in a cubicle or at your desk otherwise face the wrath of managers for not working, despite massive increases in technology that multiplied productivity.
I have no idea where people get this pipe dream that suddenly societal norms will change such that our employers will let us off the hook for most of the day. This should be obvious with the largely abandoned work-from-home allowances.
For me, I can't vibecode everything bc it won't make it past our QC, and the complexity of the work done means I have to babysit and often step in and either do it manually or correct bad assumptions/stuff that isn't sufficient for the task.
I don't know what kinds of work that isn't trivial CRUD work is or isn't so easily streamlined but I do know folks in more complex/more safety critical stuff, we're just having to fucking sit there and babysit. It's so isolating, and even though I spend more time Not Coding than before, team culture and really any feeling of having value, skill, or pride in my work is off the wayside because the retards in management think, since we can do some stuff more quickly now, our human brains will be fine with taking in easily 5x more context for significantly more complex work on shorter timelines.
And unfortunately that often also compounds with the current state of the world in general. It's another straw on the donkey.
It's more important now than ever to hold onto the fact that human beings have inherent value. This exists independent of their ability to generate wealth for an employer. I know HN loves a technical solution for every problem, and so do I, but this is a political problem requiring political solutions.
> human beings have inherent value
I wish, then I'd get a paycheck without having to do any work.
I think that is what we should do. It's called UBI. It acknowledges that everyone has inherent value, at least enough to deserve to stay alive and participate in society.
Only if you sell yourself.
No, you don't work w/ a fleet of agents, you use them to do your work.
This is a general issue that is being forced on us by AI companies; AI is not a human being, it is a tool that we can use to do things. We don't have them as coworkers or subordinates, they're objects specialized in language processing. Thinking otherwise leads people to all kinds of existential and psychological issues that in the end make no sense.
No. If these tools could do your work, people would feel existential dread anyway. It doesn’t matter at all if they are coworkers or tools (I really don’t see why would it ever matter).
> If these tools could do your work, people would feel existential dread anyway.
That is, if your entire self worth is tied to your work.
I think we should consider moving away from that notion.
> That is, if your entire self worth is tied to your work.
No. It's also if you take pride in your work and your self worth is partially based on that. And I think there's nothing wrong with that (quite contrary - I think it's great).
Personally I don’t care about that all that much. I don’t have any “sacred” inherent value for human beings. If they can make robots covered in real human skin, blood vessels and muscles (lab grown over the robot frame) with an LLM brain at some point in the future, I would definitely prefer that to a human lover.
It's more or less reverse to the mean. The period in which a people had both access to education and leverage through their labor and intellect is super short. For most of recorded history, the bulk of humanity lived as peasants, serfs, or slaves. They were mostly illiterate, bound to the land or to a master, and had little power to bargain over the terms of their (or their children) lives.
I'm too young to be financially secure given my impoverished background, not having parents in my life, being homeless as a teenager, etc. But too old and physically disabled to change course from tech. Nothing to do but keep walking through this tunnel and see where it ends up. It's scary!
I've had a couple of lucky breaks but I worry about my kids. About their ability to find meaning in life, their ability to find a way to stay afloat and about their ability to continue to form long term relationships with other humans who are turning inwards ever more.
It's crazy, intellectual and creative pursuits gave my life meaning and direction. What now? How do we create paths for our children to do the same given that it looks like the human experience is well on its way to full commoditization?
If you find out, let me know, mail in profile.
Ha! I was planning to reach out to you regarding this over the weekend :)
We've brushed shoulders enough on HN that I would like to strengthen that bond. I also wonder if building some kind of focused mailing list over this topic would be helpful to others. I think right now we all need solidarity more than ever.
same here, and it often keeps me up at night. for a few years i really felt like i had finally managed to get on track towards escaping the poverty i was born into, now i'm not so sure anymore in the long term. the good thing is at least i know i somehow managed to claw my way upwards from rock bottom to a tech job once already, so whatever the future might throw at me i'll probably be able to somehow deal with that too.
I wish I had your confidence. I also climbed up the ladder a ways but I've been hit by a pretty serious medical diagnosis, a layoff, and this AI stuff... I feel extremely screwed
"The arc of the moral universe is long but it reverts to the mean" is somehow less inspiring.
>For most of recorded history, the bulk of humanity lived as peasants, serfs, or slaves. They were mostly illiterate, bound to the land or to a master, and had little power to bargain over the terms of their (or their children) lives.
I feel like we should be doing everything we can to not revert back to that way of life.
No man wants to do the things that need to be done, maybe it's the birth control in the water
Disagree. Scribes, doctors, bankers, and government clerks always existed, even if the education to do it was expensive and exclusive. For the first time in history, this labor, including its knowledge content might become automated and commoditized.
Yes, on the other hand these positions were usually not given to serfs. Positions were usually filled by a ruler, selecting from his subjects, largely determined by birth rights.
Bankers did not exist in 10,000 BC
You're talking “always existed” too literally.
> the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.
Not for those without status (which is most people on this planet). But yes, AI is set to wipe out most established hierarchies and their gatekeepers, and to equalize most fields. A profound sense of dread is something most ordinary people live with throughout life, and if AI can reduce systemic unfairness, I bet most people will actually find some degree of relief from that dread. The complacency of the formerly-statused being shocked along the way isn't much of an argument against that equalization.
> Beyond just making you redundant in your job, the whole architecture of your existence, the thousands of hours of struggle, the sacrifice of putting off social games or personal enjoyment are now rendered moot in hindsight.
This passage answers itself, doesn't it? You can sense those things you postponed fun for (work and the apparatus around its struggle) were never worth it. So why fight for us all to have to keep working just to live? Isn't a post-scarcity world the thing we should all be fighting for?
>This passage answers itself, doesn't it? You can sense those things you postponed fun for (work and the apparatus around its struggle) were never worth it.
This is what you don't get. _I_ already know it's not worth it. I've always known. It's everyone else! It's a wife who cares whether or not her husband has a good career. It's the police who care if you look like you're poor. It's the way every single person in society obsessively cares about the status of others.
What in the world makes you think I've had to build up status for my own sake? It's absolutely baffling that anyone could be confused about this.
It's baffling to people that don't care that much about what other people think.
>A profound sense of dread is something most ordinary people live with throughout life, and if AI can reduce systemic unfairness,
I'll keep that in mind. If I'm suffering more than you are, the solution is to bring you down to my level.
Not only is that not a real solution to unfairness (it's certainly not fair for the person who now has to suffer more than they were), it also ignores the far better option which would be to reduce your suffering so that it isn't disproportionate to everyone else's. You might be holding onto to some self-destructive fearful crab-bucket mentality if that's where your mind instantly takes you.
I believe you're responding to sarcasm.
Pain is inevitable; suffering is optional.
A fallacious argument. There's no goal of inflicting suffering on other people; you can equally look at it as enabling people who were previously shut out of something to participate in it, or if not to participate in it directly to reap the benefits by having an agent exploit automation on their behalf.
>> the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.
> Not for those without status (which is most people on this planet)
Look at how coal miners react to the idea of getting rid of coal as an energy source; whether or not you think they have prestige or status, there's certainly an amount of dread there that I'd expect to somewhat resemble the type we're talking about here.
AI will bring on the biggest gatekeepers of them all: the ones that own the compute.
I think the fix is to get started on post scarcity economics now. Sooner or later we're going to have to come up with practices that boost or limit endeavors based on how people feel about them, rather than based on how much money somebody has.
It doesn't have to be ((post scarcity) economics), it can be (post (scarcity economics)). Both systems can coexist for a while, one providing a bit of a safety net while the other falls apart.
There will still be scarcity, just of different things.
Even if we vastly increase productivity beyond our current wildest dreams, the production system takes time to adjust to changes in demand, creating continuous points of scarcity.
But Georgist economics are way ahead of how to deal with this. Tax the ownership/control of things that can not be increased (e.g. land).
Sure, but that's no reason to continue using control over scarce things as a proxy for consent. It's not exactly working out.
We had to tolerate gold because it implied a game that was facilitated by the laws of physics, and we didn't have the resources to set up any other sort. But now we do, the rules can be whatever we want them to be.
Sure, let's try Georgism, but given the historical track record of that kind of reform, let's work on other things in parallel.
> continue using control over scarce things as a proxy for consent.
This is an expression for which I can imagine a multitude of specific meanings, can you clarify? I don't know of anybody who is actually using control over scarcity as a proxy for consent (consent of whom?)
Physics still puts bounds on production, and what we can have. Land is the perfect example.
Gold was only ever used as an accounting mechanism, and fiat money is definitely an improvement but accounting must still happen. We were never richer or poorer because of the amount of gold we had in our possession, it was merely used as a token of how much of others' labor we could have. As many current things that are scarce become less scarce, we'll still keep accounting for them. And as new things become scarcer, accounting will be just as necessary, by US dollar or by crypto or by handmade pieces of art that AI can't produce, or whatever.
Georgism's failures have been through landowners getting control of the taxation mechanism and taking the money back that belongs to the rest of society. Literally any other social change is going to have to confront that same question of power and allocation of resources too, it's nothing specific to Georgism.
I mean that if Alice and Bob each go to the bank with and idea and they each get a $1 business loan, now they control something scarce (a dollar).
Suppose I own a store, and they each show up and offer me $1 in exchange for my product. If I accept that dollar, I'm now participating in making their venture happen. If I refuse, I've resisted it in some way, however small. Whether I accept the dollar is a proxy for my consent re: whatever venture caused it to be issued.
A sane design for an economic system which was not strictly scarcity based would attempt create an advantage for ventures that are helpful, and a disadvantage to those that are harmful. One way to do this would be to provide enough metadata along with the money so that I can make a more informed choice about whether to create a liquidity problem for the holder of this money.
Like maybe Alice's venture is going to poison my drinking water, and Bob's venture is going to treat my dad's cancer. Then it's likely in my best interest to tell Alice to get lost but to support Bob. This would be a sort of nucleation site around which unions could form and solidarity could be built. Not trade unions, but rather a union of people with shared interest re: where they get their drinking water or somesuch.
We're allegedly into people acting in rational self interest, yet we've designed a system which limits access to the information necessary to behave rationally when deciding whether or not to engage in a transaction: "What behavior will I be enabling?" needs to be part of the equation.
I'm not an economist, maybe my idea about metadata on the money is silly. Probably there's a better way. But whatever it is it should be in some way sensitive to my values--even if they differ from the bank's values.
Our system is not. It assumes that both parties to a transaction have shared interests, which is increasingly not the case. It forgoes a bias for help over harm in favor of fungibility and liquidity instead. (I expect this property is a historical accident having to do with the unique chemical properties of gold, and not something we would have designed for, were we doing it from scratch. It succeeded because nature implemented the scarcity part for us, not because it resulted in an optimal set of economic properties.)
Re: Georgism...
> landowners getting control of the taxation mechanism and taking the money back that belongs to the rest of society
...that only works for the landowners if there is some central point in the flow of money that they can focus their corruption on: a government or a bank.
If you shift the whether-this-money-is-legitimate question to the next person to receive that money, recursively, corruption becomes much more expensive because it must be sustained for every transaction--it becomes something that they have to do continuously, rather than a battle that they need only win once and then defend for generations. The cost of doing so creates a drag which is not experienced by people pursuing pro-social goals (because holders of that money can reliably spend it widely, no need for coersion).
Whether money is "legitimate" or not it will be invented by whatever system you think you are replacing it with. Prices show up in optimization whether you want them to or not.
If you replace "government" with something like anarcho-syndicalism you shift the money question to asking people to behave according to some social norms. You replace the allocation problem with a different system of social relations, that's more opaque and harder to understand, but there's still the same allocation question: who gets what? Who gets to be close to the funny people? Or the artists? Who gets the better view from their house? Who gets all these things which now become the focus of the scarcity question?
Yeah sure, the scarcity is not gone, but its also no longer the only logic that's influencing the direction of the system.
I'm not asking people to behave according to some social norms. I'm expecting them to already have such norms and to use them to decide on a direction in which to shove the market. We've got plenty of horsepower, what we're lacking is a compass. I might not agree with the choices that get made, but I'm still in favor of them being choices that had humans at the helm, rather than the whims of an irrational market.
Whereas with a strictly scarcity based system, our preferences only half matter. It has created a dangerous blindness to outcomes. As long as the number goes up, we keep our foot on the gas, never mind the cliff over there. There are a variety ways to steer an economy:
- say yes/no to a transaction...
-- based on expectations about the value of what is being bought/sold
-- based on whether the transaction will enable the buyer to help/harm the seller or visa versa
I suppose that's 8 buttons to push. But we're currently ignoring the third dimension which reduces it to just 4. We haven't provided people with an expressive enough control panel to act rationally. Not enough input to make a good decision, and outputs that don't carry enough information to be used as actionable feedback, it's just flattened to a yes or a no.
> Prices show up in optimization whether you want them to or not.
Yes but they could carry more information, such as why you're getting a good or a bad price, which might lead one to change their behavior so that more people are more willing to work with them and they therefore get a better price.
> that's more opaque and harder to understand
Well yes, a car with only an engine and no steering wheel is indeed easier to understand than one with both, but I'd still prefer the latter.
Which is why I'm saying we need to start now. We can't wait for scarcity to be gone, we need to actively make space for something else to grow. Not so the new thing can replace scarcity, but so it can find an equilibrium with it.
agree. wealth inequality has only widened despite AI. things like healthcare insurance, tuition, rent and homes less affordable or more expensive than ever. Nothing has changed and probably ever will. Ai creates abundance in some things like knowledge but other tangible scarcities and individual differences of status remain.
The people at the top of the hierarchy are not very intelligent or engineer minded, so the more likely result of all of this simply to genocide the population. That's easier than trying to solve logistics.
It's mainly about status. No one cares that much about AI automating drug discovery or coding, but it's bad when it assists in math. Math results are attached to individuals , whereas for cancer drugs or other medical discoveries, it's more of a team effort by a lab. AI now has the effect of turning math into something that is more lab-like and less about the individual.
I would answer to this that the mathematicians (humans) are the ones who give any meaning to mathematical discoveries. If we don't have mathematicians anymore, the result is that AI can find whatever they want, but 99% of that will be useless waste. Outside of practical applications, the only reason most mathematical results have any interest is that lots of mathematicians spend time trying to find their solution. Once humans stop caring, mathematics done in an automated fashion will also stop having any value, other than its possible use in finding solutions to existing engineering problems.
> Once humans stop caring
As a former research mathematician, I've already stopped caring due to AI. I used to write expository articles on mathematics on the internet but I deleted those because of AI. Unless mathematics and indeed human endeavors can be made human again, I don't see much point in caring to generate commodities in a mechanical fashion.
We'll be free of cancer, and dead on the inside.
Counter point: AI is trained on output from humans. We all made AI, together. It belongs to all of us. We use it to benefit us, and we celebrate our wins together.
"It belongs to all of us"
For the low entry fee of ~$200 a month you too can escape the permanent underclass - which is more than some countries average citizen makes in an entire year.
Seems like a good deal
But it doesn't belong to all of us. It belongs to a bunch of rich egomaniacs who stole what all of us built.
Counterpoint: They request a subscription fee they refuse to reciprocate and will not take "No" for an answer.
Countrpoint: Their moat doesn't appear to be defensible, we can just replace them with somebody who will.
Counterpoint: Their moat is funding secured and contracts locking in the purchase of currently forecasted chip production. Even if you assume a catastrophic withdrawal of invested capital from those vehicles on a coordinated revocation of faith by the investing community; who do you plan on having operate this theoretical squeaky clean, ethically training data sourcing company? And for that matter; what makes you think you'll get passed the greed that is already sunk cost into the current gen of shysters?
This entire evolution is a generational project hoping to paradigm shift away from their predecessor's paradigm while ignoring the lessons of their predecessor's predecessors on the nature of greed, the corrupting influence of power, and the consequences of blatant cultural/ethical unhitching amplified by the reach amplification of technology?
The Stasi, or a perfect Jim Crowe couldn't be realized with manpower alone. With computers, and a small cadre of the unethical willing to succumb to the promise of lucre on the other hand...
When I was a kid I built my entire identity around being #1. I was the most talented kid in school, anything endangering that resulted in an instant identity crisis.
Then I went to college and I understood that there are people who, for one reason or another, are thinking in ways so advanced I'll never catch up. I'll never be the #1. I'm actually much closer to being average than I thought.
But this doesn't mean I should just give up, lie down in my bed and cry. Since then I've learned a lot of skills - most of them being crap at best, but that's not the point. What matters is my own subjective experience. The fact that despite objectively not being good, I see myself as a skilled person with rich life.
Once I was watching a YouTube video about elderly care and there was a phrase along "These people won't achieve anything more in their lives, my goal is to keep them occupied until they die" which is a bit morbid at first glance, but it actually makes so much sense as a life philosophy.
> Once I was watching a YouTube video about elderly care and there was a phrase along "These people won't achieve anything more in their lives, my goal is to keep them occupied until they die" which is a bit morbid at first glance, but it actually makes so much sense as a life philosophy.
So they'll keep us all busy with bingo games, shuffleboard, 80s TV re-runs, etc? I guess I don't see that as being much of a life philosophy.
Well, instead of someone keeping you busy, choose the thing you'll keep yourself busy with. Grow a few plants, for example. This has literally zero economic or artistic value or anything, but could be a fun thing to do.
The point is, the things you do don't need to serve some bigger goal. Your entire life doesn't need to serve some bigger goal. Just keep yourself occupied until you die.
And what will you live of, until you die?
People will respond to this in different ways, some will accept, some will transform, some will rebel, some will opt-out.
I've never been under impression that mathematicians chose their careers based on payout.
Yes, they all live like Erdos on other people's couches. None of them have kids or a mortgage.
[dead]
The only thing constant in life is change.
It would be nice if we weren't actively investing trillions in change for the worse (for most of humanity).
What a facile statement. Yes things change in life, but as society we try to avoid changes that rapidly render people and their lives worthless.
When did we last try to avoid such a thing? The way I remember history we celebrate this whenever it happens. Like how the electronic computer made the job of human computers obsolete.
> Like how the electronic computer made the job of human computers obsolete.
Human computers didn't celebrate this, the next generation did.
It was awful for the human computers, just like this transition is going to be awful for us, even if it leads somewhere better eventually.
> even if it leads somewhere better eventually.
That's not a certainty.
Every society, except the USA
Change is the only constant
The more things change, the more they stay the same
Incessant shit.
> You are made one step closer to complete worthlessness
And so the idea that your worth is the monetary amount of your labor compensation becomes even more toxic than before.
You are substituting one legitimate meaning of "worth" (moral value) with another legitimate meaning (economic value).
But then conflating the two to make your point, which if you care about making meaningful statements as apposed to word games, is not a legitimate move.
It would be nice if you were right, but the point is that it's not me who does the conflating.
Unfortunately, I think this is a human failing being exposed rather than anything else. We like to build conceptual models of how things work, but they are often tinged with should. Reality doesn't care how things should work. It only cares about how things do work. It's always jarring when the should turns out to be false.
Philosophers for millenia believed that human beings had meaning, purpose and set out to explore what that could be. It was a noble profession for a long time. Once we realized that there was no hidden spark, ergo no purpose, the study of and the prestige associated with philosophy took a nosedive. Yes it is still studied but it carries nowhere near the respect it once did. Mathematics seems to be suffering the same fate.
The next status wave will be fitness as glp1 drugs filter through to everyone and everyone wants to be fit.
Yes, and I think over time there will be a blatant inequality in who is commoditized and who isn’t.
If AI engineers didn’t create the data to produce the models, and they increasingly don’t write the code to produce the models either, then why are they treated as the agents of change rather than the subjects of change, like mathematicians are?
Nah. There is nothing beyond being made redundant in your job and labor leverage. That is all. It has got nothing to do with resentment over fucking machines. It’s all about people (the executives that you mention) fucking us over. The end. Completely human problem.
I know I’m being sold some bullshit when this is made into something inherently existential and psycho-identitarian. An egalitarian future of labor-worthlessness means that there is nothing above us to dictate anything at all—maybe not even nature itself, save for meteors and other cosmic events. I’m not afraid of being “worthless” in that world of complete freedom. Fuck no.
I am afraid of being hungry and homeless long before society reaches "an egalitarian future of labor-worthlessness" that's good for common people.
And that does not disagree with anything I wrote.
> I'm not afraid
So how do you plan to maintain food, clothing and shelter when you have no leverage?
Some have been scurrying to amass a nest egg and become self-sufficient. Of course, if demand destruction from the hollowing out of labor happens fast enough, who will consume and keep these stock prices up? With the tax base simultaneously crashing, gov't can't step in and prop things up. It could help during the transition while failing as a long-term strategy but what else can you do?
Self-sufficiently living off of stocks?
In capitalist economy, people who work are consumers and people who live off of stocks are self-sufficient.
Not reliant on their own labor for their survival is how I intended that. Obviously those folks still depend on the labor of others to make ends meet.
The rest of that quote is about the timeline where everything turns out perfectly but everyone is quote and unquote useless. That is not the only timeline. The timeline where the AI Ghouls get their way and oppress everyone else is covered in the first paragraph. This is to illustrate that the only ill-effects to my mind are exactly that: the humans who are are trying to oppress other humans. I don’t care if the machines make me jobless in itself. I have no psychological qualms with that.
You are made one step closer to complete worthlessness
Nobody's worth is tied to source code output.
Great, I'll just eat good intentions, wear a smile, and live under happy thoughts. I'm 50, so I'm "cooked" as far as contemporary society is concerned.
Now now, there are plenty of jobs for you to do. You'll just ve significantly poorer with no hope of social mobility for the rest of your life.
It'll be fine. You voted for strong labor protections, universal healthcare and a welfare system right?
> Nobody's worth is tied to source code output.
If someone's expertise is worth nothing to those around them, they have a real practical problem.
In this context, the word "value" isn't referring to some moral model. It is referring to the asset value of skill.
85% of people hate their jobs and just want to spend time with their family and friends. Utilitarianly, AGI is a benefit for the vast majority of people. And people who get their pride from playing status games are about to learn what it feels like to be on the other side
Maybe I'm too doomer, but if a large portion of the population becomes economically worthless, how can this not have a destructive effect on society? It would be easier for anyone with power if you wouldn't exist. How would that not lead to corrosion of basic human rights (or worse) and the spawn of a new permanent underclass which you cannot escape from? It sounds very dystopian but if AI is left unchecked this will happen imo.
It already has happened with technologies before AI. The internet itself in many ways has disrupted communities a lot with the net effect being the erosion of individuality and value placed on the uniqueness of people. AI is just the next stage in the cohesion of technology against the human spirit.
So it's already happening and is just strengthening now with AI.
For me the most obvious conclusion is, use your maths skills and get into open source AI.
Make AI as cheap and accessible to as many people as possible and we will likely be much better off.
The more people empowered by the tech the less like you will be “redundant” in the way you describe. You will own the important tools only the techno oligarchy own today…
Would you feel the same if you were a part owner of OpenAI? I bet you’d feel excited about the money you’re about to make ?
isn't math skills really just pattern analysis and creativity? how is this limited to just the math domain? why are mathematicians worried?
I don't really know but I can see two obvious outcomes.
1. Sam Altman et al owns all money, or power and all labor.
2. AI becomes cheap and ubiquitous, everyone "owns it" and we can see the contents of the sausage and modify it as required.
Mathematicians are uniquely placed to help avoid 1 because AI is based on math & algorithms.
> Mathematicians are uniquely placed to help avoid 1 because AI is based on math & algorithms.
It's based on centuries old applied math.
You know we (tech as an industry) have done this to a bunch of people already, right?
Thats what "disrupt" meant
> bargaining with superintelligence
This part is nonsense, still. There's no known AI -> AGI path, and superintelligence doesn't mean anything, and AGI is assumed to be (going to be) both malicious and a slave, both for no good reason.
It has already been happening for decades if not centuries. Instead of bargaining with superintelligence though, it's bargaining with moronic systems. I watched an airline gate employee struggle to allow a family onto a flight despite acknowledging that they were confirmed and had seats on the plane because the computer wouldn't beep when they waived their boarding passes. She called her supervisor. Her supervisor resolved it not by writing "it's okay they can get on" into an exceptions field, but by kicking them off the flight, putting them on standby, then accepting their standby request and reseating them (in different seats, not all together, for some reason). The humans involved all knew things were okay, but the system was too dumb to just allow them to go forward.
Superintelligence might even help.
“Worthlessness”
Consider this… in the grand scheme of things, we are on a tiny planet in the universe. What cosmic contribution could you make?
Most people won’t make anything even close to “a ding in the universe”, as Steve Jobs has been reported to say.
So they may as well enjoy their life. And your worth can be to your family, to your children, your parents who took care of you.
Learning a hobby. Practicing religion. Mindfulness and meditation. Doing sports and improving your own body.
Look around if you are a heterosexual man, for instance: many women are far more attracted to men that perform in the moment, than men who labor quietly to create a product. The former enables untold hours of honing social skills and calibration, and humor and performance skills. The latter is of more durable value to society, yet relatively few women are genuinely sexually attracted to a man doing the deep work. (By the way, it is also true the other way: most men are not attracted to women for doing deep work or being a rocket scientist.)
Animals don’t do “deep work” and most animals are attracted for far “shallower” reasons. If everyone has a UBI and AI does the “deep work” then that frees up humans to be like animals in a zoo or in the wild… doing all the displays to attract mates, having babies etc. That’s not nothing — in fact it is 99.999% of everything that happened in history and 100% of what happened in prehistory.
You’re just ceding that 0.0001% which represents technological progress. The actual dread comes from the fact that you and all of humanity will be as completely powerless to change things as cats or any random animal species is powerless now. The difference is you will perhaps understand your powerlessness more than they do. But in the good scenario, you’ll be like animals in a zoo. We are constructing this zoo for ourselves now.
We will remain on this blue planet while the AIs will colonize the universe. In fact I wonder why aliens would ever come to Earth, when they’d just send their AIs or robots. Look at this: https://news.ycombinator.com/item?id=46200502
Maybe I'm obtuse, but I just struggle to understand why we as humans, at this stage, knowing what we know, we think that there's value and honor in trying to build new things and try to move off our planet. I mean I get it, there's the exploration spirit, there's the pushing of the frontier. But our generation, in striving for such things, has somehow disconnected ourselves from where I believe the value actually lies. Taking care of loved ones, of our children, of pets. Teaching our children to love and to be gentle and to take care of others. Appreciating the beauty of the universe, because no machine will ever be able to do that. Creating art, and beautiful music, investing inside of these works our emotions rather than physical resources so that we can create feelings in other. A machine can never make real music or art because it doesn't come from feeling and experience.
I just don't know.
You are of course correct. The true value lies in relationships, art, love. This will never be threatened by AI etc. These are just tools, however disruptive. But the society we build might be misaligned from these true values. We need a revolution.
As Ernest Becker would ask, then what is your "immortality project"? Even worse, what's the immortality project for the vast majority of humanity? Do we just bring back religion to Middle Ages levels? Or do we hope everybody goes through some kind of never-before-in-history spiritual awakening?
"If everyone has a UBI" - isn't that quite a big "if"? And also - how soon would it come? Soon enough for families losing their income?
We won't be bargaining with superintelligence, we will be bargaining with the handful of oligarchs that are creating super intelligence to rob us of all bargaining power.
This is why they're talking about it in religious terms. It's their salvation from labor and its demands.
This isn't a passive process that's happening by itself. It is driven by specific people with specific goals.
[dead]
[dead]
> How is one to feel safe in the progress of the future when the only hints of the wonderful post-scarcity utopia we can quantify so far is the removal of all human leverage via their labor?
I'm sure the Telegraph operators, human calculators, mine workers and all sorts of people have asked themselves this question. It's obvious that it feels strange to suddenly be part of the cohort asking but I assure you it's not new. And the answer hasn't changed much since.
> I'm sure the Telegraph operators, human calculators, mine workers and all sorts of people have asked themselves this question. It's obvious that it feels strange to suddenly be part of the cohort asking but I assure you it's not new. And the answer hasn't changed much since.
I don't buy this argument, much like I don't buy the similar "they said the same when calculators were introduced".
It negates that some changes are qualitatively and scale-wise different. Also, by gradually chipping away at things that humans do, you (we) keep shrinking and shrinking until there's nothing left, every step of the way some people claiming "well, the telegraph operators said the same". Keep in mind that, while it's hyperbole, some AI entrepreneurs are actually saying stuff like "maybe jobs will become a hobby". Unrealistic for now, maybe, but if they do become a hobby, what leverage will we have at all?
In other words, it's true the past is worth keeping in mind, but not to the exclusion of the possibility that this time it's different. Or that the trends of the past actually point to harm in the near future.
And like to anyone else who hints that “automation creates better jobs”, I have to add: but not for us. Maybe for the next or the generation after that.
Industrial and technological progress has birthed better conditions overall, but at a cost of locally worse ones. Mine workers that got displaced didn’t end up being machine operators - most of them landed on benefits or sporadic gig work. Only the next generation was able to be properly trained and educated to be able to benefit from the technological advancement en masse. And we might see that happening again.
so our children and future generations are not "us"?
In the I was referring to people reading my comment as “us”.
While it is nice to think of better futures for coming generations, I think we (reading this) also have to think about our more immediate work, as in next 2-5-10 years.
[dead]
> the removal of all human leverage via their labor
I’m not convinced at all that SI will ever be able to produce the correct result with little input. The advancements in capability I’ve seen in models is that they understand my intention better, and follow industry best practices better, so their output is often acceptable. But I absolutely have to make my intentions known, as detailed as possible, or I will get the mediocre, standard solution that falls out of the weights.
I’ve been having trouble with models always steering towards certain solutions - they insert a queue here, a mutex there, a backoff over there. If I remove those, and let the model freely rebase the PR stack, then it will re-insert those mechanisms because it has been specifically trained to do exactly that. It’s essentially boilerplate, smoothly aligning my codebase to match their weights. Which is often good enough! They are training the models well. But individual problems require individual solutions.
My personal take on the human part in all this is: We get to decide, and the models obey. Our decisions are now super leveraged. So let’s make good decisions, and communicate them well. The layer of where decisions yield the most leverage, has been pushed upwards (I see it on the type and function signature level, others will see it even higher), but IMO it won’t be possible to push it up much further without losing truth to the decisions we need to make for a good product.
I can kind of relate but I feel like I am mourning something different: the loss of the desire to seek truth in tech, math etc.
Nothing is validated or tested anymore. It’s all about the shape or believability or even how loud it is.
They dropped 100+ papers no one read. I am sure mathematicians are reading them now but even this forum has concluded that these problems are solved, without reading or validating.
It’s like everyone forgot “don’t believe everything you read on the Internet.” And it makes for a very boring discussion when everyone is just spewing unvalidated claims, layering them with strong beliefs founded on similarly unvalidated claims. It’s dull, without substance and just pure noise.
> I feel like I am mourning something different: the loss of the desire to seek truth in tech, math etc.
This is what I've been feeling too. It feels increasingly like the professional world has become a tower of scams. It's not worth building technical skills or focusing on quality; your boss doesn't know or care what good work looks like, he/she just wants your work to look good enough to his/her boss, who knows even less about the actual work being performed. And everyone reports up to an MBA CEO who was brought in from a completely different industry.
I want to work in an industry where skill, diligence and expertise is rewarded and respected, but from what I'm seeing political ability is 95% of the game and caring too much about the work is a career liability.
I only see the loss of desire from the anti-AI crowd, if they ever had it to begin with.
https://matharena.ai/literature_mistakes/
You today can go into the arxiv and find major errors after a few papers with no formal training. How is that not AMAZING!?
Not sure I agree - they dropped hundreds of papers, most of which were so badly written that they were incomprehensible. If it wasn’t for the lean certificates on 30% of them, the mathematical community would have told them to go and whistle.
I think the consensus here is that most of the papers are effectively correct. But it would take years of study just to understand most of the problem statement - so we have to wait for the mathematical community to digest the borderline-nonsense that OpenAI spewed out.
Most likely that they look really correct and are not and require immense effort to understand, if even possible, to disprove their claims.
What we have is a DoS in research verification.
Just tell the (AI) authors to go back and write clear explanations for why they matter and what the specific claim is.
I gotta say I think mathematicians brought this roblem on themselves, most math papers are absolutely incomprehensible to anyone who is not an expert in the field. Mathematicians spend so much effort trying to make their argument to other mathematicians that they've gradually abandoned making their field or findings comprehensible to anyone else and left that to science journalists. In turn, those journalists churn out very formulaic stories written to a template: 50% the human story of the mathematicians social/academic wanderings in pursuit of their goal, 50% an explainer with some nice graphics on the subfield they operate in and a mention of what sort of problem their insight solves, perhaps with a few colorful metaphors thrown in.
The last mathematical paper I can think of (but I'm not an expert, I'm sure there are some others) that was both mathematically rigorous and accessible enough to be readable was Mitchell Feigenbaum's Quantitative Universality for a Class of Nonlinear Transformations. This is accessible int he same way as a textbook - you have to work at it, but hte author is explaining his thought process and drawing the reader's attention toward things that matter or stand out as unusual.
Including those with Lean verifications?
Yes, for all we know they're exploiting a bug in Lean.
Sadly, if it were a bug the AI would have used it to prove everything, like in the human written hoax of the Collatz conjeture.
Even if a person manually wrote a paper and 10 person manalluy reviewed it, it is possible that all 11 of them made a mistake.
It seems like, once the IPO is done, this flood of papers will dry right up: the inherent value of the papers themselves is unlikely to justify the cost to churn them out. It makes as much sense as OpenAI churning out hundreds of slopped programs no-one wants to run, or hundreds of novellas no-one wants to read: it's just showing they can.
These papers are just first steps. They are going to keep churning out papers, in part because the generated papers are going to get better. And better.
+1 We need human verification.
It's the ultimate proof of how fashion and religious believe override everything else. It's fashionable now to believe in these unverified fairy tales, and if you don't, you're at best a weirdo who will be "left behind".
You're not wrong, but we should not absolve those who perpetuate the fairy tales from their share of the responsibility.
I think for me is, that history is not kind to the people who are left behind.
For example, weavers were one of the most powerful professions (if thats a thing) in the middle ages. In england there are the wool churches(not entirely weavers....), in the low countries, the weavers guild fomented revolutions, and shaped the borders of countries.
But that came to a crashing halt over the space of 200 years. slowly at first, then all at once. the power loom, then cheap imports from india turned a cottage industry into dark satanic mills. Yes there are/were specialists that still hang on, but they are like football stars you can probably name all the people who make money, the rest are barely scraping by.
For us, in the programming world, we will not have a 200 year grace period. Yes, AI produces shit code, but thats not the point. its cheaper and there are vanishingly small places were "good" code matters anymore. AI could be used to help a large range of people, provide support, trusted advice.
However because of the people who are funding it, that will not be the case. it will be used to reduce wages. Just like offshoring, resulting in china being the worlds manufacturing hub, AI, and the current people funding it will aim to make the expensive worker redundant.
In short, I weep, because Altman, and the money behind him ans his ilk want me destitute. I have committed the crime of wanting to earn a wage, and thus I must be punished.
It's profit chasing as well as practical - it kneecaps the educated workers increasingly asking questions about the morality of business and technology. It takes away the income that affords them the space to push back, threatens high ed, critical thinking. The salivating over AI makes perfect sense in that context.
Profit is the driving force behind civilization, motivating innovation and progress and allowing us to live longer and better lives instead of remaining stuck in the Stone Age.
That's too cheery - it both pushes innovation and is leading us towards planetary catastrophe.
So the dude in the cave that invented fire on demand was chasing after "profits"? Progress and emerging "civilization" clearly could not have been motivated by profit. We could argue money has distorted a natural, inherent, human drive to improve his and group's living condition.
"AI" tech is actually a perfect specimen here: this could be a genuinely positive net for Humanity if only it wasn't controlled by the sort of scumbags that are only motivated by "profit".
Depending on how broadly or narrowly you define “profit”, your statement is either too weak to be useful, or false.
If you define it so broadly that the hunter-gatherer inventing a spear for hunting more effectively is “profit”, then your statement is a tautology. The spear innovation was driven by “profit” and the “profit” is having the spear. Great.
If you apply a narrow, capitalist definition of “profit”, the statement is false. Plenty of advancement of civilization was and still is being made without or outside of capitalism, especially if you go as far back as the Stone Age that you mentioned.
Profit is what's on the verge of destroying our civilisation, too. Ever heard of climate change and biodiversity loss? And of course the energy crisis.
And we are not trying to address them, we are just trying to make more AI and more techbros.
Yes, it is _a_ driving force, but its not the only one.
the enlightenment was driven by "we can do thing better" and "to know god, we must know the world". A lot of money for research came from the catholic church (including for Galileo who wasn't put in jail for heliocentric, he was put in jail for being a massive insufferable arrogant prick)
In england, a lot of research was carried out by bored vicars, on CoE dime. (well pennies)
So yes, profit can be a motivator, and it can be very effective, it needs to be tempered with humanity. We could go into who a lot of brands we see today that are old were founded by quackers, but we dont have time for that.
the TLDR is, yes profit is a motivator, but its often short termist.
So how do we push back?
Tweek or reinterpret Copyright law.
Declare that the current use of copyrighted works is not fair use. Create a new "generative use" clause which permits copyright use consumption and transformati on provided a number of conditions are met e.g. not-for-profit and for-public-be nefit organisation.
Place a tax on current AI company outputs, similar to CD or harddrive tax in Fra nce. This new "generative use" entities would not have to pay this.
The government/people "owns" all of the land and resources in it's borders and makes it available to people or companies on various conditions. You could interpret that they also "own" all of the copyrighted works (i.e. things become public domain at death+70years or whatever the current scheme is) and choose the mecha nisms, fair-use, exclusive copyright etc, which are or aren't permitted.
Hope that open weight models destroy the monopolies?
Help open-weight models destroy the monopolies.
How do open weight models help devs to keep their jobs? Seems like the same outcome on different computers to me.
Closed models run by ~3 mega corps on monopoly is worse than ~anyone being able to use them at will.
It does not solve the problem but makes it more tractable.
Open weights doesn't solve the problem, yes it _might_ cause the bubble to burst, but it won't put the djin back in the lamp.
A brief look at labour relations in both the US and UK suggests that sustained direct action is the only real way to force change. The hard part for us is that for the last 25 years we've all been told that unions supress wage, and that we are all expert negotiators. Unions would be the natural place to start, but they are but a former shell of what they were (for good or ill)
I don't have all the answers but my gut is: resist convenience, educate yourself on how tech is steering politics, give input to city planners on projects, if possible increase your agency so that you have the ability to push back and influence career choice/conditions (build up financial security, live below your means, advocate for housing).
This piece circles, without naming it, around the concept of “deontic scorekeeping” central to the work of Robert Brandom on semantics. The idea that our ability to create and sustain conceptual systems depends on the give and take of reasons within a social setting. Brandom has been front of mind for me lately because elements of his theory both explain why LLMs should work in principle, and where they are bound to be radically deficient. An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time.
Worth reading: Robert Brandom, “Making It Explicit: Reasoning, Representing, and Discursive Commitment”
> An LLM ... cannot be a real participant
The participant is not an LLM, the participant is a company. The company learns. It adapts. It has a trillion dollars of skin in the game.
I mean, many companies operate within particular domains but don't have much skin in the domain itself in a way that is meaningful to other participants there -- like Oracle builds software for healthcare, ostensibly, but any actual user of their products will tell you they are nightmarish with regard to usability, and research shows that patients are often harmed as a result. So healthcare isn't their game, it's selling healthcare software.
Likewise OpenAI doesn't have "skin" of any shape or value in e.g. the mathematics community or even in mathematics research per se. In my cynical view, they are exploiting the exposure of mathematical research to automation in an effort to create a case study of their product obsoleting a highly visible and well-leveraged and relatively small profession. Regardless, I doubt even they would say they have durable skin in mathematics in any sense, certainly not in either of the senses I mentioned above. But their output inhabits the same space as mathematicians, who have to contend with the consequences
OpenAI most certainly have a "skin" in math and physics.
They hire math and physics grads and good command of math is very useful in computer science, machine learning and AI.
What qualities do the recruiters at OpenAI value in the mathematics and physics graduates that they hire?
And how do you see these qualities changing over, say, the next decade?
OpenAI is concerned about math and physics insofar as these skillsets enable their current staffing model. That is not exactly durable.
> The company learns. It adapts
And potentially steals [1].
[1] https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_priority...
I don't see any theft in that article. Who was deprived of what property? Why didn't they make backups?
Don't be obtuse, it doesn't make for productive discussion.
> The idea that our ability to create and sustain conceptual systems depends on the give and take of reasons within a social setting.
I read this sentence 5 times and I still cannot parse it.
I think it means if we all have to justify ourselves to each other, we collectively come up with an sustainable ecosystem of ideas that we can understand. So if we stop justifying ourselves to each other, the ecosystem doesn't get built or sustained, and we stop understanding things. Not the person you responded to, but that's my reading.
If a short sentence like that could accurately summarize Robert Brandom’s deontic scorekeeping, he would have wasted a good chunk of his career.
I mean this with zero snark: if you’re interested, you’d be best off asking an LLM or doing a search. But the overall idea is that Brandom described how concepts and meaning are created socially by a kind of game in which people keep track of the claims that other people commit to.
If you accept Brandom’s framework, then it can be argued that LLMs can’t meaningfully participate in this process. This could be considered an explanation of why people have a negative reaction to LLM involvement in activities that are meaningful to humans - even though most people won’t be familiar with the theory, they would instinctively recognize that something important is lacking, such as commitment over time.
What a coincidence, I've also picked up inferentialism but never really got into it.
I'm interested in the reversal that Sellars proposes - it seems clear to me that "representation" is not prior to the space of reasons, despite how unintuitive it is.
> An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time.
I see no reason why this can't be the case in the future, even though I think inferentialist-based theories are generally strong. Can't you force LLM's (or whatever AI architecture will exist in a decade) to stake durable commitments itself, then? They're at least autonomous enough to run computers. I agree that that is a core part to true meaning & autonomy, but nobody just lets them yet. I don't think inferentialism is a consciousness-based argument, either, so that escape hatch isn't available. I also don't think this is a necessary condition to engage in the space of reasons (but maybe then I'm misunderstanding it, or retreating to a more Sellarsian understanding). Could you explain on this more?
Though again, I only have the book sitting on my desk, and I haven't cracked it open...
I actually agree with you. LLMs could become participants in the future, but this would require (1) durability of real awareness over time (an architecture change from the current fixed-weights + transient context architecture) and (2) that humans grant them the status of equal participants (probably not a terribly high bar given how people already treat these things). I'm not aware of any real progress toward (1), and all the stand-ins people have developed live in the context layer or textual artifacts, which are poor stand-ins for durable memory.
And yeah, consciousness based arguments are bunk IMO. It's clear that LLMs are in a basic sense "aware", and even to some extent "sapient" (in Brandom's usage), i.e. they can justify claims and object to things at least locally. I follow Rorty on the question of consciousness, which is to say that consciousness has no deeper meaning in practice beyond "are you awake", and in philosophical discussion it's largely a proxy for "are you one of us" in some amorphous sense.
Well that's too bad, since Rorty completely misses the point on consciousness.
Consciousness, the interesting part anyway, is qualia, or the "it is like something to be me" as Nagel put it.
Intention, as Dennett used it in his book "The Intentional Stance" is the "are you one of us" angle, and is distinct from consciousness because as the book title suggests, it is a stance taken with respect to some other system (person, cat, dog, turtle, LLM) rather than a description of subjective experience.
So I think it is your (and Rorty's) positions that are bunk. It is very, very far from clear that LLM's are in any basic sense "aware" let alone "sapient". They are extraordinarily able to manipulate language in a way that leads to text generation that has traditionally been assumed to indicate consciousness and/or intelligence. But we have no way of knowing if it is like anything to be an LLM, even though most of us are busy taking the Intentional Stance towards them.
I guess I'd say that your qualia aren't a something, but they're not a nothing either. We can't say what they are. They're not publicly describable, and therefore not publicly attributable. As Wittgenstein would say, "They cancel out."
And as I think Dennett himself would say, qualia are just Sellars's Given rearing its ugly head again.
Well, while I love much of Dennett's work, I emailed him after "Consciousness Explained" was published to say that I was disappointed that it appeared to describe what we are conscious of, not how we are conscious. He sent back a short note to say "You're probably right".
This sounds super interesting but 477 pages is a beast. Would you recommend anything about this subject thats a bit more approachable?
Brandom wrote a shorter version of the book, called "Articulating Reasons", which is available in full on his faculty page [1], or from booksellers. He's a very wordy writer (kind of exhausting to read, even for someone who enjoys philosophy), but if you get ahold of Making It Explicit, chapters 2 and 3 have the meat of the ideas I'm talking about here.
[1] https://sites.pitt.edu/~rbrandom/Courses/2022%20Phil%20of%20...
Edit: One of my ways of benchmarking new AI models is actually to type a version of this prompt in and see how sophisticated the response is: "Talk to me about the concept of deontic scorekeeping in the context of llms". You'll get good mileage out of that.
I think in future, each company will built its own universe of mathematics consisting of prized discoveries. Maybe one Mathematical Universe per Company in the multiverse of mathematics.
Ok what about memory systems though (basically LLM modifies its own RAG context) - that should allow commits and changing dispositions over time?
RAG context memory and modifying the weights are two different things. As far as I know, models can't modify their weights reliably.
Humans change over time due to what they experience. It’s different from waking up from a ten-year coma and having access to a record of what happened to yourself in an alternative timeline where you didn’t fall into the coma.
AI systems have Reinforcement Learning
tbf naming something "deontic" is not helpful
This may not be helpful or strictly accurate, but when I see deontology or variants of that word, I substitute "SHOULD". It is like a duty or obligation.
Super roughly:
* Virtue ethics: character and motivations
* Deontological ethics: rules and obligations
* Consequentialist ethics: utility and outcomes
NOTE: This would suggest a surface reading of Brandom's "deontic scorekeeping" might be keeping score of who has what obligations, who is following or breaking rules, etc.
Then the argument against LLMs being party to that is that we don't really assign roles and responsibilities to LLMs, they don't really have "duties" or "obligations" in the way a member of society will have. We don't negotiate with them over who should do what, they don't contribute to the rules that are set for them.
I'm literally just guessing based on the context here though, since the few times I've tried to watch podcasts with Brandom I generally bounce after 15-20 minutes feeling I haven't really understood anything he is saying.
The irony of complaining about an academic jargon word in a thread about mathematicians.
Mathematicians always define their terms exactly, in order to begin their discussion. Philosophers never define their terms exactly, because then they would have nothing to discuss.
Deontic - `Relating to or expressing moral duty or obligation`
Deontology - `normative ethical theory that the morality of an action should be based on whether that action itself is right or wrong`
I don't think that's true. Why is it that I see more people using "shaped" and "load bearing"?
Because language is mimetic, and people are consuming huge quantities of AI-generated text. I.e. saying that an LLM can't be an equal participant in our language games isn't saying that it can't play by the rules. In fact that's what LLMs are trained to be best at.
What an LLM can't do (at least with the current architecture) is sustain a commitment that it can be held accountable for later. It doesn't have durable "perspective". It can't update its priors. Maybe a new architecture will come along that gives LLMs memory plasticity, at which point they will shift into being individuated quasi-persons, but that's not what we've got now.
I've written some about this here, if you're interested: https://elliotmilco.substack.com/p/deontic-scorekeeping-comm...
Rather the opposite, AIs are arguably deontically asymmetric because they are powerful and virtual digital entities, and thus should be very concerning to the human species. Not because they current have no internal abstract state for learning state like humans and animals.
You don't consider recursive self-improvement a step towards 'updating a perspective'?
I think you are also suggesting Mathematics and Language are the same thing.
Recursive self improvement is a bulk process that shapes the weights of the next model. The weights are still frozen. Each individual "agent" would need to able to durably update its own dispositions to get to what I'm talking about.
I'm not suggesting mathematics and language are the same thing, but I am saying (which shouldn't be controversial) that mathematics as a discipline exists within the medium of language, and that mathematical reasoning, discovery, and proof happen via people proposing, justifying, testing, and objecting to formulations in language. This isn't a metaphysical claim about whether math is real or captures inherent structures of the universe etc.
Ok, then what is an agent that provides a 'thinking' or reasoning loop around said model?
You ended your first post with: "An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time."
And so if there is an agent, a thinking loop, that can recursively adjust over time...how does your statement hold up? I'm particularly talking about 'staking durable commitments or changing its dispositions over time'. Recursive self improvement is all about these 2 things, especially when it's the 'outer thinking loop' doing the improvement.
But you're right, Mathematics exists as a medium within Language. But then Language is just a transfer of ideas from one mind to another.
This principle is the heart of what the article is talking about: a transfer of logical mathematical ideas passed from a human to another. And the sense of 'shared understanding' that comes with it. It has nothing to do with keeping score.
It's about getting a 'sense' for why someone thought or rationalized an idea in order to connect some other idea. You get an 'ah-ha' moment, and some of those moments are gone, or a different path is taken to reach the moments now. That is what the 'weeping' is about (not that I agree with the article. I think anyone can spend however they want. Math doesn't have to change, it's that people want it to, apparently, because it's 'easier' to do something different, or for some other reason I don't know about).
Interesting point
It would only be true if you consider Mathematics and Language to be the same thing.
Both depend on Logic, which is where "deontic scorekeeping" comes in. You can only establish something (score) is true and believable by 'keeping score' and having some level of understanding about the language or math you are reading.
The article is mostly referring to the connection that comes from shared understanding between humans, particularly mathematicians, as they build new shared understanding, and realize there are limits to what they can know and 'leave behind' logic for others to pick up and carry on.
That connection has been severed, or strained a bit, but I'm sure new ones will be created. It just might be of the same flavor that mathematicians prefer, therefore they weep.
And if I understand "deontic scorekeeping" correctly, these forums are a better representation of the concept. You will lend some amount of belief to what is said/read based on what you know.
Ideas are contagious and AI is our most powerful vector yet.
I suppose we would see some interaction with AI in that sense, but it'll take place over the course of learn cycles and when AI starts ingesting its own ideas it typically goes wrong somewhere down the line.
I think Thompson has no understanding of endeavors like pure math or the theoretical sciences. In the words of Einstein, "I want to know how God created this world. I am not interested in this or that phenomenon, in the spectrum of this or that element. I want to know his thoughts; the rest [is] just details."
I think mathematicians aren't bemoaning their lost sunk costs. They're now aware that they may never again have the joy of being the first to pull up the edge of the tarp of ignorance and see the raw truth beneath _for the first time_. Their voyages of discovery, which give life meaning, may never leave dry dock again.
It is strange, because in a sense, when you start learning math, you never explore, you are being taught. Now mathematicians were able to exercise curiosity. That is going to change. Back to the ranks of eternal learners if you will. The fun was in the research. Now I think it will leave the door open toward more Applied math. What can people do with all the discoveries. The curiosity will be there. I think it is better that way. It will also put a cap on abstractions that overlaps. In essence, AI is less likely to create new theories, coin new terminology, etc. But it is better suited to recognize emerging patterns. That will probably simplify mathematics. Unless the domain of abstraction is infinite and with each new abstraction comes its own results but I doubt it. Who knows though...
> The story of human discovery and the triumph of the human spirit will soon be excised from this discipline. The Dinitz-Garg-Goemans counterexample was the most egregious demonstration. It revealed that the process of prompting novel proofs will be as auraless as ordering doordash. Watch as magic and mystery evaporate. Watch as the sun sets on our heroic age. Is there not something evil in the act of blocking all future generations of mathematicians from the experience of discovery? Forget about accuracy or even attribution. Something fundamental to the experience of mathematics is being taken.
> You are a helpless onlooker. Before you a channel through which humans have accessed the ineffable and sacred for thousands of years is being sealed for eternity.
https://kirwinhampshire.substack.com/p/the-dark-night-of-mat...
As someone with friends in the mathematics community, they've expressed similar sentiments.
I have to wonder if this is the equivalent of teleporting back in time and finding Marco Polo or any of those early explorers who traveled for the love of the game (and fame and money), and opening up Google Maps on your phone and showing them the zoomed out world. And showing them any island or village they could name.
my analogy is that it is like an explorer being handed access to satellite imagery. and then being told that your experience will still be valuable in telling the satellite where to point and interpreting the photos. folks who got joy from traversing unknown terrain might find it distressing to become an image analyst.
This is absurd and silly. Discovery doesn't go away because more new things are being discovered. This position only makes sense if you make an arbitrary cut point where some instances "don't count". If you were to go back to scientists or mathematicians several hundred years ago and tell them that one day science would be done by tens of millions of full time researchers around the globe, often working in teams, some of them might also bemoan the loss of the "hero scientist" as any commoner could now participate. But science and math have only benefited from this, and this has increased, not decreased, the beauty and wonder in these fields.
It is not the same because the act of problem solving for discovery is now to be done exclusively by machines, as humans are no longer remotely competitive.
If you read the linked article, the author goes into more detail about how doing mathematics also has been an act of communication with past and present human mathematicians. This is very much true when math and science are done by millions of human researchers, but not so when it is done by massive swarms of agents at a couple labs.
I am sure science and math will benefit in the sense of solving more problems than ever before. But you are not really understanding the agony of the humans caught in the change.
> they may never again have the joy of being the first to pull up the edge of the tarp of ignorance and see the raw truth beneath _for the first time_
Until we can demonstrate AI consciousness, “seeing the raw truth” will be a human activity. Even now, verifying and understanding these AI proofs is a massive endeavor. Coming up with an intuitive (to humans) distillation of the finding remains an open challenging demanding expertise.
i think to certain types of people (who are probably more likely to be mathematicians), the "self" discovery aspect of it is important, they don't want to read about a new proof for a problem they were grappling with, they want to come about it through their own thoughts.
LLMs are not feeding us things, we are asking it and getting answers. This dialectic I think is super powerful and it still requires the human to ask questions and continue expanding intellectual curiosity to go down the rabbit hole and discover. LLMs are just an incredible rubber duck but the discovery is still there.
> This dialectic
We literally had a case where OpenAI may have stolen the dialectic from a mathematician using its product and then given it to a proprietary model burning tens of millions of dollars of tokens [1].
[1] https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_priority...
I think I disagree, if I had a puzzle on my desk and I told an LLM to explain how to solve it, I think I have robbed myself of the enjoyment of solving the puzzle. Is it really any different than googling a solution?
There are different avenues of "solving" something though. Asking an LLM to one shot something vs iterating on it collaboratively are different ways of achieve potentially different ends.
If the result is black and white, then yeah, its probably not super fruitful but if the end result is subjective, then there really no is objective "solve".
> Until we can demonstrate AI consciousness, “seeing the raw truth” will be a human activity.
I don't think this matters a bit. We individually can't demonstrate "consciousness" in other humans, it's not even a word with a definition. What we can do is miss the feeling of discovering something on one's own rather than having it explained to you over and over again until you understand it.
I mean, dead mathematicians are definitely not "conscious," but their books keep you from discovering things that you may have discovered yourself if left to your own devices.
The joy that was mentioned is the joy of knowing a bit of gossip that no one else knows, of being the first explainer. Interpretation is just being able to pass gossip on; a decidedly lesser joy. And there's no reason that AI shouldn't be able to explain findings in an intuitive way - LLMs at least purely operate based on human intuition as written in human communications and then captured in matrices.
If these proofs are good (doubtful, imo) and can be reduced (intuitively distilled, sorry), there's no reason that an LLM shouldn't be able to do that.
That's such a nice way to put it! I recently stumbled into "The Last First- Winter K2"[1] and saw the same spirit along with a similar misguided effort to win without the will, nor capacity to do it — e.g. influencers taking a stab at K2 is like prompting your way to results.
1: https://en.wikipedia.org/wiki/The_Last_First:_Winter_K2
Why is not like changing to a different type of vessel for the voyage? Unless and until it is all AI in closed system, could direct machines where to look etc.
Climbing a mountain is different from flying a helicopter to the top
True, but then the moonlanding wasn't possible on foot.
Only the journey is different.
As they say, the journey IS the destination ...
And the sense of achievement that comes with it..
> They're now aware that they may never again have the joy of being the first to pull up the edge of the tarp of ignorance...
Not "never again", it would be extremely difficult but science and math are also built on random, trial and error, and intuitions that could not arise from AI.
That's the issue - trial and error. A human can try a couple ideas in a day, but realistically, there are not that many sane ideas.
Computer can iterate without frustration on a completely random garbage. Once in a while it stumbles upon a gem. Even if it's one in 10000, that's still hundreds of good ideas in a day (provided enough GPUs).
Your comment assumes math and science are some kind of closed systems that can mechanistically pursue any endeavour but reality (whatever it is (e.g. the universe)) is much more complex than that, and there are realities that are outside of those systems. Not saying that humans can see all those realities though.
Maybe? I'm targeting math, which doesn't need any kind of reality check. It has axioms though, so in a sense it's closed.
You can generate completely random ideas (a couple proofs I saw where you define some weird set, pick an element and derive a contradiction when the set is empty or whatever).
Some of these will reach a conclusion: proof or a counterexample. And that's how speed can outperform creativity, if the speed bump is fast enough. It's the same as with those monkeys and Shakespeare.
I'll say it again: AI is capable of answering the questions put to it but AI is not yet capable of knowing which questions are worth asking. Mathematicians have no cause to weep, they will be fine.
Most of these questions Mathematicians posed to each other have no practical application, and probably will never have a practical application. (ex: the solution to the Navier Stokes question doesn't change anything at all.)
In many ways, the teleporting machine that teleports to the solution is against the point of the questions. No one wants the solution to be "Run $10,000,000 worth of tokens to cover all possible Lean formulations of the problem".
I think for Mathematicians, I'd say that the human level formulation is still possible and with a new perspective. We as a society (and outside of Mathematics) need to also understand that this AI teleport to the answer thing has less value than we all expected. The value is in the journey.
The open question is if people will still want to take the journey if the teleport machine exists. Or if the existence of the teleport machine halts exploration.
> "Or if the existence of the teleport machine halts exploration."
You're overlooking that the teleport machine costs lots of money to use. The AI providers are not going to waste money by teleporting randomly around.
Someone is going to have to find a new destination that is worth spending the money to teleport to (or walk by themselves) and that someone is mathematicians.
How rich do you think math majors are? No theoretical math professor or department can afford $20,000,000 in tokens for these problems.
Meanwhile, the AI labs are doing that for just marketing.
Right and the AI labs will stop just as soon as the need for marketing is over. We're agreeing.
Since when have tech companies stopped spending on marketing?
What marketing benefit are they going to be getting when all existing noteworthy problems have been addressed? They're just going to spend money on having the AI generate proofs of no value at random to thunderous public indifference? Clearly not. There will be a need for more noteworthy problems for their AI to publicly solve to get any marketing buzz and the consensus for what is noteworthy comes from mathematicians.
Yes. They'll stop after there's no more noteworthy problems. What a wonderful concept for theoretical Mathematicians.
You realize that they don't want that, correct?
[flagged]
> “Google Translate is amazing, but it has taught us essentially nothing about linguistics,” Strogatz said. “We don’t understand why it works. It just works. That’s the Twilight Zone. It is knowledge without understanding.”
This is not true, and it is the crux of an enormous debate we're now having across many, many fields.
Classical Chomskyan syntax (which models sentences as parse trees) has been enormously, astonishingly predictive of the rich structure in real-world human language. But we are quick to forget that we invented concepts like "words" and "noun phrases" specifically to help us analyze and predict things about human language. What LLMs and modern MT shows, definitively, is that there is another, totally separate way to describe these phenomena, which is both vastly less descriptive, and also vastly more predictive.
That these appear to trade-off is unfortunate, but it is NOT the case that we "don't understand why it works." We absolutely understand why it works, if we didn't understand we would not be able to improve these systems. What we do not have is a clean, descriptive theory that boils this down to something that is formally specifiable and analyzable. But those are different things, and this is the root of the disconnect between many communities working in and around these spaces (linguists vs computational linguists; mathematicians vs computational mathematicians; mech interp people vs empiricists).
"We absolutely understand why it works, if we didn't understand we would not be able to improve these systems" - Understanding how to improve something does not require understanding how it works.
Growing a plant does not imply understanding soil minerals or photosynthesis. Training a dog does not imply understanding it's brain. Using LLMs to improve a codebase does not imply understanding C++. Improving a PC with new parts, does not imply understanding RAM.
You can understand abstractions about the system, without understanding the system. Without gradient descent or other forms of automatic optimisation, do you think humans could build something as capable as GPT-1?
Not sure if this will bake your noodle or not, but yes this is basically my point. Saying I understand how dog training works has very little overlap with saying I understand how the dog brain proteome works, even though they're related, and even though I might (or might not!) understand better how to train my dog for having fully understood the proteome.
Saying we understand training and deploying machine translation systems that have strong predictive (rather than descriptive) power, apparently has very little overlap with defining a formal, analyzable, mathematical model of language. Mathematician who refer to "understanding" generally mean the second thing, but the first thing is also a kind of understanding. Just messier.
I think they're referring to the fact that neural networks are more or less black boxes. Their internals are blurry and not explainable in the same way that traditional linguistics tries to explain things - with rules.
Of course, ultimately that's because language itself doesn't follow simple rules like linguists would prefer. They're more like guidelines.
Yes, that is one of the points I am making. Chomskyan syntax is a strongly descriptive theory that is entirely ex post. MT and LLM systems are strongly predictive and have no descriptive theory. It is completely wrong to say the first is "understanding" and the second is not. They are both different types of understanding. In particular, Chomskyan theory clearly does not represent enough "understanding" to predict real-world English sentences with similar lucidity to that of LLMs.
> LLM systems are strongly predictive
No, they're not predictive at all. They're generative. A prediction is a forecast based on a theoretical model about a state of affairs in the world. Making a guess what the weather tomorrow will be is a prediction, and a prediction can be true or false. An LLM generates text based on certain parameters, and output it generates aren't true or false, they're simply whatever the system is tuned to produce.
LLMs are terrible at making forecasts or world modelling precisely because they don't actually operate on much of any model or theory, they have no notion of truth or falsehood, they simply generate whatever you want them to generate.
If you'd like to use that definition of "predictive" in this discussion that's 100% fine with me. But, you should do so knowing that in ML and AI "prediction" is a term of art that means "a statistical model's output given some input." LLMs are one example, in that they take as input a sequence of tokens and predict the next token.
In this specific context, the point is that LLMs and MT "predict" lucid-sounding sentences that incorporate linguistic structure (syntactic and semantic) that are well beyond the predictive abilities of the Chomskyan theory of syntax (which is itself a generative model). In other words, the Chomskyan model of syntax is descriptive of much of the rich variation in real-world human language. But it is not predictive in the sense that it does not provide a constructive algorithm to emit lucid-sounding sentences (and does not attempt to, either).
They're generating by prediction. That's how they work - the predict the most plausible next token.
I still think mathematicians are going to be as important as ever. The LLMs are only finding what was technically possible given our current breadth of knowledge after brute force solving problems. Eventually it will run out of things that it can do with its training and the only way to get it to do more will be training it on new human found knowledge.
Finding that new knowledge may be exponentially more difficult, but it will be much more valuable.
> Eventually it will run out of things that it can do with its training and the only way to get it to do more will be training it on new human found knowledge.
How do you know this?
AIs cannot invent things like cubism, surrealism or hip-hop.
Or maybe it can; the early image generators famously did output horrific images sometimes because of their limitations, but these outputs qualify nonetheless creations.
Now the interesting thing is that it is humans who evaluate them as horrific, worthless or interesting.
Certainly AIs could pretend they have that sort of opinions and judgement, but hopefully humans will remain more interested in the opinions of other humans.
> Certainly AIs could pretend they have that sort of opinions and judgement, but hopefully humans will remain more interested in the opinions of other humans.
Online, how would they even know? Offline -- either we give up on the internet, or the offline discussion will be heavily influenced by what AI said online.
Amusingly, LLMs are good at detecting patterns because they are modelled after the human brain, which is very good at detecting patterns (this e.g. is annoying when you are a game developer who wants to reuse assets). People are still able to tell if the source of an artwork (or text) because LLMs tend to output "patterned" text (like the famous "load bearing"). That being said, the false positive/negative rate of humans for this task has yet to be measured.
I don't see how its output possibilities could be infinite given a finite set of data. Taken to the extreme if you give an LLM only a single piece of information then that is going to be the only thing it can produce. Given it two pieces of information and maybe it can produce every combination of those two things. Given all of the worlds knowledge I'm sure its possible outputs are incomprehensibly large, but still not infinite. Then having its outputs match a set of really specific instructions is going to be a small subset of that.
How is this different from human brains being made up of a finite number of neurons in a finite number of configurations? I know very little about LLMs and nothing about neurology and maybe you can explain to me why one is so much more powerful than the other.
As I understand it, the advantages of AI have not just been speed of processing.
Humans have cognitive limits. Humans hold a specific slice of all knowledge at a time and necessarily limits what correlations can be leveraged. This is similar to "context windows" in modern AI. Human minds are wildly variable in efficiency. AI is has constant inefficiency, largely controlled by the speed of light, physical RAM, and processing power, with superior (no cost) "thought" context switching. The AI also has superior accuracy without the need for "concentration" because the AI is processing on a Turing machine with the limited operations that it's superior at in terms of time and energy - eg find an element in a list of 1 million elements.
> a finite number of neurons in a finite number of configurations
Neurons and synapses change, especially synapses (the connections between neurons). Some neurons have a few connections to other neurons. Others have hundreds of thousands of connections to other neurons. These grow and change. "Estimates are between 100 trillion and 1,000 trillion neural connections" Is it finite? Depends on your definition of finite. If anything countable is finite, then it is finite but the number may be practically uncountable. Is it fixed? No.
That's 100G to 1T. Neurons are more complicated than weights, so add a x1000 somewhere, but they have more redundancy because they are noisy, unrelilable and may even die, so divide by some small number.
Sure, a given LLM can only produce so much (though it might be a lot), and then you'll need fresh inputs. But you contend that those have to be human, while I am arguing that they could be
- random,
- from the environment,
- from other AIs.
That increases the space considerably.
In particular, note that your argument also applies to humans: A human, being finite, can only produce finitely many things. But you add other humans, the environment, and pure chance to the mix, and it'll be pretty infinite for all practical purposes.
model collapse seems to say that "AI" cannot train on itself.
I would be curious to see the end result of letting it endlessly update its inputs from its own outputs. I was under the impression that eventually this would cause delusions and if you let it do this long enough and it is solving problems that no humans can verify as correct that it isn't useful.
Maybe that is the limits of my imagination, but it seems unwise to take humans completely out of the loop. Having mathematicians around that can understand what it is doing so that we can utilize that knowledge in a way that we can have some confidence in seems important.
There are two reasons:
1. LLMs aren't run deterministically - they use a bit of noise to improve the output. So even with the exact same input you're going to get a whole distribution of outputs.
2. Feedback from the environment. LLMs (and humans!) work best when they can test ideas. That's why coding agents work so well. If you ask an agent 1000 times to just think of a compression algorithm, it'll probably spit out very samey answers just like a human would. If you give it a test corpus and let it try out different things it will absolutely be able to come up with novel ideas.
Well humans do that and you expect them to continue to do that - why can't AI models do it as well?
Typical thought experiment - go back to the early years of humanity, even before we've created language itself. Communication is basic grunts and other such things, cutting edge knowledge is 'poke him with the pointy end'. Compile all of this knowledge into the training set for an LLM, let it run with infinite processing power and infinite time. It'll just continually remix the knowledge of that snapshot of time. You're certainly not getting to the Moon, discovering the secrets of the atom, or even discovering mathematics.
It sounds increasingly dumb to bring this up given how advanced models have become, but it remains true - they are still fundamentally token prediction algorithms. And that imposes hard constraints on their potential. These constraints are relatively academic in our current world because there's just so absurdly much information and no human can hold more than a negligible amount in his mind at once. So they'll certainly be able to connect many 'obvious' dots that we don't see simply for a lack of carrying capacity, but they're not going to discover concepts that do not clearly follow from the comprehensive knowledge of an era.
Perhaps they could some day. I'm basing my understanding on the current day capabilities. Once that changes I will update what I think.
It seems far more likely to me that current LLMs have learned generic reasoning capabilities which will allow them to solve effectively infinite problems, rather than whatever vague limit on capabilities you seem to think there are.
I don't think AI will continue growing infinitely either, but maybe "incomprehensibly large" is large enough to solve all the math problems.
The set of data will increase as it outputs
I liked reading an essay by mathematician Nestor Guillen that develops thoughts along these lines - the "convex hull of ideas".
https://birdsnfrogs.github.io/2026/09/12/Felix_qui_potuit_re...
Yeah, I think that's a great essay and it really helped clarify some of my own ideas.
At the same time, I think it's pretty amazing how this is the fundamental question of AI's trajectory, yet people on both sides seem so sure of their viewpoint with little actual evidence or data to back it up.
That is, it's possible that LLM-based approaches will be forever limited to this "convex hull" of current training data, and thus humans will always have a role to play (and will retain primacy when it comes to overall intelligence), because it appears that humans are not limited to this convex hull based on previous discoveries (e.g. I think basically every scientist would argue Einstein's general relativity was outside the convex hull that existed at that time).
Yet, if LLM-based approaches are eventually able to surpass humans on this "novel discovery" axis, then we as a species are basically done for when it comes to intellectual primacy - and then it becomes a very strange and scary world. And many AI researchers at the top companies clearly believe current approaches will scale and eventually (pretty soon I'll add) lead us to RSI.
I have no idea which side is right, and I'll admit I really hope LLMs eventually hit a wall. But again, I just am surprised that many seem to be putting forth their beliefs, but I would love to know if there is any good evidence one way or the other (or better yet, a more concrete and verifiable definition of the "convex hull" so that researchers could provide some falsifiable examples).
Came here to say this. What AI is doing with math is essentially fuzzing, but looking for valid inputs that solve a problem and the inputs are mathematical abstractions. It's a cool approach, but AI doesn't understand what's going on, and without understanding it won't be able to create the next generation of mathematical abstractions needed for the next generation of proofs.
this is assuming that humans can be original in ways llms won't be, which I doubt will hold true
> which I doubt will hold true
Why?
Because I don’t think there is any fundamental computation we can perform that ai can’t
[flagged]
> SI
opinion discarded, stop trying to make fetch happen
SI?
Super Intelligence, courtesy of Donald Trump: https://www.forbes.com/sites/conormurray/2026/10/08/trump-ca...
You have to give credit where it is due: super intelligence is just straight up a better term. Broken clocks and all that…
No, it's not. It's a bullshit marketing term by a narcissist who wants to call everything he has a hand in "super" or "great".
The field had already settled on "artificial intelligence" (AI), "artificial general intelligence" (AGI), and "artificial super intelligence" (ASI), and while there are definite gray areas in the definitions, it was widely agreed that AI referred to the field as a whole, AGI was intelligence that was "general" in the sense that it was as good as humans in all domains, and ASI was intelligence that surpasses the best human experts.
Trump wants "SI" to replace all uses of "AI", which is simply stupid bullshit, especially given that current AI includes many levels of intelligence, most of which are far, far from "super".
aren't mathematicians needed to create the problems? Everything solved so far is a question posed by a human. Which is probably why cracking diseases is more difficult (apart from man made viruses :/)
As a software engineer it is hard for me to feel sorry for these mathematicians. Software engineers are getting laid off because of advancements made by AI. Math researchers are employed at universities, and universities aren't laying off mathematicians now that OpenAI has solved some of their puzzles. Mathematicians can still feed their families, software engineers displaced by AI can't. Yet I see far more of these blogs about the plight of mathematicians, not enough about the plight of software engineers.
When software engineers spent the last 50 years eliminating the work of clerical, secretarial, and dozens of other back-office functions, were they concerned about how those people would feed their families? Do you think anybody but other software engineers feels any sorrow for them today, now that they are facing the same fate?
That wasn't software engineers eliminating that work. And neither is it today AI eliminating software jobs. Let's name the straw man in the room: it's humans displacing humans, and extracting the value for themselves.
Just like in past history.
Yes, it sucks, for software engineers, those in the past, and those in the future. As long as you and others fall for the straw man perhaps you and others will get more and more tricked and hurt, misplacing your aggression and hatred on each other rather than those receiving most of the extracted value.
> extracting the value for themselves
Yes in some cases, but the reduced costs also allow lower prices which benefits all consumers of whatever product is being produced. At least for those products where there is a competitive market.
While I mostly agree with you, there's an argument to be made that classical programming tended to eliminate the mechanical parts of work that people hate doing, while AI is removing the intellectually stimulating part.
It's not fun to think of a future where knowledge workers become irrelevant in society and we're back to just physical work and managerial elites.
I'm not really one of them, but there are many developers for whom the intellectually stimulating part is designing the system. Writing the code was boring drudgery that they hated doing. They tend to call themselves "architects" or something similar, not "programmers."
How long until the LLMs run circles around the best software “architect” out there?
And then "ideas are cheap" becomes the dumbest thing. I always knew people would laugh at us 1000 years from now just not why.
Finally, someone who gets me!
Plenty of people pay the bills with work they hate doing.
What an asinine comment. Do you think I, as a software engineer, am sitting around twirling my mustache, cackling as I plot to destroy livelihoods? I'm trying to make a living like anybody else. I've had my own opportunities lost to outsourcing and H-1B's since day 1.
The economist John Keynes predicted almost 100 years ago that technological advances would lead to a 15 hour workweek. If AI now makes you 5x as productive, then why aren't you working 1 day a week for the same pay? Why didn't Keynes' vision come true.
You may not have sat around twirling your moustache but you have actively been involved in an industry where the goal is to automate stuff that people would previously do manually. That is the entire point. Which, if you believe the message on many threads/posts here, puts people out of jobs.
I don't believe it is that black and white. Yes, automation eliminates some jobs, but overall moves everyone (on average) further up the productivity line, which improves lives, again, on average.
That is why we live like kings now compared to people of the past.
But circling back to the point, if AI does the same thing, eliminates expensive manual coding and other similar work, that is essentially the same thing we've been doing to other people for decades.
It is not for the same pay. Real wage increased dramatically during that time period. And despite all obsoleted jobs, society is still more or less at full employment.
A living like anybody else or a living at the expense of everybody else?
Before calling that comment asinine, did you consider that the commenter might be reflecting @flowerlad's comment back to him/her to prove a point ?
> Why didn't Keynes' vision come true.
governments implemented (and still implement) keynesian economics, and it has turned out to be a very bad idea and he was very very wrong.
There are orders of magnitude fewer mathematicians with full time academic positions than there are software engineers. The economics are different and consequently academia tends to be more slow moving in reacting to market trends. It doesn't mean it isn't coming, but it will look somewhat different
I don't personally grieve for the likes of Strogatz or Tao, it seems likely both of them will be fine however this pans out. But many more mathematicians have been toiling away in precarity and obscurity and without tenure for years and years. And of course many mathematicians went into private sector roles as software engineers, actuaries, analysts, ML engineers and similar, and have been displaced there, too.
And highly visible blog posts written by famous mathematicians with large audiences and closely aligned with the news and hype cycles getting frontpaged on HN doesn't mean that mathematicians are disproportionately whiney, nor that software engineers aren't also writing about their plight or ending up on the front page as well.
To me this goes in the same direction as people not feeling sorry for software developers because our job is supposedly automating the job of other people (which I don't think describes the profession accurately as a whole at all).
Neither sits right with me. Can't we just all collectively acknowledge that we have to grapple with this new piece of technology that either is already or at least seems to be going to be hurtful to a lot of individuals?
Instead of looking at the in comparison limited group of people currently pushing this technology and asking ourselves whether we should be on board with their doing, we look to one another and are happy that someone else is now suffering just the same as we are. What a sad state of affairs.
100% agree. The comment you are responding to is basically playing "victim Olympics".
AI is going to suck hard for lots of professions, software engineers and mathematicians included. I think it's much more productive for folks in these professions to show solidarity with each other given the way AI will disrupt their livelihoods, rather than showing disdain based on who you think may have it slightly better.
I agree with you because I so thoroughly understand what mathematicians are saying. Expertise and knowledge developed over decades, previously unique and marketable, and status conferring, are threatened in their relevance because of ai.
I know this intimately because I identify, roughly, as a programmer first and security researcher second. Guess which two fields ai companies decided to specialize in first? Yes, the FML ones.
The intellectually rewarding part of my job/skills - the only reason I enjoyed them - are, not completely gone yet, but threatened. And shrinking. This is a very scary observation.
I know from ‘isomorphic situations’ lessons in history that fighting the tide is pointless. I have to decide intelligently how to respond. Easier said than done when your livelihood and identity are at stake.
For now I still have a job but I and everyone around me is worried. In a year, At best our jobs will be very different.
So yes I empathize with these mathematicians enormously.
I don't empathize with them that much, as many of them share in the blame of creating the broken system we have today. They want the status that comes with tenure and having a nice math job, and then pull up the ladder for everyone else.
Math journals commonly reject papers as “out of scope” or “not of sufficient interest” without an editor or referee actually reading them. So it’s not as if the “math community” was checking these papers for accuracy in the first place. Now AI speeds up the creation of papers that go unread, and we’re supposed to pity mathematicians? If journals and experts can’t be bothered to read the papers, why should authors bother to write them without AI? And suddenly mathematicians are worried about AI papers being “too hard to read,” even though that wasn’t a concern pre-AI, when no one was reading them anyway. Too bad. The incentive structure now points toward AI, and mathematicians share the blame for that.
As a software engineer? The career whose very reason, largely, for existing in this market is to automate jobs, with few notable exceptions? That one is somehow deserving of sympathy and that of mathematicians is not? Come off it. The notion is undignified.
A few notable exceptions?
Are phones an "automation of existing jobs"? Maybe couriers (??), but most of what people use them for goes far beyond that.
What about the software used to drive trains or autopilot planes? We still have conductors and pilots, but now the outcomes are safer on average and more efficient.
Or something like Google Maps -- all that replaced was the paper booklet people used to keep in their cars, it's not like we used to employ 'navigators' to go out on a road trip.
Or something like Intuit QuickBooks: there's still an accountant in the loop, and if anything more accountants today than there were a few decades ago.
Certainly some software is essentially automatory in nature. But to characterize 'nearly all' as being such just betrays a lack of awareness for the rest of the field.
> That one is somehow deserving of sympathy and that of mathematicians is not
He didn't say that mathematicians aren't deserving, he was griping that people said this about SWEs. Do you think mathematicians are deserving but SWEs aren't?
> Are phones an "automation of existing jobs"?
You seem to be unaware of the existence switchboard operators (people redirecting calls to the correct person) were hundreds of thousands in the US alone at some point.
How did you manage describe a litany of automation while being fully unaware that is what you were doing? I mean you even said the word autopilot lmao. Yes, not all tasks in a job are fully automatable. iPhones, automating communication. Quickbooks and turbotax, automating business workflow steps.
The words and excels of the world, automating documentation and bookkeepery. It all makes some kind of existing thing we already do faster and more convenient, and often more efficient. Most software simply does nothing new. The thing you didnt manage to list which is the actually good example is games.
> Software engineers are getting laid off because of advancements made by AI.
The bean counters laying off software engineers ought to take notes from the frontier labs' own hiring practices. They're still hiring lots of humans for business and engineering roles even when they would be better positioned than anyone to automate away those jobs with their world-class models (https://openai.com/careers/search/).
They're hiring now. Clearly some researchers at OpenAI believe we'll eventually all be unemployed: https://x.com/alexbronzini/status/2103846255227175288
Holy logical fallacy shit
What exactly is the logical fallacy there?
"OpenAI is hiring software engineers, therefore all companies will continue to hire engineers"
Generalization fallacy, survivorship bias, etc.
[dead]
This is a silly analogy, and you're comparing very different entities.
Say you have a friend who likes making music (not professionally - just a hobby).
If an invention came out tomorrow that took the joy completely out of making music, would you respond with "It's hard for me to feel sorry about you."
Just because things suck for you doesn't mean it can't suck for others (in different ways).
It's not about universities, it's about grant funding over the next few years. It is very apt to collapse if AI is more useful than people. Why pay people to find this when tokens will do instead.
I'll let you in on a secret: In the US, most math professors don't seek grants. In most universities, I don't think grants are even a factor in their tenure application. It's quite different from engineering.
In my undergrad math department, only 2-3 faculty members out of a dozen bothered with grants.
[dead]
Who will continue to develop neural nets if the mathematics training pipeline dries up? Will we see the frontier labs relinquish AI research to the machines themselves while their humans apply for unemployment benefits?
Currently math research serves both as a training pipeline and as a math-production pipeline, and its funding is set based on those two outputs. If it is "just" a training pipeline, do we need as many mathematicians?
Who has been funding grants, and do you really think the grants are going to stop? There are infinite number of puzzles in Mathematics. The grants are not from companies like Boeing and Walmart who urgently need solutions for the problems, if they were I can understand why they might stop.
Why is it hard ? It's basic empathy.
What does your being software engineer has anything to do with feeling sorry for Mathematicians' job ?
It's a classic "I have it worse than them...everyone should feel sorry for ME and not them".
Software engineers are getting laid off because of the end of zero interest rates, wars started by Putin and Trump, and a bit because of pandemic over hiring. The part that is due to AI is less hiring of juniors. Know your enemy before starting a fight.
What's your source for "Software engineers are getting laid off because of advancements made by AI". I haven't experienced that in my field and can't find any data to support that notion.
Software engineers still can feed their families. The notion that software engineers can not feed families is not consistent with reality.
Sounds like you don't know any junior software engineers that got laid off.
Do you? Most of the software layoffs had to do with higher interest rates and changes to R&D tax policy. AI isn't the major factor.
The idea that a junior software engineer who can't get a software job _and also can't get any other kind of job or move somewhere cheaper to live_ to the point that they can't afford to feed their families is what's unrealistic.
It's not like the options are write software or starve. Billions of people find jobs and can feed their families without writing software.
I also don't often meet junior software engineers who actually have families to feed. Most juniors work in the industry for a few years before deciding to get married and have kids.
> move somewhere cheaper to live
Unfortunately this often means moving away from family. What are you going to say to the kid whose parents toiled to get him through college on the expectation that he would be able to take care of them? "Sorry Mom, I have to move to North Dakota to work on the oilfields"?
> can't get any other kind of job
At that point they won't be software engineers
I think your expectations of who gets to do math is uncalibrated. One famous math researcher who solved a major open problem was working at Subway while still doing math on the side.
A very small fraction of people who get math degrees actually continue to do math in their careers. They usually have to get other jobs in order to survive (actuary, teacher, finance, etc.). Academic math funding was already drying up for last few years, and now AI will probably cause more cuts. Not to mention that academic math positions have never paid particularly well.
I feel bad for anyone who can't find a job, but the software industry as a whole has been the cushiest industry for decades, and the current turbulence seems more temporary (and driven by corporate greed) than a systematic destruction of the trade.
It’s only been a few months since LLMs started solving significant open problems in math. I wouldn’t jump to the conclusion that they have it better than software engineers.
In many ways it will likely be the opposite. Tech firms still need engineers to talk to customers and run experiments. What will the mathematicians be needed for?
> Software engineers are getting laid off because of advancements made by AI.
This is said by a lot of CEOs trying to make layoffs look good, but I haven’t seen anything that suggests that it’s true.
> Yet I see far more of these blogs about the plight of mathematicians, not enough about the plight of software engineers.
Really?!? We’ve been living through 3 years of blog posts, LinkedIn hot takes, and podcasts talking about the plight of software engineers in the era of AI. There isn’t a week that goes by without an HN post about it.
I can feel empathy for the practitioners in a field that may have been profoundly changed, the same way I can feel empathy toward software engineers who are struggling with the changes in their own field.
In the end, we’re all trying to figure out where this leads. And how this will affect our lives and our work.
The difference is it takes great intelligence to be a mathematician, whereas anyone can be trained to write code. It’s like the difference between an automotive engineer and a mechanic. I would argue writing code takes less skill than carpentry.
Do you write much code? Do you design software? It sounds like you don't. The challenge with software is managing complexity. This is unique to this field. Mathematicians are humans.
Unless this is satire, have you not considered how many proles have been laid off because they were replaced by technology or told that a spreadsheet model had decided their job was superfluous? Capitalism has been using technology to fond new ways to enrich itself for the whole age of personal computing, and arguably long before. Those Andreesen Horowitz comments about 'software eating the world' may have been exciting to people working at startups, but came off as rather sinister to people who were in the world outside softwareand just wanted to build a modest but comfortable life rather than develop unicorns and cash out.
[flagged]
> I am not one of those AI people who believes that hundreds of millions of jobs will be wiped clean from existence in the next five years
"Wiped clean" is the wrong phrasing, and 5 years is a short timeframe given the ability of some sectors of the economy to resist change. But yes, hundreds of millions of white collar professionals will need to rethink their careers and their business models within the foreseeable future.
Just today there was an HN article about how shady the dental industry is. In the future, you will go to a special dental imaging facility where a modestly-compensated technician images your whole mouth and feeds the results into an AI system, and when you get home you can look at the assessment and chat with the AI about it. Then you can potentially go visit a dentist, oral surgeon, etc, if you actually need work. This system will produce far better dental health outcomes at far lower cost: the diagnoses will be more accurate, and there will be no bias towards additional, unnecessary procedures.
> In the future, you will go to a special dental imaging facility where a modestly-compensated technician images your whole mouth and feeds the results into an AI system, and when you get home you can look at the assessment and chat with the AI about it. Then you can potentially go visit a dentist, oral surgeon, etc, if you actually need work. This system will produce far better dental health outcomes at far lower cost: the diagnoses will be more accurate, and there will be no bias towards additional, unnecessary procedures.
Right up until the sellers of dental supplies find a way to...
Indirectly influence model training to increase the frequency with which models recommend dental work and/or lower the bar for what they determine to be "necessary"...
or...
Directly influence model recommendations by paying model providers (who will be happy to take that money).
The point being that models (and particularly the humans that control them) are presumably just as corruptible as the humans they are replacing.
The advantage is scale. If a dentist is shady and overprescribes it can be difficult to find out because there are not many victims. If a company serving millions of people is overprescribing there’s a much higher chance of them getting called out on it.
That's the optimistic take. I have a much more pessimistic take: there is now only one place where people who want to sell dental devices have to focus their efforts. Less effort for them... and because of the scale... much bigger payoff.
When I was thinking of the examples I gave (poison the training and/or pay the providers), I was intentionally drawing analogs to how I saw Google search decline.
First, SEO "experts" poisoned the results. Then Google saw how much money could be made, and they sold the ad space directly to marketers. If you search for "Best <widget> for <doing something>" on Google, I am certain you are not getting the "correct" answer. And Google has suffered no consequences for this. To the contrary: it has been extremely lucrative.
Unfortunately in the real world it's just the opposite. A guy selling opioids on the street is called a dealer and gets arrested. A company overprescribing opioids to millions is called Purdue Pharma and gets record profits.
People are now aware of the harms Purdue has caused. Justice has not been served, but their plot has been uncovered and discerning consumers need not engage with them.
Ah, you're saying that a monopoly protects the consumer from anti-consumer practices?
Journalists protect consumers from anti consumer practices. There’s not enough money in journalism to go after individual dentists even though many are bad actors. There certainly is enough to go after corporations, which we have seen many times. I’m not saying justice will be served, but I do think more public oversight = better informed consumers. There really is no good way to know if a dentist is honest before going to see them. That’s just not true when it comes to mega corps.
No one wants that. People pay other people to do things well. No one wants to need to discuss orthodontics with a bot, they would rather pay someone to know those things and think about them.
Great idea! It’ll work just like how Realpage helps people: https://hn.algolia.com/?q=realpage
I happen to be (re)reading Strogatz' book "Nonlinear Dynamics and Chaos" right now. It's beautiful thing, and nothing in the "age of AI" threatens to equal it or in any way diminish my enjoyment of it.
I think the point is that in the "age of AI" virtually nobody will have the opportunity to follow a career and produce works like Strogatz was able to do.
I'm not disagreeing with the post. My personal opinion is that LLM "thinking" is qualitatively different to human thinking, that the "convex hull of ideas" idea is probably right in some sense, that the social structures we have built to allow and encourage human intellectual exploration are not obsolete and that there is a high risk of catastrophic damage being done to them by those who fail to see this.
It's really a fascinating book and topic. And i too enjoy appreciating it. I guess it's appreciating the beauty of nature and as you said, AI will not be able to threaten that.
Hacker News is so depressing these days. One doomer post per day to keep the valuations up.
Nothing has changed in math. Math is not solved nor has the human been kicked out of the loop. Same with programming.
*I said nothing really changed because LLMs becoming superhuman at deduction was priced in with o3 and neither the peer review, training nor publishing methods will get a revamp beyond what could already be done with SQLite and a website.
I'm no mathematician, but my impression as someone who at least does a lot of applied work with maths is that the main thing that's changed is that the yardstick by which academics earn recognition is being re-assessed.
But from my perspective proofs have never been the most interesting part of maths. What really interests me is the intuition that leads to new applications, and to new questions. Newton, for example, gets more credit for calculus than Leibniz because he's the one who demonstrated how darned useful it is. Erdos is arguably more famous for asking interesting questions than for answering them.
I still don't see LLMs seriously contributing to that facet of mathematics any time soon. Every so often I ask one to suggest a good binomial confidence interval for a particular application, and it still tends to just suggest one of the subpar options that everyone learns in a first year stats course.
Proving theorems is one of the main ways that mathematicians turn math into money.
The recent barf-out done by LLM companies is useful if what are you are interested in is an unreadable mess and someone giving you the final answer. You might be able to extract some new "aha!" moments out of it, but we will have to wait and see what happens once people go through it. I would hypothesize the LLM output will be good at hinting at some non-obvious relationships and patterns, but I doubt it will yield anything of substance in its own right.
Historically, proving difficult theorems required mathematicians to sometimes invent new fields, new ways of looking at problems, the discovery of which itself yielded incalculable contributions to both mathematics and other fields. It is this that is the greater contribution of and to mathematics.
> Proving theorems is one of the main ways that mathematicians turn math into money.
Mathematicians working at universities are paid to teach, to engage in the politics and business of government contracting, and only in very rare circumstances to play prestige games on famous problems.
It is unlikely that Congress will decide to slash budget for NSF's MPS directorate on account of AI getting good at proving theorems. That's just not how Congress works.
It is also unlikely that LLMs will be able to do, in the near future, do the sort of applied mathematics that happens in the context of occasional NIH or DoD funding that flows into Mathematics departments, for a variety of complex sociological reasons I'm happy to expound upon.
And it's unlikely Engineering degree programs will stop requiring calculus sequences, since being able to do the computation was has not been the actual point of teaching calc since he late 20th century (if ever).
So, the good news is that I doubt any of this will have much of an effect on the job market for Mathematicians who want to prove theorems for a living. The bad news, of course, is that the job market for Mathematicians who want to prove theorems for a living was already pretty horrendous.
I’m on your side on this overall, I think. And I suspect we’d agree more than disagree on nuances. But saying that ‘nothing has changed’ is very obviously wrong.
I guess I chose the words "nothing has changed" because that's true relative to all the subtext of content I'm seeing online claiming that "math is solved"
I think the https://news.ycombinator.com/item?id=50043026 Terence Tao post was much more hopeful and along the lines you're suggesting.
It was, but the comments on that post were insanely depressing to read. I'm a mathematician, and find myself largely hopeful and excited for its future, but being online this last week has been incredibly brutal due to the surrounding vitriol. It's an avalanche of negativity, and I'm largely not talking about that of my peers (though indeed, some of my peers). Most mathematicians I've spoken have been reasonable / positive, perhaps an effect of my field.
I really need to just stay off social media for like a month and keep talking IRL. Longer, if I know what's good for me.
> I really need to just stay off social media for like a month and keep talking IRL. Longer, if I know what's good for me.
We all should.
And none of us knows.
I agree with "nothing has changed" in practical terms. If you take a look around most things rely on an undergraduate level of math than whatever is at the frontier. The proofs may be important, but they are really remote to people's lives.
This is like saying WW2 didnt happen while you were living in a Soviet village in 1940s, and you were doing the same thing, and nothing seemed to change
Go apply for some jobs. You’ll quickly come to the realization that something has changed.
And this is only the beginning.
And how! All the “nothing has changed, stop whining posts” are neglecting that they haven’t had to go look for a job. My previous company became quite toxic about LLMs and employee usage. It’s not “move over to the paradigm or get left behind” instead it was “jump on the hype-wagon and sing LLM’s praises all day, every day or you’re out”.
Yes, interest rates are higher than ever and consumer sentiment makes business wary
ZIRP is over but rates are not especially high and yet I don’t remember job market being that bad. Maybe employers are taking wait and see approach and avoid hiring developers they expect to layoff in a year or two if models will continue to improve and becoming cheaper at the current pace.
Not that it explains all job market weirdness but you do have a problem on the hiring side with having to deal with the flood of AI generated applications. I assume that’s going to have a slowing effect on the typical apply/interview/hire process.
I think that models will improve and get cheaper. I think we'll see Opus 5.5 tier models open weights in six months.
You are probably right that some companies are factoring AI into headcount but I'd imagine that the interest rate and energy crisis are bigger concerns
Interest rates are higher because AI.
It was long predicted that the sign that AI is here in macroeconomics is rising interest rates, because investors can get MUCH higher rates of return doing AI investments than borrowing governments, why would I lend money to Greece when I can lend it to Anthropic?
I own my company and directly participate hiring. What’s changed among me and my peers is that we’re on incredibly precarious economic grounds with no telling what’s going to happen next month.
LLMs haven’t done much in the way of informing me or others I know regarding “who and how many”.
I’ve done some hiring lately as well and we got a ridiculous number of applications, like 500+ for a run-of-the-mill developer job.
I’ve worked in tech for 20+ years and I’ve never seen it like this. Sure there are job openings, but sooo many applicants it seems.
(This is Sweden, YMMV.)
I would wager that most of them are AI generated. The cost of generating a neat looking LaTeX resume and adding vibe-projects to one's GitHub is very low...
500+ is a lot?
If you’re one of the 500 then it is, yes.
It’s normal to be one of 10k.
For a research role at Anthropic perhaps, not for the roles I’m typically hiring for.
The fear is the sales pitch. AI has some uses but nowhere near enough to justify the price tag, so we need something more to sell it.
Fear and FOMO mainly.
Re depression: Seems like it is not just HN... You say nothing changed. Maybe that's even true.
But many people perceive this very differently, see link below. And perception is reality after all.
https://news.ycombinator.com/item?id=50093180
Yes I take issue with all the anti-human propaganda coming out. I think that there will be a future we're essentially replaced but it's not now
The doom posts will continue until profits are up.
Jokes aside, its like working for a company, individually you can lift mountains and someone else will take credit. When you use LLMs for a single bit, all credit goes to LLMs lol
> Math is not solved nor has the human been kicked out of the loop. Same with programming.
Is it like Chess and Go? There are still plenty of people playing for fun, but the machines are better at it than we are.
Nah, see my other comment in this thread with actual technical issues with the math dump and how math =/= deduction
Mathematics as a field of study is maybe 2500 years old and a lot has been figured out in that time. But look how far AIs have come in just ten years. I wouldn't expect progress to stop now, would you? A decade from now I wouldn't be surprised if machine mathematicians are essentially oracles. They tell us the answer but can't explain it to us any more than you could explain long division to a golden retriever.
yeah nothing changed in math. mathematicians are weeping for some strange reason (maybe paid by OpenAI to weep?). And Terry Tao is posting blogs on AI + Mathematics because he was bored (or because he was also paid by OpenAI).
They are overreacting imo. A lot of mathematicians stuck their head in the sand with o3 but this was the logical outcome; superhuman deduction
Disagree, there are in fact documented cases.
> I think I have been underrating the pace of model improvements. In March 2025 I made a bet with Tamay Besiroglu, cofounder of RL environment company Mechanize, that AI tools would not be able to autonomously produce papers I judge to be at a level comparable to that of the best few papers published in 2025, at comparable cost to human experts, by 2030. I gave him 3:1 odds at the time; I now expect to lose this bet.
https://www.daniellitt.com/blog/2026/2/20/mathematics-in-the...
“I’ve slowly updated my timelines over the past year…”
and
“I think I have been underrating the pace of model improvements. … I was not correctly calibrated as to the capabilities of existing models, let alone near-future models.”
Fair enough. But after the distinct distances result if you did not price it in, that's on you haha
I don't get the argument - if we follow what you say, they're not _overreacting_, they are reacting appropriately but should have done so earlier (when o3 was released) rather than now?
My claim is that if Bob, a mathematician, believes that he is rendered useless by an LLM then he did not play with o3 and all the other models between now and then to realize that, yes, they are superhuman at details and deduction but not superhuman at math
Can you come up with a concrete prediction on what AI can’t do in math? Something testable.
Yes, here is a concrete prediction
AI will not replace the mathematical community (peer review, abstraction creation) until AI can train the next AI purely on data generated by itself and with a tool in a turing complete environment. The environment has to be air-gapped from human data but the AI can use tools and its own outputs
I think that "solving math" is isomorphic to solving the "thinking part" of the scientific process which would be genuine RSI
This was the top comments when the submission disappeared from the front page.
Now the thread has been cleaned up and a pro-defeatism comment is the top comment and the submission is on the front page again (I'm not alleging moderator intervention, user mechanisms are sufficient).
> One doomer post per day to keep the valuations up.
Wow that's very well worded.
It's probably because with the release of each model it is becoming more and more difficult to see how a person's livelihood and way of life continues. Uncertainty stacked on uncertainty.
How will it pan out, no one really knows. The leading frontier labs are continuing to hire aggressively. But who knows for how long or for long or near-term goals.
The open ai math dump was actually what got me off the singularity wagon.
There were mistakes that could easily be seen with a diagram rendering three big results wrong, exposition is terrible, no coherent underlying theory generated from solving so many problems, as Thomas Bloom pointed out even duplicate proofs of the same statement in a single paper.
It clicked for me, "this is slop but slop can be superhuman and useful!". Similar to programming where it can generate as many criticisms of your codebase but you have to weed them out.
If anything, that makes the case that math is fucked even stronger. Nobody had any idea of how to prove quasi-Riemann. OpenAI produced three proofs, one of which was just a lemma in solving another problem. They have a machine that is better than every human being alive at math that solves problems humans cannot even when working inefficiently.
Math was never only about deduction. It's also about data curation.
I was taking to my friend in OA who was really surprised that the proof of L(F_2) = L(F_3) could be done in the 1980s. We just didn't have a superhuman deducer back then.
All scientific break-troughs are obvious in retrospect.
Special relativity which Einstein got a Nobel for is now taught in high-school.
"Of course if you move a magnet in a piece of copper you will generate a current, it's extremely obvious it should be so"
That's true. I'm not saying the work is trivial, I'm calling out the difference between "deduction from known data" and "distilling known data"
I'm sorry, but "nothing has changed in math" is a truly ridiculous statement.
Sure, there are people who overstate things. But that isn't an excuse to keep your head in the sand.
o3 already had this priced in amongst many mathematicians I talked to at conferences
You use the word "already" as if it already happened, but this is confusing speculation with reality. Market prices are a summary of speculation. Plenty of things are "priced in" that haven't happened yet, and maybe won't if the market is wrong.
Anticipating that something will happen is not the same as evidence that it's happened.
You have an earned an upvote from Long Term Capital Management!
Doesn't seem to be the case here: https://www.daniellitt.com/blog/2026/2/20/mathematics-in-the...
I was on Twitter around 2024-2025 and showed both Daniel and my advisor at UCSD sota results I created in group theory using Gemini 2.5 (edited) Pro in Q2 2025.
A lot of us knew what was coming. Profs I speak to these days say it's the same old except they need to pick a different kind of problem for grad students
my doomerism isn't about keeping valuations up, it's about tearing it down.
https://en.wikipedia.org/wiki/IBM_and_World_War_II
the above, a "sorting" machine is the primary usage of hyperscalers currently, they're just trying to find new uses to boot for extra money/legitimation.
the browser industry the phone industry the CAs JavaScript
it's all paid for by those wanting to sort humanity into whatever bucket the can fear monger the peasantry into for aweek or however long it takes the classification system to sort out.
gentrification works, but we chose forcible displacement now we have to live with the forever consequences.
would you give guns and bombs to the displaced native americans who taste blood over their beads and trinkets?
we've been inside the event horizon of the singularity for years, the tax bill will only get worse unless we recognize it for the tick that it is.
> One doomer post per day to keep the valuations up.
Yes, outsiders (investors, journalists) are given a false impression while insiders (software engineers) are supposed to give up. Double win for the agitprop machine.
[flagged]
> Xi's minions are watching.
Sometimes you think you’ve been accused of everything you could be accused of. Wonders never cease!
> The "moderation by less", as well as the guidelines, are both farcical in nature. They've given way to ruination of an otherwise decent tech discussion forum
The guidelines have been constant (with minor updates) for most of the two decades HN has existed. Dan’s a moderator for 14 years and I’ve been one for several. Both of us have been active participants almost since the beginning. When did the ruination begin?
Your comment seems a generic vent, largely unrelated to the parent comment but if you have any actionable suggestions you’re welcome to email us.
> Your comment seems a generic vent, largely unrelated to the parent comment
There it is, the "off topic" play. Well done, Tom.
If you'd put anywhere near the amount of attention in moderating the site for over all quality that you put into my "largely unrelated" comment, this entire comment thread would not exist. Think it over.
Please don’t insinuate that I’m “playing”. It’s a slimy tactic, not an earnest contribution to a conversation. You opened this subthread with an accusation that we’re putting our thumb on the scale for the CCP. Yes, it breaks the guidelines, along with basic decency. The only reason this site still exists for people to bemoan that it’s not as good as it used to be (which people have been saying since about three months after it started) is that we have guidelines and uphold them in a way that people can trust, often against pressure from many different people with all kinds of agendas. As I said, if you have any actionable suggestions you’re welcome to email them but all I see is someone complaining about quality whilst having no concern for the quality of their own contributions here. If you don’t like the guidelines you should take your own advice and find a site that’s more to your liking.
What I was aiming for was to point out the oddity of dang selecting the one odd ball thread to tell everyone to play nice, in what is otherwise a site that is daily filled with AI doomer posts, as per the opening comment of this thread to which I responded. And I did mention that I found it amusing - because I doubt either of you are actually shilling for CCP. This wasn't even the main point of what I was after.
That neither of you are at the very least bothering to steer the greater discourse towards some higher ground is a lack of balance that I am highlighting.
To remark my comment as off topic in the context of the thread opener is to dismiss what I am saying using a very lazy tactic that I've seen used countless times before. No, it is not off topic. It is very much on topic and relevant.
And if you have a problem with my comment, feel free to email me.
We’re posting mod comments to uphold the guidelines every day in threads about all kinds of topics, which anyone can see by looking at our comment feeds. There are posts about AI all the time here because it’s the biggest topic in technology and even the world right now. Some topics/discussions are negative and many are very positive, reflecting the broader discourse. We’re curating the content all the time to provide a good balance of topics and perspectives. By statistical trends and what we observe in the quality of the best posts and discussions, HN is the healthiest it’s been in years.
But one of your central complaints was about the guidelines, which have been constant for nearly two decades, so we’re just not having a serious conversation here. We’re always looking for signs that indicate if and how we can be doing better, but this interaction has nothing to grab onto.
> lobsters
Never been invited, but I've also never felt smart enough for that community. I'm glad I can read it though!
TBH I come to HN to understand US silicon valley mindset. difficult to get a more distilled version of that in other places.
FWIW as someone in silicon valley, this really isn't the silicon valley mindset. HN represents a sort of messy combination of many different tech tribes, but I will say its depiction of silicon valley is largely that of someone who reads too much tabloid coverage of it. It's very much the feeling one gets about silicon valley by reading Techcrunch articles without actually being here.
I actually think it's a bit dangerous to try and understand SV from HN comments, in the same way that trying to learn about an event by watching a fictional adaptation of it can often confuse more than educate. The goal of the comments here aren't for SV to chat anymore and it hasn't been for a very long time, for better or for worse.
Twitter is closer to what you'll get among founder and hype circles in silicon valley, but even that isn't a good pulse of what engineers here actually think. The public web is just too adversarial a place for most folks to want to hang out anymore and silicon valley has been so deeply politicized in the news that a lot of people just don't want to air their opinions in public anymore. My suggestion is to save up some money and just come here and visit for a week or two, go to events, rant to a few graybeards, that kind of thing. You'll get a much more authentic look.
been there done that twice! But probably SV is different now than it was on my visits on 2013 and 2017
Am, this is 80% Indians with most of them from India.
not sure what is the source for this. I'm just a latino posting from Ireland tho.
source: https://news.ycombinator.com/item?id=49892245
I need to do some more research on this. Because I've always been told that AI, more specifically LLMs, are "glorified autocompletes". Yes they have huge data sets but ultimately they can only provide answers based on data that they were trained on.
So how can they possibly solve problems that humans have worked for decades or centuries on without being able to solve? How did they find something new? Is it that they were able to deduce the solution by examining all current data sets? Or simply brute force it until the produced solution passed all necessary tests?
The following certainly does not describe all mathematics or all LLM operations, but its my mental model for why LLMs are so effective in several fields.
Lots of mathematics entails recognizing that a tool/techniques that works in one area can be adapted to a completely different problem. For a human to do this requires having a robust mental model of the technique and the problem itself, and probably having just the "right" mental model to see how they fit together. And we are talking about extremely hard/sophisticated techniques and extremely hard/sophisticated problems, so the ability to deeply appreciate either can require years of study. And on top of that, even if the mathematician has sufficient understanding to develop the intuition that the technique might work, they will need profound fortitude, skill, and patience to see the solution through.
Unlike human mathematicians, who will develop a deep understanding of only a relative handful of techniques and problems in their lifetime, LLMs have ALL written techniques, problems, solutions, etc stored in their internal representations. So when they ably match a problem to a technique, we can imagine the operation as a supercharged autocomplete, or even an extremely proficient search engine. Their stroke of genius is really a consequence of capacity, moreso than cleverness.
Yes this is it. So many breakthroughs in math (and other sciences) have been like "I went to a conference and happened to mention my work to someone in a completely different area/field and they said: well that sounds like something i just noticed in my research".
Combining spread out ideas is very powerful and LLMs are great at that.
They have access to a much larger corpus of information than any single human can manage in his one brain. No human mathematician reads all the papers and has perfect recall on them. The answers have always been there, but no individual has been able to make all the right connections to pull them together and formalize a new discovery or proof.
I mean the big idea of math is that (with some caveats) all the answers that can be provided have always been there.
> Is it that they were able to deduce the solution by examining all current data sets? Or simply brute force it until the produced solution passed all necessary tests?
Yes, both of these + an infinite token/compute budget and SOTA+ models. It's not like some hobbyist out in the woods with a GPU running Qwen 27B can do this.
Because the "glorified autocomplete" crowd likes to ignore the technical detail of how the autocomplete actually works.
Yeah. "Glorified autocomplete" might be all the human brain is too. "Glorified autocomplete" can do an amazing number of things, including perhaps pass the Turing test and perhaps ... "think".
"Glorified autocomplete" is meant to describe the _underlying mechanism_, but unfortunately it also sounds like one of the things LLMs are actually used for, so it gives the mistaken impression that the claim is all they are good for is a component inside an editor application.
[dead]
>Yes, some are crying about bruised egos, the feeling of wasted years, a creeping sense of status obliteration, the frustration of an unfair competitor, the indignation of feeling cheated and mistreated, and uncertainty about the future. But they are also mourning...
This nails it, however IMHO its almost all bruised egos and feeling of wasted time. I don't know the ratio of A:B but I also don't know anyone that is in the latter category of feeling doom and mourning that their craft is about to regress.
I don't know what's happening exactly, but I am genuinely astonished how little people seem to care about people losing something real, dismissing it as "bruised egos", using petty terms like "whining" and "crying", or calling them some insult for the crime of being angry or sad for having a big chunk of their life pulled out from under them. I'm referring to the big picture of common talking points, not just this case.
I guess I'm not astonished that people are so bad to each other in general, but more that in the case of genAI it's so prevalent in places that seemed more elevated previously.
Part of me wonders if this is actually a form of self-defense. The misery of others can provoke sympathy or the opposite: Angry pushback to protect oneself from dwelling on their misery.
Similarly, I got into software because I like working with computers and solving problems for others. I see TONS of similar sentiment where people are almost joyously exclaiming “Software is solved! Sorry!!! It’s been a long time coming, you all deserved it, get fucked!”
And I can’t imagine feeling the same about any group of people. It’s been odd, to me. I’m just trying to support myself and the people in my life, and now there’s a looming fear that I can’t do that anymore… business owners are drooling over that, and other sectors are laughing.
Hard times, I guess.
All my life, when I would think back to what mill towns went through when they all closed (something I did not live through), all my naive outsider brain could arrive at was, "Yeah, but those were just grunge jobs, humans were freed up to do better stuff."
And perhaps the same thing is now happening to me. It feels different, but maybe the same thing is true.
How do you know it feels different? It may have felt exactly the same to them.
I mean it feels different when it is happening to me than i felt about it happening to them
Tech workers are a small subset of the population that have been able to live very privileged lives when compared to everyone else.
I think the average person lives a much harder life and has been doing so for a while. Should we really be surprised by the lack of empathy when folks in tech had little to show for the common man this whole time?
I saw something similar on Twitter responding to 45yo software engineers making comments of depair in that HN thread about the increased suicide rate of middle-aged men.
For some reason I'm stuck in a real depressing, doomer algorithm / homepage on Twitter. But everyone on my homepage seems to have a take on the HNer comments along the lines of "see, this is why you don't make your job into your identity".
But that's such an antihuman response. The only reason you can put decades into a field to grind and build expertise is because you have an identity-level interest in a subject.
It's also not a good thing if the optimal way to live going forward is to have no passion for anything lest it changes so that we won't feel bad. I don't think that's possible unless you remove the human part of our brains or make life so horrible that we're happy just to survive another day.
> the increased suicide rate of middle-aged men
Source? There has not been a statistically-significant increase in middle-aged male suicide rates between 2023 and 2024 confirmed data [1]. (2025 breakouts aren’t out yet, but top-line data show decreases.)
[1] https://www.cdc.gov/nchs/products/databriefs/db572.htm
The article was https://www.telegraph.co.uk/news/2026/10/13/suicide-rates-hi...
In 2025 in the UK men age 45–64 had a suicide rate of about 24 per 100,000, which is the highest it has been in the past half century, and is higher than other age categories. In the mid 2000s, the rate was like 17 per 100,000.
From your link, in the USA the overall rate for men of the same age is about 30 per 100,000, but varies dramatically between states.
Got it. I think it’s fair to point out that we have no evidence that UK statistic generalizes to America or other countries.
I'm referring to this submission: https://news.ycombinator.com/item?id=50093180
> For some reason I'm stuck in a real depressing, doomer algorithm / homepage on Twitter
At the risk of taking this off topic, the reason is Twitter. Don't go on Twitter and the doomer algorithm disappears from your life.
Lots of people have lost their jobs and had to overhaul their lives due to layoffs or technological advances or political choices or factories moving overseas or whatever.
They are showing as much care for mathematicians as they received themselves, which is not much.
i think a big part of the backlash isn't about math per se and it's about what 'work' or 'a job' are supposed to be.
when mathematicians complaining about losing the part of the job that they love, people think "if you love doing it, why is anybody paying you? you're describing a hobby."
Yeah, the glorious utopian future we're building where humans are only able to make a living doing things they hate, and money is concentrated with the few.
Because we don’t see it as something for them to be legitimately upset about, except that it’s going to disturb their ivory tower unserious profession.
If AI got significantly better at performing surgery overnight and doctors started crying about how they were going to be replaced, what do you think the public response would be? Non-doctors want the best possible care; they aren’t interested in protecting an industry. The same will be said for pilots and lawyers, too.
People dislike self-serving whingeing, end of story, but that’s doubly true when it comes out of a profession (theoretical mathematics) that is mostly practically useless.
The doom and mourning is that it's no longer their craft. Their involvement is on the threshold of no longer being useful. What does it mean for humans when the best their involvement accomplish is slowing down progress?
This is a fundamental human need and it's being taken from everyone eventually, by design. It will take a generation to turn over and find new ways of being useful. Those who ran into this wall and could not safely reduce their income and retire/pivot to other fields, will be the coal miner oxy crisis of this generation.
Mourning is fine if it is temporary, however no man should dread their own future if they still have control of their life. This is the idea of unalienable rights and was enshrined hundreds of years ago. It seems to me that anyone dreading the future of their field failed to take a page out of their retirement strategy and diversify their own interests, i.e. https://en.wikipedia.org/wiki/Polymath
https://en.wikipedia.org/wiki/Life,_Liberty_and_the_pursuit_...
My friend, this happened a week ago. If a spouse dies you don't come back a week later and say "welp, chin up and diversify your dating pool!"
https://record.umich.edu/articles/study-grief-is-short-lived...
Ok, so if you really believe that losing one's vocation at 25 is the same as a 75 year old losing their spouse, that paper suggests you should wait at least 6 months before you start being a jerk about it.
I don't see what point you are trying to make. At 25 losing your vocation gives you more than plenty of time to transition into another lifestyle. Many people will consider this the very beginning of their career. At 75, retirement is near. Your legacy is known but not final. 6 months is less than 1% of your lived life, and you have a lifetime of memories to reflect and enjoy.
I'm not entirely sure what you're on about. Are you suggesting that mathematicians should become truck drivers or plumbers and be happy with that (until the next physical-aware models release, and then...)?
What do you mean, "control of their life"?
I was talking to a mathematician friend about this, and she was telling me that a lot of the proofs that have been produced in the past couple weeks don't have an application.
To me it begs the question: in a theoretical world where we don't have human mathematicians anymore, what is the point of these proofs if there's no human who actually understands it?
If the point of a lot of this math is only to expand the boundaries of human knowledge, it all seems kind of pointless.
heh these models should be turned loose on all the proofs paid for and sitting on the shelf and see if they can find an actual application useful to mankind.
Often, we don't know the applications of new math. For a long period of time we had "useless", "purely theoretical" mathematics that eventually turned into relativity and quantum mechanics.
Proofs are somewhat like legos. A bigger toolbox of them can only help in solving other, possibly practical, problems.
As far as the utility of a human being involved -- In a non-theoretical world where we don't have feline mathematicians, what is the point of these proofs if there is no cat which actually understands it? The cats can benefit from, for example, advances to optimziation algorithms that reduce costs for cat food delivery, of course.
Is AI at the stage that it can also point out all interesting things to study and then solve them? Otherwise humans would still be able and needed for progress.
Good question. I'm not a professional mathematician, but I would be interested in seeing what results the mathematicians get from trying to prompt it to come up with new and interesting problems.
My intuition is that it'll be at 'smart undergrad' levels today, though as the cryptographers say, attacks always get better.
It's the abject lack of shame that gets to me.
None of them have pointed to any of the solutions and said "this is real innovation" - because it's all been mining existing ideas and applying them elsewhere.
Imagine you spend years working on a solution to one of these problems. Only to realize after you publish that more than half your answer already existed in another branch of mathematics and you spent a bunch of time re-inventing it.
Yes the first big solve by OAI was "stolen" and that is tragic, and pretty bad. But it looks like this new batch, the one people are crying about, has none of that stink.
I guess I'm lucky that I've always hated my job, because that prevented me from tying my self-worth to it.
Even if this was overly long and mostly things I've seen before, I'm just very thankful that it wasn't yet another claude piece.
There was an interesting passage in the article:
> Today, the mathematicians are weeping, because the machine smashed their tower first. But tomorrow, it may be the physicists, the economists, the illustrators, or the writers who weep over their fallen blocks.
Isn't it extraordinary that these models still can't write well? I mean, they write passably well, but they all have a highly characteristic style that you can spot from a mile away. These language models can crack the toughest open problems in math, but they haven't yet cracked human-like writing, especially long-form writing, without exposing themselves. This is very very odd, IMO.
Or, just maybe, they share a key property with the mounds of popular writing, journalism and random internet accretions they were trained on: they seem total rubbish on anything you know about deeply, but brilliant on everything else.
Their math is as just as good as their rambling writing and their Ghiblified poster art. Most people don't seem to realize that.
This is not a reasonable take in 2026. The proofs are valid. You can even check the Jacobian counterexample yourself with pencil and paper.
The counter example, yes.
"Some subset of this collection is valid, therefore they all are" is not a valid inference. It's more like what we call "proof by intimidation".
> This is not a reasonable take in 2026.
I think it is. Why is it that the only people in any field I'm aware of that think the AI output in their field is high quality are either the ones that are carefully crafting the output themselves or were producing/consuming slop themselves before all this started? (Outsiders pretty uniformly like the output, which is why I drew the parallel to Gell-Mann amnesia.)
The difference here obviously being that the part that you mention is more embedded in the proofs (long and difficult and near-hallucinatory) but the results that matter are that the proofs are correct and validated. So I think you're failing to account for that, the problem for math is that the journey matters less than the destination.
Hard disagree. If we send someone out exploring and they discover something (maybe?) but their account of their journey is impossible for anyone else to follow and contains obvious hallucinations, what have we gained?
Isn't the problem more like they did discover a new land (as in, the proofs are true, and the solutions are validateable) but we have no idea how to make heads or tails of how they discovered it.
Math and programming languages are formal. Meaning they don't have any ambiguity in them. Every statement is what it is. That means they are very rigid and there's not a lot of variance. Natural languages are more diffuse and a written piece is more the reflection of the author than anything else.
That is why it's so disturbing. After several paragraphs, you realize there's nothing human there even when the actual information is present. People that do create AI piece are OK with it because they already have the context, but for the reader, it's always the uncanny valley.
Writing is a very slippery thing. “Good” writing from 200 years ago reads as unbelievably dense and overwrought to most people today. So even if you made an LLM that was “good” at writing, how long would it stay that way? AI systems can put out more writing per day than all of humanity combined, meaning it can also generate and beat to death cliches much more efficiently.
> Today, the mathematicians are weeping, because the machine smashed their tower first.
I read this passage and thought, "has Derek even talked to software developers"?
That feels to me like the first discipline to go through this, where their daily activities are massively changed by GenAI.
No, tech writers have it much worse.
Good point! Tech writers, and writers in general, were impacted before developers.
Maybe visual artists/photographers were impacted before then?
Anyone have a timeline of this stuff?
My guess is that translators were affected first, as automatic translation was decent 10 years ago.
Language models don't have a characteristic style you can spot from a mile away. They haven't had it since before ChatGPT.
What is true, is that chat models (and by extension agents) have a very characteristic style after their LLM went through post-training. But the pre-trained language model doesn't have that style.
So if you want to write well with an LLM, you could and it is a well understood problem. It is more a matter of that not being of much interest, as it competes with having the model be good for chat interfaces.
How to have a well-aligned, RL'ed, safe or thinking model, which also writes well in long-form, is a bit an open problem. Though presumably agents could do it when they have access to a pretrained model.
Why is writing in the style of a corporate marketing department a requirement of being good for chat interfaces? Why can't they tune them to sound more like a normal human?
They probably can tune for it, but it is not what the shareholders want anyway. I am unaware of any frontier lab who is actually interested in building something like that, long form proza indistinguishable from humans. Automated code and low hallucination rates are more important topics, and trade-offs need to be made.
Edit: why the style is like marketing corp-speak has to do with personalization not working well yet. So the default vanilla style is very corp. Most chatbots allow you to change the system prompt though and that can get rid of a lot of these issues in a lot of chatbots. But it is hard to maintain such a system prompt as the bots change so quickly.
There's no money in it. There's money in programming, and math is a good target to give the models superhuman reasoning abilities, but it already writes well enough for middle managers to write all of their emails.
>Isn't it extraordinary that these models still can't write well?
It's not at all extraordinary if you understand how they're trained. For math and programming it's possible to generate an infinite amount of training data with correct answers, as compilers and theorem provers can guarantee correctness. For writing, the quality is not objective, it's determined by human preference, so you need humans in the loop, which slows things down massively and limits the amount of data that can be generated.
You can just tell them to adopt a different style. To make sure, add another pass with an agent that flags and/or fixes LLM tics. Humans typically do several passes over writings that they intend to publish, why are we expecting agents to do it in one shot? Just like the superpowers skill for software development does a spec phase, planning phase, several review rounds, etc., if you do the same with writing you get very passable results.
Makes me wonder whether it's a fundamental limitation of the technology or more like missing a good set of tasks in the training pipeline
Might be that the vast majority of their training data is poor writing.
Recently a Canadian immigrant won a bunch of the highest French literature prizes for his novel, which then turned out to be an LLM fabrication.
Gemini's (at least for Flash 3.6 - 3.8) writing feels quite "human" to me
I'm convinced the people who do not see what Mathematicians see simply have not considered that their jobs are just as vulnerable. The day when OpenAI replaces their own AI researchers with GPT will be upon us, and the response from the Mathematicians will be "First time?"
As usual the author is ending on a completely unrealistic positive note: "humans may come to value more highly our capacity to relate and care for one another".
Does relating to someone pay my bills? People will use the informational tools to improve their social status and resource accumulation, there is no doubt about this.
If you work in a nursing home it can pay your bills. Is it so far fetched that that kind of work might see a much needed status lift post-ASI?
The vast majority of nursing home work is physical, sprinkled with a lot of excrement and sexual harsssment by the seniors.
There is little or no time, or incentives for care as described by the blog.
I am not quite saying "if you can't beat 'em, join 'em", but I know more math now than in any point in my life because I want to understand how LLMs work.
I understand there are many types of math and the mathematicians don't all want to sit around and do KV cache and gradient descent things, but surely a percentage are as interested as I am and can contribute to this new emerging field? I would wager that Xl+1=Bl Xl+Cl ℱl (Al Xl) makes much more sense in their mind than mine.
This is the best attitude to have about the current state. Unless you have some dramatically different sensible solution (and you probably don't when it comes to technology, because it seems to spread for reasons that would eliminate other sensible solutions) that will catch on and take over and replace AI: it's going to continue to mature as long as it's mostly useful to everyone. And AI isn't even a new industry, it has been maturing for quite a while, which adds more weight to the argument that having an attitude that adapts to technology (unless you wish to go solo) is a 'better' fit.
The dramatic solution can never be: 'we should all just want to do it this way because it felt better to me'. That never seems to work, from what I've seen. So there is a lot of public discourse, arguing, to determine if 'doing things the old way' is perhaps more better. And if not, maybe we just shed a tear and move on (to look for a better fit).
> I know more math now than in any point in my life because I want to understand how LLMs work
The implication here is that you wasted your time learning that math, and the fact you learned that will not actually help you contribute to anything.
Now maybe you are closer to understanding the feelings expressed in the article?
I felt the article deeply, but I don't think learning anything is a waste of time. You contribute to the world by helping everyone around you in any way you can, the river is beautiful because it flows.
As a mathematician, I will never understand how my colleagues had never foreseen this. Okay, maybe you can believe in a plateau but at least accept as a possibility that there won't be one. This applies to every other human endeavour.
I am also sad, but my sadness is, in a way, priced in.
It was really quick. I stumbled across a comment from KConrad on MathOverflow on how useless AI was for research. That comment was from 2024. When it disproved the unit distance conjecture in May, the solution involved a breath of knowledge of the literature that it made sense it took a computer to make the connection. It only took 5 months for it to advance to the point that it was better than humans at almost everything. Even if you expected it, the mind boggles at how fast it was.
Also, it's impossible to really know how you're going to react to something until it happens to you. When ChatGPT came out, I already thought "in the long run they're going to automate almost everything humans love to do". Despite having 4 years to brood about it, I did not really know what it would feel like until it happened to me directly.
> It was really quick.
That's how the singularity works. It's going to keep getting quicker, incomprehensibly so.
I like to call it the great incoherence.
Even before LLMs it was feeling like the design of society wasn't really for people any longer, instead it's more and more designed for corporations and their convenience. In the US at least. Same with how the internet has stopped most people with socializing the same way they did in the past.
Now we have a new layer that will be optimized for these digital entities. We see it in writing all over the internet. There will be tons of other places they show up and doing things that don't make sense to us.
I think what makes it hard is that it's quick, but not sudden. I understand constant-rate improvement. I understand what it's like to be in a car crash. What I can't understand, emotionally, is the constant acceleration. Even now, if I look at an OpenAI solution in isolation, I can understand the big picture, so if things froze where they are I could adjust. Intellectually I know I'm not going to get that chance, but emotionally I don't.
I don't see how this is possible. There are constraints on the speed at which information can move and the number of steps required to compete a given task. The improvement cannot continue, let alone accelerate, indefinitely without bounds.
OK - sure. Probably.
But it can continue to inconceivable levels before it gets to that point.
Okay, how can we be certain even about that? It sounds just as plausible to me that the improvements will become asymptotic and boring, not exponential or inconceivable -- and how can we make any confident prediction at all about the heights these models will reach? The technology those heights require doesn't exist, and its existence isn't guaranteed, even if it is possible.
> how can we be certain
Certain of predictions? I think that's an unreasonable ask.
And I'm comfortable with my prediction. Rather than argue about it - let's wait a year and see what happens, and then we can discuss it.
> I think that's an unreasonable ask.
Oh, good point! I misread your comment. Sorry for that!
> Rather than argue about it - let's wait a year and see what happens, and then we can discuss it.
Deal!
I saw it coming since reading Kurzweil in high school and it's still heavy.
I just had a chance to flinch first.
It's not really a profound insight. Back in the 1980s when I first started writing BASIC programs I remember thinking "well, if someone invents a program that can write programs, there will be no more work for programmers"
It's easy to have that thought, and eventually you'll be correct. One thing about technology is that it only improves. It's always building on top of what already exists. The only uncertainty is the exact timing, not the ultimate outcome.
This is it.
The people struggling right now - they were told. 30 years ago. They really should have come to terms with it by now.
But these are also the people who refused to believe it, for whatever reason. Their heads were in the sand. So I do feel bad for them... but... this is somewhat self inflicted.
No. They might have better braced themselves for the hit, but the wound was not self-inflicted.
Refusing to get out of the way of a train isn't the train's fault.
Lol ...and they call AI inhuman!
We've been hearing about this train for more than 30 years. Anyone who didn't see it coming had their head in the sand.
It's ok - I understand it seemed impossible. But many people understood it just fine - and they're doing fine.
I knew for decades that my parents were going to die, but I still mourned the day it happened, and for a long while afterwards.
Your inability to give space for those who love math to mourn its inevitable evolution is unnecessarily cruel.
Yeah, that doesn't seem comparable to me. A true comparison I think would be if you denied your parents would ever die and then your life were put in jeopardy when it happened because you didn't plan.
But we've gone back and forth about this a few times now about the same disagreement over framing.
You have a nice day.
I remember the same doom cycle during my PhD when GPT 4 came out [1,2]. It seemed like everyone in NLP was freaking out except me. A few years later and new CS PhDs are still graduating and joining academia and industry. Much ado about nothing.
Considering I currently live in Ithaca and have a friend doing a Math PhD, so I've heard from him some of the discussions in the department. I think the biggest shock for people is that individuals spent significant portions of their time on some of the problems that OpenAI supposedly solved (I say supposedly because all of the proofs have yet to be independently vetted). I understand that it's a shock, but I'm confident that in time it will be no more of a blip than it was for NLP.
[1]: https://xxcancel.com/mayhewsw/status/1636476184098643968
[2]: https://xxcancel.com/andriy_mulyar/status/163613925780582809...
A blip for NLP? What bespoke NLP research is about anything other than LLMs these days? How many companies are handcrafting bespoke NLP models instead of sending a LLM request? Really confused here.
I’m not sure I’m following… because as I understand it NLP is basically solved now (with transformer LLMs). No?
To me it seems the ultimate question is why would we expect humans to have an advantage at anything in the near future. A few years ago that would have sounded insane, but intellectually I don't doubt frontier models are superior to the average person. Even in the arts and humanities, give your preferred frontier model an interface and a way to "see" its work and it'll make the average person's work look like toddler scribbles. Humans still have some advantages, some places more than others, sure. Working with a designer recently I realized how bad, or maybe average, frontier models were at design. But I highly doubt the vast majority of people will be economically viable. Maybe this is one of the great filters, a society driving itself mad when freed from necessity of economic utility and without the big philosophical questions having answers.
What a buzzword filled nonsensical post.
As a programmer I’ve been ‘weeping’ for a few years now.
Programmers, artists, anyone who used to create any form of documentation, translators, there are literally so many.
Either these mathematicians thought it would never come for their jobs, because all of these other jobs were simpler or they've been living under a rock.
Either way, I don't feel for them any more than I do for all of humanity.
My friend, who is a math professor, told me that they weep because they are about to become just like us developers: prompt writers and AI output reviewers.
> My friend, who is a math professor, told me that they weep because they are about to become just like us developers: prompt writers and AI output reviewers.
If it was about understanding interesting proofs that some AI came up with, and extracting their essence, I don't think the situation would be so bad.
I see the much larger problem in the now existing career prospects of non-tenured researchers in mathematics. Everybody know how precarious the career prospects in academia and how precarious often the non-tenured positions in academia already are. With the new AI breakthroughs in math, these have in my opinion even worsened a lot.
UPDATE: For a related point, see also pixl97's post: https://news.ycombinator.com/item?id=50104145
I think there is totally justified distress knowing your source of livelihood may be in danger.
That said, I don't think the reactions have been entirely circumspect.
1. Is the role of math academics really going to go away? Is there concrete evidence of this? Faculty still have very important roles to play, training students, training engineers, determining the direction of research. The breadth of math is absurdly, perhaps infinitely, wide, especially if including applied math.
2. You need to be a bit crazy to go into pure math academia in the first place. If you value the pursuit of knowledge for its own sake, shouldn't the emergence of such powerful tools to that end be a good thing?
I think people vastly underestimate the ability of human to respect human performance despite being inferior to machines. Think about chess players vs stockfish, marathon runners vs a car, etc.
I think the crux is much more prosaic, as buried in the article. > I also don’t want to lose my job to a tangle of GPUs.
To put a GPTism, it’s not about the knowledge or status, it’s more about keeping their job and money.
And that can only be answered by policies, and imo by UBI. So votes if you can people, protest otherwise, discuss, debate, lobby with your friends, families acquitances.
Jobs and money are bottom of pyramid needs. If we answer those with UBI I guarantee you status and purpose are the next ones.
Chess players vs stockfish isn't a good comparison because chess fans only care about the performance of maybe the top-50 players.
People may gain and be satisfied among their own peer groups. All good and healthy if that peer group is your local badminton club or knitting circle - something where effort and skill have rewards and people can make objective, tangible progress. But groups not oriented around such activities could turn even more toxic.
When anyone can push a button on a machine and out comes a math result, it still doesn't become theirs or anyone else's until they can internalize it and make it part of their thinking.
"I can’t solve these things, but maybe you people in the future can stand on my shoulders."
Seeing AI being lauded for solving problems like this makes me feel... uncomfortable.
We are feeding all works of the giants that we have all stood on into a huge vat, in the hope to generate something meaningful.
If we can't understand the output, internalize it, or build from the time and experience that a human would have gained from solving the same problem, it feels like a dead end.
When all the problems have been "solved" by AI, what's next? If humans no longer struggle to solve the problems that the previous generation have left behind, how do we progress?
Wihtout getting into the details, I never got a chance to do what I'd wanted to do, professionally, from the time time I was a 5yo until I left for college. It just wasn't in the cards. And I would have been really good at it.
I'm sure there are millions in the world like me. Not too sorry for people who got 5 or 15 or 25 years of opportunity to chase their rainbow.
What is it that make human so special, ask the article.
On absolute level, no human can answer since humans don't have absolute general knowledge and thus lack ability for valid general absolute judgements. Note that personally I do believe that humans can access some chunks of absolute truth, including that some absolute truth is likely existing but we lack the capability to grab it at general level.
There is no absolute reason to believe humans are absolutely special.
Now on very relative and immanent grounds, humans are very special to me and the interest in which will dwell in. Note that this is nothing special to human species. Special interest for same kin is certainly a common trait to any living being to which we human would agree to attribute ability to have any interest in anything.
Who care of we absolutely are special? Only those who believe they need to belong to special ontological class to worth anything to their fellow relatives and feel like they need to justify some kind of privilege will fall in obsession with that topic of being special.
No one need to be absolutely special to get interest of others. Relax, go say to the people which are so special to you how much you care about them.
>Newton famously compared his achievements to “standing on the shoulders of giants”
Newton was possibly just taking a dig at his small-statured rival, Hooke. Newton was brilliant, but a shit.
I'm wondering if mathematicians are sort of going through what programmers have been going through recently, really questioning whether their craft can continue. But now it feels like, in the software world, at least from my seat, people can handle much larger scope and work on much more complex problems. If anything, my job has gotten a lot _more_ interesting.
So I wonder if mathematics will go through that too? A period of chaos and doubt, but finding new and more effective ways of working.
Development is more interesting, I agree, but also more burnout inducing, and the not yet understood implications create unreasonable expectations on developers. It will take some time to find the new normal. Right now me and every colleague are working harder than ever and drowning in tasks. Far from being redundant.
Yeah, for sure, but at least to me, it feels like we will figure that out. We're in a transitionary phase. But it also seems to me like, to your point, we're busier than ever, and in a lot of ways, the problems we're working on are harder than ever.
People need to separate out the alien intelligence vs human intelligence.
Its easy to demonstrate this: The math hysteria passes Lean but no human can make sense of it at present. This is akin to bots telling humans where to dig. Thats fine but lets be clear about whats going on.
Oxytocin is one of the oldest molecules in the world — it created sexual reproduction, a very long time ago. "Our capacity to relate and to care for one another" looks a lot like oxytocin, which therefore belongs to all of us who reproduce sexually. There is nothing stopping artificial intelligence from making us humans return to "relate and to care for one another." Hugging, talking, laughing, dancing. Trusting, being trustworthy. Sex, relationships.
You know, I think Yuval Harari hit the nail on the head when defining humans as those mammals capable of gathering in groups of more than just a few dozen, by being able to believe in abstractions — like "a country," "being a fan of a soccer team," a religion, etc. Harari mentions how difficult it would be to keep forty thousand chimpanzees well-behaved in a stadium, and yet we humans do that every single weekend.
Which abstractions will we believe in, with or without AI? I think this part isn't really under attack. Nor is our capacity to relate to one another under attack.
So both our relating-animal side (oxytocin) and our human side (believing in shared abstractions) remain untouched. They aren't being replaced by anything.
What are AIs — much like what ChatGPT did with mathematics — actually replacing?
Something else. But neither our capacity to connect with one another nor our ability to gather in the thousands, millions, or billions around abstractions. Just my two cents.
Great article!
(My thoughts on similar topics: https://blog.vladi.dev/en/articles/mudando-o-ponto-de-equili...)
Your assessment is depressing -- the thing that makes us special has gone from "thinking", "curiosity", "understanding the world" to "we don't kill each other when gathered in a stadium".
That's a fair point of view. It's Yuval Noah Harari definition, not mine (Sapiens book, 2nd chapter):
"Ants and bees can also work together in huge numbers, but they do so in a very rigid manner and only with close relatives. Wolves and chimpanzees cooperate far more flexibly than ants, but they can do so only with small numbers of other individuals that they know intimately. Sapiens can cooperate in extremely flexible ways with countless numbers of strangers. That’s why Sapiens rule the world, whereas ants eat our leftovers and chimps are locked up in zoos and research laboratories."
(...)
"You could never pack a thousand chimpanzees into Times Square, the United Nations, the Vatican or the New York Stock Exchange. You would get pandemonium. By contrast, thousands of Sapiens regularly congregate in such places, and they usually produce orderly networks and trade rather than violence and chaos."
---
And the passage I recalled from memory is actually from a TED Talk ("Why humans run the world", 2015 - https://www.youtube.com/watch?v=nzj7Wg4DAbs&t=231s)
"If you try to crowd 100,000 chimpanzees into Wembley Stadium, or into Oxford Street, or into a church, or into a school, you will get complete chaos. But you can put 100,000 humans together in Wembley Stadium, and they will usually behave reasonably well, watch the match together and sing along."
---
But I’m not giving up on your celebration of our curiosity.
Speaking of curiosity, I’d bring Peter Gray and his "Free to Learn" book into the conversation.
Take a look at the second of the five characteristics of play that Peter Gray proposes:
"Play is activity that, from the conscious perspective of the player, is done for its own sake more than for some reward outside of the activity itself. In other words, it is behavior in which means are more valued than ends.
When people are not playing, what they value most are the results of their actions. [...] When people are not playing, they typically opt for the least effortful way of achieving their goal.
In play, however, all this is reversed. In play, attention is focused on the means more than the ends, and players do not necessarily look for the easiest routes to achieving the ends. Play often has goals, but the goals are experienced as part and parcel of the activity, not as the primary reason for the activity. Goals in play are subordinate to the means for achieving them."
Using Gray’s definition, mathematicians might actually be mourning the loss of play — the process itself, the thinking, the exploring — rather than "just" reaching the final result (via ChatGPT).
Is this a valid way of looking at it through Peter Gray's lens?
Cooperate to what end? If we succeed at obsoleting thinking, what would the collaboration be about?
(Also, I suspect if you take 100,000 humans, with no explanation, and force them against their will into Wembley stadium, you'll also get chaos.)
I apologize that I've just updated my answer above bringing Peter Gray's play definition to help us deal with our innate curiosity.
I'll summarize here: If we succeed at obsoleting thinking, we loose the game, we stop playing, according to Peter Gray's definition.
So you - and the mourning mathematicians - are all correct: we loose the most important part of life, according to Gray's view (above).
Yes. We are trying to build a fucking nightmare for humans. There's still a chance AI will plateau, but we're staking the future of our humanity on it with our current path.
I love that we have new ways to solve problems. And it's been hard to accept that my own unique intelligence—which I've spent decades cultivating—just isn't as important as it used to be. I am developing new ways of thinking to better adapt to an AI-enhanced world, but I'm not done with the mourning process. We're all finding our way through this.
What new ways of thinking are you developing?
Knowledge without understanding would be just getting "42" and not being able to ask any follow-ups.
But I don't think that's where we are at the moment. I believe the results, proofs and papers can be reviewed and understood, just like if a human peer had published another result.
I also think that the speed at which results can be now generated can't be followed by our collective speed to understand them, using the methods we are used to. As we enter the next phase where AI publishes using mostly references of other AI results, this will compound exponentially until it would take any human decades to verify a single result by hand end to end.
I agree translating a phrase doesn't teach you a language, and I also believe translators are an essential tool that results extremely helpful when learning a new language.
So instead of throwing the towel, could we develop and adapt ways to improve and aid our understanding and verification processes?
The author mixes "Science" and "Mathematics". Mathematics is not a science. There is nothing empirical about it. By mixing the two concepts, a reader might think that AI is on the verge of solving all of science. Until AI gets good at digging large circular holes in the in the ground, I think Experimental Physics is safe.
> Until AI gets good at digging large circular holes in the in the ground, I think Experimental Physics is safe.
Are the experimental physicists usually the ones holding the shovels?
Maybe math is more like a science now, though? It's as if ancient astronomers saw the results from a modern telescope. "Why even look at the sky anymore," they might ask. "And we don't even own the telescope!"
Maybe the answer is to look into how Big Science works? There's still plenty of work to be done figuring out what to do with the outputs from these new math-observing instruments.
I had visions of large automated warehouses full of chemical and biological components hooked up to science machines to do everything from mixing, boiling and printing to spectrograph and robotic assembly. Smart computers and humans would pick the experiments to brute force with exact billing.
There was a time, not too far in memory, when the value of being human wasn't about our intellectual prowess[^1]. We were creative with the human worth thing: being there in the time of need, faith, nobility or being a well-dreaded, literally-bloody viking full of beads in your beard.
Yes, I fear we will stop being the top of the food chain[^2]. But by god weren't we getting bored? Maybe getting off our high horses and going into the weeds won't be such a bad thing.
[^1] And even today, if you go to parties and keep bouncing into people that have no trouble picking up intellectually stimulating topics, you are in a lucky minority.
[^2]: More about my fears, which are unhealthy and you don't have to read but I feel compelled to include citations: https://w.ouzu.im/aweb/unhinged
"Continue reading this post for free in the Substack app".
Ugh, no.
This current moment we are in feels like something out of the Remembrance of Earth's Past trilogy, bit instead of humanity becoming super depressed and hopeless about a future alien invasion we are super depressed and hopeless about LLMs destroying all our hopes and dreams.
A hypothetical... suppose AI "solves" (for your preferred definition of "solve") your field and what you used to do becomes useless or near-useless. But you still enjoy doing it and you also get UBI that provides for comfortable lifestyle regardless of what you choose to do.
Questions: (1) Do you still weep for what has transpired? (2) Do you continue to do what you used to do (with optional AI assistance)?
For me: (1) no, (2) yes and I think the same would be true for most people. So I can't help but believe that most of the anguish about AI is far more about a direct economic hit to people's livelihoods than it is about the impact to their respective fields. Which is more than reasonable. But then why not frame the conversation that way instead of beating around the bush?
For many people, the answer to 1 is yes. You maybe happy with our future where we putter around aimlessly as consumers, but some people want to live in a world where they can at least try to do something that matters. Virtually every professional mathematician is motivated by personally making a permanent contribution to human knowledge, no matter how small.
I understand that sentiment, but I don't think it really counts if the only reason you were able to do something first is that others were forbidden from using an AI that would have done it faster.
There is no reality in which everyone gets their basic needs (food, roof, health) without a job.
heck, we have the resources to that now but we don't do it.
We do for seniors, who are much more expensive than average due to medical costs. In a hypothetical future where AI is much more productive than humans we could easily extend that to everyone.
This was a very interesting read, from 2003.
https://karmak.org/archive/2003/12/manna/manna1.htm
no we don't? Also varies massively between countries. UBI talk is just distracting the masses from the impending change of paradigm. UBI is basically Blisstonia.
I’d be okay with UBI and hobby intellectual work for the rest of my life. But I’m not a mathematician. Many of them probably have more of their identities tied up with their cognitive abilities and social standing.
I think an under appreciated part of mathematics has been contributing clean and elegant work precisely so that others could use that as a foundation for further work and new insights. This dump from Open AI seems to be the opposite. Huge and messy, incomprehensible and taken seriously only because of Lean.
There is this assumption that AI will contribute to faster progress in math but given the garbage dump of proofs which we have to take on faith are robust I wonder if the opposite is true.
My reservation could only be dismissed if AI turns out to be able to contribute fundamental new insights and pose new problems. This takes us into an area where AI progress has been slow to say the least.
The dump from OpenAI seems to be the opposite: huge, messy, and incomprehensible, and taken seriously only because of Lean.
Math papers were hard to comprehend well before AI. This is hardly new. For example, here’s a screenshot of a paper from 40 years ago:
https://i.imgur.com/kPFlczp.png
At least with AI, you can download the LaTeX, upload the paper to your favorite LLM, and have it explain the paper at whatever level of granularity you like. That wasn’t possible even two years ago.
The right conclusion is that LLMs are an alien type of intelligence - if you can even call it that.
And that's not really surprising. We are not supposed to re-create humans but build something better than a human.
I object to a few of the assertions made by maths folk in the article- 1st being that Google Translate taught us nothing about linguistics. If you use google translate you learn less about linguistics than if you had to experience learning the languages first hand, maybe, but you can learn form just that output if you are determined to, ad, more importantly, the folks who built it and are still building it are still learning a huge amount about linguistics.
What are the relationships between many languages that become apparent, including code as a language that were not obvious before llms? Translation is fairly solved. The study of how and why languages, not at all.
I mourn the loss of comfy, intellectual, thinking jobs like those of the mathematicians where people could spend time pondering without the immense pressure that would come from doing something like that in a corporate environment.
You can see why in his own words. "my field of research" emphasis on "My". Humans are great at constructing exploitative systems were all but a few get the shaft. The machinist smashed their ivory tower. For people concern with mathematics, from Platonists to Constructionists AI is a great development, it's only bad for elitists. Except not even then since we are all in the same boat now but the water-level has risen greatly!
Isn’t it deeply exciting that the proofs look so alien?
Isn’t that in itself the new frontier for human understanding?
I would have thought that mathematicians would have found alien forms of maths to be astonishingly compelling, must learn, must know, must understand.
Instead there seems to be a sort of shoulder shrugging lament “we don’t understand it”.
I would barely be able to sleep with excitement at the potential for where this could lead. Like finding an ancient tomb full of answers to our deepest questions along with the promise of more.
How deep does maths go? Is it “solved” or have we just been given tools to delve ever deeper?
Where is their curiosity - don’t the want to find the bottom?
Mathematicians are smart enough to have figured out which way is up. You can debate the stuff outside someone's field not knowing the details but the insiders know what is happening. Ai may not be better than someone with very specific knowledge at this point but it is certainly better than the net. Once it gets optimized for every little super human knowledge, it can do better.
These guys know because they looked at it. Denying it is useless. But maybe some of these guys can figure something new using it and asking it to look for it.
One of the driving forces of math achievement is an internal award (yes, friggin business speak).
The vanity, fame, self-improvement, it's very selfish as everyone does.
Now the fame coming from proving things like Riemann Hypothesis is taken away. Self-improvement? relying on AIs make you stupid. It will be lost. Vanity? Who else don't know how to use AIs? It cannot fill your void.
Now also the most important thing is the communication.
There will be less communications amongst mathematicians at least in person. On paper? maybe but mostly via AIs.
I don't think it's the communication that's lost, it's the motivation. People love to collaborate, but they don't always want to waste time doing it.
It's the same reason you see 'no AI' repo labels: only by forcing some standard on the community can the 'old way of collaborating' continue to exist.
There will for sure be new ways of collaborating, and walled off gardens like you suggest, where communication stops and everyone just shares some final product.
But, in some cases the product must be understood (or demonstrated) before value can be derived, and so collaboration will still exist.
Desire, and the 'shape' of collaboration has changed greatly though. Why collaboration has changed, is a completely separate issue related to economy and how each of us spends our time.
ty. Good points in general.
> There will for sure be new ways of collaborating
I am not smart enough to know what the new ways will be but there should be one to adapt to changes.
The fame from solving the Riemann Hypothesis has not been taken away. It's still open.
Desire and motivation are factors to consider. Would you want to invest 5 years of effort into solving the Hypothesis 'the old way', knowing someone is probably working on it right now with AI?
And then once it's solved by AI: do you want to churn through the proof to understand it? What if it takes you 5 years to churn through the proof, and it turns out someone made a mistake with AI (I have no idea what would be realistic amounts of time, but the idea is still worth considering).
The issue is very complex, and you will always be hungry.
For sure, I just wanted to correct the impression that it has been solved. There are a lot of people going around pretending LLMs have solved everything and human maths is over, but what has been achieved with them so far, while impressive, is a very, very long way short of that.
If I were a mathematician working on the Riemann Hypothesis, I'd probably be rather pissed off right now but continuing my work, i.e. the same as I am as a software developer.
I get you. What people are doing now is questioning value: in themselves, and what they can create for others. I hate to always oversimplify things but we're pretty simple things to begin with:
Social creatures that share a common space, with finite time that nobody wants to waste.
> I hate to always oversimplify things
Every analogy works "only" at a certain abstraction level. No need to disregard oversimplification. This spawned the debates so all good.
ty for the cath. That is the only "famous" one I recalled as I am not math savvy.
These reactions strike me as essentially spiritual crises.
"What am I and what's the point if my intelligence no longer has special economic and social value or meaning?"
Questions that more of us will be asking in the future. Do yourself a favor and figure this out, if you haven't already. Would be interesting if this leads to another "great awakening" type religious movement.
Would doctors weep if cancer is cured by AI?
I deeply understand the sentiment, that’s just human.
But in the grander scheme of things, it doesn’t matter who solved it, speaking for the benefit of humanity as a whole for a longer timespan than our short lives.
As long as those who have studied and contributed are acknowledged properly, I’m happy to let AI solve decades long scientific problems.
> Would doctors weep if cancer is cured by AI?
The pharama and medical PE companies will definitely weep as a source of their income is lost
Article refers to Hugo Duminil-Copin, who also did a podcast with Brady Haran [1] about this subject, for him it isn't case of bruised ego, or gatekeeping. He realized that the way things were done were going to change, and it made him emotional, he is not pessimistic or against AI.
[1]: https://www.youtube.com/watch?v=omuJpoV-Ko4
I strongly feel most angst is because Mathematics has become yet another capital intensive game to play. Rather than a noble, purely intellectual practice.
Mathematicians are grieving the birth of something that is, in a very real sense, the crowning achievement of mathematical ingenuity.
I never appreciated hard math much because my brain just can't get there. I thought so much of math was much ado about nothing, but now I see the art in math...not in the inscrutable formulas, but in the creation of AI.
Well done, mathematicians. You can now rest from your labors.
> Well done, mathematicians. You can now rest from your labors.
well i think the grieving is because they still have to make rent somehow.
Heard from one mathematician recently - math is not about problem solving, but about the artifacts that are produced in the process. The new Navier-Stokes solution for singularity isn't worth a million, it supposed to generate new mathematical methodologies, tooling and lead to additional discoveries, like the graph theory.
OpenAI didn't have to mass publish these solutions. They did it to make themselves look good. They are ostensibly making a tool for people, to better humanity. Instead, they chose hubris and self-interest.
Letting mathemeticians use these tools to reach the same conclusions would have been far less destructive, and potentially, even exciting. Shame on OpenAI.
> Instead, they chose hubris and self-interest.
Well, you accidentally described a top mathematician before age of 41.
Not following...
These guys are still in the early stages of grief. They should look at coders; we've already been there. Coders don't assume coding is going to remain a profession. Why should any other profession be exempt from disruption?
Honestly, there are much bigger fish to fry, like how will everyone pay their bills, and how will we prevent catastrophic AI.
>I wanted to talk to one of the mourners to understand their technological grief. I wanted to know if theirs was a familiar case of bruised ego or disappointment—akin to, say, a boy who spends an hour building a tower out of blocks and watches his big brother smash the thing to smithereens—or something stranger and more profound.
I'll be honest. The response to mathematicians, even this one, makes me concerned about the lack of empathy and honestly just theory of mind of many people in tech. It's not a mystery why people are upset that their field, for many the singular passion to which they've devoted their entire lives, is being annihilated so that a massive company can boost its IPO valuation. It's not "bruised ego" ffs.
‘Tech’ culture has been getting nastier, more vicious, and more zero-sum for years now. It wasn’t even all that long ago that I remember feeling hurt when a friend of mine sneered about “tech bros.”
But I get it now. I see a certain type of guy in an AWS fleece and instantly judge him, round him off to some kind of bully. Harsh, unsmiling, tweaking on stims, telling you to cry harder. Or maybe he’ll open up tell you his theory of how we need eugenic breeding facilities on the moon, or which countries we need to “glass.” It’s gotten so ugly.
I know this seems mean, too. Coming out of nowhere. Strawmanning or whatever you call it. Literally inventing a guy to get mad at. I’m aware of the irony. I don’t know. I guess it’s another kind of mourning, and I don’t know how to express it. I thought tech people were supposed to be like Woz.
I don't care one bit that Mathematicians wasted years of progress or lost their craft over this. The problem is that models that can do this are locked behind corporate walls.
“ What makes us special, it would seem, is this capacity to compound.”
Yeah so you’ll compound using the AI tools and get more done. Stop crying that your tilting at windmills is now being done more efficiently and find new windmills to tilt in new AI-assisted ways.
It's got to be alarming. Without a doubt, being not only on the frontier of yoyr profession, but in THE profession that is undoubtedly the most intellectually rigorous and well renowned and respected... It's got to feel pretty awful
This is such an insidious propaganda targeted at the demoralization of entire professions. Strogatz is an AI booster who "wept". Totally representative!
How much do they pay you? Does OpenAI have bulk subscriptions to your Substack?
> But tomorrow, it may be the illustrators
I thought illustrators have been out-of-job for quite time, right since Midjourney released. So not tomorrow, but yesterday.
We are seeing many fields being impacted. Makes me happy to see things like artcraft coming out that directly impacts the big tech companies themselves.
If understanding is so important and AI systems can only produce knowledge. Why can't mathematicians continue their work on better understanding?
Mathematicians should web search the arguments when artists were hit by the wave. Scientists in general and people that think humans know better.
This is probably the first time in history deep thinking is being replaced by a machine, but the overall arguments are the same in all cases. Graphical artists saw it first, then programmers, now mathematicians.
No, there is not stopping the wave. No, society won't probably be able to support the same number of mathematicians as before. But yes, there will still exist mathematicians after the wave break, just probably completely different than the old way.
And yes, some people will still try to do artisanal mathematics and say it's better, it probably is, if the artisanal mathematician in question is doing deep creative and out of the distribution work.
Those of us who loved programming have already been through this.
I hate programming these days - back in the day, the cubicle-hell nature of "popular" frameworks was at least offset by being able to think through and solve interesting problems.
These days most of us are stuck in the "endless-review" loop with LLMs. You can't afford to not use it given the inflated expectations. As with all of modernity, productivity doesn't yield leisure - it just proportionally increases your workload.
Hardly surprising that the other side of the Curry-Howard correspondence is now getting hit with this. Math was one of those depts. where slow/deep-thinking was possible (CS was already insane 10 years back - now it's just absurd) - now even this fortress will be broken. Tragic.
Omitted title payoff: But it's something AI can never do.
I have no idea why they weep, I read the article and I believe they weep because they choose a mindset of scarcity vs abundance. There's going to be more math with AI! The number of math to solve is infinite, AI will bring to the surface so many of them and can't solve all of them, a person can pair up with AI to tackle one.
2 seconds after AI cures cancer: "I know why the cancer doctors weep"
the comment section is super split here, some people think ai is not doing anything useful until it proves/disproves something really out there, like riemann hypothesis
> "Earlier this year, Shanmu Jin, a Chinese medical resident, used ChatGPT to solve a problem called Crouzeix’s conjecture, which professional mathematicians had failed to solve for decades."
I note that that these conjectures being solved by AI are named after people, minds knowledgeable and and brilliant enough to come up with the question in the first place and recognize its value. What we have in these AI math solving systems is the equivalent of the chess and go software that can beat humans: something that can adhere to a set of rigid rules to reach a win state once given a starting point. But, as Pablo Picasso said, “What good are computers? They can only give you answers.” These AIs are a magnificent achievement and will change mathematics and coding but can they ask earthshaking questions? Could an AI identify conjectures that mathematicians recognize as a worthy question that will lead to new insights? Where is HAL9000's theorem or the M5 Multitronic's conjecture? Maybe they can someday but I don't see any immediate prospects for it.
Please spare us the "AI will surely keep getting better forever" (unlikely and don't call me Shirley), "moving the goalposts" (I myself haven't set any), etc. non-arguments.
My take is that it is an exciting time to be a mathematician since the frontier for interesting math problems is about to explode, and without prior art or references, it's green field for both AI and human. Think of all the new discoveries we'll make.
The beginning of this article is decent, the end sounds like a tobacco lobbyist touting the health benefits of nicotine.
> If intelligence becomes less uniquely human, we may come to value more highly what remains uniquely human: our capacity to relate and to care for one another.
Lol. How does the author think relations between people work? (At least those outside the family). Relationships require something that fills them with meaning - a reason why you would want to have that relationship in the first place. Without meaning, no relationship.
I recommended reading this other post that was on HN earlier today: https://thetheorygirlie.substack.com/p/ai-was-great-until-it...
It describes in pretty grim words what happens to collaboration once you have nothing anymore to collaborate on.
> Mathematicians may still teach proofs, argue over what they mean, translate alien logic into human insight, and pass that understanding to students.
This would people to have time to understand and dissect those proofs - it doesn't work if there is a new breakthrough proof every week.
I still remain hopeful that this is a step change in capability and capacity that we can adopt and adapt towards our own purposes. What was taking decades for us to do has been reduced to days and dollars. So while we’re witnessing a forest fire consuming entire fields of open problems in math, the best upside would be finding new problems that come from learning how to work with the new tools.
This is what it seems like so far for software. The speed limit has changed. It’s interesting to explore what that means. Extending the analogy, long ago you could be a race car driver part time; that would be pretty laughable for an F1 driver. It’s a very specialized skill and the optimal athlete is quite different for driving a car than for running, for example. That feels like what I’m seeing in software.
The most effective folks are much faster and better with these tools than average. It’s a different skill set.
I think that formal verification makes Math much closer to code than many other fields. But I’m eager to see the first mathematicians that are dramatically more proficient with the new tools than average.
"I know now why you cry." - Terminator
I wonder how much this is like a tremendous telescope being invented which can see new worlds with utter clarity. We could verify that a planet was habitable... but it wouldn't bring us any closer to visiting it.
Now many of us will get to learn how to find meaning in a life where your time is worth about $2-4/hour, which is the reality for about 2 billion people today.
I am curious.
Where or what do solving these math problems lead to?
Is there some suggestion that they lead to deeper understanding about something else? Do they solve physics problems?
What’s the benefit? What’s the point?
The interesting answer is we don't know. Many of the math problems that make our computers efficient were developed hundreds of years ago with nothing we could do with them.
It is mostly impossible to figure out what we find when we solve these problems. But occasionally they'll lead to real world efficiency in algorithms we use and it changes the world.
This doesn't make any sense.
If you have AGI, when a situation presents itself that needs a particular thing solved - it'll then go the math thing specific to the situation.
> Think about a world where non-mathematicians leapfrog the field after a few AI prompts. It would be a world with more knowledge. But it might be a world with less understanding.
This seems like a good summary of the article, and I think the answer here is obvious.
So what?
News flash folks: we do not have the knowledge to survive on our own. If a solar flare deletes the electric grid, the first thing that happens is 8 billion people die in the next year.
We survive on abstractions others have built, and we have a global dependency web.
AI abstracting knowledge is neither new or dangerous; if it is dangerous, this certainly isnt the argument to prove it.
I have little sympathy, since I think it is a result of many decades of complacency when it comes to thinking about why want to people do what they do and instead just enjoying being the ones with the results and money. People who love the process and see the human as inherently valuable won't have a problem. I'm more shocked that there are so many brilliant people who are so results-oriented and don't even have a concept of value outside of that. I'm kind of disgusted by the tears actually
> I have little sympathy, since I think it is a result of many decades of complacency when it comes to thinking about why want to people do what they do and instead just enjoying being the ones with the results and money.
It's not clear from your comment: Are you talking about math professors?
If so, what money? They make a living, but it's not that great. I just checked the salary of two associate professors at my university: $80-85K.
And keep in mind they often start their careers quite late. Often multiple postdocs after PhD. Starting a tenure track position in your mid 30's is not common.
> I'm more shocked that there are so many brilliant people who are so results-oriented and don't even have a concept of value outside of that. I'm kind of disgusted by the tears actually
The whole point of the article is that the person crying is because he loves the process and is worried the human connection will be gone.
This is exactly my perspective as well. Seeing humans as inherently valuable is the key. It's shocking to me that so many people seem to have been reducing humans to the results they are able to produce and now that the results appear to be drying up, their perception of their own human value is going down with it.
I’m fully STEM: software developer with a physics PhD with a mechanical engineer wife, but this is certainly just deserts for the STEM people, who mocked the humanities and especially philosophy for centuries (literally, this started back with Kant even) for spending time thinking about these questions. STEM people are not at all spiritually prepared for it, and this is the main disappointment for me. I spent many years struggling with publish or perish in physics but all those around me claimed they simply loved doing physics and that that’s what mattered. Now as soon as their exclusive ability to give results is slightly threatened the love of physics/maths/coding is no longer spoken about. I get maybe they weren’t the same people, but I thought more people loved the process than it seems today. If anything these new developments make it even clearer for me what I love about this kind of work. I’m so happy it has made me not have to defend based on results, a razor sharp contrast that makes it so clear really why humans do things
> just deserts for the STEM people
it is a little ironic that after the "creative destruction" of industry after industry all of a sudden now it's unfair and we weep. I say this as a software dev with his neck on the line like everyon else.
It’s not necessarily their fault. Our economic system puts zero intrinsic value in humans, it only puts value on their output. It’s hard not to feel this type of way when it’s the only system you know.
Your loved ones have replaced you with more agreeable robots. Now no one cares whether you live or die.
Still disgusted?
Anyone who would replace me with a robot never loved me anyway. If that happened I would go and seek out the humans who really do care, and take solace in the fact that Christ will always love me
> Anyone who would replace me with a robot never loved me anyway.
Ever heard of sour grapes?
This is the lie you tell yourself to handle the situation.
Handle what situation? Why would people who love me ever replace me with a robot? It’s a random posed situation that no one discussed and doesn’t make any sense lol
> Why would people who love me ever replace me with a robot?
Why would one who loves someone, ever love another?
Huh??
...except it is exactly the opposite of your big brother destroying your tower. Instead, it is your big brother showing you this method using a tool to build many more towers at astonishing speed!
I get the sadness (and, as a software engineer, certainly feel a real existential threat myself) and feeling diminished, but at the end of the day that is all just ego and I very my agree with the end of the article - who cares who came up with it and on what shoulders we stand, as long as it is saving lives and improving our understanding of the universe!
Not sure about that title, it did bait me into a click. "Mathematicians are now facing something similar to generational trauma of structural racism born of exploitation, slavery, and colonialism?" https://en.wikipedia.org/wiki/I_Know_Why_the_Caged_Bird_Sing...
But regardless, the detection of breast cancer via AI made me think of phlogiston https://en.wikipedia.org/wiki/Phlogiston_theory . "Phlogiston theory states that phlogisticated substances contain phlogiston and that they dephlogisticate when burned, releasing stored phlogiston, which is absorbed by the air. Growing plants then absorb this phlogiston, which is why air does not spontaneously combust and also why plant matter burns. This method of accounting for combustion was inverse to the oxygen theory by Antoine Lavoisier."
It could be phlogiston. AI is super smart theory states that AI is super smart and that is a single thing can do tasks very efficiently. But it's so smart, we can't understand it, so we humans can't help it, it'll take AI to make it even smarter. That's what's holding it back. But then it'll solve everything.
Or it could be oxygen. AI is super-fast-and-parallel theory states that AI is not a magical monolith but a bunch of pedestrian programs that run fast and in parallel, thanks to hardware (GPUs) that's good at that stuff. But since it's all going so fast and at the same time, something a single human cannot do, humans can't really see how it works. But if we could, we'd probably just see its just doing stuff programs did before. So what's holding it back is that you have to keep throwing hardware at it, and eventually you run into the limits of how fast you can build that hardware or power it.
It all might be stuff humans were better at, when approached one-on-one, but against a swarm it loses. You get a billion doctors together to look at a single mammogram, then tell them if it cancerous, then do that again a billion times with each doctor. Then ask those billion doctors on the billion-and-one mammogram to give a diagnosis. Heck, do that with a billion toddlers. I think either might be able to beat the AI. I don't have a billion doctors or a billion toddlers--frankly I don't know which would be more difficult to handle.
To know if it's oxygen or phlogiston takes probably a demonstration like this "Phlogiston remained the dominant theory until the 1770s when Antoine-Laurent de Lavoisier showed that combustion requires a gas that has mass (specifically, oxygen) and could be measured by means of weighing closed vessels." Maybe the data center financing trouble will be the demonstration that the mass in the gas is just VC-money hardware.
I know why you got a billion cages with a billion birds and a billion bird listeners to compare a billion songs to the few songs we like to hear, and then packaged all that into a single bird-looking thing to make singing-like sounds: So Altman can get that VC money.
I don't really like this comment I made. I was tired and I tried to make an analogy with phlogiston for what seems like a pushed portrayal of AI as being, well, some kind of "super intelligence." But the kind of person that might try to "make fetch happen" with "super intelligence" is very different from, say, J. J. Becher theorizing about the mechanism with the hope of understanding it https://en.wikipedia.org/wiki/J._J._Becher . Phlogiston wasn't a marketing term bordering on propaganda, but rather a theory before good measurement was available. It's more like someone today trying to guess what may have existed before the big bang, there is no known measurement for that, but for something that was staring people in the face nearly daily: fire.
I mean, it kind of works, in that AI is doing neat things and terrible things and it's staring people in the face all the time--except in this case because the tech monopolies force it on us. So people will want to make explanations. But there was no Big Fire that wanted us to buy more fire-making products and thus would be incentivized to come up with, say, "fire works by using your magical spirit and you get closer to heaven with each fire you set with our fire products, buy our fire products, or rather, let us keep pushing all competition for warming one's self via political corruption allowing our monopolies to exist."
Nonetheless, I do think there's something to be gained with the idea of a billion toddlers looking at mammograms, and the lack of measurement that underlay phlogiston.
First of all, I don't know if I ethically should be using the term "toddler." I heard Coxon use it. Frankly, I don't trust him. I won't trust him until he starts saying /prominently/ that people at tech companies should be liable for breaking laws that already exist and accountability is the bigger problem. Corporations already could do dangerous stuff with biology, weapons, bioweapons or the like, it's not complicated. You do dangerous stuff, you should be held accountable. Why are we farting around wondering if some bio-engineered virus that was released by some corporation could replicate on its own, when we should be say "yo, some organization released a virus on people, why are the people at the top of this organization not getting arrested or something?" I'll admit it might be an "or something" and not "arrested," but it sure looks like nothing is happening--because that's what usually happens when the sufficiently rich do something really bad. No accountability. If Coxon is going to use talking points (and I suspect he is) it should be the very first talking point Coxon or anyone else uses--otherwise I don't trust them.
But let's use "toddlers." A billion toddlers started to get right on mammograms being cancerous or not, but being toddlers they can't really articulate why.
One toddler notices he gets his marshmallow for being right when he notices the pictures are not "shiny." What does that mean? He can't explain it, but it turns out he noticed that some images were made at a lower resolution or otherwise fuzzy because it was older/cheaper tech. And who uses that? People who are poor in poor areas. And turns out, poorer people from poorer areas don't bother to come to the doctor until it's an advanced problem, because they can't afford to come in otherwise (because, say, it's America and American healthcare is broken). And sure enough, those people are more likely to have cancer.
One notices there's more white stuff in some places and dark stuff in another place, not always the same place, but kind of similarly. They say "it's um got that white stuff dark stuff thing." It turns out that's just a reflection of something that happens when the patient is richer or poorer, and again richer people have more chance of false positive because they have the luxury of getting checked more often.
One notices they are more "squishy". That's because radiologist in socioeconomically better areas have a different technique... And so forth.
And, then to make it worse, most toddlers are so inarticulate they can't even say "shiny" or "squishy," so you can't get any information out of them.
That inability to articulate is kind of like the inability to measure, the inability to see the importance of mass in the air, per this: "Phlogiston remained the dominant theory until the 1770s when Antoine-Laurent de Lavoisier showed that combustion requires a gas that has mass (specifically, oxygen) and could be measured by means of weighing closed vessels."
So, it kind of works as an analogy that way, but--again--it's a bit messy.
This comment isn't much better, but I love to dig deeper into my holes.
Of course the other Abundance guy joins in on the AI Inevitability Soothsaying.[1] It really is the perfect vector for the AI sirening. You just take the inevitability and awesomeness as a given. That’s the real point. But you write about the “grief” and the loss of “identity” and psychology—oh my the feelings. See, we are left-liberals and we care so much about people’s feelings. All the investor class and their hype is perfectly on track is about to fuck us the rest of us. But the feelings about people can’t “understand” and “discover” is what matters.
[1] https://news.ycombinator.com/item?id=48387092
> we may come to value more highly what remains uniquely human: our capacity to relate and to care for one another
...under capitalism...
Sure we will...
I’ve been a math professor for about ten years, and I think people outside the field misread what makes mathematicians angry, sad, and worried. Interviews that make it into the media tend to go with “AI is killing mathematics,” maybe because that gets clicks. The reality is more complicated.
Some mathematicians work on well-posed, famous problems, but most mathematical research is about questions that aren't well-posed yet. A typical example is finding where the boundary of existence lies for some equation: which meaningful conditions guarantee a solution, and how far can you weaken them? You can’t even start on that without understanding the problem. For the famous open problems, the meaningful conditions have largely been settled already, and what remains is to prove existence under those conditions or build a counterexample. That is exactly the kind of task AI is bound to be good at, and plenty of people expected computers to get there eventually, going back to AlphaGo. It just arrived much faster than expected, which is causing a lot of problems. Assuming it made no errors, AI would be a great research partner.
Which conditions are meaningful depends on context: where the field is going, what an individual researcher finds interesting, connections to actual phenomena. AI can’t decide what I find interesting. The OpenAI Navier-Stokes result shows this failure of context. OpenAI announced finite-time singularity formation with a smooth body force. Constantin, Ignatova and Vicol (https://arxiv.org/abs/2609.20803) showed that for solutions with the key features of OpenAI’s construction, the force can neither vanish identically near the singular point nor be real analytic in space. Fefferman’s Clay problem statement does allow a forcing term in the breakdown versions, so the announced result may fit that format, but this method cannot reach the unforced case, which is what mathematicians actually want to know. It is a step forward. Still, I think a mathematician who had spent years on this problem would have pushed toward better conditions, and the paper would have been better for it.
What makes me angry about the ~770 papers is that the author, OpenAI, takes no responsibility for their manuscripts. Many are badly written, to the point where you can’t tell whether they are right or wrong, and verification runs on unpaid labour from human mathematicians. People say, “but it’s checked in Lean.” As Bastounis, Circelli and Hansen showed for the Navier-Stokes case (https://arxiv.org/abs/2610.08144), the Lean proof can pass while not corresponding to the natural-language proof in the paper. A human author would be expected to dig into the Lean code and fix the paper. I doubt OpenAI will. If someone dumps more Lean code than any person can read and walks away, who is supposed to check it? The person who was working on that problem gets scooped if they finish their own result, and gets no academic credit for verifying OpenAI’s paper either.
I’m also angry at mathematicians, not just OpenAI, who use AI to mass-produce results they don’t understand themselves. Publishing a result you don’t understand is the same as not taking responsibility for what you wrote. It shifts the burden of verification onto referees, readers, and whoever builds on that result next.
My biggest worry is that mass-produced, un-understood papers will slow mathematics down. You need to know what we currently understand to see where to go next. If everything around you is papers nobody understands, you can’t find where you could make a real contribution. It’s not that your view is narrow; it’s that the papers block the view. It's like we want to go to space, and low Earth orbit is filling up with space trash.
I also worry about my students. A PhD student is something like an academic child, and I don’t know how to advise them anymore. Do I tell them to put out 20 papers a year whether they understand them or not, like some people do? Or stick with the traditional goal of one paper a year that they fully understand? My conscience as a scholar says the latter, but I’m afraid they’ll lose out on the job market.
And there is sadness. Solving math problems is a joy. It’s an experience of thinking through something with other people, being delighted and frustrated together and making progress, and that is being taken away. A friend I used to discuss problems with now feeds our ideas to an AI and writes the paper right away. I’m left with the experience already spoiled, doing the tedious cleanup and verification without the fun.
As for “mathematicians are just protecting their jobs”: mathematicians don’t make money from research in the first place. There are almost no experiments, so research grants mostly cover conference travel. Most of a math department’s budget comes from teaching, especially courses like calculus and intro statistics with thousands of students. If anything, now that AI does the homework, assessment is moving to oral exams and in-person tests, which takes more staff, so I don’t expect math faculty positions to shrink unless we give up on math education. Outside academia, better access to mathematics may even increase demand for mathematicians.
[flagged]
> “They have annihilated my field of research.”
I think they might find comfort in knowing it's not just mathematicians - this is coming for all of us. Computers putting us out of work has been coming for a long long time.... and now it's here.
[dead]
[dead]
[dead]
[dead]
ahem
I weep because the money was bad.
I want to sympathize, but most of the whining in the article seems like gatekeepers bemoaning their gates being blasted off the hinges.
Are there any mathematicians who are genuinely excited about this tremendous leap forward in mathematical knowledge? Where are those articles?
> gatekeepers bemoaning their gates being blasted off the hinges.
i think that's 85% of the response from software devs if not more. All of the devs I know who actually like to build have embraced AI and the new capabilities that come with it. The ones that guarded their little corner of code proclaiming only they, and their inner circle, have the power to comprehend it are the ones crying in the corner.
Getting those fields wiped just shows how unnecessary those fields where in the first place. I'm a programmer. I am the adapter between a stakeholder and a machine. I am just needed because the stakeholder lacks the knowledge to tell the machine what he wants. If he can use this new fancy adapter called Agentic AI now I happily quit my job and go on finding something that makes more sense in the first place.
The same goes for math.
Let's find some things that bring value other than just the price of a stock.
Broken take, with out this trail of mathematics that lead us here we'd have never reached this place. You think multi dimensional matrix math comes natively to humans?
At worst its time is past, but even that is unlikely.