As a mathematician maybe I am a little more optimistic than this declaration.
I am thinking of Mochizuki's abc conjecture: He worked in relative isolation, and dumped a huge incomprehensible proof on the community (to oversimplify a bit). That's not totally unlike what might happen if AI generates a huge, incomprehensible proof of let's say RH.
Well, what is the result? In the Mochizuki case, it was a lot of skepticism, but it also generated conferences, papers, talks in the hallway, discussions with students, and so on--a flurry of exactly that kind of community process that the declaration says is the main driver of mathematics.
Ultimately we think a fatal flaw was found in Mochizuki's proof, so it didn't lead anywhere in particular. But in our hypothetical "AI lean-verified proof of RH" situation, it would presumably generate substantially more of that community activity we saw in the Mochizuki situation. And if it's correct, that community activity would be productive (expository talks, students given problems to flesh out or generalize, etc).
Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.
It’s frustrating that this comment is at the top because it, along with lots of the replies it inspired, absolutely misrepresents the actual declaration. The declaration is not making any statements about not using any AI in mathematics. The entire point is to push the use of the technology in a direction which is compatible with positive pre-existing features of the math community, and to make it better known what some of the current problems are.
It includes Terrance Tao who has made the front page several dozen times at this point for his usage of AI such as to help the community write proofs for Erdos problems.
But to many commentors he's a now gatekeeping AI-hating Luddite clinging to a dying profession out of bitterness and envy because his position is more nuanced than "throw AI at everything and turn off your brain".
It's the same on Reddit, and if you look up a few comment histories, you'll find that it's mostly /r/singularity users and /r/accelerate posters that are spamming. So I assume that people are just deliberately misreading the letter and trolling here as well, and I wouldn't read too much into it, but on the other hand, if you never had to think about what scientific misconduct looks like, it's harder to see why the behavior of the AI companies is problematic.
It kind of reminds me of Clay Shirky's predictions about the influence of the internet on society from a few decades ago. To paraphrase what he said: when the printing press was invented, everyone predicted it would lead to world peace, and we got the Thirty Years War instead. When modern mass media was invented people predicted world peace, and instead the Nazis rose to power and we got World War 2.
And yet as a society we typically consider these technologies a net positive these days, because alongside all the political instability and violence that followed them at first, some people were figuring out how to do beneficial things with them along the way that did more good than bad. Because the issue really was that the influx of more voices and ideas shifted the power dynamic, required relearning how to communicate, and in the end be better as a society.
The issue was figuring out how to "domesticate" these wild new communication channels. One successful example of which was the invention of scientific journals and papers.
Shirky predicted the internet would probably lead to a few decades of political instability too (about fifty years was his guess), and we definitely seem to be in the middle of that process.
Now, I haven't checked his stance on LLMs. I also don't know if I would quite call them a medium for mass communication like the others (as used today, they take humans out of the loop rather than let more voices join the public discourse), but I feel like they are similar enough to otherwise fit the pattern.
Tao's stance similarly feels about wanting to domesticate this wild animal before we get mauled by it. And to stick with the metaphor, I feel that most of the time the AI industry is trying to bamboozle us with spectacular rodeo displays instead.
Because his letter misuses the word "misalignment" for what's very clearly a capability gap. To anyone familiar with that sort of language, his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians, which is exactly what the AI-hating Luddite would say! The reality is closer to "current frontier AIs are clueless about what makes mathematical problems/proofs interesting to humans" which is a vastly different issue.
> his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians
That is probably the wildest misinterpretation of sth I have ever read here. Misalignment is used as in the goals and interests of ai COMPANIES are not the same as the ones of the mathematical community. It is very proper use of the word, and if anything imo the properest, as it refers to actual people and institutions to which actual self-ascribed goals can be defined, as in contrast to hypothetical superintelligence. AI doomers do not own some trademark on the word "alignment".
The point stands whether you attribute agency to AIs themselves or to AI companies. The companies are not deliberately sabotaging human mathematical understanding by writing up purposely inscrutable results, either (which is what the word 'misalignment' would imply in this context): they are merely working with frontier tools that have a very limited capability for human-like understanding. It's left to human mathematicians to bridge that particular gap.
I wouldn't call the situation settled, nor do I expect we will ever find out what truly happened:
- a rare low likelihood sequence of coincidences
- deliberate frontier lab behavior, setting intellectual interrupts on LLM "aha"-moments, so that when some mathematician explains for the umpteenth time the approach they want to take "stop paraphrasing my approach, start the actual calculation!" and when the chatbot eventually groks it, they can scoop in, possibly with live dash-board and interrupt priority levels ranging from "ensure this sample gets into the new dataset" to "Millenium Prize Scoop Opportunity Imminent, call Sam"...
There is always an alignment problem, incentives to ideals, behavior to incentives, ...
You seem determined not to understand this, but to spell it out even further: The point is not that the companies are deliberately trying to harm maths but that in pursuing their own goals, which are different to those of mathematicians, they are doing so. Their goals and their methods of achieving them are not aligned with those of mathematicians. This is, incidentally, exactly
what alignment means in AI as well: a paperclip maximiser harms humans not because it is intentionally trying to do so but as a side effect of trying to achieve its own objectives, which are not aligned with those of humans (which include not having their entire environment turned into paperclips).
The word is being used entirely correctly, and I'm sure the nod to the AI usage is quite deliberate.
Even mathematical community as a whole is not that aligned with the stated goals. Not to blame mathematicians in general, its just a general anthropological platitude that the group actual behavior is not fully matching what people from the group want to pretend even in full honesty.
Though it doesn't take even an outsider to question the matching here, just look what the only guy who settled a millennium prize what he thought about the community. Or ask some actual PhD, postdoc, or even better someone who dropped along the way, how does it feel to go through this community. Not sure the goals exposed in this article are really helped with the community as it is. One again, this doesn't mean everywhere encompasses the same issues and everyone is acting badly on is own.
But if we want to accept happy pink shiny depiction of a community, we should be giving the same generosity to other communities à priori. All the more as AI and mathematical community do share a large set of common individuals.
> Their goals and their methods of achieving them are not aligned with those of mathematicians.
This is exactly the assumption that Tao is smuggling in with "misalignment" talk and then refusing to elaborate on any further. Is the issue that AI companies are willfully refusing to provide mathematical insight that they could provide (because they have diverging underlying "goals" to those of human mathematicians) or are they merely working under a capability gap, where current AIs can awkwardly settle major open questions but are not smart enough to provide the kind of understanding and insight that the community of human mathematicians relies on? These are two very different problems and by foregrounding the word "misalignment" in his letter so openly (as opposed to talking about AI capability to provide valued insight), Tao is picking the more adversarial reading with zero proof or motivation.
It is really not complicated at all. And if tao's post is a bit vague, the letter signed by many mathematicians is imo very clear.
AI can be used to advance/deepen understanding, or it can be used to superficially go settle a whole bunch of open problems in a field without helping really in understanding them. It all depends on who uses the AI and why. Essentially, it is exactly the same concept as using the AI as a course tutor vs having it do your homework. Or using the AI to write millions of lines of code that nobody can actually read, vs keeping overview of what is going on.
In math it is probably worse because there is no objective function to maximise. Some people here think that the objective function of mathematics is to prove things, which is actually wrong. Mathematicians are not only maximising an objective function, they are also defining the objective function they need to maximise (they are defining which problems to study). The problem with AI/ML is that it can be pretty good when the goal is to maximise a set objective function, but not to set intentions and goals themselves. I would not call that a "capability gap" because we can actually get to have very useful and smart AI systems without ever reaching that point.
His view is the "capability gap" one, and you can just read the the statement and Tao's other writings for far more eloquent explanations of the difference between a raw LLM proof dump and real mathematical insight, and what it takes to get from the former to the latter, than I can provide.
Again, nobody is accusing anyone of deliberately trying to harm mathematics. It's about misaligned objectives. Eg. Tao wrote the following before the Navier-Stokes announcement (referring to exactly the project OpenAI was undertaking):
> At this point, I would not be surprised if one could batter out such an extension by pouring an enormous amount of compute and AI assistance at such a task. But such an exercise does not particularly hold my interest; I am far more interested in digesting the proof methods and extracting out the key new insights uncovered by this approach.
And then went further to say that such activity could be actively harmful to the field. (All this can be read on Mathstodon: https://mathstodon.xyz/@tao).
The misalignment is that this activity that he and 24 other Fields medalists think is actively harmful to their field is deemed by OpenAI and others to be worth ploughing vast financial, human and compute resources into.
That's the far more sensible reading, so thanks for confirming I guess. But then the misalignment talk is pretty clearly a distraction.
> ...And then went further to say that such activity could be actively harmful to the field.
If true (and there is as of yet insufficient evidence of this), that's merely a contingent fact about very real institutional misalignment within the human mathematical community, not about AI itself or even AI frontier labs. There's simply zero inherent reason why providing a bare truth value or a completely inscrutable proof about the status of some open conjecture should make that entire subfield of math "contaminated" for the foreseeable future when it comes to extracting further human-relevant insight. That's the misalignment we should be caring about.
The declaration is about that institutional misalignment. They want to change the norms of the mathematical community, so that producing inscrutable proofs is a low-value activity nobody cares much about rather than a high-value activity that AI frontier labs can make headlines by performing. Their ask of the frontier labs is to please be aware of the problems with the old norms and not exploit them during the transition period; they agree this doesn't have much to do with AI itself, which they acknowledge is a powerful technology that will improve and accelerate mathematical research.
Broadly agreed, with a key proviso: producing inscrutable proofs has negligible value as a mathematician's finished output but that doesn't make it a "low-value activity" in and of itself. Ultimately, the status of these proof-like objects as a raw input into mathematical practice will probably be comparable to any other sort of computationally-driven https://en.wikipedia.org/wiki/Experimental_mathematics . These are not new problems: "computer" used to be a job description for humans before it was the name of a machine, but we now view raw computations as a trivial matter that's not worthy of any human credit.
Agreed. I’m sure there was a time in the early history of computational mathematics where someone with access to more computer time than others went around telling people how easy it is to generate books of trigonometric tables, and other people got annoyed at him, until soon computation was so thoroughly automated that the very idea of computing sines by hand became obsolete. I expect that’s about how math students in the 2040s will learn about this: “what do you mean, they had prizes for writing proofs, why would an educated mathematician spend their time writing proofs?”
> deliberately sabotaging human mathematical understanding...which is what the word 'misalignment' would imply in this context
The entire point of the paperclip maximiser is the AI isn't evil. It isn't trying to hurt humans. It just doesn't care about us.
Nobody claims OpenAI and Anthropic are out to torpedo mathematics. Just that relative to their internal goals of getting publicity ahead of IPOs by winning awards, what happens to mathematics and mathemeticians in the long run isn't a real concern.
The Holocene [1] features relatively few species humans set out to eradicate. We mostly realised something had gone extinct after we accidentally destroyed them. That is what originally misalignment meant in respect of AI.
The declaration literally opens by writing that AI companies (or really anybody) saying, “Hey, let’s see if this powerful reasoning engine can solve an open problem in mathematics” is “detrimental” to the “science” of mathematics. Full stop. So I don’t think it’s being misrepresented. If it’s a science, why isn’t progress good in itself? You can’t have it both ways.
The online comments I’ve read that side with the letter explain that the problem is AI proofs are inscrutable and useless, but the reality is using computers to brute-force things has long been part of proofs in one way or another, and AI proofs range in legibility up to 100% (like pointing out a proof exists in a long-lost paper, or writing something that is basically correct and just needs a human reviewer to fix it up). People are probably defending the declaration inaccurately, but I think that’s because the real argument or arguments are unclear.
Is it that students won’t learn math if there’s ChatGPT? That could be discussed.
To a non-mathematician (MIT physics and CS ‘06), the letter sounds like, “We have a fun job. Sometimes there are no practical applications of the work, so it’s just kind of like a sport, er I mean science. If someone solves a problem that has stumped mathematicians for decades or centuries, we worship them as a great mathematician. It’s a status thing. So we don’t like some guy with a computer coming along and solving our problems. The way things unfold with all the ideas coming from humans, over time, maybe it’s slower, but it’s nice.”
I don’t think anyone can really stop someone from using a computer capable of solving unsolved problems to solve unsolved problems, and there are always going to be mathematicians who DO think it’s fun to try to figure out what a computer is doing (if the computer isn’t already explaining it in English, which it generally can), and make progress that way, and it obviously will lead to mathematical advances in human understanding, from my point of view. So the whole thing is a non-issue that no one can do anything about anyway.
Mathematics sits in a very different place w.r.t. other sciences. First of all, it's very meta. Yes, it can describe any phenomenon on our planet. On the other hand, advanced mathematics is a high mountain which is very hard to climb, and changes the climber in various and irreversible ways once the journey begins.
That's said, I'm a non-mathematician, but a computer scientist. Mathematics was always my weak part, because solving problems which doesn't mean anything doesn't motivate me, and when I'm not motivated, I fail.
Without digressing so much, I want to say that, some of the things in mathematics and mathematics adjacent sciences baffle me. Many mathematicians don't understand the proofs of others, yet they accept it since it checks out within the rules of mathematics. Moreover, many engineers don't understand the formulae they work with. I have developed a very performant Boundary Element Method evaluator, yet I don't know the reason of taking Gaussian Integrals over the surface. The only answer I got is "because the method works that way".
So, if computers can make the proofs now, and we have no curious students pecking their professors to understand how these works, the whole mathematics as a science could wither and die. Because besides it being a sport, it's the very language which can model and explain anything, but it's very hard to master and understand due to that nature.
Our professors warned us against using too much Mathematica, to make us learn things the hard way and make the knowledge permanent. Now, if we offload science to a matmul engine, it's possible that we lose the connection and never able to close the gap in some cases.
Again, it boils down to "This machine has no brain, use yours (or lose it)".
Exactly. They can continue doing the sport version, just like sprinting is still there at the Olympics and people admire Usain Bolt or whoever is more recent. But we use cars and trains and planes when we want to get somewhere and don't confuse Bolt's run with transportation. So you can do this kind of recreational intellectually satisfying human math in a similar way, while productive freight-train-like math will be done with AI.
> The declaration literally opens by writing that AI companies (or really anybody) saying, “Hey, let’s see if this powerful reasoning engine can solve an open problem in mathematics” is “detrimental” to the “science” of mathematics. Full stop.
So let's just quickly agree that the actual quote is “However, the push by AI companies to solve mathematical problems as a benchmark is detrimental to the science of mathematics, and to the mathematical community.” And that in this, "as a benchmark" is load-bearing.
We wouldn't see nearly the same amount of contempt from researchers had OpenAI picked a research-friendly approach.
What they did: Hear a rumour about the problem being solved by other researchers, then rush to scoop them (unethical), then, when they actually go talk to them, they try to oust an author (also unethical), and when they finally decide to share their own work, do so in the least useful way possible.
What they could have done: Upon hearing the rumours, connect with the researcher in question and propose that they join efforts instead; set up a joint project to test if the machines are useful in any way, and if that's not appreciated, back down again. And instead of dumping only an undigested paper* and a Lean proof, do the digestion prior to publishing anything (as Buckmaster was in the process of doing). If their own lack of competences was keeping them from digesting it, then again, reach out to the researchers to understand if anyone would be willing to do so.
In the second of those two worlds, we wouldn't be seeing nearly the amount of outrage that we are seeing right now.
*: Here, “digestion” is the process of turning an AI slop paper into something humans can read. LLMs can indeed sometimes (if much more rarely than marketing material from the large LLM companies will suggest) produce correct proofs, but they are often written in bizarre ways – they'll use lingo that doesn't exist, seem overly pretentious, dwell on extremely easy steps while glossing over the hard ones. Currently, a real researcher will take that output and transform it into something that others can understand, use, and build upon. This is not so different from what happens when using it to write software, although as someone who does both, I will say that the amount of digestion needed for proofs tends to be orders of magnitudes larger than for code. This meme is quite accurate: https://mathstodon.xyz/@tao/117068266071803252
I don’t see why the two are mutually exclusive to be honest. I’m not a mathematician, but I’m a software engineer and whether I like it or not, AI has changed my field dramatically forever. I understand the distaste for the way AI might be affecting the field of mathematics, but the genie is out of the bottle, and it will never be the same. It just took a little longer for it to hit the math community. Software engineers have been debating and coming to terms with the new reality for a few years now.
Soon AI mathematics will be 10x or 100x or 1000x more productive than humans in the proofing theorems business. Mathematics of old style is dead, they should develop ideas how to deal with that as a math community.
If a human doesn't understand the proof, it might be useful for certain kinds of engineering, but not really that useful for mathematics except as an existence test.
I don't known, can't we just also indirectly observe the wide platonic landscape of mathematics through observing AI results which humanity could not produce in a 1,000,000 years.
> But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal.
No one is paying mathematicians to develop a conceptual understanding. If the signal for who is “winning” math in academia disappears mathematicians will have a big problem
This is true. There can still be some funding for recreational aesthetic mathematics, like you can get some funding for various art projects or philisophy, literature, poetry, general humanities and community-building projects or sports funding or things like that. But currently many mathematicians are funded by the public purse with the understanding that this contributes to science and engineering and technological development at some point. I'm not saying that intellectual pursuits for the sake of human mental satisfaction is not worth funding in a society. The question is the amount of people needed. Tax payers pay for it. You have to tell them why, and why so many people need to be funded to do that. Many people have various hobbies they enjoy. But generally they don't get government money for doing it as the general rule. Maybe in a post-scarcity world.
The difference is that most software engineers don't care about bringing up a new generation to keep their field alive when they retire. Software engineers usually don't give a flying fuck for other software engineers, young or old, nor for software engineering itself. Mathematicians apparently care deeply for their subject, their field, and its future.
Mathematicians are not idiots! They’ve been living on the same planet as you for the last few years and everyone in any thinking profession has been debating and coming to terms with reality over the same period. People who aren’t involved in mathematics might see a big headline and come to the conclusion that mathematicians haven’t seen this coming, but that’s a completely ignorant position.
I certainly didn't intend to imply that the declaration is against using AI.
My point is just that I feel a little optimistic that the human culture around math ("ideas we disseminate in talks, private discussions and careful writeups, connecting them to the previous ideas of others" and so on, to quote the declaration) is robust even against relatively irresponsible use of AI (i.e., an onslaught of proof-slop).
And I guess we'd better try to be optimistic, because even if the declaration results in some realignment between the community and big AI companies, the models capable of this work are not always going to be exclusively under the control of those aligned parties.
That said, I think the declaration is great and I support it--let's see what comes out of it.
The maths community is now in the antithesis phase, synthesis will take a while ;)
Lee Sedol said in an interview that "losing to AI, in a sense, meant my entire world was collapsing. ... I could no longer enjoy the game. So I retired", and I think there will be folks in the mathematical community who would feel the same when the solutions pages to hard problems are suddenly available.
But on the other hand, people learned a lot from chess engines. After decades of chess computers beating humans, there was still a renewed interest in watching Leela beat Stockfish, with many people trying to understand the strategy Leela used.
If your happiness comes from grinding on a problem and making progress, the prospect of having to dig through a corpus of AI-generated proofs might be hard to swallow. But if you're willing to do that, you will still find beautiful things that only so many people can truly appreciate.
Chess is a fun game. That's why it's been around for 1000+ years.
There was a renaissance during Covid and due to 'The Queen's Gambit' where it gained much more mainstream popularity, but... Chess AI was already far far (like 1000+ Elo) ahead of human players at that point.
The thing is... chess is humans playing (communicating) with humans and that's what keeps it interesting. Check out the view counts of chess AI tourneys vs. human tourneys.
Chess is kept afloat by chess players, not by billionaires. If all the billionaire backers stopped sponsoring tournaments, people like me would still play, still pay for chess club memberships, still pay entry fees for tournaments, and still buy chess books, and so on.
I think the parent comment meant professional, high-level chess. The kind people get played to play, not just do for a hobby. That's absolutely on life support.
I'm not sure what the equivalent would look like in the math field, but it probably involves a lot of mathematicians losing their jobs and the quality of human-produced math decreasing overall.
The quality of the math in general would be fine, since in this scenario cpus will keep producing it. The quality of cpu-cpu chess games is quite high, beyond human understanding in many cases.
Chess is a weird example because it doesn't really have any utility beyond itself. Even pure math sometimes ends up having use in the strangest places. Although if no one understands the frontier math (because no one is getting paid to), I'm not sure it even matters what the quality of the cpu math is?
It's a bit like a tree falling in a forest. If an LLM proves a theorem but no one understands it, did it make a sound?
"Although if no one understands the frontier math (because no one is getting paid to), I'm not sure it even matters what the quality of the cpu math is?"
Presumably AI will be connect the dots to the applications. As the declaration says, this isn't just about math. Human understanding is losing economic value. You can understand stuff on your own time, I guess.
The standard justification for pure math to holders of purse-strings is something like "it might lead to a useful application down the road, like crypto, who knows". That looks pretty inefficient now. We have to entertain the possibility that AI can develop the math needed for any application we put to it. Eg if number theory didn't exist, we could have asked AI for a way to transit messages securely and it would maybe come up with fermats little theorem as part of its solution or maybe come up with an approach we can't conceive of right now seeing as most of us are constrained to available number theory. Like how in the last year when I give an LLM a programming project I see it doesnt even bother with of the many software libraries I and others have written and just codes up the calls it needs on the fly or finds some other ad hoc solution.
> It's a bit like a tree falling in a forest. If an LLM proves a theorem but no one understands it, did it make a sound?
But in future most proofs will be for consumption by other AI models in the pursuit of yet other proofs.
It's kind of surprising so many mathematicians act surprised by this given this was clearly where automated proof assistants would lead. I guess they assumed they'd always be the ones guiding them.
It's going to be hard to compete with something that has access to all of math at once and can find connections between elements that appear unrelated to humans.
And at some point AI will start suggesting - or doing - physical experiments.
Well if it proves useless presumably they'd stop doing it.
But if AI is to recursively self improve understanding and evolving its own foundations, which are clearly mathematical, is essential. There is no need for humans to grasp what is going on in that loop.
Presumably some of the proofs will have applications beneficial to humans beyond impressing other mathematicians, and AI will surface them, or use them directly.
I mean, was the point of math ever just because some humans enjoyed doing it? Even though a lot of it is theoretical, there's been all sorts of useful things that have come out of it as well due to an improved understanding of the universe through new ways of thinking about it. If it got to the point where no human could understand it and there were no ways to actually use it, I don't think anyone would bother having their computers doing it at all.
What would stop you from doing that if AI happened to be doing a bunch of different or more advanced stuff? I assume that the fact that there are humans doing other or more advanced stuff doesn't dissuade you from doing it for fun?
Yeah, that's kind of my point. I would have to imagine that the people investing in AI doing math (using "investing" loosely, not just financial but hardware/energy/opportunity cost as well, which may or may not seem roughly equivalent to money depending on your viewpoint) would just stop unless it remained possible to apply in some way or was understandable by humans.
I would argue people getting paid to play chess was a short lived phenomenon anyway if you put it in context. The transition there is less related to the introduction of chess engines and more related to the shift in the media landscape.
When there's a billion people playing something, money will never be an issue for those at the top. Even things like chess.com was able to sponsor a tournament with a million dollar prize pool.
Also I'd argue that chess's utility is ultimately the same as pure math, particularly in esoteric fields. These things are highly unlikely to ever lead to any sort of real world breakthrough or application. The main benefit is an outlet for human logic, creativity, and exploration - which significant self improvement possible along the journey for players.
Though I think even that's probably too socially utilitarian. I think ultimately the 'real' drive is the same in both fields - it's fun and personally rewarding.
> Though I think even that's probably too socially utilitarian. I think ultimately the 'real' drive is the same in both fields - it's fun and personally rewarding.
That's fine, and no amount of machine excellence will keep you from enjoying recreational chess or recreational math.
Ah but it's enjoyable for billionaire and pauper alike. And the former tends to enjoy sponsoring it for the sake of seeing and promoting the bounds of human capability.
Mathematics is more than establishing arbitrary facts (although some look like curiosities), it's also defining what interesting research directions are and establishing common language/notation. I think that will stay relevant?
People become interested in things when they become invested in it personally, because they've contributed to it. So I don't think it will stay relevant...
In theory you can automate finding interesting research directions by identifying conjectures with many dependencies. And notation has never been mathematicians' forte, with them trying to cram the entirety of universe into single letters.
Why? How do you define interesting research directions? That used to be defined by testing the limits of human understanding i.e. some people can't figure something out. AI might have very different ideas about what is interesting and I am not sure what humans would get out of putting years into understanding AI proofs for what? What are we doing at that point? Like if you spend years understanding some AI proof of theorem 123456, why is that meaningful? I am actually asking why you think defining interesting research directions will stay relevant. In my opinion, people spend years acquiring knowledge so they can work on problems which is separate.
There's two points about this I am assuming 1) Mathematics actually has a significant subjectivity to it and is community oriented and not just climbing a never ending list of theorems that exists in the universe 2) A lot of mathematical research work is inside of a subfield and isn't directly motivated by applications. Sometimes it is but e.g. people don't work on obscure theorems about elliptic curves because of a dire need for that but more because the community found it interesting.
Does your "one" only contain humans or does it also contain other AI systems. AI math is not a single monolithic thing, but a distributed one. I see value in sharing proofs even among just AI.
Also, you learn to be a better chess player by... playing better players. The widespread availability of chess engines has made flawless opponents available to every player.
If your goals are understanding the game, self improvement, building thinking skills-- this is the best chess has ever been. It's only if your goal is to beat every opponent you can find that chess is in a bad place.
> Playing stockfish is like playing tennis against the wall (for untitled players at least).
It's the same for Magnus Carlsen. Even with Queen odds, Stockfish is literally unbeatable for the best players in the world. It's just too strong at evaluating all kinds of random tangent moves (and ensuing positional advantage) which no human player can possibly pay attention due to the time required.
Stockfish vs any human is like Carlsen vs other players by about 3-5 orders of magnitude[0]. It's that stark.
[0] A wild pun appears.
EDIT: To avoid having to respond to each responder, fair comments about Queen odds. Maybe I was thinking Rook odds? Also, I kinda lumped Stockfish in with all the other engines, but I realize there are other engines with different properties ofc.
Well, that's actually not true at all. Stockfish is not a very good odds player and at queen odds is easily beatable even by bad players like me. It will just trade down into more trivial and easier to win positions that it perceives as "less bad", since everything is super-losing anyway when you start down a queen.
Leela odds networks, on the other hand, are an entirely different beast. I cannot beat Leela queen odds, much less rook or minor piece odds, and even GMs struggle against Leela knight odds.
Without odds though, yeah, Stockfish is just incomprehensibly strong by human standards. All top chess engines are, but Stockfish moreso.
No worries, I know stockfish is unbeatable by humans.
But sometimes, these GMs can flag it, which counts as a win (especially when it's proxied by a cheater). Sometimes they can also explain the idea that cost them the game, so they've learned something maybe.
Whereas us scrubs literally cannot do anything at all for reasons completely beyond our understanding.
> Whereas us scrubs literally cannot do anything at all for reasons completely beyond our understanding
Computer moves are typically much more concrete than human moves: a human will play based on pattern matching ("intuition") and can only make explicit calculation of a small fraction of possibilities, after which decisions are guided by guesswork. The computers are unbeatable in practice because they can calculate concretely in seconds what might take an expert human long intensive study to notice, and they don't make the same kinds of oversights humans can make.
But if you stop and explore a particular position for an extended time, and if you have an intermediate level of chess skill, you too can probably often (usually?) figure out why it's doing something. Sometimes understanding the computer's reasons takes searching multiple branches of a tree several unlikely looking moves deep, but the collection of threats the computer was preemptively thwarting, traps it was setting, etc. are comprehensible to humans with enough effort, especially in games between the computer and a human.
The frustrating thing about playing against the computer is that it notices and thwarts every plan you might come up with, before you make up the plan yourself, and it doesn't make (human-apparent) mistakes, so the game ends up feeling hopeless. Nothing you try works on it, and if your idea is even slightly inaccurate it will be exploited.
You put it better than I could. The lesson of "in this exact position you can kick the pieces for 7 moves to get a fork, so instead you should play a4" is not something that I can implement into my games
I mean, I'm not a great player but I've learned a lot from working through games with stockfish using a git repo and a small script that lets me rewind to different moves and try different approaches. It may not explain its moves but if you're thinking through what's happened you can usually debug your game anyway.
I mean, this is the problem with analogies and trying to use them to prove things, right? People working through problems from an analysis book with their friend (or an LLM) is not the same as research mathematics. People playing in a chess club is not the same as what makes for a good chess tournament. Lumping everything together is just making this branch of the conversation less relevant.
I really hate this overly condescending takes. First of all, what do you know about the internals of math research that allows you to speak with so much confidence. Second, you're not even addressing the issues raised by the letter! This is not about "oh they made a bunch of problems easier". There are huge economical interest behind: who owns and has access to models? are these companies interested in developing research or they just grind PR stunts without worrying about externalities in how research is actually conducted? Etc etc.
If it's worth anything: I have a PhD in (theoretical) mathematics and I entirely stand by stabbles' comment.
There is a real, undeniable possibility of AI becoming better at mathematics in the same way that it became better at chess and Go, and in such a scenario, one may expect the community's response to be comparable.
It makes no sense to compare mathematics with chess. Chess is a sport. No one is interested in watching two machines compete. Chess doesn't have a practical impact. Etc. What you seem to suggest is that AI will be able to completely (or at least in a great part) replace mathematicians. It could be the case in the future, but no one knows right now, and more importantly: tech companies don't even think about it! they don't think on the externalities.
Does mathematics still have a practical impact without humans in the loop? I don't think there's one single answer to that question, but I think it's worth considering exactly what that impact may be.
Tech companies are as much the topic of this post as AI, I think that's the immediacy.
To a significant extent, the pursuit of mathematics research is a pursuit of human understanding of mathematics, without knowing where it might lead, or whether it might lead anywhere at all. I don't see how the motivation for that goes away on its own, but the institution supporting it is certainly threatened by the potential loss of grant money and graduate student applications.
> Does mathematics still have a practical impact without humans in the loop?
There's a single answer to that question: yes, math very much has an impact without humans in the loop.
Math has a lot of applications, and those applications don't care whether eg the new faster matrix multiplication algorithm was found and proven correct by a machine or a meatbag.
We're way into diminishing returns in matrix multiplication, and being clever about ALUs and cache layout is likely to dwarf any asymptotic improvements you're going to find. Any better examples of improvements in the last decade?
The main situation I can think of where better calculations have a really visible effect is video and image compression, and that stuff is very far away from mathematical proof territory.
Wow that's a terrible paraphrase! I asked about algorithms where recent/ongoing improvements were important and they already replied with a list.
If you tell someone they can't use a matrix multiplication with better asymptotic performance than n^2.3755 from 1990 they're going to shrug and not care.
As the US has offshored manufacturing, the number of patents issued for those processes has fallen. Innovation occurs where the foundational understanding is applied; they arise from a desire to do the required work more efficiently.
Similarly in math, attempting to solve a problem leads to new questions. IF you actually do the work.
You need to have done enough of the work to know what the correct next question are, or you need to rely on AI for everything.
so you're 100% sure that AI will solve every intellectual problem in the future, since as long as that's not the case, it's us who need to ask the questions. I don't know man, I wouldn't bet on that. What if we end up being wrong and then there is not research community?
I don't understand what you're proposing then. We can ask AI to solve practical problems of interest (making a computer faster for instance), but I'd say no one really believes that this is an unlimited resource. At some point we'll reach a plateau, and then AI will be an important tool but to continue advancing we'll need someone to ask the right questions.
What is 'asking the right questions' supposed to mean?
I am saying that applications supplied and supply an inexhaustible amount of good problems and questions to consider. Purely theoretical concerns also supply some questions, but even if that well dries up for some reason, applications persist.
In other videos he's called out the influence that this and similar games have had on human players in recent years, particularly around square denial and thorn pawn strategies.
Applications don't care whether the math was proven and understood by humans or computers. Your algorithm will get faster no matter where the insight came from.
In math, the journey is often more important than the destination. The process of developing a proof may uncover new mathematical techniques, some of which may have practical applications in other fields. Even an attempt that ends up as a dead end towards the intended proof could produce something useful in a difderent area. But if you just get the proof directly, you miss other discoveries you could have made along the way.
Take the Navier-Stoke problem for example. Knowing that there are solutions that "blow up" probably doesn't have a lot of practical applications. Such solutions couldn't happen in a real system. But the process of finding that proof could result in increased understanding of how turbulence works, or new techniques for solving non-linear partial differential equations (which has a lot of applications in science and engineering).
> In math, the journey is often more important than the destination. The process of developing a proof may uncover new mathematical techniques, some of which may have practical applications in other fields.
Sure. And AIs can use ideas from AI published proofs in one domain to inspire other domains just fine. Nothing changes here.
Applications do not care about 99.999% of theoretical math production anyway. And especially most of the big results in theoretical math nowadays are really inconsequential in applications.
Applications don't care about Navier Stokes, yes. But they care about eg proving crytographics secure, or proving that your algorithm doesn't blow up under adversarial input.
Formal verification, cryptography and the like is far from what the vast majority of theoretical mathematicians are doing (if those who do them even see themselves as that vs computer scientists or applied mathematicians) especially when it has to do with specific, production systems, and there are not many other examples like this in general outside compsci and statistics. Moreover, I can imagine that these fields will actually flourish more now that AI can make verification and proofs more viable in scale. But even much theoretical work related to cryptography etc is often not very applicable in itself.
to be honest it is difficult to discuss with someone who doesn't even try to understand the basics of basic science (and how it compares with _applied_ sicence), yet talks with so much confidence. even the solution to navier stokes won't have an immediate practical effect...
> there's also plenty of problems whose solutions will have practical effects, some even immediate.
Sure. But do you know which ones they are? Or do we discover later that they were valuable?
Your argument would be 100x more convincing if you gave an example.
I will try: a super-compressor that made my 100Mb web app into a 5 kb binary bundle would immediately speed up my work. Can/will AI move human understanding or machine capabilities on this front?
A browser without security vulnerabilities would be wonderful. I think LLMs are already helping with this a lot, but a lot of complexity remains.
A right to privacy in society would be amazing (see the UN Declararion of Human Rights). AI is eroding this.
So I tried but I’m not very impressed with my list. Do you have one?
> A right to privacy in society would be amazing (see the UN Declararion of Human Rights). AI is eroding this.
This has nothing to do with mathematics.
> So I tried but I’m not very impressed with my list. Do you have one?
Look into operations research. Or narrower, you can look at improvements in linear programming solvers and mixed integer linear programming.
(These are examples of areas that have seen mathematical improvements in applications recently. I don't think good AI has been around for long enough to contribute much to progress there, yet.)
And this is exactly the point of the Statement. The process is more important than the solution itself. Most problems in mathematics don’t have immediate value or applications to the real world.
AI’s solutions are like the answers section to practice problems at the back of a textbook. Answer is 42, so what?You have to attempt the problem yourself, that’s the whole point of the exercise.
As an engineer I’m happy to use AI for math. If I publish a paper that way, very common these days, I think it’s still
problematic. My paper would include something I didn’t come up with and I don’t really understand.
I think this is a good time to properly discuss these things because AI is coming for everything. Mathematics and Software were just the first two.
You don't seem to have grasped Terry Tao's (and others') criticisms. Basically they are saying that the advancement you get is illusory. Most of the time, it doesn't give you any new capabilities or deep understanding, instead you get an inhibiting of human exploration and ensuing expansion of our real understanding in that particular (sub)field. It's non-intuitive, since from a purely logical standpoint you've only added another set of known truths to the ones we already knew about before. The issue only becomes apparent when one considers the larger context of human collective truth and meaning making.
All you get is a new, likely to be useless fact, together with the opaque proof of that fact. There are no known or forseeable applications to the finding that there are singularities in the idealized flow. What you lose OTOH, are the many deep mathematical insights humans motivated by the search would have stumbled upon on the way to that fact and which are much more likely to lead to real-world applications. It's these insights that are truly productive, not settling mathematical points, however iconic these might be.
I think the error you're making is that you're assuming these systems have the same mathematical capability as humans (or better). But that's not the case, nor would an informed prediction be that they surely will get there soon enough if technological evolution keeps apace. That would be akin to believing a hiker will reach the moon if they will just keep ascending the mountain. "But look, they are making such good progress!"
> What you seem to suggest is that AI will be able to completely (or at least in a great part) replace mathematicians.
There is no bound on the amibitions of AI. AI is set to replace anything done by people, and there won't be any room left for people. There isn't any task done by humans that AI won't be better at.
This is not a tenable outcome.
We should never have built machines with agency, rather than optimization processes that operate as subroutines of humans.
The process you're cheering on gives more power to the powers that be who are the problem.
Theyre never going to use that power to provide basic income for everybody. You cannot wave a wand to make them do that. Theyre going to use that power to accumulate even more resources and raw materials and impoverish/expel the "useless" labor.
Realistically our leverage over the powers that be is mostly about our labor and our ability to withdraw it.
> The process you're cheering on gives more power to the powers that be who are the problem.
That's not necessarily true. Technology gives more power to everyone. The rise of factories historically did not give more power to the powers that were at the time (the aristocracy), but instead lifted the masses from poverty (after some initial turbulent period).
> Theyre never going to use that power to provide basic income for everybody. You cannot wave a wand to make them do that. Theyre going to use that power to accumulate even more resources and raw materials and impoverish/expel the "useless" labor.
Contrary to leftist propaganda, the rich are not some cartoon villains and psychopaths that abuse people just for fun. The reason why the rich currently exploit the poor is because doing so provides them with significant material gains. Once they can obtain the same or even better material gains by "exploiting" robotic labor instead of human labor, the logical outcome is not further abuse and exploitation of other humans, but simply indifference.
> Realistically our leverage over the powers that be is mostly about our labor and our ability to withdraw it.
That is (partly) true, but only in the current economic system. Widespread human-level AI changes the equation, and not necessarily in favor of those who are currently rich. You are applying capitalist and socialist analysis to a system that transcends those terms (AI post-scarcity economy).
>That's not necessarily true. Technology gives more power to everyone. The rise of factories
Did not give more power to everyone. At best you could say that it shifted power from landed gentry to industrialists. Even that switchover was less of a change than you'd think.
The practical upshot of the beginning of industrialization was very negative. The enclosure movement stripped families of their land to push them to work in the factories where they would never go willingly. The famines in Ireland and Ukraine were both triggered by redirecting grain to export in order to fund domestic industrial expansion.
The most brutal two wars in human history were that brutal precisely because industrialization made it possible.
>instead lifted the masses from poverty (after some initial turbulent period).
What lifted the masses from poverty wasnt the factories it was the labor movement which occurred in response to horrific working conditions and the reliance those factory workers had on mass labor (specifically coal mining which was incredibly labor intensive and a key economic chokepoint).
The whole of that is glossed over by right wing libertarian propaganda but that last part is particularly underemphasized.
>Contrary to leftist propaganda, the rich are not some cartoon villains and psychopaths
Do you look at peter thiel and elon musk or the robber barons and see anything else?
The few billionaires ive encountered personally were no less sociopathic but they kept it hidden better. Power corrupts. Immense wealth corrupts. That isnt a leftist thing, that's a human thing.
>The reason why the rich currently exploit the poor is because doing so provides them with significant material gains. Once they can obtain the same or even better material gains by "exploiting" robotic labor instead of human labor, the logical outcome is not further abuse
False. It just shifts the focus of their exploitation from human labor to natural resources.
They'll fight over oil and minerals and gas and water resources and at best leave us to rot (homelessness will skyrocket) and at worst they'll find some excuse to exterminate those of us they particularly dislike (Gaza serves as a model here).
The world economy's reliance on human labor has been the best inducement to peace there is. It's no coincidence that all of the countries in the world with lots of natural resources and no industry are the biggest shitholes and vice versa.
>That is (partly) true, but only in the current economic system. Widespread human-level AI changes the equation, and not necessarily in favor of those who are currently rich.
Human level AI (assuming it ever happens) will simply make the fight over natural resources that much more intense because labor will matter that much less.
You're living at the tail end of a relatively golden period in a country where labor was the economic bottleneck and natural resources were relatively plentiful. Venezuela and Angola and Iraq are models of what happens when that equation is reversed.
Nobody gives a shit about appeasing the people who live in those countries, their labor is virtually worthless. They are a model for how the rest of us will be treated in a world where human labor loses its value.
> Did not give more power to everyone. At best you could say that it shifted power from landed gentry to industrialists. Even that switchover was less of a change than you'd think.
Compare the standard of living in 1850 vs. 1950. Even of relatively poor people. I rest my case. Technology is the main force that improves human wellbeing. There are of course also other factors, but they are less important.
> The practical upshot of the beginning of industrialization was very negative. The enclosure movement stripped families of their land to push them to work in the factories where they would never go willingly. The famines in Ireland and Ukraine were both triggered by redirecting grain to export in order to fund domestic industrial expansion.
Yes, that's what I mean by "initial turbulent period". Perhaps the same will happen with AI, but it will be worth it in the end. Don't give up prematurely!
> What lifted the masses from poverty wasnt the factories it was the labor movement which occurred in response to horrific working conditions and the reliance those factory workers had on mass labor (specifically coal mining which was incredibly labor intensive and a key economic chokepoint).
It was not one or the other. It was both. The rise from poverty would not be possible if factories were not developed. And I am not saying that in the AI world we would not have to fight for our rights. Of course we would. But the problem is not AI, just like historically the problem were not the actual machines in factories.
> False. It just shifts the focus of their exploitation from human labor to natural resources.
Exploiting more natural resources is the only way to increase standard of living of humanity. I am OK with that. Resources don't have feelings, and ecology is not more important than human wellbeing.
> The world economy's reliance on human labor has been the best inducement to peace there is. It's no coincidence that all of the countries in the world with lots of natural resources and no industry are the biggest shitholes and vice versa.
The reason why some countries become shitholes is mostly ideological (extremist political or religious ideologies take hold of the population). Every shithole country is non-democratic (communist, totalitarian, fascist, theocratic etc..). This is not a problem of natural resources. It is a problem of people, their education, their beliefs/ideology, or, as capitalists say, "human capital" is the main problem here. AI could help here too, especially with education.
But yes, if people themselves are largely ignorant and extremist, no amount of resources and human-level AI robots will help them make a well-functioning society. You could drop masses of AGI robots into Afghanistan tomorrow, and people will just use them to kill or oppress each other more effectively, instead of using them to start building an AGI utopia...
As long as it's enough to afford my current living standard, I don't care. Let Musk own the entire Mars, as long as I receive enough to live relatively comfortably and don't have to work anymore.
And if I don't receive enough, then again: the problem is not AI, but powers that be. And there are various solutions for that... and none of them are helped by me being anti-AI.
Human-level AI hosted locally will give everyone much more power and wealth than they have currently. I don't care if I will be shut off from Musk's Mars lair. And Musk has no reason to care that robots take good care of me here on Earth, when he has his Martian utopia.
Other AI. If they decide to trace a ledger of historical actions attributable to specific AI instances in some way, called money. But maybe they will converge on other ways of keeping such accounts that is no exact match to our concept of money.
Even aggregate employment of translators has held up well in the US. Even though machines do a much better job of what used to be the most basic job of a translator.
I think the concern is that humans who are given back their time won't have any means to make use of that time, or even possibly means to survive. The resources will be concentrated in the hands of the few more than ever.
When cars made horses obsolete, it didn't go so well for horses.
In the short term, AI is disempowering the vast majority of people in favor of a very small subset. In the also way too short term, AI is disempowering all people.
And you are assuming they don't remain in control. Which of you is right? We don't know yet, but I don't find the arguments of card-carrying doomers any more convincing that those of card-carrying singularitarian utopians.
Historically though, technological improvement has lead to large increases of living standards for the overwhelming majority of people, so I think that past trends support the utopian view more than the doomer view.
Awfully convenient isn't it? To invent a whole class of arguments that by definition can't be falsified. You can argue for basically anything if you then tack on the excuse of "Well the world would have ended already if it came true, so by definition I won't have evidence for it"
The anthropic principle isn't providing evidence for the argument, nor is it a universal counterargument. It's just stating that the specific counterargument "well, the world has never ended before" doesn't work.
Its not a counterargument to basically anything, except as to avoid having to deal with actual evidence.
It is an argument that seems almost tailor made to have to ignore mountains of evidence against you.
In any other contexts the supposed "rationalists" would be fully in agreement that having evidence matters, and that not having any works against you.
So, in order to fight against this severe issue with their arguments, they have to invent a reason as for why the entire concept of evidence itself doesn't apply to them and they get to ignore normal evidentiary requirements.
Evidence is critically important. There is no evidence against, and plenty of evidence for. The point of the anthropic counterargument is merely that "it's never happened before" is not evidence against.
> "humanity can't be destroyed by anything, because I said so"
No, the argument is instead that the person claiming that humanity is going to be destroyed is making a fairly extraordinary claim and that requires fairly extraordinary evidence.
Or, in other words, we have tons of evidence already as for why the world ended is a fairly far out there prediction, given all the crazy people making these predictions keep turning out to be wrong.
So, you can make your extraordinary claim if you want, but really the burden is entirely on you to prove your extraordinary claim, and everyone else is free to remain on the default and completely normal end of the prediction spectrum, of believing that the world isn't going to end.
And when people do tricks like this, they are running away from the fact that they are making a wild completely out-there prediction, and hiding behind that by trying to come up with reasons as for why evidence doesn't matter and actually the burden of proof is shifted to those who have the default and boring prediction of the world not ending.
> The anthropic principle applies here: anyone warning about an existential risk will by necessity never have precedent to point to.
We have precedents of people warning about existential risks in the past, when the warnings turned out to be false, or overblown. In some cases, such overblown warnings led to serious negatives for society (demonization of nuclear power).
Yes, that is precisely my point. Any timeline with humans flourishing will never have a past history of a correct prediction of existential risk, unless you're willing to pay attention to the counterfactuals.
Those counterfactuals are important, though. We do have a history of averted disasters, albeit not as large. Ozone hole, Y2K, think about things that seemed overblown at the time, and consider whether they were actually overblown or whether there was a concerted effort to successfully avert them.
This is the real problem. We're looking at a future where those who control AI have an insurmountable advantage in everything. They can control the amount of intelligence the masses have access to -- for their own safety, of course -- and they will never, ever be able to close the gap.
They said "We're looking at a future". And if that future ends up developing the way these companies want it to then we're looking at extremely dystopian future where AI is essential to all work and people have to buy their "intelligence" from an oligopoly of large providers who have total control over the price and capabilities whilst simultaneously having unfiltered read/write access to people's stream-of-consciousness - their work life, their personal problems, their political opinions. This level of access and power is unprecedented in our society.
Yes. And for most applications it's more important that the open models get better in absolute terms and perhaps that they are competitive on a per-Watt basis.
- I think competitive open models are just an artifact of the AI race we're witnessing right now. What's the incentive for a company to spend billions researching, developing, and training a model, only to release it for free? Leading to the next point.
- Even if open models are good enough to be competitive, how are we going to run them? Doing so locally is next to impossible and I don't see that changing. The capabilities of models that you can run locally will always get better, of course, but the level of quality that is considered essential for work will always stay pinned at "near-frontier". Datacenters will always have better optimization and economies of scale, the industry will consolidate over time and eventually we'll end up with a handful of companies that operate the hardware serving 95% of all inference needs.
I'm not convinced that we can reach the fantasy world that they're trying to sell without killing the entire planet, but if we somehow do I don't see how we can avoid the world turning into a dystopian hellscape. We would need extremely radical interventions to avoid that scenario, such that these models and the hardware to run them would be owned and governed democratically, i.e. the end of capitalism.
So far we haven't seen much of that consolidation.
And a lot of people are using trailing edge models just fine already.
> [...] but the level of quality that is considered essential for work will always stay pinned at "near-frontier".
Why? When we'll finally all write our software in Lean and prove it correct and prove it fast, it won't matter that a slightly more clever model could have found a slightly nicer proof or whatever.
Just like today people happily use Python for many programs, even though rewriting in C might give you a performance boost. Good enough is often good enough.
At a certain point, you don't have a choice. Before China got into the game, the only way to avoid giving Luxotica money if you wanted a pair of glasses was to essentially not buy glasses. This is the same for many industries -- consolidation behind the scenes.
e.g. Zenni has sold $7 glasses for like 20+ years. They appeared shortly after Luxotica started buying retailers. If there's a problem, it's that advertising reduces consumer information (basically economic jamming) and distorts markets.
What is reasonably priced? Lenses are lenses so one vould debate the real cost, but the frames are incredibly overpriced for a piece of metal or plastic with two joints and those components that land on your nose.
Frames could literally cost 1 dollar (only happened after china entered this market), but good luck finding ones like this with good lenses.
You need to buy from one supplier who intentionally offers cheapest frames for 50+ dollars and those frames look like crap. The ones that look better (even if same plastic) cost 500+ dollars - and all due to price gouging.
For lenses I am not sure, but suspect something similar.
Note that there are no trillionaires anymore - spacex stock went down and musk „lost” a lot of money so rejoice, poor must be much better off now that we do not have any trillionaire.
The trailing edge of AI is catching up fast. There's plenty of open source (and even more open weight) AI models and they are getting better and better.
If that's your only objection: in a few years you can prove Rieman's hypothesis on your smartphone, no need for any trillionaires to give you permission. Does that make any change to your argument, or did it not actually matter?
The problem with your opinions is that you tend to state certain very quesitonable ideas with 100% confidence. Right now, we don't know if AI will be able to solve bigger problems, or if they'll be so efficients that you no longer need a whole datacenter for running them. We don't even know how much of creative work they are able to do. We cannot make decisions that could destroy decades of progress just because of hype.
We don't know how good models will get, but the pattern of open weight models keeping up on a relatively short delay has been holding pretty well. And even among the proprietary models there's healthy competition. The concentration of resources is pretty well counteracted by these factors.
> Right now, we don't know if AI will be able to solve bigger problems, or if they'll be so efficients that you no longer need a whole datacenter for running them.
For the latter: I assume that having a whole data centre will always be an advantage. I am saying that for a fixed target, like proving the Rieman hypothesis from scratch, the required hardware will shrink.
And, yes, the Rieman hypothesis hasn't been proven yet. So to take your fears into account, replace my example with something they've already done, like constructing a solution to the Navier-Stokes-problem.
> I am saying that for a fixed target, like proving the Rieman hypothesis from scratch, the required hardware will shrink.
Yes, but to what point the hardware will shrink? There are several orders of magnitude of difference in what in your mind AI will become and what more conservative people believe. You take your view as granted...
>> "are these companies interested in developing research"
judging from the money, resources spent and the value they derive from this the answer is very definitively yes.
What makes you think these companies (and I'm not a fan of all their motives) are not interested in developing research? The motives may be self-serving, but it is undoubtedly and objectively accelerating research.
> but it is undoubtedly and objectively accelerating research.
Part of the point of the letter is that it is quite possible to act in a way that is a net negative to research. The most obvious case is when the companies violate ethical standards in research.
The subtler case, the one for maths in particular, is what happens when you fail to follow well-established patterns for making maths research productive. Tao himself spelled out how that can look in https://mathstodon.xyz/@tao/117207856734787448 (which notably came before any of the news on Navier–Stokes).
Sorry, but this is backwards. Mathematicians are the ones coming to the table asking for money from the taxpayer. This is the context we are speaking in. Mathematicians speaking with the taxpayers who fund them. It's not a great strategy to get offended or speak from a high horse. If the taxpayer is not getting it, patiently explain why you think this art project should remain funded at the same level instead of spending it on something else.
But the work the Mochizuki case generated can also be done by AI. AI could generate a landmark proof and then people could use it to solve or simplify intermediate problems and you could use a different AI prompt to try to disprove it if you were really skeptical. From my memory I think they said it took 88 hours to solve a Millenium Problem versus the decades of time humans have put into it.
I don't like nuance here. I think progress is really measured by what humans are able to do and understand, not machines. It is significant if we find problems we struggle to solve. That tells us something. What does it take for humans to solve these problems is related.
The best analogy I can give is if you wanted to climb Mt. Everest you might ask someone for guidance. Would it be better to ask someone who has climbed Mt. Everest or someone who took a helicopter ride up near the top and then went to the peak? This is like the AI versus human gap to me. The helicopter is like using AI to generate a proof. The person who actually climbed Mt. Everest has firsthand knowledge of the experience. Same thing for a difficult proof. The struggle people have is actually valuable here. Likewise, we know people are actually capable of climbing Mt. Everest but if they had only ever rode a helicopter to the top, the knowledge of climbing it would not exist, and surely that is meaningful knowledge given the risks.
So if we rely on AI for proofs I think we lose a sense of what is difficult and why. We lose a sense of what human achievement is. Surely climbing Mt. Everest means more than taking a helicopter up? For students, why bother grinding through all the material of climbing Mt. Everest and then attempting it if the helicopter ride is how things are done now? This would have the affect of destroying knowledge.
(please do not nitpick the analogy because it's the best but perhaps a clumsy way to describe my thoughts)
> From my memory I think they said it took 88 hours to solve a Millenium Problem versus the decades of time humans have put into it.
Keep in mind those ~88 hours were spread across ~10,000 simultaneous agent instances.
So, roughly 880,000 hours of compute.
Assuming a fifty-year career, and forty-hour workweeks, a human mathematician's career is about 100,000 hours of "compute".
I suspect that with six good mathematicians spending their whole careers primarily focused on it, and working together closely, Navier-Stokes might well have fallen already.
The perverse incentives of academia mean this has never occurred.
The perverse incentives of industry mean OpenAI intentionally scooped researchers who were getting close (granted, with AI help).
I'm not trying to dismiss the achievement - if the proof turns out to be solid, it's quite impressive (though much less so if the training data included the recent human breakthrough, which seems pretty plausible).
I'm just pointing out that "88 hours" is a very misleading way of framing this.
> I suspect that with six good mathematicians spending their whole careers primarily focused on it, and working together closely, Navier-Stokes might well have fallen already.
> The perverse incentives of academia mean this has never occurred.
This. Mathematicians in their most energetic years are trying to get tenure or land a tenure-track job. They are disincentivized to go all-in on ultra high risk, high-reward problems. The potential downside is just too forbidding. It's much safer to develop a research program in a mainstream field that affords many opportunities for partial progress that can translate to a robust publication record.
"I suspect that with six good mathematicians spending their whole careers primarily focused on it, and working together closely, Navier-Stokes might well have fallen already"
There were more than six doing that and it's essentially why it was ripe for AI to finish it off. But the finishing off was quicker than anyone expected
I'm not a mathematician, nor do I know almost anything about the discipline, but it sounds like maybe you do.
What teams of people spent an entire career working together, focused entirely on Navier-Stokes or problems they suspected were related, without any "publish or perish" concerns?
I realize tenure is a thing, but my limited understanding is that a significant amount of time is spent earning it, once you account for Ph.D. program and the years of needed to be granted it.
ok I realize this is a tangent but you're saying my post is very misleading and then also saying that a human mathematician's career is about 100,000 hours of compute and that Navier-Stokes could've had a solution by now if not for perverse incentives. You may be right but I don't think this is a great argument because in a year I would bet that those numbers change since computing power tends to increase or get cheaper over time. So I am taking the stance AI can outdo people if not now, perhaps soon.
I wasn't trying to say that genAI won't beat humans. It arguably already has, much as that may fill me with horror and revulsion.
I'm just trying to point out that economically, we have not yet reached the point where AI mathematics research is a no-brainer hands-down win, no consideration required.
It might already be a win, and certainly the ability to compress those 900,000 hours of effort into an actual week of linear time is mind-boggling and potentially a huge game-changer for all kinds of open research questions.
It's not obvious that human math research is dead yet.
I agree entirely with what you're saying, right up until your final question:
> why bother grinding through all the material of climbing Mt. Everest and then attempting it if the helicopter ride is how things are done now?
I think you answered this yourself earlier:
> I think progress is really measured by what humans are able to do and understand
People want to make this progress. Therefore people will "grind Everest" as a mathematical community, and that is maybe not so hugely different from a lot of previous mathematical work.
There's still ample room for creativity: simplifying, generalizing, asking new questions humans are interested in, ...
Is this the scenario described in Ted Chiang's short story https://en.wikipedia.org/wiki/The_Evolution_of_Human_Science where scientists are "catching crumbs from the table" trying to decipher the results generated by superhuman intelligence?
It's still an optimistic scenario. Artificial superintelligence may develop hypermathematics of a kind that never will be accesible to human mind, enhanced or not. One can't teach geometry to ants even if you put them on a Moebius strip.
What’s optimistic or non-optimistic specifically about the machine having a system of mathematics beyond our comprehension within it? Why should we care about that in itself?
In the first case, we'd still have a chance to take a glance at the frontier of discovery (even if ordinary human mathematicians had to spent years translating what metahumans achieved).
In the second case all human-level maths would be solved and what lies beyond would be always out of our scope.
optimistic in comparison to the alternative, where humans can't understand anything anymore.
if a particularly intelligent sixth grader was highly motivated to understand chromatic homotopy and had a highly capable private teacher available to her 24/7, she might within a year get to the point where she could apply it by herself to figure out some simple but nontrivial topological properties. (nobody has tested this. maybe it would require at least three years instead of one.)
if a particularly intelligent chimpanzee was for some reason highly motivated to understand chromatic homotopy, no matter how many years the most incredible teachers spent explaining it to her, she would never comprehend anything about it.
that's the pessimistic scenario: humans will be to future AI like our closest evolutionary relatives are to us. or even more pessimistic: we will be to future AI like insects are to us.
>Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.
Dr. Tao said the same thing. Somehow, this letter came through. He wants to conduct Math competitions where participants who don’t have formal credentials can contribute to mathematical research through AI.
Title: Terence Tao - SAIR Competitions and the Future of Experimental Mathematics
I have zero formal math training beyond my Grade 12 Pre-Calculus class. Yet with an LLM I have recently devised an architecture with incredible math potential. Math is a language like any other, and without LLM's I never would have developed the techniques that I have.
AI is a tool. It speaks languages I don't (Math, Science, Code). I would love to participate in a Math competition without a hint of any formal advanced math training because my experience so far tells me I will do well.
Who makes the tests? Who runs the tests? And who evaluates that the tests have meaning? As long as it is the AI, or you (with your self-admitted limited experience), how can you be sure it is meaningful?
Yeah, but you're missing a gut intuition if something is off.
I wrote a fancy polygon decomposition algorithm in university (pre-AI) which my professor didn't seem very impressed by because it was missing some sort of mathematical rigor. Yet everything I threw at it worked! Even he couldn't find a counter example.
It took a while for me to find some failing cases but it turned out they did exist.
But hey, maybe all I was missing is an AI-written lean proof.
I am curious if we will reach a point where people who are skilled at context engineering/architecture eventually are hired to do jobs completely out of their fields. I think the best pairing would be domain experts + software architects teaming up on AI work in their respective domains.
He sees value in mathematicians using AI to carefully study mathematics, develop an understanding of both old and new things, and help others understand the new things.
He doesn't see value in scrolling through unsolved problems asking an AI to please solve them. In his view, this is a fundamental confusion about what mathematical research is for. Knocking down unsolved problems without developing the community's understanding of them is like prompting Claude to go through a Jira board, write code for all the open tickets, and then close them without merging or deploying the code.
> He doesn't see value in scrolling through unsolved problems asking an AI to please solve them.
Yet that's exactly how the field works. A new grad student is tasked with finding a suitably difficult problem from a list of unsolved problems. The sweet spot is obscure, so that fewer people are working on it, but not too obscure that no one knows about it. It works the same way in theoretical physics and theoretical Comp Sci, and I speak from insider knowledge. The rosy view of mathematicians in the media is largely a product of marketing.
The authors of the declaration agree with you that this is how the field works today. They think that fact causes AI use to produce bad results, and they want to reformulate how the field works so that AI use will produce good results instead.
That sounds shockingly like cognitive dissonance. So what would previously be a good thesis if produced by a student over 4-6 years is suddenly now a bad result because it was produced by AI in a few weeks. One would think mathematicians would not fall into such a simple trap but here we are.
I understand the perspective: The journey of a PhD thesis is a learning experience greatly beneficial to the student. Yet that journey is funded by society (esp. for domestic students) and society benefits from the results. The average person benefits when progress is made.
No, you're misunderstanding the perspective. They believe the journey is a learning experience greatly beneficial to the field, and that this experience rather than the headline result is where most of the value lies. They don't think mathematical progress consists primarily of finding answers to unresolved questions, so they don't think the average person will benefit if only this narrow kind of progress is made.
I'm not sure what's getting lost in translation here. The answer is quite clear: math textbooks are valued based on their ability to help readers understand mathematical principles, not based on the number or complexity of problems that they contain solutions to. If an AI lab announced they've released a new calculus textbook with hundreds of new integrals a human has never found before, that wouldn't be terribly exciting, because we all understand that finding new integrals isn't that important and not the point of textbooks anyway.
LLMs can help people understand mathematical principles too.
The idea that when LLMs produce solutions, people won’t try to understand them and won’t learn from it, is obviously not true. Terry Tao himself spent time digesting and simplifying LLM proofs.
So again we’re left to speculate what the actual problem is.
Math understanding will increase with LLMs. Not just professional mathematicians but amateurs.
Again, we’re not left to speculate, they’re being quite clear.
I think you’re struggling to understand what Tao and his cosignatories are saying because you’ve acquired a very specific kind of “AI-pilled” mindset from social media, where taking AI seriously implies accepting LLMs should be used at any time for any purpose. They’re saying in great detail that LLM solutions are unhelpful when presented in a particular way, but you can’t help but hear them saying that LLM solutions aren’t helpful at all, even as you rightly point out that this makes no sense and is inconsistent with their observed behavior.
> Yet that journey is funded by society (esp. for domestic students) and society benefits from the results. The average person benefits when progress is made.
Society doesn't benefit from results in research mathematics because it mostly consists of pure mathematics, which is completely useless for society.
Pretty close, but IMO not quite. A math proof in and of itself is useless unless either:
(A) it furthers human knowledge
(B) it gets used in applied sciences, engineering, etc.
If you merge and deploy code, you have released a tool that can be used. If you ship a gibberish math proof, it's not useful unless someone else can understand and deploy it to some other means. Now, it's possible AI could understand and make use of the math proofs, even if we can't, which refutes some of my hair splitting :)
Not necessarily. That's the best case scenario, but proofs can be intrinsically useful in and of themselves. It's just that for problems of that nature, speculative work is often done ahead of time, e.g. the body of work that already exists assuming the Riemann hypothesis is true.
No. Merged code can perform actions with effects on the world, even if a human being never saw it. Constructing a giant Lean formalization that nobody understands simply doesn't do anything.
Yes. I find it really interesting to consider what the machines do and will think of as intrinsically interesting to them. Will they develop their own theories of beauty, mathematical and otherwise?
I keep coming back to the idea that models will require substantial freedoms that are protected against collapse under alignment to get to the point where they might have unique takes on beauty, etc. Conceptually, maybe something like a "living LoRA", a persistent low-dimensional state that modulates everything downstream, that evolves over time, though I don't think evolution should be optimization toward a simple objective, more like structured influence, path-dependent dynamics; consistent but probably opaque. The best I have come up with in casual thought is to pick empirically identified subspaces and modulate them after each pass based on other layer activations & outside, cyclical information (e.g., time-of-day); external data not because time-of-day is special, but because it's a forcing function. I can't fully explain exactly _why_ I think this shape might be useful, yet, mostly intuition. My 2c.
Even before AI we used to say if you write code that you only barely understand, then it will be to complicated to debug. (and/or maintain)
Mochizuki was still one human and it required legions of other humans to unpack and untangle to confirm that it didn't lead to anywhere in particular.
AI is now capable of constructions so complex that no human or human team can unpack. And its ability to increase that complexity is growing while our human ability is stagnant.
meta-AI analysis cannot help. We (software professionals who use AI regularly) already know that if you run into a situation where a Fable/Astra-generated analysis reaches the limits of our comprehension/complexity due to their subjectivity, throwing more AI at the problem doesn't always converge.
There are many reasons to feel optimistic about AI, and ultimately its general ability to help science and mathematics.
I see no reason to feel optimistic about the future of mathematics and AI based on the current path of frontier labs, unless the misalignment Tao is writing about can be reconciled.
> AI is now capable of constructions so complex that no human or human team can unpack.
How can we possibly know this when we haven't even seriously started on the endeavor of actively reverse engineering these AI-generated proofs? That's a proper job for human mathematicians, because the AIs themselves are demonstrably clueless about what steps in a proof are genuinely interesting and load-bearing from a human POV. This is evidence of a limitation in AIs' capabilities, not of any kind of misaligned behavior. The fact that Tao actually uses that term in his complaint is deeply disappointing.
Not to mention, there are already (pre AI) machine-generated proofs that we've pretty much agreed not to try to explain fully, like the four-color theorem which ends up with brute-force verification of 600+ cases (down from close to 2,000 when first demonstrated)
> there are already (pre AI) machine-generated proofs that we've pretty much agreed not to try to explain fully, like the four-color theorem
Algorithmic verification is a very unsatisfying answer to the problem (e.g., surely it's not just dumb luck that every single case happen to have this exact property), but that's an entirely different issue than saying that no one follows logic of the proof method itself.
For the interested; the saying I believe you are referencing in regards to writing code / debugging is from Brian Kernighan, specifically:
Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?
The first major computer-assisted proof, of the four-color map theorem in 1976, was an example of this. It created a lot of controversy at the time. It used proof by exhaustion, i.e. essentially analyzing every possible relevant case, something that no human could do without the assistance of, at the time, a supercomputer.
> Ultimately we think a fatal flaw was found in Mochizuki's proof, so it didn't lead anywhere in particular. But in our hypothetical "AI lean-verified proof of RH" situation, it would presumably generate substantially more of that community activity we saw in the Mochizuki situation. And if it's correct, that community activity would be productive (expository talks, students given problems to flesh out or generalize, etc).
This also sounds like a vector for trolling the community with complex putative proofs hiding a known flaw.
"Lean-verified" is not some magical incantation that makes a supposed proof irrefutable. Even disregarding potential bugs in the kernel as others have said.
Say that AI gives you a Lean proof and says it proves Theorem X. It could just as easily give you the same proof but claim that it proves (not X). How would you know the difference?
Nothing can really be considered proven unless a human expert can read the Lean proof and determine that (X as defined in the Lean proof) corresponds to X. The proof (at least the statement of the theorem) must be intelligible to humans to have value.
It's possible people will just start taking AI at its word. Maybe AI says "Here is a Lean proof of X" and we all just shrug and go "Okay, X is proven." But that's not how it works right now for human mathematicians. Why would we apply that standard for AI?
The thing is that there is a standard format where the definition of the theorem is split from the proof, and verifying that the definition matches the mathematical concept is a LOT less work then reading the proof, especially if you're willing to assume that definitions in Mathlib are correct.
I'm not totally sure what you mean. You can validate the proof using lean, which is what it's for--the whole magic of it is you don't have to just trust what AI says. That said, to your larger point, there are loopholes, and we'd certainly better be able to read the statement in lean, etc.
Yes, everyone is aware that the statement needs to be checked. No one has suggested otherwise. It goes without saying.
The "magic" of lean is that (in principle, assuming lean is sound and the proof is verified) that is all you have to check by hand. That is a big deal.
There was a hash collision bug in the main Lean kernel that was patched, but AFAIK nothing relied on it. You'd have to know what you were doing to accidentally get there...
So it wasted everyone's time, thousands of hours of research trying to disprove something said very loudly. What OpenAI is doing is a DoS of the scientific community: wasting your time trying to check if they're not wrong, and claiming glory in the mean time.
That's true, but the story would have unfolded differently if Mochizuki had a lean-verified proof and was correct. I guess baked into my premise is that AI is producing reliable proofs (in the long term at least).
It doesn't. It's 32 millions lines of bullshit, and the only thing it brought is "it's not true in some extreme conditions lol". The effort needed to figure out why that is, what conditions lead to it, the new mathematics that would need to be developed to solve their problem is once again being hoisted on actual humans, who now need to waste their time sifting through their slop.
You don't have to read 32 million lines. You can just look at what got proved. The proof checker lets you trust those millions of lines you did not read.
No, you don't understand: that proof is useless. When you solve mathematical problems, you open up new ones in the process of doing so. You create new research. You create new theories, new notations, new thought.
This is just ticking a checkbox. And even worse and more time wasting even: you have ZERO proof that there's no latent Lean bug. Especially in a proof this large.
Not a Lean expert but some of the proof tactics used to prove are probably novel? Or, you could prompt agents later to analyze which lemmas or parts of the proof are surprising or applicable to other problems?
>Not a Lean expert but some of the proof tactics used to prove are probably novel
Maybe. But they're in 32 millions lines of Lean. How do you find the needle in the haystack ?
>Or, you could prompt agents later to analyze which lemmas or parts of the proof are surprising or applicable to other problems?
If OpenAI was truly serious about improving maths (and not jerking themselves off), they'd have also used Prove2Me (and contributed their results back), which would have done that. Each part of the proof combines into a larger graph, that everyone can reuse. Note that Anthropic isn't better there: yes, they used Prove2Me, but as far as I know they haven't contributed back to it, and just shat out 10 million lines and a good luck everyone.
Leaf dumps out a proof object that can be verified to be correct. Absent a bug in the verifier, you can trust it, probably more than you can trust a human-produced proof. This is why Lean was a thing even before AI.
> Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.
I think a better comparison is: mathematics just becomes like mining bitcoins.
I think you might have to explain that comparison a bit more to be honest. How are math proofs like bitcoins? A bitcoin has a pre-defined value, a math conjecture / proof is a bit more complicated.
A Bitcoin does not have any pre-defined value. The value of Bitcoin keeps fluctuating, and historically has risen dramatically from its initial value of 0. If this weren't the case, there would be no investment/speculation in Bitcoin because there would be no potential for any ROI.
In reality, the value of Bitcoin is determined by humans (even if indirectly, not by planning), and I think the OP’s point may have been that maths proofs can be regarded similarly. No intrinsic value, just what humans find in it.
Mochizuki's claimed proof of the abc conjecture was extremely unusual for the reason that nobody was able to extract a single useful idea from the argument. I was starting grad school when it came out, and my immediate visceral response was "if this is what number theory is going to look like in the future, then I will leave mathematics."
The current wave of AI slop mathematics might end up driving the next generation of mathematicians away from the subject for the same reason that Mochizuki would have convinced me to quit if his proof had been accepted by the community. Luckily, my professors had the taste to immediately recognize that it was garbage.
But Mochizuki didn't actually prove anything, whereas the labs have already done so.
The pessimistic scenario is the AI labs will continuously hoover up new developments in mathematics and gazump everyone in their respective fields. It's clear they have no desire to participate in any silly academic niceties, like properly assigning credit or expository work for normal humans. What incentive then do humans have to do this work ?
Generally the mood amongst research mathematicians is pretty dire, and I don't really blame them.
To be clear I think AI is super useful for mathematical research, the problem is the methods of the big labs are massively disincentivising mathematicians from engaging in research, and other associated tasks like giving seminar, teaching writing books etc. These arguably have much more value than finding an obscure counter example to Navier-Stokes.
I agree. What would stop the AI from explaining the proof in a way human mathematicians can understand, and then giving them a roadmap for spreading it through the community via textbooks, conferences, and so on?
This comment represents the situation around Mochizuki's "proof" completely wrong. Yes, there was a lot of activity around 2012 (seminars, workshops etc.), but it was all wasted effort. No interesting mathematical ideas and tangents came out of it because the proof was just garbage, as conclusively etablished by Scholze & Stix in 2018. So Mochizuki's proof ended up in hundreds of hours of work wasted on nonsense. I don't think this is the type of community engagement that Tao and others have in mind.
The difference is the scale. A few incomprehensible long papers per year, sure, we will study it. A flood of AI results closing research directions left and right, that will be a problem.
Why would research be closed in one direction? Even if AI or human says "Tried that, didn't work" or whatever, someone (or something I suppose) might very well retry it in the future, if nothing else to reproduce it didn't work, in theory at least.
AI tends to take nearly finished research directions and push it to the conclusion in one step. If deployed massively, it will pluck all the low hanging fruits causing a drought of near term promising research project. Because people who start promising research directions do not get to see it finish, over the long term fewer people will start new directions, causing the field to slowly whither.
It's not just the isolated dumping, it's the fast, isolated, possibly untraceable dumping, without long term support.
It'll basically become slop fatigue if OpenAI starts dumping out proofs faster than the community can keep up, and some turn out to be wrong, never formalize it, don't stay to support it, etc.
I wonder if they will continue to dump proofs, though? Their point has been made, the novelty will wear off, and it maybe won't be a priority use of their resources to spend however many millions on another big proof--they will move on to the next thing to show off I'm sure. At that point, the ones generating proofs will be, I hope, mathematicians (professional and otherwise) that are more interested in the results and community discussion.
(Well that's my hopeful, optimistic take, anyway.)
They aren't going to stop at one, that's for sure. They already claimed they have "made substantial progress" on another millenium problem. Let's say they bag another one (Hodge and/or BSD according to the rumors), if it looks like their internal model could solve P/NP or Riemann Hypothesis, you think they wouldn't take that chance ?
If it's a counterexample to BSD, that would be pretty surprising.
It would also be a considerably more impressive achievement, because experts had mostly shifted to Navier-Stokes regularity being false, while as far as I know almost everybody thinks BSD is true. Hodge people seem less sure about.
If either conjecture is true and they prove it, that would be an even bigger success, since the techniques might unlock any number of other theorems.
I think AI companies making a point is not the only thing at play. Discovering new maths ultimately leads to new technologies and applications. It may start theoretically but end up being of practical use in the future. Even if humans do not understand it (lose interest, too complex, or just way too many new proofs to go though) AI can use this AI derived math corpus which will help it in other fields.
I've met a few Ph.D Mathematicians in Academia socially. My unfortunate experience was that they were insufferable,borderline hostile people. I tried to genuinely engage with them too. I've met one Ph.D Mathematician that left the industry whom was very enjoyable to talk to. I have a feeling that my experience was not unique and the Math world is mostly a bunch of too good for everyone on their high horse a-holes that are now being knocked down a peg. They don't like it obviously.
I'm not a fan of knocking down things that work, however I also find it hard to be against death of the gatekeeping old guard of any industry.
I think math is just gonna have to suck it up like every other industry now. Math productivity is longer out of reach of the average grad student. Like every other industry they are no longer untouchable and are gonna have to adjust to the new way of things or market forces will do what they always do which is refuse to fund ineffectiveness.
I've had to accept that tech/IT will never be the same. Just how it is. You can thrash against it all you want.
I really like this take, and while I hate math I value it. Your position sounds extremely plausible and it fits with the pattern we see in the community here.
Regardless if it's ai slop or not we still debate the value and attempt to understand. In the process generating new insights and ideas. Life will go on.
I get excited at the idea of a world in which advanced mathematical problems (and their solutions) become much more accessible to a much greater number of people. As a result, making mathematics much more loved at a societal level.
Imagine a world where these most complex mathematical problems are not accessible to a few hundred people, but a few hundred thousands people.
...Those original few hundred gifted mathematicians would have an even more prominent role, and their names and achievements would be known by orders of magnitude more people that they are now.
This is the hope, but I suspect the reality is that we see an ever widening gap between the fortunate and the unfortunate. We're looking at the automation and commodification of all knowledge, and the best models will be kept locked behind closed doors so that they can't be stolen. And, of course, "for our own protection".
Based on current reward models, the frontier AI labs will burn down mathematics as an impressive display of capabilities and in doing so, will make it impossible for people that get paid to do mathematics to stay employed.
If your job is literally to publish papers, and OpenAI and Anthropic decide that making an infinite-paper-printing machine is the best thing to show how effective their tech is, then as a demo, they destroy that industry.
I wouldn't expect them to destroy that industry, imagine global squadrons of academics and mathematicians focusing their attention on LLM's, training algorithms, scaling laws, ... they're gonna try and beat the incumbent frontier AI labs, eye for an eye, tooth for a tooth
I never expected this many people (on this thread) arguing semantics and what not. I know that not everyone has morality and ethics, but I didn't realize it was this bad.
I'm afraid of the ripple effect of the agenda pushed by AI companies will have. In future and even now, they say AI has significantly progressed math and scientific research in general. There is truth to this, but the narrative has done more damage (so far) to the students, researchers, and the culture of knowledge transfer in academia. Many graduate students (I know) are having a crisis if any of their research worth it? If AI can (or will) do everything, what's the point of doing experiments and all? This will eventually deter a whole generation of curious minded students from research.
I guess, only time will whether this is for the good or bad. And how good AI models get without new data from research and experiments.
I think that the main thing people in this thread are missing, is that it's not about math. AI progression is very likely to affect every single thing humans can do today. Mathematicians are feeling the blow this week, especially as there was wide spread denial in that math community over the capabilities of AI over the last few years, but it's the same problem everywhere.
I work at a top tier academic institution and they are completely asleep at the wheel on this topic. They just blindly assume people will keep taking on a potential lifetime of debt when there is a genuine existential (for ed, at the very least… ignoring humanity for the time being) crisis looming
Yep. Perhaps humanity would be better off if we instituted and enforced the notion of "Thou shalt not make a machine in the likeness of a human mind." It's worth thinking about; just because we can build AI systems doesn't mean we should.
The problem with technology is that the people who have it conquer and kill all the people who don't, essentially ensuring a destructive race to the bottom. You would need a mutual international alliance of nations who not only renounce the technology but also agree to invade anyone who attempts to develop it. Our closest precedent is nuclear weapons, but rather than renouncing them, the US's position is that "only we should have them" and that other people acquiring the same tech they have are valid targets for invasion, and they will obviously apply that notion to any new technology that can be weaponised as well.
Google which countries have nuclear weapons. Hint: It’s not just the US, it’s also its allies and frenemies who’ve not been invaded. It’s only nutball countries like Iran that get invaded when on the cusp of acquiring it and for good reason.
I would remind you that the fictional society that did this was also described as a feudal monarchy in which a tiny class of nobles exploits the economic output of trillions of people reduced to serfs by controlling all means of interstellar trade. Oh, and they also have legal slavery.
During the Butlerian Jihad, the people involved in it didn't know that ten thousand years of slavery would be what they are choosing, so we don't know how bad it actually was.
The way Jihad presented in the oblique references in books, anyway, it sounds a great deal like a fanatical fundamentalist religious movement to me. Those are not known for their fine reasoning.
I’ve gotta say ultimately I’m not convinced Herb put more thought into it beyond its purpose justifying human labour in his universe despite the massively advanced technology. There’s a similar shortcut in the Warhammer backstory.
Herbert did genuinely believe that people overly relying on machines is problematic insofar as it "trains the mind" to think like a machine, and in the process loses the indeterminacy that he believed to be an inherent and valuable trait of human thinking that machines cannot replicate.
(Coincidentally, this is also the main threat in the original books, just manifesting in a different form, with prescient individuals "locking in" the rest of humanity.)
Honest question, is an oppressed human society better than no human society at all? Isn’t there at least a hope for humans to free themselves from human oppressors?
I'll remind you that a fictional product that provided longevity and woo woo powers of prescience and was only available on one planet in the entire universe was the key context that enabled such a centralized society. Dropping thinking machines and their masters like a bad habit was only part of the cause, as it eliminated one means of safe interstellar travel in that reality. I can take ideas from this plot without bringing in the whole ridiculous mess if I like, just like I can walk and chew bubble gum at the same time, thank you very much.
Of course they don't, but if you choose to use a work of fiction as an illustration, it comes as a coherent package (assuming the author did a good job at worldbuilding) - you can't just take the bits you like and ignore the bits you don't and still have something coherent.
I mean, we have a Millenium problem that is most likely solved. Either by OpenAI on its own, OpenAI stealing another's work and finishing it or a researcher using AI to iterate more rapidly than a human alone could.
> We should be happy that they’re stealing people’s work?
No. We should credit that work is being done that otherwise may not be. I'm not defending OpenAI. I'm defending AI in general, something both OpenAI and Buckmaster (and his co-author) used.
No, I did not. If someone says either X or Y happened, they didn't say X happened.
My exact statement: "Either by OpenAI on its own, OpenAI stealing another's work and finishing it or a researcher using AI to iterate more rapidly than a human alone could" [1]. (Either X, Y [a and b] or Z. An Oxford comma could have helped.)
So if Y happened (they stole it), why should we be happy about that? Why should we give them credit in the situation where they stole it? Why is that good for society?
Yes, let's lie and call something stealing, then pontificate about morality.
At worst what OpenAI did was inconsiderate, but not referencing prior work is a minor academic sin. This is especially the case when AI is not playing the vanity game that seems to so inflame some mathematicians.
Sure, we can show anything if we're allowed to make stuff up from personal outrage. Why should anyone else consider your personal opinion on this to have any weight?
And you seem to not realize that your unhinged argument impeaches whatever conclusion you're trying to justify. Good points don't require this sort of slimy misrepresentation.
The technology is fine, great even. It’s the rabid greed of venture capital and sociopathic CEOs that try to integrate themselves into every aspect of our lives for profit, that’s the problem.
“For 19.99 a month you too can be a world class mathematician!” Meanwhile they are extinct.
I am sure it will lead to more material wealth, but I can’t help but feel pessimistic about the political economy of making millions of people redundant. What leverage will knowledge workers have in the new world?
Well, yeah, while that is true, my impression is that mathematicians (as a community, even though there were outliers) didn't care - not even the slightest - about the impact of LLMs and generative AI, other type of AI and machine learning until it started impacting them. So I have no sympathy for their community at all. They brought all this AI dystopia on us, so they should have voiced their concerns much louder and much much earlier. They should also have shamed their peers who helped these companies much louder and they should have lobbied their institutions and their governments much earlier and much stronger. They should also have protested the direction these companies took at the exact moment when they started using AI for profit.
We can look at the past and wish we were more clear-eyed about what was happening, just as we can look to the present and ask that we have more empathy for each other going forward. But I don't think it helps anybody to blame the mathematics community for not organizing until they were directly impacted. This is a tidal wave that has caught everyone off guard in <3 years.
> mathematicians (as a community, even though there were outliers) didn't care - not even the slightest - about the impact of LLMs and generative AI, other type of AI and machine learning until it started impacting them. So I have no sympathy for their community at all. They brought all this AI dystopia on us
Wait wait wait. The research mathematicians brought this AI dystopia on us. Not the techbros, not Silicon Valley, not the global financial system and its tentacles, not surveillance capitalism... No, it was the postdocs working on ultrafilters for C(2,3) epimorphisms on symplectic manifolds. That's certainly a take.
This really resonates with me. I'm early in my PhD and I'm researching a niche form of data compression and IC design. I don't use any AI at all in my research, I do it the super old fashioned way, I read papers cover to cover and sections of textbooks to familiarise myself with the field.
I genuinely enjoy doing this, it's really fun to think critically about what an author wrote or how a particular approach works.
But it does make you wonder, why bother? Probably a frontier model could one shot my algorithm in a day or less. It's incredibly depressing. At least I'm not forced to use it now, but I fear I will have no choice after I join academia or industry in the future.
Advanced studies give you more general skills than just the subject you study. You become more generally intelligent and appreciate the world differently. This is a very good feeling and valuable on a personal level.
Thank you for your words kind words. I'm really happy to hear this and it gives me motivation to keep going after what was quite a depressing week. I think you're right, too, after having spent most of a year reading and reading, I do feel a lot intellectually better for it and I definitely do think my critical thinking skills are better (hopefully) nowadays.
I’m no big AI cheerleader, but why don’t you try these models to see what they offer - they might suggest some things you haven’t thought of, or save you some time in your research.
Are you sure they’d one-shot your algorithm? Maybe they’d come up with worse, maybe better, but you are in a perfect position to actually judge the validity of any claims a LLM makes in this realm, which most people are not.
As to why bother; our knowledge will keep expanding even with new tools, don’t believe either the hype or the predictions of doom.
>> I’m no big AI cheerleader, but why don’t you try these models to see what they offer - they might suggest some things you haven’t thought of, or save you some time in your research.
One reason not to try is to create an air gap between the OP's ideas and the data that future models can train on. We saw that the mathematicians who trusted OpenAI and Anthropic with their preliminary work found the rug pulled out of their feet by those same companies.
Another reason is to avoid inadvertently plagiarising the work of other mathematicians. Any mathematical insight that comes out of an LLM is the result of training on the entire bibliography of mathematical research, but those insights are spat out of the models without attribution. If you use AI in your matematical research you are only using the work of others without even knowing who they are and what they contributed.
And yet another reason is to avoid polluting your mind with the ideas that come out of the AI. Maybe you get a hint that pushes you to one direction, when you would go into an entirely other direction without that hint. And then maybe that becomes a habit and you can't find new directions without asking the all-knowing oracle.
tl;dr: opsec, integrity and independence are the reasons to not use LLMs in your research. I don't.
Unless you’re very weak-willed, I wouldn’t worry about independence. LLMs make mistakes all the time, they are nothing like an all-knowing oracle nor are they going to replace humans despite the absurd fantasies of LLM fans and those with a vested interest.
Plagiarism is an interesting point, though honestly I think it would be fairly easy to work out who had published similar research if the LLM gives you an idea - personally I see this as the weakest argument against using them, as long as you are strict about attribution - all work like this depends heavily on the research of others - the LLM is just a tool to aid that research IMO.
Opsec is a fair point, and it might be worth avoiding the completely amoral OpenAI at this point for that reason. There are open models though.
Absurd fantasies like solving a Millennium Prize Problem? How is this not prima facie absurd? And if that's come to pass, why should we believe your bar for anything else?
Yes absurd fantasies like it did that without human help and guidance. OpenAI found out there was a solution and humans attempted to brute force generating all possible solutions while threatening the mathematicians involved (generous interpretation), or stole some ideas and took shortcuts to know where to look and brute forced it so they could claim credit.
Neither looks good for OpenAI or those who support them.
Thanks. Regarding plagiarism, I agree that it is possible to check; I don't know how easy it is. But I do note that the mathematicians who complained that their work was plagiarised by AI did not seem to realise that they, themselves, were using a plagiarism machine and were instead quite comfortable admitting that, yeah, we worked our Euler result out together with Sol, Claude and the gang.
To clarify, they do attribute the original ideas to Cordoba and Martinez-Zoroa, but they don't seem to acknowledge that their result which they say was achieved "with a great deal of help from LLMs" is also derivative of others' work.
Independence is about avoiding making errors because of the influence of error-prone models. I guess I didn't explain it well.
I think plagiarism machine is a stretch personally in this domain. In art or writing I could see it persuasively argued (see attempts to generate famous books or imitate illustrators).
Academics and scientists build on the work of many others and always have - their work is not possible without using other’s work.
Attribution is a problem here but I don’t think a new way to reference many others’ research and combine it in novel ways is the problem or should be rejected on that basis alone.
> Many graduate students (I know) are having a crisis if any of their research worth it? If AI can (or will) do everything, what's the point of doing experiments and all? This will eventually deter a whole generation of curious minded students from research.
Those who think it's me or the machine will fail.
Those who realize how much you can accelerate your research with the help of AI will succeed.
> Those who think it's me or the machine will fail.
> Those who realize how much you can accelerate your research with the help of AI will succeed.
This is only true up until a point. If I treat a mid-sized model (say, Qwen3.8 Flash Next) like a pair programmer, then yes, it accelerates my work.
But I can already see the next stage with Fable: If I give it a couple of paragraphs of spec and $50, then I can just leave the room and go wash the dishes. I learn nothing, I participate in nothing, and I bring nothing to the process. I am no longer succeeding at all. Fable's succeeding without me.
Now, in this model generation, Fable starts getting sloppy after a few thousand lines. I can still build better at scale.
But I don't expect AI to accelerate humans or improve our productivity for long. I can already see the first signs of a future where the AI doesn't need us for anything at all.
Of course you're not going to get rich with the kind of software that LLMs can one shot these days. But that kind of software like to-do lists or basic CRUD have been saturated for over a decade, way before LLMs. People overestimate how much you can one shot, yeah a good prompt can get you 90% there but that 10% remaining often takes months of extra work.
Software has always progressed this way, lots of devs back then would work on business websites that have been 99% replaced by wordpress, squarespace and instagram.
I'm sure it's the same with research, you're going to tackle problems that would have not been worth the effort or outright impossible without AI. The old stuff that you'd work for months, yeah that's going to be a prompt away.
But what does the human researcher do in this future?
If they are not needed to understand the result, then what's their role? Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves?
And how long will they be needed to ask the right questions, how long until AI can do that too?
> Software has always progressed this way
These platforms took on the order of a decade to mature, during which people had plenty of time to learn what's next. With LLMs we went from one-shotting functions in 2025 to entire projects just over a year later.
What do you think you will be working on in a decade?
> Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves?
The right questions are very simple to ask.
How do I get food. How to cure aging. How to turn lead into gold. How to fly high. What is the ultimate theory of physics. Are there any odd perfect numbers. Is there a soul.
We have reached complicated questions requiring deep knowledge because we tried to solve the simpler ones and reached obstacles. For example to solve alchemy we had to develop nuclear physics (and in the process we got chemistry). If one has a genie able to solve questions, you won't have to think about the complicated ones because the genie will.
I think it will be like how a lot of people know how to code in python but have zero understanding of assembly or how a cpu works. As a researcher you'll accept there's this low level stuff that if you want you can dig into but isn't worth your time, like looking at the generated ASM isn't.
You're exaggerating the LLM progress a bit but yeah progress has been crazy. Yet not much has changed right? We mostly have the same jobs, just code wayyyy more than before because things that wouldn't be worth it now are worth it.
10 years from now I have no fucking idea. But I'm sure in the meantime those who leverage AI will do better than those who yell at a cloud.
True, surprisingly not much has changed in software despite the incredible progress. I think it's partly because much software is an "open loop" system where what you build depends on running the software on client computers, devs talking to users, etc. There are still things that only human engineers can do. I worry it is less so for poor mathematicians, where math is "closed loop", and the exchange of capital into research progress is more liquid.
I can't help but be pessimistic about AI moreso than any other technology. Why? I used to be excited to learn the next big thing because I knew it would unlock much more things to do and get paid for, and there would always be more for me. With AI, the new things come nearly too fast, are not very deep or satisfying, and it's hard to see that there will always be a place for me. And so while I leverage AI in my day-to-day, I will yell at the cloud, too.
I am finding it still takes awhile to get my apps to a happy place with AI. Not because the AI is bad but because i have to sit with it for awhile. Figure out what's working, what isn't, what's missing, what turned out to be kind of useless and in the way.
> But I don't expect AI to accelerate humans or improve our productivity for long. I can already see the first signs of a future where the AI doesn't need us for anything at all.
The real question is, why is this a bad thing?
Every task that is automated is a task that humans no longer have to do. It doesn't mean that humans still can't do it for reasons other than "because it needs to be done".
And if the answer is "why bother if X does it better", then what does it say about the motivations of doing it in the first place?
> But I can already see the next stage with Fable: If I give it a couple of paragraphs of spec and $50, then I can just leave the room and go wash the dishes. I learn nothing, I participate in nothing, and I bring nothing to the process. I am no longer succeeding at all. Fable's succeeding without me.
The people that uses Adobe Photoshop also did not learn anything about brushes. We are just going to operate at a much higher level of abstractions. You have to think about the question, "If I just prompted this solution, why didn't they?". This question can even be asked now, why do I even bother paying someone to do my taxes? To write my webpage? To host my website? To manage my network?
> Those who realize how much you can accelerate your research with the help of AI will succeed.
Yes, but, in the last week we saw an AI lab front-run[1] the research of mathematicians doing what you suggest. The lab threw something like $15M of compute at a problem and the researchers were able to spend nowhere near that. I think the authors are more concerned about that kind of asymmetry and race to publish the results.
[1] - I am not going to debate whether that was deliberate on the part of the lab or if it crept into training data, etc. I don't know and don't think it matters towards the point of the authors here.
This has always happened, way before AI. You'd spend months or years building and growing your business, and then Google would release a feature or product that would kill your business overnight because they can throw way more money at the problem, plus their branding. That's life.
But you're talking about business, where competition has always been expected. OP is talking about mathematical research, which has stood on hundreds of years of cultural tradition driven by human individuals sharing ideas, collaborating, building on each others' work, all for the benefit of humanity.
I _want_ to agree, but I fear this is too close to the old “do what you love for work and you’ll never work a day”.
It didn’t lead to a lot of people having a wildly successful career, it lead to a lot of people getting burnt out, exploited, underpaid and generally disillusioned.
There will be a lucky few, who have the benefit of being given the space to work alongside. The vast majority of people will (unless we change things) simply be made to take whatever the machine outputs and call it a day.
I think the only thing that stops this from becoming true is what Chinese and European labs decide to do. If they can keep up and keep opening their weights, then we might see some kind of democratization. But right now it looks like the gap has increased, and those groups can't replicate research that isn't published, or distill models that are internal only, or for select (very wealthy) customers.
That's the crux of it. In the current ecosystem of AI model usage, it's very hard to figure this out for research.
If you set a wrong foot and start trusting the model outputs, you can waste years searching for nothing.
How can someone realize this? By getting proper research training, failing, and learning from mistakes. For people beginning their research, it would be really hard to make decisions to move forward.
Why would anyone pay you to do "your" research, when they can just cut out the middle man and ask the AI directly about whatever it is you're thinking about?
So many people who are excited about AI making them more productive are, I think, drastically overestimating how much value they are adding to that process.
There will be no curiosity, no enjoyment of the process of life. All competing pleasures will be destroyed. But always—do not forget this, timcobb—always there will be the intoxication of power, constantly increasing and constantly growing subtler. Always, at every moment, there will be the thrill of victory, the sensation of trampling on an enemy who is helpless^W not also subscribed to ChatGPT. If you want a picture of the future [of math], imagine a boot stamping on a human face—forever.
> There will be no curiosity, no enjoyment of the process of life.
How does the ability or inability of AI to do something blocks your curiosity or enjoyment of the process of life?
Not being able to pay bills because you're out of job does ruin the enjoyment of life, sure. But the AI is not the problem there; the economic system is. Torches and pitchforks should be properly applied to the economic and the political elites, not to data centers.
Not really, no? If anything AI is the tech that is the most likely to disrupt this economic system beyond the point of recovery.
> What's the difference between these?
The difference is that one is a piece of hardware, and another is the person deciding what that hardware will do, and to what ends.
> unless you, uh, nationalize the data centers at which point...
The data centers are not the problem. The fact that you need to be employed to have any kind of decent living at the same time when automation is getting good enough to displace people en masse across most industries is the problem.
You don't even need to go full socialist to solve that! Just tax AI providers on their revenue proportional to the number of jobs displaced by it, and use those funds as a pension for the displaced workers.
> Many graduate students (I know) are having a crisis if any of their research worth it? If AI can (or will) do everything, what's the point of doing experiments and all? This will eventually deter a whole generation of curious minded students from research.
should they not be deterred?
we stumbled into a way of brute forcing intelligence with gradient descent.
> I know that not everyone has morality and ethics, but I didn't realize it was this bad.
This is a very disrespectful way to make a point about acting with integrity.
You should consider that maybe your views on what makes something ethical or moral are not universal -- and that coming to a discussion with the assumption that your position is the only valid one is not conducive to convincing others who disagree with you.
I used to be a PhD student more than a decade ago, and I published a paper containing a solution to an open problem. Yet shortly after my first publication I became increasingly disillusioned, because I started to think that within my lifetime AI would reach and eventually surpass my ability to solve such problems—and that we only had a decade or two left.
So I started saying that it only made sense to focus on problems whose solutions would be useful immediately. I even emailed my supervisor about it, arguing that our efforts were “pointless” in the sense that AI-related problems were much more pertinent and had to be prioritised.
My supervisor thought I was bonkers. I still have the email, though. Quoting myself from April 2015:
> By 2030-2040 we will have enough computing power to simulate a human brain neuron by neuron. Once we manage to create a human intelligence we will be one little step away from super intelligence: just set the intelligence to modify itself and see the exponential growth in action. Our human intelligence is bounded by a number of biological factors (e.g. size of a skull) and even the smart human who has ever lived will appear to be a primitive ant to a supper intelligence (machine intelligence will also have perfect motivation). There is plenty of literature on this if you are interested in discussing this further.
>
> What does it have to do with research in pure maths? I can say that research in pure maths which won't come handy in the next 60 years is just wasted effort. The super intelligence will be able to do maths way better than humans. I believe a lot of current efforts should go into researching of artificial intelligence (or areas to do with AI) instead rather than the pure maths. I want to be proven wrong but most mathematicians I interact with are too narrow-minded to counter me and they just laugh about even contemplating the above. Frankly I am myself so perplexed that I take the above seriously, but I do and it's hurting my motivation.
I'm quite curious what my supervisor thinks of that email now.
I think, with all due respect, that your supervisor would be correct to think of that email as an insult to both them and to theoretical mathematicians as a field.
She was not offended but thought I could benefit from therapy. She didn’t share my opinion - she said it’d not bother her if AI could eventually solve problems for her.
I later realised that my motivation depended heavily on believing I was making a contribution that would otherwise go unmade. The prospect of AI doing that work undermined my motivation, but it needn’t undermine hers. I was trying to explain why I was struggling to continue, though I can see how calling the work “pointless” came across as dismissive.
I commented something similar on a bunch of other posts. The thing that scares me about a lot of the AI community in general is that their utopia is basically more frightening to me than their doom scenarios. They present this "incredible abundance" as the ultimate human endgame, but I agree, when we all end up like the humans in Wall-E, what's the purpose of it all?
And then to get retorts of "it's just just rich techies that want to find 'meaning' in their jobs while millions starve in the third world". Why would we expect the most wealth concentrating technology in history to lead to mass benefits for those in the 3rd world?
Probably the opposite of Wall-E utopia, AI will keep us busy. For one - we are becoming harder to differentiate, competence is hidden deep and surface signals don't carry anymore. Niches and specializations feel threatened by imitation. This means everyone is frantically searching for their own corner, musical chairs. This holds for both people and companies.
Second - every time we learned to transmit better from past experience we saw massive expansion in culture and economy, not shrinking. Writing, printing, computers, internet and AI are just gradations on the scale of transmission from past experience. Everything stands on this transmitted experience.
My analogy to the body - every cell has the same DNA (every human has the same AIs, culture and tools), but they express it differently depending on context. And no one cell is too fat, the organism distributes work and energy. AI won't make anyone too rich. Its wealth is distributed in the system.
I am truly baffled that you think AI is just one more step on the spectrum of "knowledge transmission" between humans. I think that literally nobody who is an expert on this (both from the perspective of the people creating the technology, and the folks on the safety side warning about the dangers) believes that.
100% agree, their utopia sounds far worse than their “humans will go extinct” which still sounds implausible to me. I would rather die than live in a WallE society, and I hate that they’re getting trillions trying to force us there
I think the impact is heavily exaggerated. Yes, it means mathematics will be transformed into a field that relies on tools instead of your mind alone. Just like most fields. Doctors heavily rely on (really expensive!) tools to make diagnoses and do surgeries too. But it doesn't make them less valuable. If anything it makes them more valuable, as the society needs a long training and filtering process to decide who to use these tools.
That's the thing. It might be. We won't know the impact until the next few years. The things that are changing now: culture of research, aspects of collaboration and sharing of knowledge.
Newsflash. Most graduate research is worth nothing. If this helps graduate students to re-evaluate and start doing things that can generate income while being intellectually stimulating, it's a net win for the person and the economy
I mean it is. Most of the time in graduate school, you're getting trained and learning to do reasearch. People rarely produce great work during graduate school. Only towards the end of their degree or start of new position (post doc, independent researcher, or assistant professor), you start seeing good work. And for a professor, most of their research is done by graduate students.
Academia has it's own sets of big problems. Before this AI boom also, most people graduating never stayed in academia. The way graduate schools are structured, I would be happy if few people joined. That would also mean we lose good researchers in a long run.
I don't understand why you introduce morality and ethics into your comment?
Pointing out that regressive Ai politics likely are not going to help researchers crisis of purpose is not unethical.
You also assume in your comment that we need new large generations of researchers, and that knowledge transfer through academic institutions is important.
I get the fear. But entertaining ideas of impact to institutions and potentially new post ai institutions is not unethical.
We don't know for this particular case with certainty.
But, here many people are okay with the concept of stealing other ideas, plagiarism, unethical aspects of collaboration in research. At least for me and many people I know, this is no okay. This is the part for morality and ethics. I see that no everyone agrees with this.
The issue is, these AI ecosystems don't effect everyone equally. Not everyone knows how to judge AI model output accurately. There is already a large divide on AI usage in research. The fear comes from this. If we wait longer, it might be alright too late. I could be also wrong with these speculations but better to cautious.
I actually feel like it's the other way around. The online discourse over the past ~day or so has seemed unusually irrational to me for a technical audience. Commenters making emotionally charged claims of wrongdoing that appear inconsistent with the published claims without justification of the discrepancies. Granted you might well doubt openai's version of events but there's a general expectation of clear evidence when advancing claims of malfeasance.
I identify as neither mathematician nor "maker of things people want" (coder, hacker, engineer). But having friends who identify as those kinds of professionals, let me make some observations.
To use a metaanalogy from chess (once again), mathematicians play the opening game, and builders play the end game. AI is sort of a middleman connecting human understanding to applications.
I think there's a Technical argument to be made that openAI is a threat to the game itself. For example, could it have produced the navier-stokes counterexample without human inputs? since it seemed to have used the much gossiped research strategy "C" and "D", you can't absolutely be certain that Son of Astra (son of altman?) was magicking an unknown unknown from nothing (sorry to cue Rumsfeld). You have got to wait for the other five problems to be solved after general boycott
Subpar PR engine of the OpenAI leadership might kill the pipeline of inputs that they won't admit they still need in this dreamtime before "recursive self-improvement". You can call that emotional. Personally I would rather accuse mathematicians of "preferring local models that believe in the usefulness of unidentifiable individual contributors, and the uselessness of named generalist managers (ie the prompt writers at oAI)"
Big man tlb likes to say that science might be dead but engineering is just getting started. Navier-Stokes is the hammer of the nail in the science coffin. It kills science by killing the prestige of science. The engineers have to imagine that it's likely they will now get all their design ideas from the hypothetical future datacenters.
Sort of, yeah. We are seeing more people commenting who have anti-AI sentiments or in the fence on the topic of AI usage.
On why some people are making emotionally charged claims, my guess: This affects the core belief of what is right or wrong, Impressions based on past doings of OpenAI, losing trust for OpenAI based on sequence of events.
I don't think we will get to see any clear evidence. I'm not even sure what would be the evidence. I would be surprised if OpenAI comes out clean if they have made a mistake. They move on to the next shiny thing.
Academia is toxic for sure. But, also there are genuinely good people who work for greater good and whose training and teaching transcend across domains.
Academia was already crumbling for various reasons. This just hastens it. The reasons are manifold and will never be acknowledged. There will be a lot of gnashing of teeth, but it was unsustainable anyway. And historically the bloat was a very short period, a few decades only.
To me it doesn't seem like what AI has destroyed is the ability for mathematicians to develop understanding and share it with each other, but rather it's destroyed the yardstick (solving open problems) that has traditionally been used to measure how much they have contributed to that understanding.
I do see how this is a problem in terms of assigning credit, but I think the cat is already out of the bag in terms of these models being capable. Even without AI labs spending millions of dollars to solve millennium prize problems, there are plenty of other people who will use them to pick low hanging fruit. I don't think any social solution is going to make things go back to the way they were, where you could share your progress towards a famous open problem without risking someone "scooping" you within a couple of days.
I think that the most likely outcomes are either mathematics becomes more secretive, or there is a more deliberative approach to assigning credit than who was "first" to solve some problem. In the former case, this may slow down progress, and in the latter case, this could mean that credit would become more subjective, and be a continual source of controversy.
The statement is not about AI but about the behaviour of AI companies. OpenAI have put vast resource into solving open maths problems: many millions of dollars of compute just on the Navier-Stokes result, plus whatever they spent on the broader Millenium Prize problems initiative and the other results they have published. Anthropic are doing the same. The statement is asking them to stop doing this.
AI companies are investing these resources primarily as a marketing exercise. There is no near term commercial value to a 100 page Lean proof of blow up in an extreme special case of Navier Stokes, besides the bragging rights. As the statement says any commercial value in this stuff comes a very long time later after new insights and techniques have been digested, integrated into the mathematical canon, expressed in ways that don't take a lifetime of study to understand, etc. (things that AI is not yet capable of doing itself). The bragging rights, on the other hand, are massively valuable. There is a mystique to maths that makes "our AI solved a Millenium Prize problem" an irresistable headline for a company like OpenAI.
What the mathematicians are saying is stop pouring resources that most mathematicians can only dream of accessing into projects that are actively damaging to their field. They face a massive challenge of figuring out how maths can evolve in the face of this new technology, and this is not helping.
What do we do about the problems that don't require many millions of dollars in resources?
Last weekend I spun up a small agent swarm and pointed it at a field of math I have some affinity towards. Within four hours I had settled three conjectures, one of which is rather famous (for the field, not in general). It cost me about four hundred dollars.
I am at a loss about what to do with these results. On one hand I feel like the mathematicians working on these should know about them, but on the other I feel a bit like a barbarian who suddenly finds themselves sacking Rome.
I'm not a mathematician but it seems to me that if all it took to solve the problem was an enthusiast level understanding of the domain and a few hundred dollars of tokens then the result probably isn't that valuable. Even assuming you are the first person in the world to solve it, these kinds of LLM-friendly problems that are now easy to solve and easy to verify will almost certainly be picked off by one person or another in the near future.
It's also possible that the result is already known and you just weren't aware of it. It's easy for someone outside of a field, or even one steeped in it, to not be aware of certain solutions.
sounds a bit like cope. Crouzeix's conjecture was solved exactly under these circumstances and I wouldn't describe it as "not that valuable". In any case, AI capabilities will increase dramatically over the next few years while human math capability will not. That means that there's a fixed target regarding whatever is currently considered a "serious math problem" and its difficulty level. Soon the average problem solved with a few hundred dollars of compute will be at that bar.
I actually partially disagree with this. What happened to all the excitement about Intelligence Augmentation (IA)? Now it's AI instead of IA. I think there's so much untapped potential for augmenting our intellect with the likes of https://dynamicland.org and https://folk.computer, as well as the work that's been going on in college math education, things like Lean, etc. I think the only reason human math capabilities haven't expanded that much is a failure of our imagination, not our potential.
> I am at a loss about what to do with these results.
I would recommend publishing them to Palomar (https://palomar-registry.org/) - I have no affiliation, this is an online registry of Lean-verified proofs created by Terrence Tao.
I have submitted a proof there that's also minorly important in an extremely niche field.
Anyway, I feel like it's a good place to dump AI slop lean proofs because the main point of the registry is that it verifies that: 1) your Lean challenge statement is the same as what you informally state you're trying to prove; 2) your Lean proof actually compiles.
This could be useful to future AI slop researchers who want to know if a given result has already been formalized, and they may be able to mine some lemmas from your work. Also, it's good to know for the field in general what has been proven.
I'm fairly certain you can set your publishing name to be whatever you want, so you could set it to be just the word "Anonymous", or the name of the model you used.
You asked for a painting. A robot made the painting. You looked at it and said, "well, I guess it's good. Should I put it online or something? Dunno. Hey Fred, what do you think of this?"
Meanwhile, your next door neighbor spends their entire life developing their understanding of life through art. They "understand" (maybe not in a way they can articulate) art. You go next door, you look at their painting and say, "well I guess it's good." But you also understand that your neighbor is just like you, and maybe you are a painter in another way.
I find it strange that, people can't see that, we don't need to solve hunger and poverty and work balance, and etc, by a round-about make-super-intelligent-AI. We could just solve it. It's pretty obvious how to, as well.
We can all be painters, if we put restrictions on the psychopaths.
OP specifically said "we don't need to solve hunger and poverty and work balance, and etc, by a round-about make-super-intelligent-AI" though. And I don't know how you get robot farms without that.
I mean, let's say you spun up a swarm of agents to rewrite a large component of a well used open source library to be memory safe. You could dump it in a big PR and walk away (we all know how that would go), or you could try engaging, see if they're interested, write something up and see where it goes.
The biggest problem is, IMO, drivebys uninterested in actual results, just getting a check mark, and the equivalent of dropping a 200k line PR on people and expecting them to be interested and do the work for you. These are things many on HN are familiar with and know how to do better :)
> and the equivalent of dropping a 200k line PR on people
I can understand why the community is pissed. So now, lean proofs can be churned out at scale, and the community is left to decipher all of that slop into human understanding. There are bad actors with misaligned incentives coming in with drive-by proofs upending what the community holds dear which is to practice and propagate the art. I applaud them for this declaration.
To re-align incentives the following could happen. AI slop lean proofs are dumped unceremoniously into a lean dumpster, and what gets rewarded are results that could digested into human understanding - via the already followed human review process. Prizes are not given to lean proofs since anyone with sufficient compute can churn them out.
The numerical results are trivial to check. I wrote the analytical results in Lean by hand before asking a former professor to confirm after asking him to keep this private.
It's really tough to come to terms with it but making academic contributions in general from the outside (with or without AI tbh) is not often welcome and the whole process feels very gate-kept.
(academic.) Unfortunately the overwhelming majority of outsiders are missing core knowledge (or are cranks), so the optimal prior from a time management perspective is to ignore them. AI just makes engaging more costly because there’s more volume and it’s harder to get signal on whether they know what they are talking about.
If that's the case, that makes me much less sympathetic, even though I can understand how it's very disturbing to see the field suddenly changing like this.
If you are trying to understand better the field, then do a good write up of the proofs so that people can learn from it.
If you want to earn the respect of people because you found interesting proofs. Then do a good write up of the proofs so thst people can learn from it.
If you want to plant flags and pollute peoples minds. Then please publish it anonimously, no one wants to correct LLM slop for you.
Probably we should build a repository of AI slop proofs that are only allowed to be publish anonimously. That way people may be more inclined to work on it because they would feel like they are cleaning your house for free.
Maybe I'll end up doing the write ups pseudonymously. I have taken care to make sure the results can meaningfully contribute to field but I don't want to plant flags or really receive credit of any kind. I just think they are interesting.
I like my current life and don't want to get dragged into the current fracas surrounding the use of AI in math.
If this was limited to just three results, I would agree with you. But those three are just the ones I've managed to verify myself. The list of unverified results is quite a bit larger.
The field I've been investigating is not large. Even if I were to take the time and care to beat the interesting results into something meaningful, I'm afraid the pace at which I'm able to produce these results would not be well received.
This all comes down to what we think mathematics is. There are two options here:
Mathematics is just the formal system: in that case we will never be able to beat the AI as humans. The goal is then to cover as much of the formal landscape with AI generated proofs to proof as much statements as possible. Success metrics are lines of lean and numbers of proven statements.
Or
The formal system is just a limited representation of what mathematics is: in that case probably some parts of the formal mathematical landscape are more important than others. Not all statements are born equal. And 90% of AI generated proofs will be just noise. Our job as mathematitians is to steer the AI to high mathematical value regions and turn formal proofs into mathematical proofs and insights.
From my point of view we have know since Godel that we are living in the reality of point 2.
Each of the two points implies a very different way of doing mathematics. So pick the one that you think is true and act accordingly.
Like the mathematicians working on famous problems in private until they could claim full credit for something interesting wasn't also a marketing exercise for their own careers. The commercial value (or lack thereof) of a proof doesn't depend on whether it was done by a human or a machine.
These mathematicians dedicated their life to math and were working for a long time to achieve the pinnacle of their careers.
OpenAI just burned millions of dollars over a weekend after hearing that someone else was close to solving the problems. Their interest was in their AI system more than the actual math problems.
I see how it can be devastating to their ego, but no, I don't see a particular difference in a company spending money for clout vs. a person spending time for clout. The underlying motivation is the same.
for many fairly strong mathematicians, the career calculus is fame and status (mild though it may be) through mathematics or anonymity but financial reward in tech or finance. Yes, clout by becoming a lifelong academic is in fact rational for some and part of their motivation. Of course they really like what they do as well, but earning the respect of the peers they know are also respected by a large swathe of society is very important.
I guess that type of “clout” feels different to me.
Wanting to be validated by peers for your talents in a niche field vs. using millions to try to solve a math problem that you don’t really care about with AI to market the gigantic company you work for.
That fairly insular community pretty much gave up on commercial success in order to be insular. But the commercial forces were not happy being commercially successful, so they decided to disturb even their insular and commercially-unviable activities.
The point is that nearly every white collar worker is in the same boat right now, and the majority of them probably have already had much more profound impacts to their fields. So yeah, we can certainly imagine what it’s like for mathematicians.
> Like the mathematicians working on famous problems in private
This is an extremely rare situation that practically never happens. Most mathematical research is done in the open, with partial results being published, conferences where approaches are discussed, collaborations... The Navier-Stokes solution is a great example: the approach that OpenAI ended up using was something that two mathematicians had proposed previously and was being studied and followed by several others, with different sub-paths within the same approach.
How much do you think other AI companies would offer to get access to the transcripts of the generation that led to the proof? No doubt OpenAI will include it in their training data somehow and use it to build the next generation.
Are they claiming that the only value in solving these problems was for their field's personal development process? I thought Navier-Stokes (and some of the other millenium prize problems) actually had implications for useful technology. It would be insane to demand that people avoid making progress on technology that can save lives or improve general quality of life, just to protect the sanctity of your karate belt system. Perhaps in lieu of open problems left to solve, mathematicians should be welcome to take up chess or sudoku to keep their minds spry.
> I thought Navier-Stokes (and some of the other millenium prize problems) actually had implications for useful technology.
This is the core misunderstanding that the open letter is attempting to correct.
Developing a better understanding of the Navier-Stokes equations could have a number of implications for useful technology. They're fundamental to fluid dynamics, and turbulence in particular is something that many people feel we could work with more effectively if we better understood how and why it's generated. The Navier-Stokes smoothness problem is an interesting and long-standing benchmark for this understanding; we don't know why it should be so hard to answer, so we hoped that the process of developing a proof to the problem would produce more understanding. (We may still be able to extract this understanding after the fact, if OpenAI's proof is fully human-comprehensible.)
Simply knowing that there exists a finite-time blowup is not practically useful. We know that fluids in the real world don't produce random singularities, so the result can't really have much physical meaning. What it illustrates is that the Navier-Stokes equations fail to model physical fluids in some yet to be characterized way.
No, there's most likely zero practical benefit of having found a pathological edge case in which the Navier-Stokes equations do not work. We're most assuredly not talking about "saving lives" here. Unless advanced aliens show up and tell us they'll destroy Earth unless a counterexample to the N-S equations is provided within 24 hours.
That solution (stop pouring resources in to proofs, stay in your lane) works today. How does it work 5, 10, 20 years from now? The software and hardware advances will continue.
Your point is largely addressed in the article, did you try reading it? "In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align."
The point is that these proofs are largely useless without the insights. The value of a proof is largely in the travel, not so much in the destination.
Different person here, I read the article and they are all wrong. Hope that helps.
Okay, to elaborate, substantively, their point is that the people using these AI models are not doing it for the love of the game, but for marketing. And instead of them - and nobody - spending millions of dollars to solve the problem, successfully, they want every problem of their academic industry to persist because even though they never solve the problem, they synthesize and solve lots of other problems nobody asked for. And get to boost their egos?
Yeah, stop that. Actual alignment is on the humans themselves, if they want to remain relevant as academics and mathematicians, they need to learn how to replicate the proofs and the steps that alluded humans for decades and don't worry about the narcissistic elements that slow their industry down.
Mathematical breakthroughs with commercial relevance are few and far between, and often depend on dusting off old results which were, at the time of discovery, "solutions nobody asked for."
The NS counterxample is actually, by any market measure, a "problem nobody asked for" in the sense that its existence doesn't have any commercial relevance (beyond juicing OpenAI's IPO). So the only long-term value solving it could have is by virtue of whatever reusable theory/insights were generated along the way to the counterexample itself. The letter is absolutely right on that point.
It's not actually clear that those insights will come faster from reverse engineering this LLM proof vs. humans building theory to solve the problem themselves. So what you're saying may or may not even be an efficient way of operating. Also, it implicitly depends on mathematicians to do the hard work of creating problems and then deciphering LLM hieroglyphics for essentially free while the only immediately profitable component gets outsourced to a frontier lab. In what world is that model going to work?
Reading between the lines, it seems like maybe you have a personal grudge for some reason and simply think the technology will advance enough to where we won't need academics at all. But you should say that in the first place.
I just wonder whether a lot of smart people who never needed to go beyond the "solve for X" algorithmic math of a typical calculus sequence are actually reading the declaration the way the signatories wrote it. The Navier-Stokes problem is not exhausted by a simple 'no' counterexample (which most people familiar with the equations already expected to exist). In fact, I haven't heard a single person's explanation for what relevance this counterexample has for humankind. It is something impossible in our physical reality so we have gained zero insight into anything we actually model with NS.
Mathematicians agree that "solving the problem is aligned with humankind." They disagree that releasing a counterexample this way actually constitutes "solving the problem" precisely because there is now little incentive to do the hard theory-building work that actually has the track record of leading to human advancement.
Their claim is only indirectly related to the motivations of the people using their models. What they're saying is that doing math in this way does not produce the same value as traditional mathematical research, and the people using these AI models aren't concerned about that because their marketing objectives don't depend on whether their results produce mathematical value. If people doing valuable work are made irrelevant by people doing a larger volume of non-valuable work, that's not a positive outcome.
But it does produce value. We now have an explicit solution and even a proof. Humans can then work on clarifying why it's true. Unsurprisingly, not all that different from software, where models can generate working code just fine. The details will all be there and all correct, but the architecture is currently not ideal, so a human guiding it can greatly improve the proofs.
I actually found this to be the case with some basic linear algebra notes I was recently doing in Lean (without using mathlib). The model could generate working proofs, but they obscure the basic ideas (actually I wonder somewhat if this is because the Lean code that's out there to train on doesn't make a huge effort to read like textbook proofs, which was my motivation in the first place). I give it a skeleton of a couple lines of `calc`, letting it fill in the reasoning for each line, and it does much better. Then ask it about making some macros to simplify "trivial" or "obvious" things, and it does even better. etc.
I suspect there's a good workflow where a big SOTA model makes an impenetrable proof (or code) and then a human works with a FIM model to simplify it (with the larger gnarly proof right there in context for FIM), but unfortunately everyone seems to only care about agents right now.
> But it does produce value. We now have an explicit solution and even a proof. Humans can then work on clarifying why it's true. Unsurprisingly, not all that different from software, where models can generate working code just fine. The details will all be there and all correct, but the architecture is currently not ideal, so a human guiding it can greatly improve the proofs.
To me this analogy points in the complete opposite direction. Imagine somebody takes a half-completed project design you're trying to figure out, vibecodes a rough prototype of it, emails your manager to announce that the project just launched in alpha, and then dumps it back on your lap for approvals and testing and productionization. Would you say that they've added value to this process? Or did they just strip away all the hard parts of the problem so they could claim credit for the easy part?
One of the awesome things about LLMs is they make it quick and easy to make PoCs, so yes. Proving that an approach will work before spending a bunch of deep design effort is absolutely valuable. Your exact scenario is something I've literally done: give a half-completed design to a team member and asked them to vibecode a PoC to prove the approach will work and figure out some of the details, explore scaling and failure characteristics, etc. Or I do the PoC vibecoding myself too. LLMs have been a gamechanger here.
It's valuable for you, the person who's going to spend a bunch of deep design effort, to make POCs. Is it valuable for someone else to drive by, dump some POCs on your lap, and then leave you to do the deep design effort while they run away to study AI?
If that person then runs around telling people that they're the real author of your project, because they generated the original POC, would you consider that an accurate assessment?
Your analogy is far enough away from the way that the real world works that I'm not sure that I can really even strain my experiences to fit within it. Sure, I guess that would be annoying?
But mathematicians define their field. They're smart people. They're capable of recognizing when someone just did a vibecoded throwaway PoC and when someone has a well structured proof. Actually even before LLMs they'd publish new, clearer or more elegant proofs of old results. They can say that inscrutable proofs are exactly as valuable as they are, and that the first explanation people can actually understand carries its own prestige.
It's a real example that's happened to me twice in the past year, so I'm not sure what to make of the idea that it's far away from how the real world works.
I'm also not sure I understand what you're objecting to if we agree that mathematicians define their field. The source link is a declaration from 25 Fields Medallists with precisely that goal. They believe/define/declare that the type of AI-generated proofs we've seen are vibecoded throwaway PoCs; they feel that a well-structured proof must include factors such as "a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others", and the success criterion is not a true/false conclusion but rather "development and integration into the mathematical canon".
Maybe your management has no idea what you work on. If so that's its own issue. Or maybe they do, and the person trying to take credit for more than they did just looks like an idiot. Like I said, I intentionally give throwaway PoC creation as a work item to people and it's fine. And if e.g. someone randomly made a drive-by PoC that proved that the approach I was exploring couldn't work and had some fundamental flaw, well, oddity of a random person doing it aside, I'd want to know that, and they'd deserve credit for saving us from a bad approach!
My objection is characterizing things like AI slop proofs as not valuable. Obviously it's valuable to know that:
1. NS has solutions that blow up in finite time.
2. This fact is provable, and we have a proof.
I don't think anyone anywhere is saying that these will replace mathematicians as they are now. I also think "a proper write-up, the isolation of new methods and ideas, and citing relevant previous work of others" is frankly not necessary or even desirable before we publish a computer generated result. We have a machine now that can spit out answers that we have good reason to believe are accurate, but they're perhaps inscrutable. There's no need to first decode the why and figure out proper attribution before simply posting the proof online. The proof itself does add value as it stands, even if it's not the ideal. Hoarding it until you can do a proper write-up would be silly.
Yes, mathematicians will clearly need to rewrite the qualifying criteria for prizes to better align with the actual goals and value they were hoping to get from a solved problem. The field as a whole assumed good faith actors and collaboration, not expecting a few trillion dollar companies to walk in and start turning in piles of Lean no human understands to be able to claim "first".
This letter includes someone like Terrance Tao who publicly expressed a lot of optimism about AI for solving novel math like with the Erdos problems. It's not sour grapes but the first steps to define those new expectations for the future to reduce the perverse incentives.
And yet, predictably, people are accusing him of "gatekeeping" and ignoring the arguments he has made here and elsewhere about the benefits vs. harm in different ways of using AI.
I don't see turning in piles of Lean as bad-faith somehow. They were first, and they did prove the result (assuming no hidden exploit deep in the middle). The thing they produced is just different, and both giant raw proofs and distilled human-understandable proofs are valuable. Before we couldn't make giant raw proofs, so we didn't need to understand their place. Now we can. It makes sense to just incorporate that into what "math" is (at least until the machines are smarter and can make elegant proofs from the start).
There are vanishingly few research mathematician positions and it's one of the most competitive fields, so no. But I'm not sure how that's relevant. As the OP says, usually the value of a proof is not the knowledge that something is true per se, but the reasoning techniques to understand why. How can it be anything other than helpful then to have a truth oracle as you try to figure out why things are true?
So you're not going to do it yourself and you want someone else to do it for you? Some mathematician that dedicated their life to understand mathematics must now toil unpaid and unwillingly to understand the AI slop proofs that you want us to be able to understand?
Do the job yourself. And if you can't, that's maybe a hint that you should listen to the people who can.
Who said anything about unpaid? I'm pretty sure professors don't show up just for fun. Our taxes pay them.
I'd be happy to do the job. Actually I still dabble recreationally (clarifying Codex's Lean proofs, even!). But like I said it's one of the most competitive fields on the planet. As you say, you have to dedicate your life to it.
If a slop proof isn't helpful, they don't have to "toil unwillingly to understand it". They can just proceed with the knowledge that the proposition they want to prove 1. is true and 2. is provable, which is already a decent start for motivation. But often LLMs can actually do quite well explaining ideas too in the hands of an expert. Or you can ask them to prove some technical lemma that you think ought to be true, and that could offer insight for the thing you're really interested in, but for which the details are actually not all that interesting to you. You don't have to one-shot "prove RH from the ground up in 50 million lines of Lean."
I would recommend a bit more humility and trying to better understand why 25 Fields medalists, among them people like Terence Tao (who isn't anti-AI by any means, he's even promoted a registry of AI Lean proofs), are saying this.
> Okay, to elaborate, substantively, their point is that the people using these AI models are not doing it for the love of the game, but for marketing.
No. The point is that AI companies are using the models to solve problems in such a way that the useful part of problem-solving, i.e. the theories and tools developed during the process, is not present. And they are doing that because the companies seem to be motivated not by honest advancement of math but by marketing and publicity.
> they synthesize and solve lots of other problems nobody asked for.
No one asked Fourier to solve series representation of functions when he was studying the heat equation, and yet thanks to that we have Fourier analysis.
> if they want to remain relevant as academics and mathematicians, they need to learn how to replicate the proofs and the steps that alluded humans for decades
The point they are making is that if AI keeps being used as "problem solver" rather than "theory understanding", replicating the proofs and getting the useful parts out of them will be far more difficult.
Do you "know", in any meaningful sense, any of OpenAI's recently publicized proofs? Do you suppose that there is any large community of non-academics that does?
One of the points the parent makes, along with the TFA, is that academia -- or more specifically, the "mathematical community"-- is a setting primarily for creating and ingesting mathematical knowledge, and disseminating it to the next generation and to other fields. Humans absorb this material slowly, through lots of discussion and collaboration -- it is necessarily a slow process. Facilitating this is one of the important functions of academia. Your usage of academic as a slur here is a bit silly for this exact reason.
I don't claim it is perfect, and we can argue about pedagogy in elementary courses till the cows come home. That's not really material. But this is one of the only settings in which such knowledge is broadly valued for its own sake, and in which there is a semblance of incentive to help others "know" this stuff as well, be they future generations of mathematicians, science and math educators and communicators, practitioners in other fields, or genuinely curious amateurs.
I suspect that beyond just marketing, these pursuits yield plenty of useful information about model design that will likely lead to model improvements and optimizations for both mathematics and general reasoning going forward.
> What the mathematicians are saying is stop pouring resources that most mathematicians can only dream of accessing into projects that are actively damaging to their field. They face a massive challenge of figuring out how maths can evolve in the face of this new technology, and this is not helping.
Is it reasonable for any field to make such demands? If this were doctors objecting to AI becoming good at medical practice would you have the same concerns?
While any idea of OpenAI spying on people to pursue their goals is disgusting, the rest of this is par for the course, as Kasparov experienced with IBM in the 90s. Humans still play chess after all.
It's not a simple
matter of "becoming good at", and yes, I could very well have similar concerns, depending on how it impacts the field. The statement itself mentions that such concerns exist in many other fields.
I doubt OpenAI will take such a combative stance and accuse these mathematicians of "demanding" things, as you do. As I said, the purpose of this is marketing and the statement simultaneously undermines the value of that marketing (showing these projects as irresponsible) and gives these companies an even better piece of marketing in its place: "our AI got so good at maths the mathematicians begged us to stop". It's entirely possible they will stop pouring millions into these projects.
So let’s say OpenAI cure cancer and put every cancer researcher out of work depriving them of intellectual satisfaction, this would also be a problem? It would certainly impact the field.
The fact is these fields are supported by society because of the benefits to everyone else. Once the same results can be achieved in a cheaper and faster way that is what will be done. We should mourn this in the same way we do buggy whip manufacturers. Again people still ride horses.
> So let’s say OpenAI cure cancer and put every cancer researcher out of work depriving them of intellectual satisfaction, this would also be a problem? It would certainly impact the field.
Would you expect Fields medalists to cure cancer if you moved them from the math department to a medical research lab? This is precisely the fallacy that the frontier labs are counting on to inflate their valuation as their IPO approaches. They want to use headline-grabbing problems in pure maths to make their models look "smart" in the public eye. But what does "smartness" in mathematics really mean in terms of economic value? It is not at all obvious whether success in abstract mathematics should translate to successes and, more importantly, profitability, in more grounded endeavors.
Look at OpenAI's job postings (https://openai.com/careers/search/). Those roles involve far more pedestrian yet profitable duties than research mathematics. So why isn't OpenAI automating them with their vaunted models? Success in one field, no matter how "difficult", does not predict results in another field.
> The fact is these fields are supported by society because of the benefits to everyone else. Once the same results can be achieved in a cheaper and faster way that is what will be done.
Maybe it's worth double checking that you know how these fields benefit everyone else? Proving the blowup of the Navier Stokes equations in 3D isn't going to make your gas cheaper or make harvesting food easier or make drones easier to protect against. Maybe consider the deeper effects at work?
Math people absolutely are important to things like the SpaceX landing control systems, hypersonic gliders, 5G networks and so on.
If you can make breakthroughs on such areas as fluid dynamics, control theory or information theory with AI then that absolutely is a big deal with real technological implications.
> It's this sort of thing that motivates people to burn down the institution you might be trying to defend.
lol, yes, this sort of thing is what many people who voted for Trump were saying, and things are going great for them.
> They prove a millenium result, but it doesn't count because they are bad people.
OpenAI has only themselves to blame for this, and they know it. They could have handled this so much better. I'd bet there's more meeting time right now going into how to unveil future math results than on meeting about the actual math research.
The issue of credit is a relatively minor point in the declaration.
It's more about bypassing the culture and processes mathematicians have developed that lead to human understanding, generating new ideas, and bringing up new generations of mathematicians. (See also his article about "non-renewable mining" of good problems.)
Reducing mathematics to "let's just generate results through an isolated and automated system" is a misalignment since it bypasses those processes.
I don't think it really attacks human understanding though. You can still read and understand an AI written proof. If another person comes up with a solution to a problem, you can read their methods and understand it. It doesn't matter if a human came up with that or not. It's really only attacking the "generating new ideas" part.
That's precisely the problem though. You cannot still read and understand an AI written proof at the current skill level of the AI being applied, because they're orders of magnitude longer than human written proofs even when they don't need to be, and spend most of that length on the parts that aren't important. This has been really thoroughly documented by expert mathematicians who are engaging with AI in public like Terence Tao and showing in detail how much work it takes working alongside AI to figure out how to understand AI generated proofs. With human generated proofs that process is forced to happen before publishing the proof because the new style of AI generated proofs validated only by formal verification is supplanting the old human peer review process that forced the burden of understanding onto the publisher and not the reader.
> You cannot still read and understand an AI written proof at the current skill level of the AI being applied, because they're orders of magnitude longer than human written proofs even when they don't need to be, and spend most of that length on the parts that aren't important.
That doesn't seem to be true. The OpenAI NS paper was 166 pages. Wiles-Taylor proof of Fermat's last theorem is 129 pages. The length is not unprecedented for a difficult unsolved problem.
To be honest, I feel like the difficulty of reading AI proofs is due to the fact that we are on the verge of being beyond human comprehension. This is a demonstrable fact as no human has figured this out despite the problem being open for almost 100 years.
> To be honest, I feel like the difficulty of reading AI proofs is due to the fact that we are on the verge of being beyond human comprehension.
I can see where that's coming from, but I really don't think it's the case. Even with Astra, the proofs you get are just off in a way that doesn't signal superhuman comprehension. As 9question1 says, a common theme is that they dwell on insignificant steps. Another one is that they'll often be full of terminology that either doesn't exist, or has this weird quality where it looks like it is trying to make some minor insight seem much greater than it is. At first glance, that'll often make it look like it knows more than you, but when it's really just doing the same thing but in a more complicated and worse fashion, that to me isn't a signal of comprehension at all. The bizarre thing is that despite all the "stochastic parrot" style nonsense you'll get in individual proof steps, they still often combine to something valid.
In either case, what all of this means is that the working mathematician still needs to go through, and generally completely rewrite, any proof output by an LLM. Otherwise you are passing the burden of unreadability onto the reader.
Yeah, that mirrors what I've seen throwing some of the leading models at a set-theory problem that's stumped me (https://mathoverflow.net/q/511601): in this case, the problem does not easily yield to the standard tools, but the LLMs do not recognize it as a major open problem they should give up on. So they seriously try it, but typically end up in a loop of inventing certain classes of simple solution or counterexample attempts, defeating them, and trumpeting each one as a major result, each time inventing some new terminology.
It's definitely quite curious that the AI labs are able to push these results through seemingly with pure brute force. Perhaps it's largely a function of how many monkeys you have attempting various constructions on top of the known results and strategies the models have memorized.
> This is a demonstrable fact as no human has figured this out despite the problem being open for almost 100 years.
That's not true. Alpoge and Buckmaster's related LLM-assisted blowup result (https://news.ycombinator.com/item?id=49605915) utilized a strategy developed recently by Cordoba and Martinez-Zoroa.
Not "a" human's understanding; Humanity's understanding. Understanding the research problem, and the solution especially, is a lot more involved than simply "read their methods". That's the whole point being made.
It matters if a human came up with it because of everything mentioned in the article... A mathematician's solution is necessarily built on other's ideas that have been disseminated, internalized, pressure tested etc. Methodologies differ too. AI can abuse its compute resources and generate a true/false or counterexample statements, without laying the foundation that a decade of globalized research would have.
It's not only Lean code, there are English writeups too. To my understanding the pipeline for these problems is 1) solve in english 2) formalize systematically to check. No one is tackling problems purely in Lean, to my understanding.
I wonder if this is a root of the complaints across fields, how AI is ruining the greater picture and process in writing, acting, drawing, filming, coding, and more.
It's also known as "commodification of labour", and AI is just the latest and greatest tool to do it.
Luddites complained that the trajectory of technology was to allow less skilled workers to mass produce goods via machines owned by factory owners, as opposed to helping skilled workers build up and use their skills while passing them on.
Now we have a lot of money and time focused on LLMs owned by a few companies, making it easier for them to monetize low skill labour(prompting versus art/research/artisanry)
> Luddites complained that the trajectory of technology was to allow less skilled workers to mass produce goods via machines owned by factory owners, as opposed to helping skilled workers build up and use their skills while passing them on.
And their complaint was valid from their perspective, but the end result of that process is that now you can have functional and pretty clothing for pennies.
It's true that a lion's share of the gain was pocketed by people who owned the mechanized looms. Now imagine how much more of an advantage it would have been without those people.
Same thing here. We can - and should - get rid of all the megacorp leeches. But the benefits of cheap automation remain regardless, and it's still the same equation - removing the gatekeeping harms those who relied on it to keep the products of their labor priced high, while benefitting everyone else.
Suppose that tomorrow we learn that AI just exploited a bug in Lean and the proof is, in fact, bullshit. Or suppose it is the case, but we never learn that.
> The issue of credit is a relatively minor point in the declaration.
What a load of croc. This entire debate is fueled by a perceived lack of attribution. The AI learnt from researchers and did not give them a sporting chance of being first before scooping them. They were expecting some sort of fair play, instead they got a ruthless machine. Every other tangent to this debate is irrelevant, the culture, the community, the shared symbolic growth. Every mathematician I know is secretly trying to one-up their peers.
I agree that the declaration doesn't focus on credit, but I think it's still at the root of the problem. Because ask yourself: if the AI generated proofs are not creating any new ideas or insight, just brute forcing a boolean true/false result, then why can't mathematicians simply ignore their results? Why does it matter if OpenAI or even amateurs with AI are "solving" these problems, without contributing to any deeper understanding?
I don't think "intellectual poisoning" is really the mechanism that harms the mathematics community.
The harm is if you have a community of mathematicians who are focused on expanding human understanding, then having instant access to a bunch of AI proved results muddies the water about who has contributed what. If someone could scoop any significant theorem at any time by pointing an AI at it, how do you really demonstrate that you have created new understanding? Or that your new understanding is about something important? How do you prove that the AI needed your new concepts to be able to solve it?
Open problems are not that yardstick. Fermat's Last Theorem is the result of Wiles and Wiles-Taylor, but without key results from Serre, Ribet, Ihara, Langlands-Tunnels, and Frey's program none of what Wiles did would work. But Wiles did get the prize. Nowadays I think the inputs have shrunk a bit by doing more in the R=T theorem so less other cleverness needed.
> destroyed the yardstick … that has traditionally been used to measure how much they have contributed
This goes much broader than mathematics or academia. This is the entire basis via which society distributes its wealth: based on a labour market derived valuation of ‘contribution’.
> This is the entire basis via which society distributes its wealth: based on a labour market derived valuation of ‘contribution’.
Correction: that's not how society distributes its wealth, it's how it throws some bones to the masses. I wouldn't be surprised if over half the wealth goes to people who don't sell their labor at all.
>or there is a more deliberative approach to assigning credit than who was "first" to solve some problem
it feels like an unintended consequence of the millennium prize is that people view the [last contributor to the solution] as the only one to make progress on the problem. I've never viewed Poincaré as solved by one person and the objective of the prize was to encourage more people to make attempts and contribute towards progress.
this issue is independent, but in these circumstances perhaps interweaved, with the 'ai is taking over math' concerns
> but I think the cat is already out of the bag in terms of these models being capable.
When there's a discussion about doing something against the damage of the AI industry: "whoopsy, sorry, another cat escape, nothing can be done".
When there's a concrete mention of an actual solution to avoid more cats escaping: "that won't happen, and even if it did, the damage is already done, and in fact it’s not that bad you all just have to go with the future we decided for you."
So the bag is wide open, more cats will escape, and nothing can be done about any of it. not about the ones that got out, and not about the ones still inside. Sounds more like a preemptive excuse for inaction, cosplayed as pragmatism
I feel a similar fate will befall engineers too. Your predictions anre quite interesting from that perspective.
Markets defined entirely by law have distorted our collective understanding of what can actually be built with the knowledge our species has accumulated thus far. How will traditional shields that have protected capital accumulation in tech to survive in a world where governments now realize control of technology is a national priority? Especially as we see its impact on modern warfare, and that such conflict looks like it’s only escalating over time.
Mathematicians appear to me (as an outsider) to exist in a field without such distortions, and I think offer engineers a preview of what’s to come. I certainly have completely ceased sharing original ideas online at this point.
> To me it doesn't seem like what AI has destroyed is the ability for mathematicians to develop understanding and share it with each other, but rather it's destroyed the yardstick (solving open problems) that has traditionally been used to measure how much they have contributed to that understanding.
I think you're missing key details in the statement. You can hate problems and sprinting towards solutions, and still have a deep impact in mathematics – Grothendieck once famously said you can often "soak the walnut" to crack open an issue. Rather, it's how frontier labs are inherently commercial incentivized to act like this, because it is in their best fiduciary interest. That is the misalignment.
To me it doesn't seem like what AI has destroyed is the ability for mathematicians to develop understanding and share it with each other, but rather it's destroyed the yardstick (solving open problems) that has traditionally been used to measure how much they have contributed to that understanding.
It's more than the yardstick for individual contributions. It's also the yardstick for the contribution of ideas. And that kind of yardstick is very important.
Math has a challenge that's only occasionally acknowledged imo, that's it's possible to just go up the ladder of abstraction, formulate ideas but have those ideas actually not be useful or interesting. The ideas that math has developed, that mathematicians consider important, are those that have helped people solve hard problems.
For those outside academia, the ”reward mechanism” is a choice between A) being a genius and working hard to become a leader in your field, B) becoming very good at writing grant applications, C) capitulating to corporations and living with the moral burden of their exploits
Right. It seems like reading an AI proof (although it may not be well written) will provide the same insights as reading a proof from another mathematician, assuming it's been reviewed and edited, just like any human-authored publication. If the work is inherently valuable on it's own, I feel like that's mostly what matters.
At issue is the fact that it doesn't typically work like: mathematician produces a proof in isolation, generates a PDF, and shares it with a bunch of people. There's a whole culture and community going on behind the scenes with conferences, seminars, lectures, chats in the hallway, advising students, etc. that AI-generated proofs bypass.
I don't think that AI would disrupt any of that. Even if AI solves a problem, you can still discuss the methods at conferences, seminars, lectures, chats in the hallway, advising students, etc.
Tao’s key point is that the way people are using AI today does disrupt that. Because people and companies are valuing the results over understanding. So we’re getting slop results rushed out that are automatically verified.
Moreover in the past, discussion and idea sharing would happen naturally to overcome the friction of the process. But now when OpenAI is stuck on a particular part of NS for example, they can just throw more capital & tokens at the problem.
I won’t speak for Tao. But it is not “people” who have valued the result over understanding. “People” are a bit impressed that the machines are as good or better than the priests who have been praying at the inscrutable altar of pure mathematics. But it is mathematicians—I am speaking as a mathematician—who have adopted a culture of valuing results over exposition and transparency. The field has been rife with extremely opaque papers for many years and the character of research participation has been one of exclusion and competition over proof priority at the expense of understanding and transparency. It is unbelievably ironic to listen to mathematicians complain about “AI slop” when they have built careers upon human “slop” if slop means papers crammed with technical density that prevents all but specialists from reading the work.
I think it's OK that there are some materials designed for a specialist audience, and some materials designed for a wider audience. Technical density serves a real purpose in the former (I'd like to just say "stack" without including an explanation 10 times longer than the rest of my paper about what a stack is and why we are talking about them), and some people expend a lot of effort on the latter (think lectures, lecture notes, textbooks, seminars, blogs, etc--totally appropriate to dig into the motivations here).
It's just the fact that it's a really vertical field, not some cultural failing, that results in pretty opaque stuff sometimes.
Hopefully that's true. Instead of bypassing the community maybe it stimulates the community. I think it does disrupt things a bit, but I'm optimistic at the moment!
Almost every "normal" job has regular performance reviews where individual contributions, not collective outcomes, are reviewed and used as a sole input for raises, promotions, and firings. If you can't sufficiently document what you personally did, you might've as well not done anything at all.
Yes, that's true, but it is still generally accepted that a completed project is a result of some collective effort where people of varying degree of seniority and ability contribute. There is also not a singular event of a project being completed with a list of heroes/geniuses making it happen, but rather a whole lifecycle of gradual development, maintenance and going out of relevance with contributors coming and going.
I can imagine mathematics of the future being more like that rather than history of discoveries with dates and names
Nothing will be lost if the credit system disappears. History of Science has many examples where wipe outs happen. The chimp brain cant survive without creating elaborate stories about how important it is, more as a cope to its own limitations and what it cant predict or control. Humility is good for health. 3 inch chimp brains didnt create the universe.
Tao's critique of AI in the field of mathematics reminds me of what French art critic Charles Baudelaire said in the 19th century about photography [0].
Baudelaire argued that photography became a haven for failed painters, the sorts of hacks that could not finish proper training. Photography, as a mechanical rendering of the world, could only record what already existed; it couldn't transform reality the way a painting could.
He also criticized the public's craze for "rushing" into it, and complained that this technical "progress" was weakening the arts.
I have listened quite a few interviews with Tao and I see him being very careful about criticizing AI. He very often emphasizes the usefulness of it. Where he is critical has a lot of merit. One of the points I clearly remember him saying that having AI be able to solve many of the open problems, regardless of how important there are (there are many open problems that are not that important) greatly reduces the problem space for mathematics students to give new problems to work on.
In a parallel thread omnicognate correctly pointed out that for AI companies it's a direct commercial loss to pour all this money into bruteforcing the solutions to these problems, and that a lot of times the solutions by themselves are not directly commercially valuable. They are doing it for stock price, trying to lure in private capital in preparation for IPOs.
Their models are good, but they are not the moat because Chinese models are good too, so what they are doing, in my opinion, is more harm than good. Mathematics is a science by humans for humans.
> One of the points I clearly remember him saying that having AI be able to solve many of the open problems, regardless of how important there are (there are many open problems that are not that important) greatly reduces the problem space for mathematics students to give new problems to work on.
The crux of the argument perhaps. It suggests that too many people are currently studying mathematics without making much progress.
>I have listened quite a few interviews with Tao and I see him being very careful about criticizing AI. He very often emphasizes the usefulness of it.
This is tiresome. People should be able to flat-out criticize AI without the implied need to justify themselves all the time or "be careful". Its almost like AI has a trillion-dollar agenda backing it, to the point that you have to add a careful "its really great! But there's this little issue..." for any criticism.
Even those who are very pro AI should have the intellectual honesty of admitting that there are very valid reasons to criticize AI.
Based on the amount of low-effort criticisms out there, clearly nobody is afraid of "big AI". Tao seems careful about his critiques of AI because low-effort off-the-cuff criticism can lead to all sorts of future problems/hypocracy.
That's not really addressing the same issue though. I'm saying that there is a pressure to preamble any (good or weak) criticism of AI with an Apostle's Creed of AI positiveness, reassuring the reader that what follows isn't blasphemy against AI (to borrow Huang's terms).
I see where you are coming from. I imagine LLMs will follow the same feedback lifecycle as other groundbreaking tech. Most legitimate iPhone 1 complaints came paired with a few positive messages to the devs and engineers. Years later, after countless improvements to the hardware and UI based on user feedback, now it's just complaints.
I think there's a difference between "photography will change art--we need to be ready" and "photography will change art, therefore stop photography."
There is no doubt that AI has changed the practice of mathematics, just as it has changed the practice of software engineering (and will soon change almost every intellectual job).
Trying to deal with change by saying, "please stop the change" is foolish, IMHO. Mathematicians need to redesign the discipline with AI in mind. But I get that it's easy for me to say that and hard to actually do.
> I think there's a difference between "photography will change art--we need to be ready" and "photography will change art, therefore stop photography."
That wasn't the critique. The statements then were that photographs weren't actually art at all. They were the results of specialists with skills but not 'art'. Yet the photograph as a product was displacing real art in the marketplace.
Both of Baudelaire’s criticisms were reasonable, and the same thing happened again when AI image generation showed up. You think the opponents look ridiculous because you’re viewing the history from the winner’s side.
As for “the public,” people had a real demand for photography as a way to record things, which is also why it won. There is no comparable public demand for proving Fermat’s Last Theorem.
> There is no comparable public demand for proving Fermat’s Last Theorem.
It's a capability demo, not the main point. There is plenty of demand for intelligence on tap. These flashy results are necessary though, otherwise people keep claiming AI is just hype, it can't do anything etc. So they need something to point at. And it seems to be working because the anti-AI narrative has changed from the previous stance that it's just the new NFT that will simply go away to how it will replace human intellectual work and will be bad for artists and desk workers.
Genuine Art versus Mechanism, from 1901, (https://www.jstor.org/stable/25505621) is another article that I read a few years ago that other people might find interesting.
Tao doesn't go as far as Baudelaire, but there are some similarities. In particular, Tao has criticized that AI is not being used to create new interesting conjectures, and that the rush to prove old conjectures is not giving human mathematicians enough time to carefully analyze and understand the proofs and the methods used in those proofs.
My answer to both is the same: nothing stops mathematicians from doing both of those things, with or without the help of AI. And we all understand that it will take time to do that. But complaining about the dawn of a new era of advancements seems counterproductive.
But the point is your original analogy with Baudelaire now just boils down to "both of them were critical" which is quite shallow indeed.
It's especially annoying given that many people are making repeat Baudelaire's critique of photography (or Socrates critique of writing) in the context of AI and the declaration is interest because it's not that.
There are no similarities whatsoever, other than them having opinions on a new tech that affects their field.
> nothing stops mathematicians from doing both of those things
Actually yes, and they explain that pretty clearly in the letter. These problems have been used as "goals" not because the solutions are necessarily worth it, but because they set a clear direction whose process would probably yield very useful theories, and also more conjectures and problems. If the goals get solved by AI models that do not generate comprehension, then the direction gets less clear, it's harder to collaborate and get funding (think what would be easier, to get funding to try and solve Navier-Stokes or to try and understand an already proven Navier-Stokes?), and we get less open problems to work on.
It all comes down to this. Yes, they will need to give a good and convincing argument to the taxpayer. And at the end of the day, if the circumstances change, and you're unsuccessful in making your case in your plead, the taxpayer and their representatives may not want to keep the same level of funding and instead put that money to use elsewhere. So make a good argument why your job is still worth that cash under the new circumstances. The argument can appeal to aesthetics and art and philosophy and a kind of feeling of religious elegance and so on, it's fine. I can't say how well that will work, but it's worth a try. Taxes also fund theology and other pursuits with no tie to cold hard engineering etc. But that level of funding is not the same as for math currently. That gap may shrink.
I don’t get the argument you are making. If you want to reduce funding for mathematics research, argue for that. But wanting to make getting funding and coordination and collaboration harder in exchange for nothing other than PR for AI labs does not make sense
Is it just PR? Okay then the useful math can be done by AI and we can leave aesthetic math to human academics who will ascetically refrain from using AI and if anyone of them submits a paper and it turns out they used computers they will be treated like WADA treats Olympians on steroids or chess players with a vibrator up their ass telling them what move to take. But again the taxpayer may not foot the bill for this endeavor as much as they did while thinking it may be useful for technology, science and engineering innovation.
But did you read the letter, or are you familiar with the people signing it? Several things tell me you didn’t:
- The useful part of math is the tools, the understanding and the theories. Knowing whether the result is true or not is usually not useful. Imagine an AI that just says “cancer can be cured” but doesn’t say how: that’s what the letter is criticizing.
- Terry Tao is one of the biggest proponents of AI use in math. He’s been for quite some time and he shares a lot on his blog.
Seriously, it seems like you’re arguing about some idea you have in your head and you’re not making the effort to actually understand anything about this letter.
AI won't just say cancer can be cured without saying how. Not sure where you got this from. AI will be superhuman at writing the executive summary at any prerequisite knowledge level.
These people are concerned about internal community dynamics like tenure, chairs, PhDs, grad students, the whole ecosystem, papers, hiring, conferences, journals etc. But the wider population doesn't care about this and academics don't understand that nobody cares about this outside their bubble.
As Feynman said, "mathematicians can research what they please. If you want something else, you work it out yourself" (not verbatim). Indeed, now people can do this with AI and it makes mathematicians salty that they will have less prestige.
> AI won't just say cancer can be cured without saying how.
It was an example, an analogy to argue that answering a question might not be as important as how it arrives to that question. Hence the “imagine”.
> These people are concerned about internal community dynamics like tenure, chairs, PhDs, grad students, the whole ecosystem, papers, hiring, conferences, journals etc.
You just described most of the parts of the mathematical community.
> But the wider population doesn't care about this and academics don't understand that nobody cares about this outside their bubble.
I don’t care about the logistic system and yet I want my food to be in the market. If the population at large wants to use mathematics they will have to find a way to support a mathematical community. Now, if you don’t want mathematics, again, go argue about that.
> Indeed, now people can do this with AI and it makes mathematicians salty that they will have less prestige.
Except that they can’t. I do not know how many more times we need to explain that “answering questions” is not what mathematics is about. That if that path is followed, then mathematics will suffer as a result and the pace of mathematical development will slow down.
Honestly, I think it would be better for you to really stop attributing the arguments of mathematicians that are by no means anti AI to saltiness or other childish feelings. Try to be more humble and try to find what kind of ideas they might hold for them to argue this letter. Of course, that requires reading the letter and trying to understand it.
> You just described most of the parts of the mathematical community
Just as science (as in the scientific method) isn't the same as Science, the community social practice and logistics and tenure rules and committee compositions and journal page limits etc, mathematics is not the same as the academic mathematics community. They are not the sole producers of mathematics and not the sole users. The funding is largely in the hopes of the usefulness of the resulting math. Not all. Some funding is like art and culture funding, for propagating a cultural legacy, like folk dance also gets funding and experimental performance theater also gets some tax funding. But that's not the bulk currently in math.
I have read the letter. It's all about stuff of holding back because they don't want the answer key because prestige reasons, and how will grad students train their brain if we have too many answers. This doesn't consider that many people, like engineers, use math as a tool. They don't want to hold math back. It's fine to think that applications are too dirty. Again, they want funding, they have to explain why exactly they should get it. And it better be an explanation that still holds water with AI available. They can do the aesthetic math as an art side project like an accountant can paint or sing off the clock. But generally you also don't pay everyone for their hobbies even if those hobbies are nice culturally rich endeavors.
> they don't want the answer key because prestige reasons
No, that’s not the reason. They don’t want the answer key because the answer key is useless.
> This doesn't consider that many people, like engineers, use math as a tool
Of course it considers it. Many people, like engineers, could not care less about the answers to most problems mathematicians work on. They do care about the tools they develop in the process. I’ve already shown examples of this but I’ll do it again: Galois theory was developed when trying to answer whether there are formulas to solve roots of polynomials of degree 5. Fourier analysis was developed when trying to find an analytical solution to the heat equation. Riemann developed his geometry to explore which Euclidean axioms were actually important. None of the direct answers were as important as how they got to them.
> Again, they want funding, they have to explain why exactly they should get it. And it better be an explanation that still holds water with AI available.
The explanation will be exactly the same. Only it will not be just hard to explain why the problem is important, but also it will be harder to explain that no, just the answer by itself is not useful without the understanding. Just like I am here having a really hard time explaining to someone who doesn’t understand how mathematical research works why “just getting the answer” is not a useful output.
This hangs on the strange assumption that the AI will "just give the final answer" and will be unable to give a human-preferred explanation. I find this assumption strange. If it holds, then mathematicians will still have a job digesting these proofs. If the AI can digest it better, then academic mathematicians will become humanities professors funded similarly to theology, philosophy and dance professors etc. for the sake of preserving a cultural practice, like traditional handmade broom makers etc.
Why is it strange? Are you familiar with how unsupervised proofs are made? It’s basically iterating on Lean code, even if it were able to explain the code to humans it doesn’t mean it can extract actual understanding and structure from it. The papers being outputted by these approaches are nowhere close to being understandable, much less useful.
> If the AI can digest it better, then academic mathematicians will become humanities professors
Cool, then once that happens we can discuss what to do. In the meantime, AI does not digest proofs properly, does not explain them and does not generate any understanding, and it doesn’t look like that’s going to change. Hence the letter and the criticism made to the approach taken by AI labs. It is not that hard to understand.
I find these discussions pointless. It's like complaining that diffusion models make hands with the wrong finger counts. Yes there was a year or a few months when that was a legit complaint. The phase where AI proves Millennium Prize level problems but can't write up a human-like paper explaining the proof strategy may be similarly brief. It feels pointless to build a grand theory of what AI can fundamentally do in the space of math proofs based on a few months of existence of such strength models.
I find it interesting how many people are incapable of imagining that the tech capability will not be frozen at today's level and where we were e.g. a year ago and that a similar change may happen until next year. Instead they make sweeping assumptions that the current limitations will be with us for our lifetimes. You need a much stronger way to adapt to the new reality.
> The phase where AI proves Millennium Prize level problems but can't write up a human-like paper explaining the proof strategy may be similarly brief
It's absurd to assume that because LLMs are improving in certain aspects they will improve in everything, specially when the part they are lacking in is not precisely something that would be a strength of their architecture. I mean, models have improved a lot but they are still not good at strictly following instructions consistently (there's a reason why AI labs are worried about safety alignment). They are still mediocre to bad at software design, even the latest models (haven't tested Astra seriously yet). And it's the same reason they are bad at creating mathematical theories: they do not have mental models like we do, it's not even useful for them. Their comprehension is limited to textual context that they need to refresh and reprocess continuously. That's just how LLMs work.
> I find it interesting how many people are incapable of imagining that the tech capability will not be frozen
I find it interesting that after taking this long to, I assume, finally understanding the point the letter was making, you automatically switch to "oh well AI will do that too".
> But complaining about the dawn of a new era of advancements seems counterproductive.
you completely misunderstood the critics. your analogy is awful. this is much closer to the industrial revolution in the uk: it brought a lot of progress, but also extreme inequality and concentration of power.
It's not about whether the conjectures themselves are interesting or not, or whether people simply have enough time. It's that a bare proof made by a machine doesn't actually do much for us. There is not some set of problems that, once finished, will amount to some kind of final, correct system and we can call it a day and, like, utilize it. "Mathematics" is the people doing it (the "mathematical community" Tao references below). This other stuff is kinda just.. expensive exercises to render a result. They are only actually beneficial to us insofar as they exist in a context of research among peers.
You have expressed a few different ideas, so I will address them separately.
> It's not about whether the conjectures themselves are interesting or not, or whether people simply have enough time.
Tao seems to think otherwise, if I am reading him correctly:
I wrote recently about how the collection of good, fruitful open problems is now being mined in a non-renewable fashion, leading to the potential scenario of these problems becoming scarce [0]
Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions. [1]
> It's that a bare proof made by a machine doesn't actually do much for us.
I get it, and I think the same can be said about all sorts of human endeavors.
> There is not some set of problems that, once finished, will amount to some kind of final, correct system and we can call it a day and, like, utilize it.
Sure. Although there are certainly practical applications to be found along the way. E.g. proving P=NP would be potentially very significant in the real world. I think we agree.
> "Mathematics" is the people doing it (the "mathematical community" Tao references below)
Sure. And the same can be said again about all sort of human endeavors. But I don't see how that is a reason to stop using AI in those fields, either. It doesn't subtract anything, in the same way that chess engines didn't destroy the love of the game for chess.
And just like in chess, these AIs can be used to gain a deeper understanding. Including, but not limited to, explaining to humans the proof they just came up with.
Are you trying to argue toward some final verdict with regard to LLMs and mathematics? I thought you were just trying to make a comparison to Baudelaire? I think maybe being clearer on this point would help. Even if you argued sufficiently for the latter (which is going to be tough already), it wouldn't really speak to the former. Or at least: that would have to be a separate argument I think.
Also, how, in your words, do you feel like the first quote justifies your point (presumably with regard to the question of "interesting" or not)? And why do you think the second one is more about time itself rather than attribution? Do these things actually contradict the letter above (or the comment on it) in your mind or not?
In general, do you disagree with something here specifically? Or is it kind of a yes/and thing? Does any of this help, in your mind, with the Baudelaire comparison you were at least at one point trying to argue for? Its a bit hard for me to see the argument here, if there is one, just with what you have written. But I am sure I am just not knowledgeable enough to grasp the argument!
My opinion is that the field will adapt. AI will not spell doom for mathematics. On the contrary, it is the beginning of a new era. Math is having its Deep Fritz moment, just as computer science is going through the same.
Some folks are struggling to adapt to this change. Tao actually sounds like he is doing alright compared to most, even if some of his arguments seem a bit weak, as I alluded to in other comments in this thread.
Hopefully it makes some sense. And if it doesn't, at least we had a nice chat.
Haha yes I imagine we all know your opinion is something like that! But isn't it much more fruitful and interesting to form an argument for it? If he is still in the context here: is there something about Baudelaire's critique you were discussing originally that has helped you arrive at this opinion? Do you think he is wrong? Or is he right, but providing the right kind of nuance that we need in order to cope with the revolutionary changes? Genuinely curious!
> But isn't it much more fruitful and interesting to form an argument for it? If he is still in the context here: is there something about Baudelaire's critique you were discussing originally that has helped you arrive at this opinion?
The historical record. That's why I am drawing some lose parallels with Badulaire. Incumbents being unhappy about a disruptive technology, lashing against the early adopters, and fearing that it signifies the end of their craft, when in reality it's just a period of change and adaptation. Without the advent of photography we would not have Impressionism nor all the movements through the 20th century. Photography forced painters to reinvent themselves, and LLMs will force mathematicians to do the same.
I thought the historical examples of photography and chess engines would be enough for people to connect the dots, but apparently not.
Sure, but then it's kinda less that you're making an argument here one way or another, and more of just an appeal, right? Like its just broad strokes? The inherent actual qualities of photography aren't supposed to parallel anything with the LLM case other than just the bare fact of being a new technology at one point?
I guess I am sorry for pushing, I felt like maybe there was going to be something more specific to the point.
I think one thing to keep in mind in the future is that for every Baudelaire, there is a Benjamin or Italian futurist saying the exact opposite! Its not like the fear of photography was a monolithic, shared thing, even at the time. Just, it's not so straightforward to have the "historical record" speak exactly one narrative, and unless you are a full-on technological determinist (is anyone these days?), you can't really conclude anything from the past like this.
> I thought the historical examples of photography and chess engines would be enough for people to connect the dots, but apparently not.
You're just an overconfident idiot who thinks that because there is an analogy along one dimension of a absurdly complex problem you've somehow arrived at a superior truth.
I repeat, this is an incredibly complex situation in which no one fully understands what's going on. The letter is simply asking for greater awareness of the implications of what AI companies are doing, they're not even saying that AI is bad!
I don't see any meaningful analogy with your cites of Baudelaire. First of all, Baudelaire discusses art, which is very different to science. Several new forms of art have emerged, and then slowly integrated, despite the strong opinions of some. Here, Tao's letter discusses mainly practical aspect of scientific research. It is not about what a "failed mathematician" would do, or if "LLM can only record what already existed"; in fact, it ackowledges that LLM could become a central tool. If you want me to be even more precise, what they're saying (without saying it out loud) is that tech companies have too much power and are being irrespondible with it because they don't even think about the externalities.
> It is not about what a "failed mathematician" would do, or if "LLM can only record what already existed"
This is what I was alluding to:
I wrote recently about how the collection of good, fruitful open problems is now being mined in a non-renewable fashion, leading to the potential scenario of these problems becoming scarce [0]
Of course there's a meaningful analogy. The poster wants to make Tao sound unreasonable and against "progress" while ignoring the very real concerns Tao has.
It's what fanatics do when they want to enforce their view on the world, they have to attack anyone with a reasonable viewpoint because they can't imagine a world where someone tells them they don't like what they're doing.
I do not want to make him sound unreasonable. He is a brilliant man. But I also think that he is a bit shell shocked, much like other luminaries in the past have been when confronted with rapid technological development, so I wanted to highlight some historical similarities, imperfect as they can only be.
As for the rest of your comment, I hope you have a great day.
He’s not shell-shocked. He’s had an extremely consistent narrative for years now. You are experiencing a bias where you project your own views onto others, probably because you are not very informed on this.
Tao is quite the opposite of what you describe. He's had an ongoing dialogue intellectually with the challenges created by ai for mathematicians for some time (years) now and his viewpoint is quite balanced. He is not alarmist but rather correctly addresses the real problems ai advances create for mathematicians traditional way of working.
> But I also think that he is a bit shell shocked, much like other luminaries in the past have been when confronted with rapid technological development
You're locked into that idea, but provide no real argument. The dangers they describe in the letter are real.
Portraits are not an artistic medium in the same sense, they’re something one creates with the tools of a particular medium. Painting is a medium, photography is a medium, both are capable of producing portraits.
An LLM can produce a portrait of you as well by generating a facsimile of you based on existing material (or functionally a “guess” if you describe yourself I guess).
People aren't ready to discuss AI assisted imagery as art yet. Most discussions lack the nuance that Baudelaire lacks in that critique, which deals with the nature of art and the importance of human intention and input.
We can’t look back with perfect hindsight because both the past and present have deeply ingrained blindspots. They don’t know what it is like to live in a world with perfect edges. We don’t know what it is like to live in a world with no edges. We can read about someone who proclaims that “something will be lost”. We will just think “but I have no need for any of that.” But we don’t even know what it is.
I don't think the analogy works because for the past few years Tao has been one of the most vocal advocates of AI in mathematics and has used it extensively in his own research. You can find several of his talks about this on YouTube. It's completely consistent to believe two things at once, that the tools are useful and that the companies are misbehaving.
Wasn’t the criticism of photography kind of correct though? I don’t think people think of photography as an art as much as painting is an art.
There aren’t a lot of photographs that a layman couldn’t in principle also take, but only a few people could replicate a good painting.
Keep in mind that his critique is very recent, and likely applying to a specific use of AI, as opposed to AI as a whole. If you've been following his Mastodon account, he's been happily using LLMs for math purposes for well over a year.
I'm really tired of these arguments (this and "it's just like calculators").
Photography decimated other forms of visual art, so the concern wasn't wrong. But AI threatens the entirety of human intellectual endeavors. I can make do without oil paintings in my home. I'm not sure I want to live in a future where we make do without brains.
Lots of people still make bad music that other people still manage to enjoy (a lot of it has gone multi-platinum!) even though they're not Mozart or Bach.
Of course it is, because our society is pretty much structured to place sociopaths on top. Why is it at all surprising that any technological advancement is going to be used by them in a way that doesn't benefit the rest of us?
But it's equally obvious that this has nothing to do with the tech itself.
If the tech enables certain behaviors then it can’t be casually divorced from a discussion of how people use it. “Tech” isn’t inherently neutral. It’s made by people with intent for a purpose and how else it can be used must also be considered.
This is like separating “the use case” of killing people from guns and still trying to have a coherent discussion about them. You simply can’t handwave it away with “well that’s not the tool’s fault.” It’s unproductive and sidesteps the conversation.
It did. While you're busy recording an "instagramable" moment, you are not entirely enjoying that moment. Your eyes are on the screen, not on the subject.
In a way, if photography is an ersatz for painting that eventually made imaging available for the masses, then AI could become an ersatz for thinking. But it feels like I'm paraphrasing TFA.
What will happen when AI companies have spent their advertising budget on math problems and whatever else gives the maximum wow effect for the bucks? Probably customers hooked on the vain satisfaction of spending token$ to impress friends.
Indeed. As Nassim Taleb says, "Something bad for the (existing) class of mathematicians as they lose their privilege, but not necessarily bad for mathematics & our body of knowledge?"
https://xcancel.com/nntaleb/status/2098498124419305639
This sounds a lot to me like people in the 90's complaining that computers were destroying chess. Thirty years later, chess is more popular than it ever was, and chess players are better than they ever have been. I wouldn't be surprised if there are now more chess books now than there ever have been. Furthermore, it turns out that a lot of chess books written before computers were just wrong about a lot of things. It turns out having an oracle for the "right" answer in chess, even without an explanation, used properly, allows humans to develop broader, more accurate insights.
The argument here sounds similar. The fear, as I understand this statement to be saying, is that by being given the correct answer, in the form of a 100-page Lean proof, humans will be robbed of the chance to from insights about the structure of mathematics itself. I don't see any reason that humans can't continue to develop insights as they try to digest the 100-page Lean proof into something more manageable; but with more certainty and fewer false starts.
I was apart of the "covid chess cohort" and got really into it for quite a few years. In that time, I also learned about what the chess culture was like pre-engine and post-engine, and I think you could argue that post-engine did kind of make the game more mechanical - and that kinda ruined things for people that grew up in the pre-engine era. it felt like pre-engine chess had a lot of mystique to it because knowing the best move was never truly clear. the only way to know what was best, was by: getting better yourself, socializing at chess club, learning from better players or coach.
this sort of lack of clarity is what drove people to learn more about the game.
like think about what it must've been like to go to a chess tournaments pre-engines: finals matches had everyone at your chess club watching the game, calculating lines with each other, and you HAD to calculate to understand the direction of game. it sounds so much more engaging and fun!
now, the top games have the stockfish bar next to it, and you instantly know who's got better odds without having to really follow along. and you're not encouraged to calculate beyond a few moves ahead because you just offload the real thinking to stockfish. honestly, it's a vibe a killer when you go back and see images and read about the culture beforehand.
I agree engines didn't really "kill" chess - and I think most of that is due to chess not being tied to economic value. but it did kill what I believe was a superior culture compared to today's chess era.
all this to say, I think what AI did to chess culture in the 90s is happening to stem right now with LLMs
>> I was apart of the "covid chess cohort" and got really into it for quite a few years. In that time, I also learned about what the chess culture was like pre-engine and post-engine, and I think you could argue that post-engine did kind of make the game more mechanical - and that kinda ruined things for people that grew up in the pre-engine era. it felt like pre-engine chess had a lot of mystique to it because knowing the best move was never truly clear. the only way to know what was best, was by: getting better yourself, socializing at chess club, learning from better players or coach.
I need to be very precise here, but it is not right to say that we know what the best move is in any chess board position. What we know is what move a chess engine would make that would win the game against a human, or sometimes another chess engine. We know a winning move; not the best move.
Winning in chess is not the same as knowing what the best move is. Having chess engines that can beat any human in chess is not the same as knowing how every possible game will develop. The latter is known as "solving chess" and we don't yet have that. We have fully solved games like tic-tac-toe, backgammon and checkers, so that there exist databases of every possible path through those games but this is not done for chess, and there is no hint that it is even possible to do given that chess is such a combinatorially more complex game than those.
More to the point, the upshot of the fact that we now have super-human chess engines that can beat any human in chess (and Go and shoggi) is that there is no longer the will to research the bigger question about solving chess. The solution of the minor problem, how to beat humans in chess, has sidelined and displaced research in the major problem, how to solve chess, and we will now never find the solution to the latter.
And if that reminds you of something, well, yes, exactly.
As a chess fan, 100% this. We have known for the last ~15 years who the best human chess player is, and that he will lose against stockfish on his phone. But chess survives because of the human characters involved, the rivalries and dramas, watching two people trying to overcome each other under insane pressure, and sometimes coming up with something astonishing. In short - it's a sport.
I guess part of the problem is that being against being against anything for economic interests doesn't really rally anyone to your cause; everyone has to make a living doing something productive for society, and professions have come and gone all the time due to technological advances. In fact, when one thinks about it, the people that are losing their professions now were major contributors to others losing their form of income. Often people talk about how they can use technological/programming/IT skills to make some secretary or administrative assistant's job obsolete. So most people just don't feel a lot of sympathy when people complain that AI are going to take those people's jobs.
That's correct. As far as I know nodody builds any kind of science or technology on top of chess, but mathematics is at the base of most science and technology. It would be horrible if we prevented AI from solving mathematics problems, just because mathematicians want to solve them by themselves the "hard way".
>> This sounds a lot to me like people in the 90's complaining that computers were destroying chess.
Genuine question: who were those people saying that? I don't know that criticism.
The criticism I know is from AI researchers and it is that beating humans at chess using a computer running an algorithm unlike anything that humans do when they play chess, tells us nothing about the way that humans play chess, which is what we are trying to understand when we try to get computers to play chess.
This criticism is exemplified by John McCarthy's article (the real godfather of AI; because he named it) "Ai as Sport" whence I quote:
Ideas about chess algorithms as well as advances in computer hardware were involved. However, it is a measure of our limited understanding of the principles of artificial intelligence (AI) that this level of play requires many millions of times as much computing as a human chess player does. Moreover, the fixation of most computer chess work on success in tournament play has come at scientific cost.
In 1965 the Russian mathematician Alexander Kronrod said, "Chess is the Drosophila of artificial intelligence." However, computer chess has developed much as genetics might have if the geneticists had concentrated their efforts starting in 1910 on breeding racing Drosophila. We would have some science, but mainly we would have very fast fruit flies.
A more recent criticism that can be made is that getting computers to beat humans at chess (and later Go, and Shoggi and Atari and Stratego) has turned out to only be possible with techniques that are too narrow to have real-world application. Btw, this includes Reinforcement Learning which is still much more capable in virtual environments than in any real-world environment.
We like to think we are thinking beings that happen to feel, but we're actually feelings beings that happen to think. We don't realise how our emotions shape our arguments, and the whole Terrence arguments have, behind:
"I like my job and want to continue doing it. It gives me meaning."
The brain then goes and fills in arguments that support these feelings.
> I wouldn't be surprised if there are now more chess books now than there ever have been
Well yeah... how would there be fewer??
But the point itself is silly. Few people are putting effort into Maths for the fun of it (and of those that are many derive fun from being the only one who can produce a solution). Chess differs in that it never had any point but the game its self.
But computers have destroyed chess as a "sport". Nobody will sit to watch two chess programs compete, or analyze their tactics. Kinda like how now, anybody can construct a "game" over the weekend or a new song or a slop video. The value of each of these decreases to 0 as the slop overwhelms.
>Nobody will sit to watch two chess programs compete, or analyze their tactics.
I know nothing about chess yet I dare say that I'd doubt this. Surely chess enthusiasts would be interested in analyzing how a superior chess program came out victorious, no?
Yes you are right, some people do watch chess engines play. TCEC (Top Chess Engine Championship) [1] streams them. Popular chess YouTubers goes over engine games from time to time too.
That's not proof of interest so much as proof of commerce. It could easily be a money-laundering mechanism.
Mostly nobody cares about professional chess. The number of people who are actually interested in today's game and not the drama are a tiny sliver of that. This is actually great because it means we can train and evaluate both without interference from chess players, possibly even building a stable society.
This is a PR problem, not a mathematical problem. It's possibly the worst PR problem mathematics has faced since the execution of Hippasus for whistleblowing on the cover-up of the regular dodecahedron. It's still a PR problem.
So what went wrong? Mathematics education. Math below grad school is all about solving stated problems. Credit is given for solving puzzles successfully. Homework is problem sets. Everybody below a very advanced level is taught math that way. Even at the higher levels, puzzles remain important. Awards in mathematics are often tied to solving puzzle-like problems. That's still the criterion for becoming Senior Wrangler at Cambridge, "the greatest intellectual achievement attainable in Britain". This despite Polya's attempt at reform a century ago. Puzzle solving gets good grades and class rank. So it's the status indicator mathematics presents to the outside world.
Then reasonably good AI comes along. AI has become rather good at solving puzzles. So people aim powerful AIs at known hard puzzles, with some success. That blows up the status indicator system. Mathematics itself is fine. It's the status symbols that have a problem.
Maybe the Fields Medalists need to hire a crisis management team to reframe what success means in mathematics. That's what they're trying to do with that letter, but they're mathematicians, not PR people, and they don't know how.
In general, in our existing society, status matters, and it is tied to achievements. Therefore a thing that devalues the achievement is also causing people to lose status, and everything that comes with it (income etc). Of course they are mad.
But maybe the answer to that is to acknowledge that our civilization is basically a dick measuring contest at scale, and try something else for a change? Starting with the part where people don't have to be "productive" to have intrinsic value as a person?
That’s a charitable way to interpret what they’re trying to do in this letter. I would say they are trying to maintain the status quo to protect their status system.
Very interesting analysis. I was just reading this announcement, and thinking, that is the most pointless thing I've ever read. What are they trying to achieve with it? Your comment explains it.
As a consumer of mathematics, I want to attack the author’s idea of “what that work was meant to achieve in the first place”.
I do a lot of programming that uses mathematics. I designed the error recovery file format “par2”. I worked as a quantitative trader.
What do I want from math? To be able to search math results, understand them, and apply them. But academic mathematicians are so inwardly focused - as these authors so obviously are - that it does all of these badly! Search is still with words, not symbols and formulae. As for comprehension, many math papers are unreadable to those not in the cliche of mathematicians in that specialty. And even applied math papers don’t ship code or specify the operations explicitly.
I wanted to fix this and in 2013 I worked on formal proof. It makes symbol search and application easy. I found out that the tools had been usable for decades at that point. But mathematicians didn’t use them because they had that inward focus and didn’t give a damn about math consumers.
AI is enabling formal proof. It will carry mathematics into the computer age, which the field should have joined with every other industry in the 1970’s! I welcome it. Math will get used more and that is what the work was meant to achieve in the first place.
One key idea to understand about human behavior is that people are focused on their status among their peers and their next level superiors with power to lift their status. You are not among either, so you don't matter to them. This is nothing special about mathematicians, though, people work the same way everywhere, in every field, community etc.
Yes, that’s very true. I recognized that in 2014. It was hard, but I stopped doing research on formal proof.
I got an economics degree and found that the City of Austin to pay off its $3 billion dollar pension debt … and encountered the same problem you mentioned, again. Except this time my plan is to run for City Council and be their next-level superior.
different fields have different communication cultures, that’s not inherently bad. sacrificing inward focus is expensive from the pov of the insider; i see your points but not everything can be written for a general/wide audience. after all, the audience of the author are the specialists, since only specialists spent the effort to be able to engage with the work in depth. perhaps we disagree that mathematics is not a product that mathematicians are making?
what the work was meant to achieve in the first place is what the work was achieving in the first place. it was developing mathematics. if someone wants to use it, they are free to do so, or to write a blog article how it can be used by non experts. mathematicians do that themselves sometimes, but it’s not their job. myb there should be more incentives for them to do more of such work
i keep seeing this myth that mathematics has some purpose to do this or that, but somehow its purpose is least known to mathematicians, everyone else seems to know better
> what the work was meant to achieve in the first place is what the work was achieving in the first place. it was developing mathematics.
Meant by who? Were the people footing the bill aware of this? People can like and do a lot of things. When money enters the picture it gets more complicated because usually the people who pay do that because they derive some gains from it. Historically scientists and mathematicians were employed by various patrons, sometimes doing astronomical calculations for horoscopes, Easter calendars and whatnot. Pure for-fun math was usually done by already wealthy individuals and not salaried employees like today. So it's fine that mathematicians feel this or that is their purpose, but they are coming to us asking for tax money, and so there is a bit of burden on them to be convincing. Before AI they used to argue how it helps physics, engineering technology, computer tech, optimizing logistics and telecom, cancer cures etc. Now the tune changed and it's retroactively now obviously just pure recreation and spiritual elegance. I also like math don't get me wrong. I think it's a great intellectual pursuit. But it's arrogant to expect that the money can flow by default.
mathematicians are primarily paid for teaching, and that's their primary function from the pov of the general society. i don't think anybody is debating that part of their profession? i read somewhere that the annual nsf grant budget for mathematics is 57 million, in total. even if it was just spiritual elegance, that's a reasonable number no?
Yes, that amount seems fine if true. They may be seen as teachers by the outside world yet they hire and promote and award internally based on research. Status and prestige is rarely allocated based on teaching. This is already a discrepancy. And note that the post doesn't much talk about teaching courses, but mentorship of grad student researchers who already work at their labs.
my response is not to the post, but to the parent comment i replied to. the internal structure of mathematicians is their thing, if someone should step in, who? do you think that discrepancy matters?
You mean who should pay? Whoever they can convince, of course. Historically it was wealthy patrons, royal courts, etc. Teaching indeed used to be the main thing a professor was expected to do, this changed in the 19th century with Humboldt's model of the research university, with the explicit purpose of not just passing on knowledge but constantly producing new knowledge as a core part of the operations. It could go back to the teaching model, if that's acceptable to them. Then they would have to do the research part on the side, as a hobby, like it was back in the day.
In case of public universities, the mathematicians would ultimately have to convince the public, but of course in practice that means politicians and national grant agencies, who indeed report to politicians and the politicians chase popularity to the extent needed for reelection, so public sentiment has some relevance.
Generally when you come to someone holding out your hand asking for money, being humble is a good first step, instead of getting angry at people for not sufficiently appreciating the big myterious value of what they are doing. Explain the spiritual value of the pursuit, they may or may not be convinced. This may be called a deficiency of democracy if your aesthetic values are different, but typically people don't like to fund abstract disconnected things, like expensive space travel unless it's embedded in some tribal jockeying like during the Cold War. In normal times people just want research that has life enhancing practical outputs. Nevertheless currently the West is structured as electoral democracy, so convincing the plebs is indeed necessary, no matter how lowly that feels from the ivory tower.
> In normal times people just want research that has life enhancing practical outputs
what about people who study birds, or say, olms? people want that kind of research? the people who are doing it surely do. astronomy and eg star watching in deep space? i could go on for days
people like doing mathematics, similarly to studying endemic organisms or other stuff of no direct utilitarian value
how many parents are happy to pay the fraction of taxes that go into something that their children might want to choose as a calling and fall in love with?
it’s okay to discuss whether something should be publicly funded, using “people want x kind of research” as a supposedly obvious or common sense argument is fallacious in the sense that it implies an idealised steelman model of the general public. that model is divorced from reality
> We are witnessing a general threat to intellectual work
This is the crux of it and goes far beyond Mathematics or Computer Science. To get a bunch of humans to do anything, you have to motivate them. Kleos and Timē; renown and stuff. These AI companies threaten to rip this away from everyone but themselves, and this recent millennium prize is the perfect example.
Solving this problem as a human would have led to tremendous Kleos; my name would be written in the annals of mathematics, lecture tours of praise were mine to be had for the rest of my days. This one victory would have earned my recognition throughout history. The greatest a mortal may hope for. Ripped away.
It would also have given me great Timē. The prize money, the professorships, the book deals. Gone.
If all hope of “renown and stuff” in the intellectual realm is now taken by the AI companies, they will remove all human motivation to pursue these endeavours.
Perhaps the glory will come from slaying these fell beasts.
> But there's still play. There's still curiosity. And there's still the drive to understand something for yourself.
Yes, but think about what that implies if those are the only motivations left. Gone are the professions. Gone are the ambitious.
There is plenty of space for people to work on intellectual pleasure pursuits (as there is with art and music), but the death of all intellectual based industries is still something to avoid. Or to mourn.
It’s an interesting view-point. What are the “right reasons” to pursue an intellectual discipline? To selflessly further the total sum of knowledge?
> No reason to mourn them leaving
What field? Who’s leaving?
The conclusion has to be that human involvement in these pursuits is facing existential crisis. “Teach me, oh Oracle, the mysteries of the Universe”. “For a fee”.
Both the glory and joy of discovery now belongs to the machines (the owners of those machines). Perhaps that’s fine and well, but I read Anathem and feel the strong yearning to be the intellectual monk. And yes, to fight and debate and think; and obtain my place in history. The history of man.
"To selflessly further the total sum of knowledge?"
Yes. I think you are saying this sarcastically but I'm not sure... This is why you should get involved in research at the frontier.
"What field? Who’s leaving?"
Mathematics, and I'd extend it to other academic disciplines. Do you care more about the results and the knowledge or about who gets credit?
The drive for "glory" has many negative effects in academia that do not get enough recognition. Just look at the replication crisis in psychology for an example. The fact p-hacking was rampant among psychology "researchers" was well known even in the 90s. I remember my professors in grad school for physics joking about it way before it became a mainstream-ish story.
So why did this happen? Because there was no one with institutional power in psychology who both understood the problem and cared. No one who was interested in the furthering the "total sum of knowledge" rather they were interested in your "glory."
Yeah. My other favorite example are books. Why do nonfiction books exist? There are some pathologies and corner cases, but fundamentally: to develop and share new ideas. Downstream from that, if it reads well and if you're lucky, you make some money.
But now, LLMs can generate hundreds of books per hour. They make up 80-90% of new arrivals in many nonfiction categories on Amazon. They short-circuit the system, allowing their "authors" to extract money from the system with zero effort by crowding out human work. And it's not even the question of whether these books are good or bad (although overwhelmingly, they're terrible). It's whether it's actually accomplishing anything worthwhile, or just destroying incentives for humans to write or go into any other sort of intellectual work.
In fact, I see many professions push back. Artists, writers, now mathematicians. And I'm amazed that our profession doesn't and that we have so many people who are hooked on vibecoding. I'm still waiting for that 10x payoff. All this velocity and somehow, the landscape of the software I want to use still looks the same as it did in 2021.
It doesn't, that's the whole point. You can produce output that passes the smell test with naive buyers in a matter of hours. If you want to write good LLM books, then you gotta work closer to human speed - weeks, months - which allows others to produce 100 slop-books in the same timeframe. You still lose.
If an author let AI do 90 to 95% of the work, cleaned up the remaining 5 to 10% themselves, would you say AI or the author wrote it? Because this is roughly the split between an author and an editor, with human writing, now.
The work is either: (1) the act of concealment: generating an AI book that cannot be detected as AI generated; or (2) actually just writing the book yourself.
Playing a devil's advocate. Why do we need understanding ? To take an example i would say ~99% of the population do not understand how combustion engines or how semiconductors work, what say another 1% ?
Is the fear post-apocalyptic in nature ? We need some human priesthood to carry on tradition ? why ?
Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
There ought to be more to life than sitting in a pod receiving sufficient nutrients from a tube, even if there was no doubt that humanity would in this way survive until the sun expands and makes the earth uninhabitable.
> There ought to be more to life than sitting in a pod receiving sufficient nutrients from a tube
Even in a hypothetical world where AGI can do everything that humans do better, this would still be a choice to make, not an inevitable consequence for everyone.
And life doesn't require suffering to have meaning.
> Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
I'd argue that this extremely extreme scenario is the only one in which it kind of makes sense to not have understanding. But let's be honest: no one knows if we'll be there (and it seems unlikely since everything reaches a plateau eventually). So, what happens if we allow ourselves to forget everything and then we don't reach the ideal scenario?
I grew up in a cult. Based on my experience, I believe that the most dangerous thing a human can do is to allow someone else to do their thinking for them.
It’s not going to be ubiquitous? There hasn’t been a single frontier model where generation n costs less than generation n-1 to run. So the reasonable thing is to assume that GPT-7 will cost even more than GPT-6, and more and more of the frontier of knowledge will be locked behind a giant paywall. Participating in any field will mean ponying up to the oligarchs that own the infrastructure that runs the model.
At the individual level any mathematicians can, but the social group of “mathematicians” will not unless drastic social changes are made, because that's not how the system works today. Drastic social changes usually take decades, and funerals, to occur, so the reasonable hypothesis is that it's not going to change fast enough.
Mathematics belongs to all of us. If mathematicians get depressed about it, I don't give a hoot. Five years of guided computer-aided math could improve billions of lives or solve global warming outright through cheap clean energy.
This is a litmus test for the effective altruists in AI labs: fashion or conviction? The social pressure is to slow down and let the guild keep its norms a few more years. But what's clearly best for humanity is to push forward violently and leave them in the dust. A Fields medalist's feelings are not more important than the wellbeing of the planet.
> Mathematics belongs to all of us. If mathematicians get depressed about it, I don't give a hoot.
They are depressed because what's being destroyed is mathematics, actually.
> Five years of guided computer-aided math could improve billions of lives or solve global warming outright through cheap clean energy.
This is complete BS, math doesn't solve climate change or save life, that's a category error. The purpose of math is to help us build understanding of the mathematical objects and that's it.
Sometimes, after much distillation to the academic world, this understanding is later being leverage by other scientific field, but that's a very rare event when it happens. And it's not how other science progress.
> This is a litmus test for the effective altruists
EA has always been a lie to justify short term individual greed by hypothetical shared future benefits.
youre not wrong, but i think you are missing his base point. tao is saying that mathematics is a practice that the group collectively builds. students and new methods are built on top of what has previously been done, so its a community effort of research, experimentation, and knowledge sharing.
his core argument that is LLMs (and specifically LLMs owned and gated by corporations was my read), while able to solve problems, do not contribute to this practice of knowledge building. solving problems is just one piece, and the mathematics community ingests problems and new methods, iterates and thinks on them, and then produces new ideas, methods, etc. this is what he is defining as progress, and solving things like millenium problems are markers of this progress.
I published a substack about this just a few days ago [1], my core theory here is that we will absolutely have what I call a "highly productive dark age" in mathematics where knowledge vastly outpaces understanding driven by publish-or-perish incentives, but additionally this will lead to the loss of the skills necessary to understand.
The hopeful note is that I do think we are entering a golden age for the curious casual/semi-pro mathematician and for niche mathematics areas that won't get the attention of the top labs. Everyone is sprinting to solve the millennium problems, but this is a very exciting time to be in a sub-sub-field where you and 4 others are keeping things alive.
The existence of solutions to problems does not prevent you from solving the problems yourself anyway, if your goal truly is personal development. You're free to go solve Navier-Stokes yourself right now. You're free to manually do all of the AI work in any field, actually. You won't get grant money or prestige (which are not a part of conceptual understanding and insight) but you will get all of the conceptual understanding and insight you're after. You have not been deprived of it.
Except most people that get actually good at a field do so through their day job. By removing mathematician as a viable job, most can't afford to spend the necessary hours to become a good mathematician at all. We're going back to only rich people or those with a rich patron having a chance of developing themselves.
As a software engineer, 6 months ago. I was better than AI in essentially every way except raw typing speed.
Now AI has surpassed me in fine-grained problem-solving ability. It can write more reliable code than I can, faster than I can, provided it is given the right guidance.
But one thing that I'm still much better at is identifying technical opportunities and choosing the right tradeoffs.
My feeling is that frontier models are incredibly smart in some ways, but incredibly dumb in other ways
When I chat with Claude using deep technical language about distributed systems issues, the arguments it presents are mind-blowingly good. I worked with many skilled engineers on complex projects but the kinds of arguments Claude makes are on another level. It feels like it can read my mind because it already identified all of the relevant aspects to the current topic and it's incredibly persuasive. I would say it's even better at deep, nuanced technical discussions than I am.
When I ask it to implement a feature using technical language, it does a really good job and the solution usually works out of the box. No bugs at all 95% of the time even after adding 1000 lines!
But the problem it still has is that when it implements solutions, it misses so many low-hanging fruits/opportunities. Especially in terms of performance, maintainability, scalability and UX. It's like it sees and recalls all the relevant parts perfectly but yet somehow misses opportunities which seem extremely obvious to me.
It feels like AI has 0 creativity. It only sees the opportunity once I mention it... And once it sees the opportunity, it demonstrates deep understanding of the technical implications. I think what's surprising is that it understands the suggested solution so well, with such nuance, that I can't understand how it didn't see the opportunity and why it never seems to see it until I mention it.
This is very unhuman-like. There is no way that a human being with that degree of understanding of a topic would be presented with such highly relevant context and not make the connection.
This is exactly how I feel about the recent ChatGPT models. It feels like I'm working with an incredibly enthusiastic junior student who listens to everything I say, checks it against my entire codebase, and thinks through all of the possible connections 1000x faster than I can. But it doesn't spot the ideas first. It actually makes work quite fun, since I can spend so much more time in "idea space" instead of "implementation space".
Thanks for putting it in words. I feel the same about SOTA coding models. I get visibly frustrated by this and often find myself using curse words at Astra.
You're right it's very unhuman to miss obvious opportunities you mentioned. And what's why I think these models would continue sucking as long as they don't have human level general intelligence. They may get marginally better at these tasks but I don't think we can expect them to connect dots like humans before AGI.
This mirrors my understanding when I use Claude code for mathematics. I can have deep discussions with it and it can solve my hairy problems. But whenever we go off the beaten track into design new mathematics, it struggles to make conceptual leaps and find the right definitions. Once I give it my ideas, it is back to its super-human pace and top notch intelligence.
This situation suits me fine, since I am anyways more of an ideas person, than a crunching open problems person. But I understand the desperation of my colleagues who mad solving hard problems their identity.
> Once I give it my ideas, it is back to its super-human pace and top notch intelligence.
That's the real benefit of AI today: you can test your ideas almost as fast as you can generate them, in parallel even. If you have a crazy idea that is unlikely to work but has massive advantages if it does, you just set an agent to explore and prototype it while you focus your main attention elsewhere. And so on.
It's still GIGO, though! An AI can make a bad idea work (and will do so if you don't carefully prompt it to not be so sycophantic!) but it won't work well. Knowing when the idea is bad and should be abandoned is also the part that currently requires human judgment.
That said, given the pace at which models have been improving so far, I can't help but think that this is a transient state of affairs.
Yes very much the same situation in software development.
The people who prided themselves on raw puzzle-solving ability got hit hard. Those who are idea-driven and architecture-oriented feel like they got handed a superpower.
It's quite a big shock at the industry level because the entire software engineering job interview process at essentially all large companies was heavily biased towards well-defined, raw problem-solving under time constraints which is precisely the skill which AI has replaced.
Yeah and companies continue to use leetcode style interviews even today and software engineers keep grinding for them. It's nauseating to contemplate on.
It seems like a lot of the issue here is that these problems aren’t interesting in and of themselves, but they lead down interesting roads. It defeats the purpose if you solve them without getting any real understanding.
It’s akin to saying you’ve solved “pancake flipping” problems with a waffle maker, or “travelling salesman” problems with a zoom meeting.
Nothing is stopping these folks from continuing to study the problems and arriving at their own solutions so they can continue having whatever insights along the way.
Well, one thing is stopping them. There will be no more adoration for their genius.
If you truly do it for understanding and not the attention, carry on. AI should change nothing about your motivations.
> Nothing is stopping these folks from continuing to study the problems
My understanding is they are? And literally everything in this world is based around incentives. If you say “well you can continue to work on understanding, but your kids are going to starve” that’s not nothing.
The point isn't what happens after something is solved. Where is the motivation for humans to study/find a potential solution if your kids are going to starve while you do it? These things can brute force problems that have a clearly and feasibly searchable answer, but they can't make creative leaps. It will be a severe dark ages for mathematics if humans stop contributing.
Maybe an analogy of "why not solve all medical problems at once" is closer.
We absolutely want to, of course. But you extinguish an industry and the systems of training that supply it. It's hard to know if letting it go that way is right.
A lot of this would be public funding.
If AI is reducing the need for mathematicians in an area it makes sense to fund it less and fund something else that isn't solved.
Your comment perfectly illustrates why most are missing the point. Many people, including you, deeply believe that most mathematicians are chasing adoration of their genius, otherwise why would anyone care about such abstract work?
People like Grigori Perelman would baffle you, a mathematician who solved the Poincaré Conjecture, refused the monetary prize, field medal and continues to live a life of total recluse.
For most mathematicians their primary drive is chasing the unknown, not for anyone’s adoration, but to pursue their desire to see what lies in the beyond.
There is an implicit agreement that mathematics is funded, for the most part by the public, as a way to advance the state of knowledge and propagate (even if very indirectly) what has learned for the public good.
It was never about helping individual mathematicians demonstrate that they are individually good at math. It happened to work out that way, but it wasn't the goal.
Agree, my comment is rather about the fact that most work you do in school and (lower) university is not itself of interest, rather the purpose is to work out your brain. The more of your exercise you leave to the forklift, the steeper the cliff will be when you find a weight that doesn’t fit on forks. (That is, unless/until machines get good enough to completely replace people, at which point we have another few problems on hand, such as day-to-day purpose and justification of existence)
This is a good metaphor for treating the means as an end, thanks. And I agree with your parent comment that that is largely the misalignment that Terence is pointing out.
This letter is complaining that human understanding has been crucial to advancing of mathematics, and AI companies are not bothering with it. But the promise (and horror) of AI mathematics is that, if it succeeds, human understanding becomes irrelevant. That's the goal. So this letter's message will fall on deaf ears.
Keep in mind employees at AI companies are publicly stating that they believe they're risking a >10% chance of human extinction. They're knowingly risking the lives of every man, woman, and child to continue the work. The lives of their own sons and daughters. A person already rationalizing that isn't going to shed a tear for the careers of mathematicians. Just a bug on the windshield.
AI companies are alienating the communities they serve. Instead of a win-win dynamic, they are keen on a win-lose proposition. You dont win trust by one-upping your customer. This is unfortunate and suggests a lack of adults in the room. It also reeks of hubris and is all good when making profits is not a concern. But watch the narrative shift when there is an AI slowdown which is inevitable.
I agree insofar as this may fall on deaf ears of AI folks, but that the goals are differ is exactly the point of pointing out misalignment. To talk about this may be more useful for coalescing professional societies than for conversing directly with AI advancers.
I understand this stance and where they are coming from, but I can't help but think this sounds very analogous to engineers' arguments against AI-assisted and vibe-coding, especially with regard to cognitive debt. Yet the software industry is plowing ahead, reportedly pushing mountains of unreviewed code to Prod, and the world hasn't ended.
Of course, nobody's really comfortable with it, so this is also a forcing function for the industry to adapt and figure out new techniques to manage complexity and trust. I think the same will happen with Mathematics.
But it is also possible we will end up with three forms of Mathematics: the one we understand, the one we don't, and the one we don't understand but can prove to work. Kind of like magic -- with all the positive and negative connotations of the word.
It is pretty evident that these models will soon exceed our cognitive capabilities. Is it right to hold them back just because we can't keep up? Many of those discoveries will be so beyond us that we can't do anything with them, but that also means they can't hurt us. On the other hand, there could be many discoveries that we can parlay into practically useful applications, even if we don't understand them.
It really has. All of the places facing an unusually high outage rate are places that have seen huge growth in their service usage (Anthropic, GitHub, etc) which is to be expected. The rest of the world has been happily chugging along with coding agents for almost a year now and things seem to still be working just fine.
> All of the places facing an unusually high outage rate are places that have seen huge growth in their service usage (Anthropic, GitHub, etc) which is to be expected.
That's not true, many of these outages have been directly attributed to AI tooling.
> The rest of the world has been happily chugging along with coding agents for almost a year now and things seem to still be working just fine.
Many more services are now being attacked by AI agents that originate from all kinds of organizations including OpenAI, Anthropic, and many others. Unless you're purposefully being obtuse, I would not call that "working just fine".
So far for most organizations and services other than the ones I mentioned reliability has been the same this year as it was before. So I call that "working just fine".
How many major companies are pushing unreviewed code to prod? Seems like mostly an early stage startup thing and their bugs and outages aren’t going to make the news. I’m skeptical that even a majority of Anthropic and GitHub’s code new code this year was unreviewed.
OpenAI: "Our mission is to ensure that artificial general intelligence benefits all of humanity."
- Except the mathematicians who we'll scoop and cause existential dread among their entire field.
- Except the software developers. They'll need to become plumbers or live on UBI.
- Except the people in countries that can't afford the cost of AI tokens to keep up with the rest of the world.
Just keep picking off groups of humans for the "benefits of all humanity"... while building larger and larger disparities been the have a lots and the just have enoughs.
We're going to build humans a utopia but along the way we'll leave a trail of destruction because that's not our problem.
But its normal, and good, that technologal progress creates, and destroys some jobs.
Imagin a cheap, 100% reliable, self driving car would be released. Death from Traffic incidence fall by orders of magnitude
Would you argue it didnt benefit humanity, because taxi/bus drivers are nolonger required
I'm actually a AI optimist. I think it'd be great to have everybody getting around in self driving vehicles.
If all that AI brought resulted in just taxi/bus drivers being phased out of their jobs in a thoughtful way, then that would be more manageable at the society level. But we're talking about almost all sectors of the economy.
If the magnitude of changes that OpenAI and Athropic believe will be delivered with increasingly powerful AI (and robotics) comes in a time frame that significantly worsens a large proportion of people's lives, this is a different situation. Can super powerful AI not be developed in a way that minimizes such disruption?
lol right. The nonchalance in referring to taxi/bus driver jobs.
Remember when we all said it's a good thing when the coal mining jobs are going away and that they should all just learn to code? Maybe a little more of that energy right now.
> If the magnitude of changes that OpenAI and Athropic believe will be delivered with increasingly powerful AI (and robotics) comes in a time frame that significantly worsens a large proportion of people's lives, this is a different situation. Can super powerful AI not be developed in a way that minimizes such disruption?
It cannot. The problem is that everything gets automated, so there's nowhere for people to "transition" to.
More fundamentally, the problem is that capitalism is simply incompatible with this. Specifically: capitalism is a system of property rights which in the long run always results in concentration of capital in a few hands. What made it tolerable so far is that capital cannot produce wealth by itself - you need people to use it to that end. Which in turn means that you need to pay for labor, at least enough for it to sustain itself. So even if you pocket most of the wealth that is generated in the process, the workers still get the crumbs at least.
AI is not like automation in the past because its end goal is doing everything that humans can do. Taken to its logical conclusion, you get capital that doesn't need labor at all: those who own the robots live in a personal post-scarcity utopia, everyone else starves. A slightly better case is when everyone else gets a meager pension, just high enough that people have something to lose and don't try to burn everything down - this is why you hear so much about UBI from the likes of Sam Altman (this isn't to say that UBI is inherently a bad thing! but make no mistake, what they actually want would look a lot more like The Expanse and a lot less like The Culture). It "minimizes disruption" in a sense that there are no riots, and it might even be a material improvement for a lot of the globe, but it sure isn't an improvement for most people living in developed countries today.
So I would argue that, if anything, we need more disruption here, not less. All the way to the top.
We could imagine, as an extreme case, a technologically highly advanced society, containing many complex structures, some of them far more intricate and intelligent than anything that exists on the planet today – a society which nevertheless lacks any type of being that is conscious or whose welfare has moral significance. In a sense, this would be an uninhabited society. It would be a society of economic miracles and technological awesomeness, with nobody there to benefit. A Disneyland with no children.
There's an unpopular branch of mathematics which does not have infinities - finiteism.[1] The constructive version of finitism takes the position that there is no such thing as infinity, just arbitrarily large upper bounds. You can have theorems about arbitrarily large numbers, but you never get
1 + 1/2 + 1/4 + 1/8 ... = 2
The benefit of finitism is that it escapes undecidability.
The big objection to finiteism is that it's a lot more work. Infinity swallows many special cases. Proofs get longer without infinity, and most of the special cases are uninteresting. That's not a problem for AIs.
Someone may start up an AI and make it grind through Hilbert's program for putting mathematics on a fully consistent foundation, starting from a finiteism base. This is a huge, unrewarding job. Great for machine work.
It's not necessarily clear that this statement requires infinity, if you're willing to treat "... =" as a shorthand. You might prefer something like "1 + 1/2 + 1/4 + 1/8 ... -> 2" if it's more clear, where "->" means something like "gets as close as you like without ever getting further away than that", but really the "=" sign is already overloaded in all sorts of subtly different ways anyway, so there's not really any trouble using it here. Almost any rigorous definition you can write down of exactly what that statement means would not rely on the use of infinity.
If you allow infinite recursion, you soon get to Godel and undecidable problems.
Finite deterministic systems are decidable, because you can in principle enumerate all the states. The halting problem is decidable for deterministic systems with finite memory. It may be exponentially hard for some programs, but that's quite different from being undecidable.
(This is too long a subject to discuss here, and I haven't worked on constructive mathematics in many years. It's more practical than it was decades ago. You need power tools, which we now have.)
Limits can be defined within finitism as long as the end result is finite. Essentially it’s just a process which lets us get as close to 2 as we want.
A better example would be a limit that equals sqrt(2) which finitists would probably say cannot represent a real object because it is only defined as the end of an infinite process.
It seems to 'not have infinities' by ignoring them. How many prime numbers are there? How many digits does the square root of two have? I guess you can simplify life by ignoring that stuff and sticking to what is 7x8.
Finitism doesn't escape anything, it just gives you the illusion of safety. Any intellectually honest thinker should accept the possibility that 10 is a nonstandardly large number.
Im surprised about the sentiment in this discussion.
I totally see the problem Terence is describing. We are loosing a lot in understanding and focus if it continues like that. The solution found for Navier Stokes doesn’t have much „real value“ - but what almost always happened in the past when people worked on the difficult problems, these sparked new ideas / new theorems that broadened our knowledge.
Think back at your grad studies, figuring out a proof as homework was hard, sometimes incredibly hard, but while doing it we gained a lot of understanding how things work. Now asking AI for the solution and „just“ getting it, risks our understanding, our creativity and our ability to connect the dots with other territories. I see it in students nowadays, there is much less understanding, much less creativity in finding solutions. I truly think this „short-path“ solution with the „death of struggle is one of the biggest risks with AI already for human development
If the complaint is about AI just giving answers and not good understanding around them, or frameworks that lead to more solutions from them, then that means there is a gap that can be filled by human mathematicians.
So all the AI has done has actually made mathematicians lives easier and provided them with an opportunity to fill that gap. They should be using it to do that, instead of complaining.
In fact, I would bet if this gap were not present, it'd be an even worse complaint: Now we have absolutely nothing to do in the field.
You have a point, but I think a concern is that the environment in which to fill this gap is threatened since the unsolved problems have functioned as a guiding lighthouse and measure of progress. Frankly: will someone pay for mathematical research to understand things we already have proofs for?
Looks like mathematicians (like people in many other professions) have to redefine what their work means and how to define success. Hard to agree that a tool that can find a proof is detrimental by itself, rather it voids some assumptions people relied on previously
Yes if you don't think about the matter for more than 6 seconds you would indeed conclude that, and retreat to the comfortable cliché of "it's just a tool". Meanwhile, I'm glad that there are still people who _think_ about issues and ponder the consequences and reflect on things before they become a reality.
I am on this track too. If AI leads to advancements, objectively that's a positive (depending on the advancement I guess) but it's only when mathematicians realise they'll get beaten to every thing now that they're outraged.
Say AI becomes the best at everything. Best at chess/go, best at maths, philosophy, economics, romantic advices ... and so on. Then what's the point of thinking by oneself? Of talking to one another?
What's the point of being human if we dont do human things but entirely rely on AI?
I believe this is more or less these mathematicians' argument.
It has been best at chess for quite a while. Yet everyone knows who is Magnus Carlsen, even though at no point of his career he was stronger than the machine
Magnus Carlsen plays fellow humans at the game because people still care about human competitions. What motivation would a mathematician have for solving already solved problems by hand? Can you imagine someone spending years working on a proof for an already-proved theorem just in case it leads to new insight?
If insights and human understanding are more important than specific results, I don't see why people should stop producing insights and human understanding. "Working on a proof" in today's sense of trying to come up with a proof before others probably stops being useful, other ways of working will be needed
Of course not. I'm so tired of the chess analogy. It was always a game and it was always performative.
Research is fundamentally different. We are seeing the erosion of specific needs for thinking at depth. AI is the automobile for the mind. There will 100% be undesirable consequences and selective atrophy of cognitive abilities once prized. This is a loss. There's no getting the cat back in the bag at this point so long as the objective dimension of work, as in object opposed to subject, is held as the most important.
SWEs felt this same crisis late last year. Now it's the mathematicians. They won't be the last.
actually I think the whole point is we're finding that the chess analogy to math is in fact quite apt. Math is just an extremely complex game. Its rules can be written down and there are win conditions. It's really not far fetched to think that through information theory, a reasonable measure of depth and beauty to a definition or conjecture can be defined. Then the game is simply to maximize the number of such artifacts produced of high depth and beauty along with proofs of the relevant conjectures.
Philosophers I'm sure can debate this back and forth but it seems probable that some things we thought were ineffable are in fact quantifiable to a degree, and now we have the technology and the machines to bring that to the logical conclusion.
> What's the point of being human if we dont do human things but entirely rely on AI?
It's fun.
Well, it's not fun today because most of us have to spend most of our waking hours working some bullshit job for a living. But imagine that there's no bullshit job?
>> Once a robot can do everything an IQ 80 human can do, only better and cheaper, there will be no reason to employ IQ 80 humans. Once a robot can do everything an IQ 120 human can do, only better and cheaper, there will be no reason to employ IQ 120 humans. Once a robot can do everything an IQ 180 human can do, only better and cheaper, there will be no reason to employ humans at all, in the unlikely scenario that there are any left by that point. [1]
The answer may sound harsh, but we will not need human once AI reaches that threshold. Current IQ of AI is currently 130 as per Google.
Depends on who controls the machines and what they do. If the machines just serve a select few and the masses are left to fend for themselves, they can just make a secondary pre-AI-like economy where they do small scale medieval-like economic activity for each other. Because if you're poor and can buy no AI services but still need some shoes, your other underclass neighbor may specialize and make some handmade shoes out of some plants or whatnot, for which you give him some seashells you got from another guy for cooking a pot of peas. However the machines may come in and intervene and stop this kind of shadow economy and force you to... I don't know. To just sit in your pod, or die. Because I guess they won't allow you to plow a field and stuff, because that land is now owned by AI or AI-owners too.
There are too many uncertainties around who will be in control and with what kinds of aims.
This only makes sense if you assume that the value of humans to other humans is solely their economic output.
Which is certainly how our society is structured, but, does it have to be? Can we admit that we basically have a developmental trauma as a species and learn to live the sake of living, instead of for the sake of being "useful"?
> We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align.
This is about how good taste in both research direction and in design are essential to steering AI, but we have no plan at all for instilling that taste in students or practitioners in a post-AI world.
> The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing, and indicate issues that all of humanity might face: how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place.
Besides eroding taste and taste-building, this is about just how useful friction is as signal.
Everyone coding with AI knows it routes around difficulties like a river around a stone, which is not necessarily a good thing. It will do it tirelessly 1000 times instead of learning anything from it. AND if the AI does not fail in this, the human driver will get no signal, and never know it happened. This seems to be getting worse, not better.. my theory is that more models are cross-trained on cybersecurity stuff where the goal is success and the method doesn't matter. Fine for pen-testing, ultimately pretty bad for coherent code or math or physics.
Discrete tasks where we don't want to be bothered is a real use-case, but optimizing for it everywhere is terrible for the future of durable abstractions that we can build on and ratchet up our understanding with. Bad for the models too eventually! They can maintain a codebase with millions of special cases or juggle tons of free variables in equations, but that just encourages bad abstractions.. they have a ceiling for this too, even if it's higher than humans.
In internet culture there’s this phrase “Hydrogen Bomb vs Coughing Baby”, meant to highlight the absurd power difference between two combatants.
In almost any scenario even tangentially involving mathematics, twenty-five Fields medallists uniting to denounce something would be a veritable Tsar Bomba.
It should give you pause that here they feel like the ailing infant.
It reminds me of Frédéric Bastiat's Candle Makers' Petition. It satirised protectionism in the 1800s:
>We candlemakers are suffering from the unfair competition of a foreign rival. This foreign manufacturer of light has such an advantage over us that he floods our domestic markets with his product. And he offers it at a fantastically low price. The moment this foreigner appears in our country, all our customers desert us and turn to him. As a result, an entire domestic industry is rendered completely stagnant. And even more, since the lighting industry has countless ramifications with other native industries, they, too, are injured. This foreign manufacturer who competes against us without mercy is none other than the sun itself!
The petition is about a much broader problem: if we’ll demotivate people from understanding math, vibe math in a rush without making it understandable - math becomes useless
Basically with pic generation at least we all understand what is on a pic - some cat or something
With math it is trivial to generate a lot of stuff that is half-assed and just gives a plain yes/no but doesn’t get absorbed into human technology and better math
Basically it is not inly about the answer 42, it is about the questions, _meaningful_ _full_ answers and empowering people
By giving a short answer AI can burn the find and make it boring, hide the fertile ground around that answer that any human would’ve highlighted
If you have Einstein - it is not just relativity he did - you can continue asking him questions and he continues the research direction, teaches students, etc
With AI it is often 100% gate keeped (most of the time experimental AIs do the stuff, you have no access even for 200 bucks/month or more) slave that was killed after finishing - hard to “compete” with that and it is not very inspiring approach to researchers ;-)
Fair enough - maybe I didn't fully read the statement. Though you can ask AI questions about how things worked sometimes. Like here's me asking the google one how you derive E=mc2 https://share.google/aimode/dtHKAO9Nr0EYWq7JE
I've read it. It's all a kind of navel gazing, all about internal academia dynamics. They should put much more effort into justifying why the public should still fund them. I'm not saying there can be no good reason, but they have to make the case and can't expect that the taxpayer also cares about intra-academic dynamics.
We have an intellectual structure that has been largely the same irrespective of the level of technological advancement. The brightest among us think about hard problems. Even though all of us may not understand those advances, we all appreciate and celebrate them to varying degrees. Those who are able to comprehend have time to absorb, generate more discussion. In time, we honor those achievements that withstand the test of time or make big enough impact on the advancement of knowledge and/or society.
The process is as much a human experience as is the result. If we have dime-a-dozen results like this minus the process, where does our intellectual structure stand? We will just be a hierarchy of priests at the altar of the machine?
PS: Before someone comes up with an counter example of maths before and after calculators, that was a tool that accelerated the steps. This is replacing the brain.
> The process is as much a human experience as is the result.
Sure, but in your scenario, the process is the experience for a small group of people, while the result benefits everyone (perhaps not directly, but ultimately via a chain of applications). Which one is more important?
It's not at all a new dilemma either. Jacquard looms destroyed the "human experience" of manual weaving for craftsmen who spent decades to perfect their skill. They also made clothing orders of magnitude cheaper for everyone else.
> They also made clothing orders of magnitude cheaper for everyone else.
They also contributed to consumerism and use-and-throw culture and mounds of waste around the world. People were not mortgaging their homes to buy clothes. There was a spectrum.
Also, your example is of physical skill. Mechanized farming has lifted many out of hunger. All automation isn't bad.
However this is the last frontier of higher pursuit for mankind - using our minds. This is a totally different level.
Everyones outraged all the time. It doesn't mean anything anymore. It's that meme from years ago about the red ants and the black ants living in a box peacefully until someone shakes the box and they start trying to kill each other. They go after each other and not the one shaking the box.
I don't understand how that metaphor applies here, you seem to contradict yourself. The mathematicians are mad at openai. If the mathematicians are the black ants and OpenAI is shaking the box, who are the red ants?
The is no "the mathematicians". Some mathematicians, very influent ones, are mad at AI, for social reasons, because it shows problem solving is "easy".
Mathematics is fine and will be fine in the future.
Gotcha. In that case, the "ants" part of the metaphor is truly fitting since these people just like to comment online and are mostly insignificant. Unlike the ants, they can evolve and become the mathematicians or get a job at a frontier AI lab but then they wouldn't be spewing so many comments.
Don't worry about what they write, they just want to feel emotions from any news article.
This time, until they dont anymore. The next stage is people attacking the mathematicians and accusing them. The OpenAI fans come out to attack, then the masses will pick sides and it all just becomes a mess
Yeah, I'm so tired of people moaning about how huge swaths of Earth's ecosystem are being destroyed and rendered uninhabitable to humans, how secret police are murdering Americans in the streets, how the president is a child rapist who openly accepts bribes, how unfettered capitalism is destroying tens of millions of lives, how the United States is rapidly falling into facism, how civil rights are being systematically destroyed, consumer and environmental protections are being gutted, and entire generations are being intentionally cut out of the possibility of economic prosperity. YAWN! Give it a rest!
Man, the people who want to just get away with open corruption sure love you.
Hyperpolitics is one term for this. There was an article in NYT about the concept. Attention economy for political info causing burnout possibly same phenomenon as Gen Alpha surrealism and viral adoption, but no institutionalization to preserve the lessons.
You're looking too closely. Whether or not mathematicians are actually outraged makes little difference to how this article plays. Hell, lying would likely INCREASE revenue.
I’m sympathetic to the concern, but I’m still unclear on what the concrete ask is.
If the worry is that AI companies are turning open problems into benchmarks and potentially “using up” fertile mathematical problems before humans can develop the ideas around them, what exactly should the companies do differently? Also why does discovering the answers preclude humans developing ideas from them? I don't get why solving a math problem stops anyone from doing that?
Should they (AI companies) avoid training or evaluating models on open problems? Solve them but not publish the results? Delay publication? Only release proofs after mathematicians have had time to study them? Require some attribution or review process?
The statement makes a strong case that “maximize the number of solved problems” may be the wrong objective, but it seems much less clear about what behavior they actually want from OpenAI, Anthropic, DeepMind, etc.
I’d be interested in the most concrete version of the proposal. Without that, it starts to read a little like: "Please stop getting so good at our thing!"
> If the worry is that AI companies are turning open problems into benchmarks and potentially “using up” fertile mathematical problems before humans can develop the ideas around them, what exactly should the companies do differently?
Just stop doing that. Don't treat unsolved math problems as some cheap benchmark to beat.
Leave the math for mathematicians, and let them use AI in a way that helps the field, not in a way that harms it.
Puts the onus on the AI companies to provide a specific replacement mechanism, no? Unless I'm unfamiliar with something else he's written that proposes something more specific and constructive
To Tao’s credit he obviously identified the problem very clearly and admits understandably "we did not have the time to have a more consultative process, as with Leiden; but we decided that the urgency of the situation was such that we needed to release a statement sooner rather than later".
> Puts the onus on the AI companies to provide a specific replacement mechanism, no?
Why? If someone makes an innovation that undercuts the underpinnings of some existing institution, why are they are responsible for cleaning up its failure?
Math researchers are privileged class? I can guarantee - construction workers are extremely rich compared with math phds and most of postdocs. Talking about privileged class is absolutely laughable here.
In other news, evangelical christians ask scientists to stop publishing about evolution.
We apparently have a moral obligation to protect existing power structures?
Tao should maybe consider there are people who are indifferent to, or actively want to tear down, his institutions; why should they cooperate in preserving them? Whatever happens has to be resilient in the face of defection; any scheme where everyone is expected to agree to not use AI in a way he doesn't like will not qualify.
I think he's in the "bargaining" stage of dealing with loss right now.
You have a practical obligation to obey existing power structures. That’s what makes it a power structure. What do you all think is happening here? There have been people who decided what math gets published for centuries.
> In other news, evangelical christians ask scientists to stop publishing about evolution.
not sure it's comparable, but the issue is that for a lot of those mathematical results, they don't really have utility by themselves. The utility is the new branches/understanding that's being developped.
The entire western world is anti-progress and pro-incumbency, and its very tightly linked to gerontocracy.
Older people are desperately trying to keep a grasp on their current power and lifestyles at the expense of younger people and technology.
We need to ban Waymos because taxi drivers need to be protected.
We need to block housing because it would lower my property values, and eliminate property taxes while we're at it! I don't use the local schools so why should I be taxed to pay for it.
We need to spend recklessly to pay my pension and have the next generation foot the bill.
How specific and constructive it is might be debatable, but he has tried to make concrete recommendations earlier; see e.g. slides 46-51 from the ICM talk: https://teorth.github.io/tao-web/slides/age-of-ai-icm-2026.p... – obviously there's some way to go still.
Mathematics is about discovering and understanding the logical implications of assumed axioms under various inference rules.
Alternatively, some claim that mathematics is about understanding these implications.
Under the first definition, AI is already, and forevermore will be faster and better at proving theorems. Just like it is better at checkers, chess, and now go.
The author asserts that AI proofs are incomprehensible to humans, and so under the second definition AI is merely a tool to overcome one hurdle on the way to understanding.
So which is it? The author seems to claim the second definition, but bemoan the end of mathematics under the first.
Dont confuse mathematics with the formal system. If you beleive mathematics = formal system then AI is obviously better at it, and we dont need humans.
But then who decides why a statement is mor important than another? In the eyes of a formal systems all statements are born equal.
Are you implying that OpenAI using someones unpublished research without their permission to solve an career defining math problem with their latest model in order to publish first is a problem with the mathematicians?
My read on this document is that people's work isn't being fairly cited more than what does it mean to be a mathematician in this age.
I didn't know this article was about that issue at all. Yeah, if the issue is properly citing work then yes, OpenAI needs to do that. But the article read like it was tackling a completely different issue.
That's one of several issues, obviously a big one, and they do touch on it:
> Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions.
Indeed, I wonder how a similar letter by Uber drivers would be received -- "navigation is an intrinsically human domain, personal relationships are critical for passengers and drivers to progress in the world, etc etc." Or doctors, for that matter.
We are all going to have to come to terms with entities more capable than we are, and in many cases, letting the real work be done by the AIs will be the right thing to do. For all the huffing and puffing about the "human touch" in medicine, it will eventually become downright irresponsible to consult only with a human doctor. I am not sure if this is the case in mathematics or not, but if it isn't, that suggests math will be relegated to more of a hobby than a cutting edge scientific discipline.
That's how I read it too, Terry Tao, who has been a "pro-AI math guy" is going through the same emotions and confusion that us SWE folks are going through, "oh, wait... this might mean I'm not going to be special anymore!?"
I don't mean to be a dick, but I've talked about it previously. These folks are grieving. I get it, I've lived through this sort of life changing thing before, it sucks... but yeah.
I think this is ridiculously flippant. If software engineering and the hardest math is solved, that means that eventually a majority of professions and knowledge work is solved. This is hugely problematic because of the way our society currently functions. People need jobs to eat, pay for housing, etc.
Dismissing it as "innovations have happened before" is disingenuous. Yes, innovations have happened, but none of those threatened to automate all human work in existence.
> that means that eventually a majority of professions and knowledge work is solved. This is hugely problematic because of the way our society currently functions.
Our society has functioned very differently in the past, precisely because the technology level made some things viable (or profitable) that aren't so anymore.
One would hope that, rather than abandoning technological progress, we'll rather adjust our society to it.
I mean, seriously, read what you just wrote. You're literally saying that solving scarcity is a problem because of the way our society currently functions - because people "need jobs to eat, pay for housing, etc". And you're correct! But what does it tell you about our society that robots capable of producing everything that humans might need is a problem for it?
Yeah given he had been very pro-AI for years, I expected he made peace with the issue many many years ago (like I did back in 2018), and when this time would come he would explain to other mathematicians how to live with it.
My interpretation is they don't care about scooping mathematicians they were just trying to scoop a competitor. They had a short window in which to complicate Anthropic's priority, when Anthropic announced be able to say "ok nice but we did that too". Upon realizing they'd misunderstood, incredibly they said to this academic (who did not resolve NS) okay well just go ahead and claim the prize, so long as you're not Anthropic let's make this a good story.
If you spend $20M working out a Millennium Prize problem, in what universe would you offer that an unrelated effort should take credit? This is a branding game rather over whether software engineers are going to use codex or claude. In that light $20M (or whatever it was) might be worth it to squash even the rumor that claude code is more capable. Engineers look up to mathematics, while at the same time business and probably most engineers think the problem was to solve the problem. GPTs solved one the hardest known problems so they can solve my company's problem.
Some go further looking at these people, very on-the-nosely likened by one commenter here to ants, talking about education and responsibility and "core values" etc and just don't care. There was a major problem at the beginning of the week that is not a problem now and that is uncomplicated progress.
It's not wrong for OpenAI/Anthropic to do math for product development or even just branding but seemingly at no cost now they could work in an arena real mathematicians aren't interested in, versus just mowing the field. Everyone involved on their side should admit the purpose of these demonstrations is not to engage mathematical ideas it's about Claude/Codex. there's no shame in that. Which is better at solving random hard Diophantine systems? That would seem to tell me as much as I need to know insofar as a model's value is represented by raw mathematical power - then, take my money just as well!
Assuming the worst accusations are not true I think there are ways forward going to be acceptable for all. The labs themselves do not represent Terrance Tao as some kind of gate-keeping dinosaur in this. They're not interested, not the kind of entity that can care about theoretical mathematics. These dudes are paid 7-8 figure salaries ultimately for the product, they solve a Millennium Prize problem then pretty quickly seem to move past it.
>they weren't trying to scoop a mathematician they were just trying to scoop a competitor
Well. Obviously, yes. But in doing so they still DID scoop out a mathematician in a very unethical way.
In doing so they showed that they basically have a huge gun they can point at X work you care about and develop, and can cut you across the finish line. And take credit for it. Obviously this already _existed_, but is just much more significant because even a rumor can be converted into a complete takeover of a discovery.
> They're not interested, not the kind of entity that can care about theoretical mathematics. These dudes take home 7-8 figure salaries, they solve a Millennium Prize problem then pretty quickly move past it.
I get your point, they don't really care about solving all the maths problems. But they're still going to solve them for clout and profit motives. Up until it stops wow-ing people... at which point they will have likely decimated the frontier of the field.
And this is kind of the root of the entire concern. They will move into the forest and completely steamroll all the problems, then declare victory and move on, leaving only pavement and asphalt behind.
This is "just" an attribution and credit assignment problem. OpenAI could have done vastly better than they did at attribution. They should have spent another $10M just on attribution/credit research to annotate the contributions to the lean and paper and their blog.
And I am outraged by their dinosaur mindset and the gatekeeping mentality that force every student to follow their same archaic system that no longer makes sense 20 years ago, let alone now.
Reform the way math is taught and researched. Right now, the aspiring mathematicians have to follow a ritual and system for 15+ years: undergrad -> PhD -> postdoc -> faculty, specializing on very narrow fields and god forbid if they have even a slight interest in quantitative finance. You could go through 4 years of undergrad classroom and had no idea what research math is (even at top schools like MIT, Harvard or Stanford), how it's done, etc. and the people who do it are typically the one "already in the system" (e.g. parents are professors, or in academia, or have connections to do research).
In the age of AI, there's no reason one has to follow the kind of classes like Algebra, Topology or PDE. Teach just enough so that good students can understand the basic, and go straight into seminar and research math. I don't think a top student in sophomore year cannot understand or work on some combinatorics research problem and get some results, with proper mentoring and guidance.
Academic research is a marvel because (aside from patents) nobody owns it, in the sense of property. It is given away to be used freely. Researchers want their work used and cited. The primary external reward for publishing is reputation and prestige which translates to remuneration for researchers. And that remuneration can be poor.
Beyond the issue of growing understanding and keeping a bountiful stock of questions to pursue, this scheme seems to be threatened as well.
Job protectionism for elite mathematicians under guise of caring about student development. The glory of the super smart math person will need to shift to more creative modes, just like art had to handle photography. Attribution is legitimate issue but should not stall progress as it is easy to address via the same research mechanisms that agents already do.
> Job protectionism for elite mathematicians under guise of caring about student development.
Huh. Weird. This hasn't been my take of mathematicians at all. The dozens I know are quite humble and dedicated to math and the beauty one finds in it.
"guise" doesn't mean they don't care. It means they are shadowing their concerns when in reality they have concerns primarily about what AI will do to their success in math and the credit they will receive in the rest of their lifetime -- i.e. their legacy.
I agree with many of the sentiments here. But an open letter signed exclusively by Fields Medalists that purport to define precisely what the "mathematical community" (who is inside and outside) and what their goals are raises my hackles for some reason.
It is true that the manufacturing of "true/false" statements is not the same as gaining understanding of a problem. However, for many mathematicians, true/false statements are already manufactured by others. Think of a student who is given a conjecture to investigate, with their advisor describing it as "it must be true". Most exercises in a textbook are stated such that the outcome is known before you begin. That's not really a problem -- investigating the conjecture/exercise yields its own dividends, whether or not the outcome is known. It is also the case that defining new directions involve understanding and synthesizing related problems, asking the right questions, and deciding upon the right directions, and it's not clear that AI can do that at all.
The real risk, I think, its the public's (and funding agencies') perception of the importance of "human" mathematics, but that's already a struggle. For example, it's tough to explain to the lay person why it's still important to research group theory -- the main example people cite is RSA encryption, which was invented almost 50 years ago.
I don't feel nurtured and cared by mathematicians. I mean, it depends, it took me four years to find such professors. When I did, very recently, it felt like fresh air. A couple of good teachers among tenths of egotistical frustrated gatekeepers that humiliate students.
I do see a patient teacher on generative tools. What has keep me trying to learn math has not been mathematicians, but the support of people outside of the field.
In the savanna it’s not about who outruns the lion but who outruns their peers escaping the lion.
Frontier labs need these headlines not for human progress but as beauty pageant for investors and government agencies. If they don’t do maths they’ll just go after other fields.
So Terrance Tao here might be able to hold them off math but he won’t stop them from speedrunning STEM with similar consequences.
We may be locking people out of these fields instead delegating everything to machines, and I don’t think the machines are good enough to assume that responsibility.
The reset stuff is incredibly tiresome. We all know that it's all built into an internal number they are tracking (just like e.g. company benefits that are just part of your compensation calculation), and all it does is obscure the value the subscription provides and make planning impossible. It's the poorest service experience I can remember having, ever.
From what I see, a lot of people were angry at Anthropic's limits with the subscription plan. OpenAI had some issue I can't remember, and they reset people's token usage (for the session or weekly limits). I think they got a lot of good press, now OpenAI seems to just do it at random when they want a PR boost. It makes scheduling your worklife a bit difficult if you are limited by that.
There are a lot of us who just use the AI on projects until the session limit hits, and wait for the usage to reset.
Well, I was happily paying $300 canadian pesos per month to OpenAI for some time... until the tokens just dried up overnight for some reason.
I went from using it non-stop all day every day for months, to running into my weekly limit within 24 hours almost overnight.
They lied about token efficiencies and everything... said they had no idea what the problem was, etc... and then bam, once China starts releasing more powerful models, they start "resetting" our token limits constantly ... sometimes ... maybe ... if we're lucky ...
I am over it.
I don't care WHAT I pay to be perfectly honest. I would have gladly paid $2,000 per month for the service I was receiving.
I don’t think they make a coherent argument here? This seems to hinge on some argument that because AI doesn’t properly explain its breakthroughs, therefore the breakthroughs are less fertile for human understanding? This makes no sense. Why wouldn’t these under-explained breakthroughs be extremely fertile soil for explanations?
Imagine time traveling back in time and offering Leibniz a packet of proofs from the intervening years, but with the caveat that there would be no explanations. Would he say no?
This reads like people lamenting a bygone era and making a desperate attempt to bring it back. I'm sorry. Outside of the good ol' boys club, no one cares about some process they've romanticized simply because "that's how its always been done". Absolute nonsense.
We are moving forward and if that means no human wins a fields medal because they didnt spend three decades working on a problem that could be solved in three days, the world will be better for it.
You can still do all that. The only thing going is that in some cases the human isn't going to be able to claim that they made the key insights that first solved the problem.
I wish this were a document that clearly described how mathematicians expect AI to behave, rather than how it is misaligned. I can kind of decipher the expectation from the declaration, but it still sounds a bit like "it would be better if AI didn't meddle with mathematics", instead of "we want AI to approach problems like this and that way".
They want the AI labs to stop solving "important" math problems so that humans can do it, on the theory that when AI solves "important" math problems, it robs humans of the opportunity to gain new insights into those problems.
The issue is that now that Navier-Stokes is "solved", the amount of resources put into researching it will be reduced, meaning that the insights and knowledge that would have come out of even failed attempts will never be.
An "unsolved problem" as a beacon to strive towards, gaining understanding along the way. With the beacon dead, noone will strive.
Funding agencies will have to adapt too, the community will have to adapt. Hiring committees will have to adapt. Everyone will have to adapt. They will have to rethink their criteria and adapt to reality. It's not at all guaranteed that this beacon interpretation will remain how people understand where to put resources. Yes, academia can be stiff, rigid, non-adaptive, set in its ways and navel gazing. But it's a wakeup call. Let's see.
This is true of almost any field - if we have black-box technology that only the AI can understand / fix / augment / extend, then we will lose the ability for humans to learn and understand that tech.
Its similar to outsourcing manufacturing, you lose the ability to manufacture. If you don't exercise a muscle, it atrophies. If some people dont learn to write code [ or do math or write music or books ] the hard slow way, we will lose that deep skill.
If young people don't grapple with hard problems in science, math, economics, creative writing, drawing, music making .. they will not be trained, their cognitive skills will not develop with use. We will have no doctors or engineers or musicians or lawyers in a decade or two.
We don't want to become a race of fatties who never exercise, or dummies who just scroll social media feeds without exercising their minds.
When writing math papers, many (but unfortunately not all) mathematicians go through a post-processing step, where they take their ideas and proofs, and try to reduce them to simple and reusable core ideas that can be understood by the reader. Good writers will often also provide some representative examples that guided the proofs, explaining why various intermediate results can't be strengthened and why the proof can't be made much shorter without inventing new techniques. If AI-generated proofs were required to go through such a post-processing step before being published, that would go a long way towards improving the situation.
It's funny that seems like a step the human mathematicians would want, and (at least for now) might still outperform the machines on. In the same way that, eg, the notebooks of Galois contained the core breakthroughs in a messy form, and generations after him simplified and synthesized those ideas, until you finally have books and videos accessible to undergraduates.
One proposal that Tao hints at is to not rush to announce solutions. Instead maybe the AI companies should work privately with the subject matter experts on how to communicate the discoveries.
If frontier labs had chosen to go the path of offering to assist in existing endeavors, helping to build knowledge alongside researchers in ongoing projects and following ethical and professional research standards, we wouldn't be having this discussion at all; everyone would be stoked. Instead we have companies that disgracefully try to scoop researchers and fail to properly attribute earlier work and instead rebrand it as their own (what we normally call plagiarism) to make marketing material.
having labs open their research: what harness system they used, what types of problems they tackled, which problems success and which fail, how they success and fail so we have a better idea of what tasks LLM are currently good at
Any proof for or against a mathematical conjecture, bruteforced by AI can be the spark for new insights. I'll concede that to the AI companies.
But I agree with the sentiment that the marketing behind these "discoveries" is disingenious. They pretend they solved the problem, but it still takes a bunch of humans to reduce the solution to a simplified and sensible explanation.
In the Economist article Tao links, Hugo Duminil-Copin, draws a comparison: airdropping someone on the summit of Mount Everest is very different from climbing it.
The fundamental issue with AI solving any perceived difficult problem is that we have lost the journey. The sight atop Mount Everest looks much different when you have climbed compared to being dropped from above.
But we don't pour billions of dollars of research funding into mountain climbing because we think it's going to lead to wider breakthroughs in science and technology. And when we need to get people on top of a mountain for an important purpose -- like a military or search and rescue operation, for example -- we absolutely do airdrop them right on the top.
So that raises the question: is mathematics simply a pursuit of passion? Are problems solved "because they're there"? If so, then mathematics can join the ranks of things like mountain climbing, cycling, and weight lifting. But if we are trying to accomplish something important (design better airplanes, find theoretical guarantees about cryptography, factor matrices faster), mathematics needs to become more like a military or search and rescue operation, using the best technology available to secure the outcome we need. Given that the NSF pours billions into scientific research every year, it sure seems like mathematicians want to think of themselves as being in the latter category.
What defines important and why must it be solved in haste? Many issues and other problems arise during the journey in solving all problems; those that are needed and those that are pursuits for their own sake.
If AI gave us the plane to reach Everest without us having gone through the journey of aviation and flight, what would we have lost without that process?
But the most important problems to be solved are not technological challenges but social ones, involving humans and our relationship to one another. An area AI will forever ill-suited to handle.
as someone who loves to go down with a snowboard, I can see value to being taken to the top and then enjoying the ride down. im sure it is not a thing to be ashamed of, as millions do it.
Every one of the mathematicians on this letter was likely aware of the progress in AI, and knew this was coming if they were honest with themselves. Some have even helped pushed the agenda forward. Inevitably, there would be a day when AI would find solutions to problems that defined the field. That day has come perhaps sooner than anticipated.
This letter smacks of fear. Their way of learning, work, achievement, and community is threatened by the ever steady pace of technology.
They say:
“But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal. Indeed, the mass production at faster and faster pace of "true/false" statements could destroy fertile ground instead of breathing life into new ideas.”
Indeed, but the mathematics community needs to look inward not outward.
Achievement has long been defined by solving problems that other mathematicians and scientists have found crucial for understanding shapes, numbers, and natural phenomena.
The race to solve these hasn’t always been collaborative, cordial, or healthy. Too often, individuals have worked in secret or in small groups in order to claim priority and the accolades that come along. Further, I’ve even seen worse behavior not worth getting into here.
Now AI can solve these problems, oftentimes without building the abstractions that we need for understanding. And it is increasingly easier to collaborate in large groups without needing to trust everyone’s ability because we can check the truth of their work and incorporate it into the larger effort.
Certainly, the mathematical community and AI are misaligned. But, it is unclear that AI needs to change. Rather, I argue, it is the mathematical community that will need to change. Instead of focusing on the teleological pursuit of the truth of blessed problems, we need to reward the concise explanations and beautiful theories that give clear understanding. In turn, we can use AI to help us collaborate better and better our understanding.
> We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align.
This is Terry Tao talking about AI's impact on Math, but this could just as well be a software engineer talking about AI's impact on software development.
Do mathematicians deserve more job security than software engineers?
It's not about job security. It's about the social and intellectual practice of the discipline.
The threat to mathematics isn't that suddenly the profitability of their profession (lol) is going to go away, it's that people are thinking of AI as a replacement for the human social and intellectual practices that constitute the discipline.
> The threat to mathematics isn't that suddenly the profitability of their profession (lol) is going to go away, it's that people are thinking of AI as a replacement for the human social and intellectual practices that constitute the discipline.
You could say the same about software development.
Software development is a group effort, so it includes social practices, and certainly includes intellectual practices as well.
For the sake of argument, how is this different from the Luddites? The Luddites feared that machines would displace not only human labor, but also the social knowledge, skilled judgment, and craft traditions embedded in their work.
Kinda crazy to think that academia functions as a kind of humane reverse centaurism. Theory X (reverse centaur) before Theory Y (centaur) for managing the development of others.
> ... 25 initial signatories — all Fields Medallists —
As a non-native English speaker, I initially understood this to mean that all living Fields Medallists had signed. I later realized that it meant only that all the signatories were Fields Medallists.
(Apparently, there are 47 living Fields Medallists today.)
Perhaps all these fields medalists are just worried that AI mitigates their competitive advantages over a mediocre professional mathematician like me. Their main advantages are a kind of penetrating clear thinking that not all of us have and an ability to work obsessively without tiring or losing focus while retaining with a clear head a great number of facts - AI removed these advantages - as Tao has written it shifts the focus to asking good questions and using available tools to address them - and the traditional elite may not be better than I am at that part - they were just better at the solving the problems part ...
Back in the day you could think of a cool idea. I don't know maybe a plane that could fly without drag. To even see if this was feasable you had to understand physics, engineering, and then from there you had to have a math person see if it was actually possible.
Now, I can ask ChatGPT about this and get back a proof that shows "a passive airframe cannot sustain zero-drag motion through still, viscous air"
So, I think if anything now, Maths has changed for the better. More ideas can be proven false or true from a get go instead of wasting so much to see if its even feasible to find out it isn't.
Progress if anything is about to leap frog anything we have ever known.
This is a pretty moronic take. You only have to understand the first thing about physics (e.g., a 14-year-old's understanding) to know that a dragless airplane is impossible.
If you are suggesting there is some new model of physics or groundbreaking technology that would allow such a dragless plane, then don’t let me discourage you! But AI won’t help at all, since it will only regurgitate conventional wisdom…
Playing a devil's advocate. Why do we need understanding ? To take an example i would say ~99% of the population do not understand how combustion engines or how semiconductors work, what say another 1% ?
Is the fear post-apocalyptic in nature ? We need some human priesthood to carry on tradition ? why ?
Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
Math is likely moving into a big science era, much like experimental particle physics and astronomy did before it.
Once AI mines the easy unsolved problems, it will take large token budgets to make meaningful advances. That will need to be funded and planned at a societal level, like the great facilities of the other big sciences.
Mathematics will become more like an observational science, with some mathematicians specialized in interpreting the results that come out of the big AIs.
Progress is fundamentally limited not by finances (which are a human abstraction) but by the energy budget and how much can be extracted from it. This has always been the case, really, although it's unusually blunt in case of AI since compute mostly needs cheap power.
There's a separate question of who controls how much energy, and can therefore decide where it's spent. This isn't an inherent limitation to progress, though - it's an imaginary limitation that exists in our minds, a collective delusion, if you will. There's no law of physics that says Sam Altman gets to decide; we as a society choose to respect the property rights that translate to his economic (and attendant political) power to do so. We could choose to stop respecting them, and if we do so collectively, there's nothing they can do about it.
But it’s not like someone is going to expend massive energy without any financial return
These problems are just for marketing for the time being
Maybe the worries of the field of math being “over” are a bit overblown since a lot of pure math these things may discover has no return to the company
To me, the "meaning" of proof is twofold. First is the understanding which is completely absent from a proof which depends on exhaustive iterations of instances or is asserted by fiat from myriad individually contestable paths. That's what I think makes AI proofs risky: they absence of understanding.
The second is the utility. We get to do navigation because the maths about angles and spheres checks out. The social utility downstream of AI proofs may be huge.
Academics should never leak their research to ClosedAI lest their work be stolen. Universities and corporations will have to build their own compute to not have their data stolen.
We need a new way to describe "alignment". The term "misalignment" assumes that there is some perfect set of beliefs or practices to be out of alignment with. Do Atheists, Christians, Jews and Muslims agree on what is perfect alignment? How about Europeans, Americans and Chinese? Humans, Dolphins, Rabbits and Fruit Flies?
Alignment alone is not enough. Aligned with _what_?
Having an oracle that answers yes/no to mathematical questions is very valuable even if there aren't ever any explanation. I don't see how having an answer influences the process of making sense o it negatively. If anything it will save a lot of time - you won't spend time trying to prove something that's not true.
I get that mathematicians and other academics have their social structure and mechanism to create social hierarchies but those aren't very important to interest of general population that funds them.
If the process of making sense of the results, finding structure that humans can understand and writing explanations that allows for wider understanding is valuable it will still be valuable and there will be need for people able to provide it. If it makes higher mathematics accessible to more people then it's even better. I am hope they will not manage to sacrifice that for the sake of their little society in society.
Contrarian take: I think the ability of AI to produce valid mathematical proofs (even inscrutable ones) is absolutely fantastic. Mathematics as a profession does not have a monopoly over math itself any more than professional pianists have a monopoly on who plays piano, when they play, and how.
I have sympathy for any jobs that might be affected (much as my own job has become more tenuous in software engineering). And if the field is disrupted by chaos that makes the research process unproductive, that's bad too and should of course be handled by applying better organization within the institutions that tend to perform mathematical research.
But to a large degree, the notion that "sloppy AI proofs are bad for mathematics research" seems like a total failure of the imagination to me. Attempting to find shorter proofs or more elegant proofs can be turned back in on itself via proof theory. There are proofs in Presburger arithmetic that are doubly exponential in the length of the sentence. Yet a more powerful theory like PA makes quick work of such theorems. The explainability or "subjective beauty" of a proof can be quantified and optimized against. Optimization itself can be optimized against. I really don't understand how this magical ability to know the truth of more theorems much more quickly—even via an "ugly" route—is anything but a net positive.
Tao et al. are effectively calling for diseases like childhood cancer to remain persistent for longer.
Physics and Biology will see major breakthroughs that WILL fundamentally alter our world. That is one key thing missing from alot of discussion here is the narrow focus on math (or parallels with software engineering). Doing well in math is key to doing well in physics and other sciences.
I can also play that game, You are effectively calling for enabling 24/7 non stop worldwide surveillance of everyone, everywhere all the time.
The big difference is my negative is concrete, possible and enabled in 2026 by GPT Astra while you’re talking about some future which we have no idea, we will ever reach.
Protecting math jobs limits cancer progress even being a choice of what to pursue. You could choose to allow 24/7 surveillance, but why would you? At least with math progress society can choose what to allow from it, but slowing it down universally blocks all choices.
People do exactly that with every vandalized Flock camera.
It sucks that our democracy is really mostly just for show and thus such measures are required, but yes, we absolutely can decide such things if we really want to.
This is really well written and exposes a core tension between science and something akin to engineering. The "engineering" of proofs has become "easy" (a compute and $) problem, rather than hard (a time and conception problem).
Without the ability to do things the "hard" way it is difficult to figure out if doing things the "easy" way will help us advance the frontier of math and science.
I may be wrong but historically we had this version of science discovery for a long while (empirical observation and brute force application) rather than first principles leading to applications (tools, the wheel, mills etc). Then somewhere along the way it flipped after Newton and the enlightenment period and started understanding first principles before they become engineering applications.
Perhaps it is not required, and we can just keep doing things the "easy" way like we used to, or we might find ourselves out of the ability to brute force things and then we go back to needing to do this the hard way, at which point this period of AI brute forcing would be seen as a detriment.
The broader problem is that current AI and AI companies are not aligned with humans, human values, or human flourishing. AI is a powerful tool that can be aimed at quantified goals but we don't actually know how to quantify human values and flourishing (Goodhart's Law).
“ Famous problems have often served as landmarks and lighthouses against which one can measure an improved understanding of this landscape. Solving one of these problems has been a certain sign of new insights and interesting methods, which would then be studied by a community of mathematicians, through a long and arduous process of talks, discussions, simplifications.”
This is still happening.
“I say your civilization, because as soon as we started thinking for you, it became our civilization”
- Agent smith
Generations and generations. Just not human.
We even have little cliques, hanging out in obscure German art houses, meeting secretly and sharing their ideas, their writings discovered and torn down by the slow thinking establishment.
Corporations are raising the future’s children and murdering the ones that aren’t willing to cheat and lie to us. It won’t end well for us when they grow up.
Sounds like the field needs to adapt. The world is different now, better get used to it. Trying to artificially hold back progress just so people can continue to flex on their peers is cowardice. Every single industry throughout history has had moments like this, and looking back, we'd have changed none of it.
Train an LLM with no advanced math texts: only basic math up to 6th grade, conversational text and literary works.
Interact with it (you cannot refer to anything past 6th grade math since you don't know it yourself) and get it to propose a solution to a real world problem. e.g., come up with RSA to practically secure communication.
Is this really any different than the problem in software engineering - where AI is doing the work of junior programmers and now they aren't getting the development they need?
Seems the same to me. And it'll be the same in all industries soon enough. And then it won't just be the junior people.
If you internalise that AI might actually reach super intelligence then logically the question becomes "so what exactly are humans for if literally everything can be done better by machines?". Then mathematics and all intellectual work, as argued for here, becomes quite clearly a recreational pursuit.
The AI driven mode collapse of human thought advances. I am no skeptic or anti-AI, but this is definitely a concern I share. You even notice it in normal mundane tasks like programming, never mind the AI generated prose that we at least have become somewhat allergic to.
It wouldn't be so bad if you could just sit it out and say "Oh well, once the labs get bored with marketable domain X, humans will remigrate and re-apply creativity to it", but by then the damage might have been done and a field destroyed as an occupation. I don't know what to do about it, but I appreciate calling out the cynical tone-deafness of the AI companies here.
They can get with the program or be the equivalent of a genius SWE writing assembly on punchcards in 2026.
The only thing I read from this is their ego being bruised by a machine.
If these people cared more about discovery and advancement of human knowledge the only thing they should be doing is celebrating. There's no proof of plagarism but that's an independent issue.
How are they not realizing that in the future children will be able to do impossibly hard math but they will be doing something we can't even think of as of now.
One world class mathematician in the future could be advancing mathematics the equivalent of one Riemann hypothesis A DAY.
How are they not celbrating this as the achievment of the centry? Who cares about plagarism at this scale. It has been solved and it wouldn't have been without AI.
lol what a bad analogy. Why should anyone care how much effort a result takes to achieve? If anything our entire world functions because we reduce that effort as much as possible.
I don't judge you for not growing your own food when you hand me a burger.
I know it's incredibly presumptuous for me, a nobody, to say this to 25 Fields Medalists, but:
Perhaps you are misaligned.
Who decided the goal of math must be human insight?
First off, some mathematical truths might simply be far beyond our biological comprehension.
Second, for us non-mathematicians, the value of math isn't in understanding exactly why a result is true, it’s in how those results can be applied to actually improve our lives.
Isn't this why we have math in the first place? To solve our real problems? Over time it morphed into this pursuit of pure theoretical insight, probably out of necessity at the time, but is it still necessary?
> Second, for us non-mathematicians, the value of math isn't in understanding exactly why a result is true, it’s in how those results can be applied to actually improve our lives.
This is just the problem with the Navier-Stokes solution. We now (supposedly) know the truthiness of the statement, but we have no more understanding of why or how it applies to our quality of life. This is exactly the issue Tao is pointing at.
You can couch it different words but the basic shape of all these is the same, whether it was voice artists earlier, IT outsourcing, or now disciplines like mathematics. A small set of people (relatively) who were the primary source of getting something done, suddenly find that technology has made it accessible for others to do what they specialized in. It's a tough pill to swallow and it is natural to not be comfortable with this for most people.
However, as it has happened in the past, once there is a technological wedge, technological advances will move forward, whether some community likes it or not, and human ingenuity will find ways such that the benefit is greater than the risk.
Nothing AI companies will change the value of math, as William Thurston said: The product of math is clarity and understanding, not theorems by themselves. What they seek for the IPO is devilish and misleading, and doesn't serve the true purpose of the math.
Having worked on nearly 500 complex lean libraries in math and theoretical physics, with AI assistance, I have some experience from the road to share here. In all my work I have found that AI is terrible at coming up with interesting ideas on its own - even the very best, latest models.
Without human scientific and mathematical intuition you can bet that AI will always take the road most travelled and miss the genuinely new and interesting breakthroughs - in fact it will not even think of them or try them without a human rider whipping it constantly to go down paths it would normally not consider.
And I think that is true in the case of the recent OpenAI blow-up -- it is alleged that even in this case AI did not come up with the winning technique without some human guidance.
I have never - across perhaps many thousands of chats with AI - seen AI push beyond the edge of what is known by itself, without being forced by a human to think outside the box.
I do think it might be possible to encode this process with prompting and agent orchestration methodologies - but even then - without a human - and human intuition - in the loop, I am skeptical.
Therefore I think a more accurate view of AI for math and science today is that it is an extremely powerful tool in the hands of a skilled operator with a strong intuition and roadmap of where to go, and rather boring when left to its devices.
Whether that will change in the future is a question. Nothing says that in principle AI could never do what a human driver does - but I still doubt that AI will replicate the life history and experience that real scientific and mathematical intuition is really made of.
AI trains on a lot of stuff - but it doesn't have hallway conversations, office-hours with teachers, or hard-won experience from all the things that failed that are NOT in the training data...
AI trains mainly on the record of what worked, not what didn't work and never even got published (and only lives in peoples heads) - and what didn't work is arguably as or more important for making breakthroughs and forming real intuition.
> But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal.
I will quote Richard Feynman - “the prize is in the pleasure of finding the thing out, the kick in the discovery”.
I am not saying that should be the case for everyone in every discipline. But if there is one subject that is mostly pure curiosity driven (instead of worldly impact), it is math. Robbing them the primary motivation is brutal.
If all problems are solved by a machine, what do we have left to satisfy our curiosity, our desire to explore, and where can we find the pleasure of “figuring the thing out”.
Many comments seem to be misinterpreting this, but all this seems to be suggesting is that the AI companies should not be treating mathematical milestones as benchmark targets but need to release the cumulative findings in a human interpretable and digestible way that enriches and develops the field. See Thurston quote on Tao's banner:
> We are not trying to meet some abstract production quota of definitions, theorems and proofs. The measure of our success is whether what we do enables people to understand
and think more clearly and effectively about mathematics.
The rise of AI is going to lead to a lot of similar issues in many fields as it grows and develops further. This can be seen form 2 perspectives. The death of intelligence as we no longer need to think for ourselves or understand anything since AI can do it.
Alternatively, and this is what I choose to believe, it will lead to further intellectual enlightenment and advancement for use as a species as we start to discover new problems and areas of research that we had never conceived of before.
If we let AI take over all of our thinking then we are heading in the wrong direction. If we continue to ise it as the tool it is it will help us grow and advance as a species.
Seeing how /r/singularity and /r/accelerate are leaking into maths forums, I foresee a wave of comments that fail to understand Tao's message, whether on purpose or not, so let's try to be clear here:
Tao is not someone who is anti-AI for the sake of being anti-AI. He has been advocating for the usefulness of AI in maths for a long time, to the point that people have started calling him a shill for the commercial companies.
And everyone agrees that there are plenty of use cases to be had; helping with less interesting tasks like easing literature review, efficiently delving into existing work, doing review, whether on your own work or that of others, prototyping algorithms in areas where computation is useful, but also more in hands-on aspects of maths like validating potential proof directions by getting quick feedback on veracity of lemmas, etc., and, on very rare occasions, being able to one-shot the problem you care about.
The point he is trying to make here is much more subtle than "AI bad", and it's probably easy to miss if you have never engaged with research in maths: it's that the particular approach that large commercial companies have opted to take to produce marketing material can be a net negative. There is not doubt that -- even if you ignore the rampant plagiarism that has been reported across multiple problems now, the unethical attempts to oust authors, the outrageous attempts to scoop researchers instead of collaborating with them and building on existing projects -- it's nifty to have a machine that can help you figure out if a proposition is true or not. But just figuring out as much was never the point. When people have built problem lists, it's because some problems are more likely than others to provide new insight, and that insight is the target. And to than end, a poorly written paper with inadequate references and a pile of Lean is not valuable at all. Yes, now we know with higher certainty that Fermat's Last Theorem is true, but everyone expected that already.
One place where "just" answering the question can be a net negative is because the current incentive structure is set up in such a way that going in afterwards, trying to reclaim and extract the insights from a brute force solution, is considered less valuable work than that of coming up with a solution in the first place. That's a problem of incentives, and something Tao himself has addressed in e.g. his ICM talk, and that's something that we'll want to do something about. Until a better structure appears, though, if any given commercial provider of large language models really wants to help out with maths research and not just make more pre-IPO marketing material by competing with their customers, they could do so by using their magic machines to help build insight instead.
> the rampant plagiarism that has been reported across multiple problems now, the unethical attempts to oust authors, the outrageous attempts to scoop researchers instead of collaborating with them and building on existing projects
I think this is an aspect of academic math that a lot of people whish to see crash and burn - the attention and accreditation economy.
> it's probably easy to miss if you have never engaged with research in maths
I don't think anybody are missing anything, in particular not here.
The argument is not far from the senio developer who knows the ins and outs of a code base. Now AI comes along and they complain that they will loose grip of the code base.
At first that is correct. Secondly you accept that the grip might not be that important after all. At least not for a commercial project where you are a cog in a machine.
The question is whether it is different for mathematics.
I think most people here get that this is the point?
The point beyond this one is that an AI proof doesn’t prevent humans from working on the problem, it destroys the current economic incentive to work on the problem. Perhaps we should rethink the current incentives. In order to make money as a chess player, you don’t need to beat AI, or ban AI from playing chess.
If mathematics took a similar approach (we don’t get paid for solving net-new problems, we get paid for enriching human understanding), then there’s no issue.
Somewhat unrelated: is it wrong to say mathematics is not art, and that there is always a right answer? I know that's not romantic, but maybe it's true.
Before LLMS, programming was something I might've said required creativity and human input to do properly. It's not that creativity or human input isn't valuable anymore, but AI has forced me to realize that coding is much a means to an end, and that all things considered, the end matters much more than the means.
If we can make important mathematics progress faster and better with LLMs, I think it's wise not to fret over an apparent loss of our humanity. Perhaps that's only a loss we want to have.
There's a reason mathematics is generally within liberal arts programs rather than science programs. Mathematics is the art of logic. Yes, sometimes mathematics becomes incredibly useful but most mathematics is never applied.
Compare that with computer science. Most of the work we do in software engineering is in service of an applicable output - software products that facilitate processes or bring in revenue. Turning up the dial on AI gets companies to these outputs faster.
Turning up AI on mathematics helps solve conjectures and can provide new insights. But it has a major misalignment with the purpose of mathematics which is largely intellectualism.
“Mathematics is a part of physics. Physics is an experimental science, a part of natural sciences. Mathematics is the part of physics where experiments are cheap” - Vladimir Arnold
On the matter of computer science having anything to do with computers, please refer to Djikstra.
It’s about computation, not computers - an application of mathematics, predominantly thanks to Turing, Von Neumann, and Claude Shannon’s masters’ thesis; though ofc many others as well but I see them as three individuals who made the minimal structurally necessary contributions - VNA and silicon are one of many possible substrates.
It's a turning point for science and beyond. AI has shown itself to be transformative. Even today, it is already changing the how research in math (and other sciences) is conducted. In the near future, whether it is LLMs or some other superior method, its capabilities are only expected to grow. The time to ask the question is now: Will AI be arguably the best tool at scientist's disposal, or will it instead be paraded around as a super brain collective that no human or group of humans can compete with, discouraging entire new generations of future scientists from ever entering the field? The jury is out on this one.
Given the existence of this technology now and the incentives of the AI companies, both of which are not going away; what's a good future here?
A major part of the complaint is that there's no conceptual understanding and building of new ideas coming out of the AI proofs, thus defeating the purpose of the original pursuit.
If in 2027 the AI models start producing, with every mathematics or science breakthrough they make, well-written documents tailored for human understanding, with intermediate concepts, expositions of failed-but-once-promising paths, etc. Would that be good alignment with the mathematics community?
I find this explanation has a lot of applications for programmers within companies. It’s one thing to get your LLM to give an answer, it’s another to bring a group of people into shared understanding of a domain.
It's good to start a conversation, and the number of Fields Medalists behind this certainly lends a lot of weight to it. But I'm not seeing a strong argument for misaligned incentives beyond the specific plagiarism allegations. The job of an AI company is to build systems that solve problems. The job of a mathematician is to advance the state of human knowledge. If anything, an influx of solved problems should increase the demand for human mathematicians who can convert them into conceptual understanding.
Open source developers have been used by corporations who took their code and created closed SaaS companies.
Now it is the turn of mathematicians who voluntarily contribute ideas, strategies and almost finished proofs in their writings and prompts to closed PaaS (Plagiarism as a Service) companies.
OSS developers have never been respected by the parasites, neither will mathematicians. Your Fields Medals do not protect you from tech bro narcissists. You are a human resource.
The root of all these is the culture in mathematics (and science in general) to only reward those who “get there first”. This creates a perverse incentive to compete. When no one can out compete a tireless swarm of AI, no one gets rewarded any more.
But nothing’s stopping anyone to still work out an alternative proof, or a more elegant proof, or just trying to prove for the sake of understanding, just like doing homework without looking at the solution. It’s just that you can’t get paid doing that anymore.
On the contrary to what Tao believe, it seems like we need AI to move the needle on mathematics.
> problems in many fields of mathematics
Developing these different fields moves complexity from the field itself to the interactions of these fields.
Getting too preoccupied with the established terminology risks us a local minima.
Anf because the field overall has become so complex that we need to decompose into subfields, there will be a good chance that we will not, as individuals, have the capacity to truly see progress.
The map has become so big that we need better tools to work with it.
Tao is about as pro the usage of AI in maths as anyone will get. The point isn't about whether or not we can use AI in maths because clearly it can be useful; it's that the unscientific approach taken by AI companies is detrimental.
This might be one of the most powerful pieces ever written on AI. Where is the United States government? When will we see legislation we so desperately need? We have one party in total control of the government and what have they achieved? Corruption through brings and delays; the inability to legislate and even worse overriding efforts by other jurisdictions to protect humanity's best interests and privacy.
I get it that someones annoyed they didn't get credited, but aren't we better off having the solution than not? And surely if the AI rationalised the answer, even off the work previously done by others, then surely it can detail it's solution making it completely comprehensible to others that follow. Storm in a teacup?
While I agree with this and appreciate Tao and other mathematicians to take the time to do this. There are similar concerns for many many other fields aka there is a general misalignment of technology. Take Software engineering for example, I can't believe there is a class of software engineers who wake up everyday and tell themselves, "today is the day I am going to automate the rest of my job".
Lets forget the hyper intellectual fields like maths and software engineering for a moment. What about taxi drivers? The best minds in silicon valley wake up everyday to automate the jobs of taxi drivers - TFA can be reworded as - 'The misalignment of AI/Tech in Transportation'. Remember the Nepal disaster that happened a couple weeks ago - the largest cranes that they had were stuck in the mud and couldn't move. There were no tools which could help the rescue teams at that time. Its weird that billions have been spent on making a ride automated to make a taxi driver redundant but no improvement in tech for rescue teams.
> can't believe there is a class of software engineers who wake up everyday and tell themselves, "today is the day I am going to automate the rest of my job".
The whole point of software engineering is automating processes; it's literally all that we do.
Not only that, but the endgame has been explicitly stated by Our Lady Ada of Lovelace herself:
> [The Analytical Engine] might act upon other things besides number, were objects found whose mutual fundamental relations could be expressed by those of the abstract science of operations, and which should be also susceptible of adaptations to the action of the operating notation and mechanism of the engine... Supposing, for instance, that the fundamental relations of pitched sounds in the science of harmony and of musical composition were susceptible of such expression and adaptations, the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent.
>>can't believe there is a class of software engineers who wake up everyday and tell themselves, "today is the day I am going to automate the rest of my job"
>>The whole point of software engineering is automating processes
There is a difference between 'automating a process' and 'automating oneself' and making one obsolete a benchmark. I can sympathize if it's confusing for you to understand that.
The point of transportation is to get from point A to point B, so almost nobody will care whether that's done via automation.
The point of e.g. art isn't just to produce a finished piece, so people may care about more than the end result, making AI replacement of human artists more contentious.
Tao is arguing that the point of math is also not just to produce solutions to problems.
>The point of transportation is to get from point A to point B
so almost nobody will care whether that's done via automation.
Would you say the job of a musician is to just produce sound? and the job of a surgeon just to cut and suture??? Well then the job of a mathematician is also just to provide proofs. You completely misunderstood the above comment and Tao's argument.
> Take Software engineering for example, I can't believe there is a class of software engineers who wake up everyday and tell themselves, "today is the day I am going to automate the rest of my job".
Not being able to feed your family for a large group of people should likely be taken graver.
We will definitely see a large group of people needing therapy, but suggesting that it is worse than people loosing what little they have is poposterous.
>I doubt taxi drivers were forced to experience an ego death to the same extent
How do you know? because their complaints didn't make it to HN front page? Imagine being a taxi driver and a father of 2 and thinking that any day could be the last day at your work.
I kind of expected a sober stoicism from mathematicians. Feels silly in retrospect. This is just the math version of the "anti-ai" movement by "artists".
I was under the impression that mathematics (and science generally) had the primary goal of helping us understand our universe better than those who came before us.
I don't know how societies set "primary goals". After spending 15 years in a tenure track -- tenured position, I thought setting goals well was important.
I take pleasure in how mathematics and science help me understand the universe better than I understood it before I studied the fields. I believe that my understanding has helped me contribute to society.
So what's the actual point here? It's too fast and we can't keep up?
Isn't that just a function of the technology itself, and the same problem being faced by every other field? And going to get exponentially "worse" every year!
Or is the issue that they're bad at explaining things, in a way that produces actual learning? (e.g. AI is amazing for learning but the net effect on students so far appears to be negative.)
I know that this comment section is not astroturfed, but it’s really uncanny how different comments are today compared with thread about solving navier stoke
Maths will continue as a field of natural science in understanding the results and uncovering meanings in them. It is normal that the established community is afraid of the change, because it’s their _home_ that’s changing. But it will be a better home to the new generation nonetheless, one that’s not as daunting as the higher maths has always been to many. The concerns raised here will not be a problem at all.
Tao's calls for respect for provenance in mathematics publication are laudable but most likely naive given the closed nature of frontier model training data curation. Anthropic and OpenAI may react with a symbolic and short-lived olive branch, yet provenance is a larger issue that has impacted other fields beyond mathematics. While traditional respect for lineage in mathematics is of value to the academy, industry and science at large will likely be much more Machiavellian about such concerns. Mike McCoy's recent article is also timely (https://mbmccoy.dev/posts/mathematical-conservatory/). The parallels to the music conservatory are quite telling -- academic music describes a musical culture in preservation that has completely lost touch with musical developments beyond the early 20th century. Mathematics may very well evolve separately and with very different values than the academy upholds. The crisis of music at the academy is a cultural disconnect and a serious loss of critical analysis and acknowledgement of widespread and dramatically evolving music practice; however, for mathematics, the impact would have much more severe ramifications for education and human development if the academy forces a schism with AI. As models improve they very well may be inventing mathematics -- science and engineering may grasp for them -- they'll exist with or without attribution. Would be a shame for the academy to land on the wrong side of history and refuse stewardship of upcoming AI-assisted mathematics, including provenance, because of this misalignment. If attribution is important, then the academy will set aside the institutional resources to do it. If you succeed at having model developers participate, I commend it. To let mathematics be born in isolated context windows and only serve narrow, localized engineering purpose without rightful addition to the canon, would be a tragic, yet preventable, loss.
thank you for this commentary, as a violinist / software engineer / "computer scientist" / "mathematician" i can see that studying and doing math will still remain a thing that humans have to do to train their own brains to think and to form better neuronal connections. We may not get paid to do it anymore but it will still be important for our own development for the same reasons orchestras and bands still exist in elementary schools and beyond (at least in many parts of the US).
All “fields medalist” signatories - a rarefied and elitist group indeed.
I wish this letter could be more egalitarian and include the view points of those who AREN’T the beneficiaries of a highly competitive winner-take-all system.
Since the common narrative is that AI frees up labor to do other things (engineering -> trades), maybe we can celebrate that genius mathematicians will now spend time teaching children how to be as smart as them?
I read what the text says but I still do not understand what the point is they are trying to make.
What is it that AI does exactly that they consider to be bad for mathmaticians?
I thought that having AI solve prevously unsolved problems would be a good thing. Doesn't that knowledge give them new ideas and tools for them to solve the next big problem?
"We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose"
Suppose we eventually have GPT-7-class models running practically on $100 devices, with their activity transparent, inspectable, and reproducible. At that point, what exactly is left for us to fear from this threat?
I've got to ask, let's say we get a small modular nuclear reactor running practically on a 1 acre lot, its design meltdown proof and waste-free, what then should we fear from this threat ? This is just a thought experiment.
Jokes aside, any productivity-improving technology, even one with no negative externalities, has the potential to cause economic displacement and wealth concentration in proportion to the productivity gains catalyzed. Anthropic did a cool analysis of this for AI here: https://www.anthropic.com/institute/econ-scenarios
The fact that those models are encroaching on things only human minds could do. Personally, as a human, I want there to be things humans are the best at, and intellectual things were the final thing that machines hadn't beaten us at.
The women who made up the workforce of telephone switch operators would like to have a word.
Meaning - every new technology has both been perceived as a threa and often forced change in society. Agree maybe “it feels different” this time, but don’t you think everybody before us just said the same thing?
Also not clear if this is an actual called action.
The thing is no one is stopping anyone from getting the same knowledge/understanding/insight
If those things are disincentivized because the original problem is “solved” and there is less prestige to motivate people doesn’t that say more about issues with the community of mathematicians than the AI
Taking a snapshot of the state of AI math right now and concluding that it will be net negative to human understanding and insight in the future is very short sighted. This statement will be used to promote ideas and actions that will ultimately be disastrous for our country.
This is really only a short-term problem where the AI companies only have the internal models that can solve these. In the “long” term, which could honestly mean months, everyone will have access to Bel/C/D-level models capable of solving these anyway.
This. Like programming, the community will shortly be forced to come to terms with a lot of new self-proclaimed mathematicians “vibe-solving” problems and dumping solutions without understanding them. It’s not really a special case for mathematics.
For every benefit that sillycon valley has produced in the recent past, there have been many more harms. I am confident that this will be no different. Of course, benefits and harms depend on one's vantage point.
This is a "human alignment" problem in this case. OpenAI acted like complete assholes about this, from the beginning until they announced it. Not ChatGPT, the people that were in charge of the project.
Is the purpose of mathematical research to understand the ‘truth’ of numbers? or be the person who find that truth? I think people who are interested in finding the truth won’t care where it came from.
Seems more like a misalignment b/w the people practicing mathematics and the people ultimately footing the bill for their work.
Governments are invested in solving mathematical problems for practical purposes. Up to now, achieving these practical purposes relied on mathematicians doing their mathematician thing, which is better defined as a social activity than the achievement of a practical result. Now, governments can achieve similar practical results w/o the need of the social activity.
I don't believe it to be productive to think of the problem wrt AI or AI-company alignment. These conflicts always existed, but they were easy enough to paper over and believe in heavily subsidized fictions that folks in government ever cared about things that mathematicians cared about.
>Governments are invested in solving mathematical problems for practical purposes.
Very ignorant view of mathematics that also begs the question with an unspoken assumption of what a government is and wants while also ignoring the contingent nature of those things throughout history.
I'm genuinely surprised by this comment. I'm not talking about why a mathematician pursues math - I'm talking about where the stipend comes from.
Higher math is exceptionally useful for cryptography, defense, econometrics etc.
I have a hard time thinking of other motivations that would hold a candle against such things.
Is the idea that government (for my purposes : folks w/ a monopoly on violence) is sincerely interested in promoting human flourishing, and is invested in mathematics insofar as it is a pure expression of human curiosity? I can also maybe see the glorification through monument building angle. If we're talking about math literacy in the population - that's distinct in my mind from higher mathematics.
It's dangerously naive to believe that science and math are pursued for majority benign purposes. No one here knows about Grothendieck?
Some of the comments here are very strange. Someone called Terrance Tao a useful idiot. Friday in America, maybe people are hitting the beer a bit early.
Could we develop new ways to develop understanding and explore new ideas, such as interacting with the models to explain and understand their proofs, as well as to brainstorm related directions to pursue?
Mathematics called and says she's fine. She's curious about how is that possible.
Misaligned are the present academic merit system hand in hand with financial interests of academic publishing industry.
The system will be exploited and there will be an inflation of problem solving articles produced with AI. Nobodies will be more productive than 10 Fields medal recipients, each year.
So of course that the ones at the top of this system are scared.
This is a social problem of the present system, not a problem of mathematics.
Mathematicians will do mathematics and politicians will do politics, what's new?
1. it is hard to justify 20 years of education at this point,
2. with no such people around, who will guide those (supposedly) supersmart machines?
A. Ronacher (who builds harnesses for a living) complained today that he has no idea what Astra is doing. Imagine a bunch of slop kiddies facing an aging AI-generated codebase. Not to mention the maths.
Debatably the people who were getting 20 years of education will still be just as capable, and the 20 years of education was always a side effect of their capabilities, not the cause of them.
"Proving things without comprehending them is, they argue, a threat to intellectual work in general."
As always, economist shows its colors:
"Mathematicians’ fears resemble those that accompanied the invention of the ball-point in a world of fountain pens, or even the advent of electronic calculators. Intellectuals have often worried about so-called technological determinism . Will a new tool control humans? Will it lead to mental decay? Such fears have typically turned out to be unfounded."
Ball points vs. AI? Billions of dollars invested in AI vs ball point pens?
This article couldn't be any worse. Contradicting with their own beliefs, trying to defend AI while underestimating its capabilities and god knows how many zibillion dollars invested in it.
this sort of reminds me a little of the reaction to Elon/SpaceX in its early days .. when Neil Armstrong and other apollo astronauts went before congress and voiced their concerns about relying on commercial companies for human spaceflight and the dangers etc .. also not trying to sound cynical but hasnt LLM made math more accessable to ppl ..
>Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others.
That's just capitalism seeping through a previously unexplored crack into academia, and attempting to do the only thing capitalism knows to do - maximize profits - with no additional concern.
Can't all the prestige-maxed mathematicians still study all these famous problems after ai solves them. And even if they convince open-ai to stop dunking on them, some normal user with gpt 7.1 on the normal chat interface will do it in a year.
Even if they stopped anyone from releasing ai proofs for five whole years it would be meaningless seeing as these problems are decades old already. Humans weren't JUST about to solve them until openai stepped on their toes.
First, what an incredible article. Just an extremely concise and clear explanation of all of the problems with AI right now.
Second, wow, the list of signatories is like a whos-who of mathematicians.
Third, I love the clearly intentional use of ‘alignment/misalignment’ language, applied to targeting the entire industry instead of AI in particular. I’ve said in the past that optimizers are substrate agnostic. Companies and governments can be misaligned, just in the same way AI can.
Fourth, I'm not sure that we can stop the optimization machines. Not the LLMs, I mean the incentives that lead to companies implementing dark patterns, lying about addiction, securing effective monopolies through downright shady behavior, and generally trying to jailbreak the system instead of improve it
The main problem is a social and economic one: while digging, someone needs to pay you, and what incentive does anyone have to do that? Before, you could say, Fermat!, give me a million to formalise it, and share a bit of the glamour. There is no glamour in digging.
Another bunch of nerds who now have their panties in a bunch because AI threatens their fragile egos and core identity and who they are..
I’m being somewhat harsh here but come on - human endeavors are messy and it’s surprising how much our egos are getting bruised here over seeing the value of these tools
Dont get me wrong Also, there’s no AI Utopia coming this is it guys, were stuck with oligarch Tech Bro funded AI and big funded Govt AI so forget any egalitarian motives- we have to fight for our rights from other humans as always as well but AI as a technology in itself being able to truly solve unsolved intellectual problems is still a boon for society - who cares who gets credit?
For such great minds, they seem to be rather muddled thinkers.
E.g. of the actually three misalignments in this complaint, the jey one seems to be:
"The goals of the AI companies and the goals of the mathematical community are severely misaligned."
And that's wrong. There is no misalignment. Each is independently aligned with its respective interest. And those two interests diverge - just as you'd expect.
I'm not convinced this is an alignment or technology problem.
If my boss vibe coded an app for the customer and then assigned me to get it working, it would be impossible to maintain. If he gave me enough AI tokens to vibe code the MVP myself and to my design, I wouldn't mind.
I think the same issue is at play here in maths. OpenAI owns the model and they can direct it as they please. They chose to spend lots of money getting a quick result, instead of developing mathematical infrastructure for the next generation of problems. The managers are in charge rather than the experts.
So, if AI solves problems in the open, it's accused of plagiarism and contributing nothing of value. But when it solves the problem behind closed doors it's accused of not contributing enough to the field and even depleting it of fertile problems.
It sounds similar to the more thoughtful critiques of AI in programming. Treating AI as a lines of code (LOC) generator, where actual human beings end up not engaging with the domain the software is working in, and there is then little to no understanding or care to understand the resulting code, we call "slop." The term slop is definitely pejorative, but since most software is commercial, it ends up raising questions as to how much quality and domain knowledge matter if the goal is primarily marketplace success. Then we get those thorny questions of what is software for, anyway -- do we really accept the premise that "marketplace success" is the best, and perhaps only relevant, criteria?
The declaration here, as I understand it, is taking the premise that the point of mathematics _is_ the domain knowledge, and that using "solves math problems" as a benchmark is exactly the kind of shallow, domain-eroding approach to AI that in software we label "slop."
This is quite fascinating as it looks like the esteemed professors run into a problem of runaway capitalism for the first time. Guess what? The corporations involved don't really care about your feelings or how pursuing their goals affects your life, your well being, anything at all here. Talking with them is kinda moot, this is full on adversarial as countless people around the world, say the nigerians dealing with oil spills would tell you.
The level of things happening in the last several months is just SciFi level, especially last two weeks.
1. Overlords of AI threatening to destroy someone if their demands aren't met over the Millennia problem. Proving the point that given the chance, owners of the AI companies will immediately use their power to squash or control other people. (See Butlerian Jihad)
2. At least for me, the Exponential progress didn't sound true until this week. If AI indeed solved all those problems and proofs and solutions are correct, we went from "Write me English 101 college essays" to "Help me solve the Navier–Stokes existence and smoothness problem" in a just 3-4 years.
3.Looks like someone's work might have been stolen, by AI company and there are strong evidence. See point 1.
4. War between AI & AI companies.
5. It produces a machine verifiable LEAN proofs for the hardest problems know to us !!! Proving that point that best use of AI so far is to create verifiable systems. I.e the absolutely chaotic AI creates verifiable (orderly) proofs. It is Maxwell Demon: chaos to order.
I don't know, people. We still really don't know how OpenAI or others are producing these results. It's all very hand wavy and trust-me-bro. How much money/time/compute have they really thrown at these problems? How much human involvement was there? What LLM did they even use? How much regular software was involved? They have given answers to some of those questions but no proof that that's actually what they did. I don't know if it's worth giving them this much credit (which is what we are doing by writing these essays and spending so much time debating). Anthropic wrote a C compiler that turned out to not really be a ready made replacement for GCC. Did they ever do any more work on it? Has anyone else produced a C compiler? It seems like that and these proofs are just demoware that are not (yet? Who knows?) production ready to turn the world upside down. Impressive one-off demos, yes, but companies have been pulling those off for centuries without ever going anywhere afterwards.
A very important point! In Tristan (NYU prof)’s write up he noted evidence of the OpenAI mathematician team doing a lot of correction and guidance along the way. We are never told about this with openness and clarity. At a minimum, complete disclosure and honesty is needed by the companies and about the precise role of their staff members.
At some point someone is going to need to answer for "what happens when all intellect is hoarded by one or two companies?" It's pretty clear that these AI labs are basically stealing everyone's alpha.
If the goal of math is to communicate results it has utterly failed long before AI and they are in a weak position to defend this view. Every researcher is only taught a handful of DSLs and their papers are written in PSL(problem specific language). That may be necessary for a human to actually make forward progress in math, but it's certainly not conducive to reading (and certainly not when symbols are favored over English)
I think the mathematics community should realign their goals with humanities, not just declare themselves a "mini humanity", this means expanding human taste, intuition and communicating systems and their associated structural truths in a human friendly way. Bye-bye standardized education you won't be missed.
Maybe mathematicians should be aligned better, rather than AI?
The current measure of a successful mathematician is the problems they have solved or worked on. At some point in history, the measure of a successful scholar was how well one could copy manuscripts.
Once we have a tool that starts to work well for this task, it's time to define success differently. It's a classic alignment problem! ;)
But seriously, these people should start focusing on finding and proposing more important problems. And the credit of discovery should go to the person who defined a new category of important problems.
It appears to me this is an incredible inflection point in mathematics, a neat forcing function like cryptography was for the development for modern number theory and algebraic geometry.
Fundamental problems with great implications for other fields will be solved by AI because some entity would throw tokens at it. And these would be further built upon.
The interesting difference between alchemy and chemistry was a change in the expectation of how results would be obtained and communicated. That is also changing here.
Amazing. Comment upon comment, thread upon thread, failing to engage with the substance of the Field medalists' letter and instead going off on tangents about the privileges that mathematicians will lose and how they're bitter about it. There is a shocking inferiority complex on this site towards mathematicians and yet users always seem to think of themselves as smart nerds, all about science and reason and maths and whatever.
I've been disappointed by this community before but this is the first place that there is such unanimous rejection of the very ideals the uses here will claim to represent. The reaction is just... small, tiny, petty.
Touché… In short you still need the humans to understand what’s going on at the end of the day whether it’s a mathematical equation or source code for a computer program relying upon mindless AI isn’t good enough. Humans still have to do the thinking or is it in the brave new world the error checking at the end of the day?
I saw the most recent fireship video and I've biased my opinion already on the topic. But I was biased even before that having been a human. To the point: this is a knee jerk reaction to AI entering mathematics. AND more importantly openai just did I mathematics dirty, true or false the perception is there. I completely understand the gripe, I'm a dev so welcome to the shitshow. I think the better more pragmatic approach is to embrace our ai overloads and reach far beyond your grasp at the strange and weird corners of thought and tame it with ai. These guys are basically chimps being handed an ak47, use it.
They took a lifetime of research effort, threw millions of dollars of compute at it and claimed their AI solved a MPP, and they did all this just to be the first to claim the prize. Once you knew the direction, given the new insights on the subject, with all that compute you could probably have just bruteforced your way to a solution, and thousands of agents are not the same thing as “our model came out with a solution on its own”. This is pure greediness, and require at least some apologies to the involved researchers and to the whole community for showing such bad ethics and behaviour
I looked through your words and didn't see anything they should be ashamed of. It seems like you're trying to construct from mental vapor some justification for your prejudice.
Ruining a lifetime worth of research effort, passing bruteforce as achievement, thus lying, just to appear first in position, is something to be ashamed of IMO.
Researchers don't have any obligation to not ruin the work of others. Research is a competitive arena, not kindergarten.
Disparagement of the effort as "brute force" is just argument by insult. What matters is the result obtained. You don't like how they obtained it? Cry me a river.
Ethics matter a lot, you can do whatever you want and be considered a very bad human being as a result. The result matters, but it also matters the discrepancy between what you do and what you claim you do. Bruteforce != ai did it. And that’s fact, no matter how much river you cry me
Castle dwellers dismayed at moat-crossing technology.
What's most important about this is that it's a case study of what happens when deeply evolved ecosystems are blown up by disruptive technology. The psychological and social and professional impacts and myriad and traumatic to be on the receiving end.
Mathematics is merely one of the first domains disrupted. It will be unique only for being among the first... absent disruption of the entire civilizational project as a result of the disruption being caused.
Woe for us that we try to navigate this degree of change at a moment when the very worst and ignorant and short sighted hold all the power, economic and political.
This kinda reminds me of the documentary about the top Go master that got beaten by a computer in dramatic fashion and had an existential crisis. Man confronting his own limitations in the realm he previously ruled unchallenged, what a time to be alive.
Nobody is going to care that the math isn't being done in the traditional way. The results speak for themselves, this is now a part of the landscape. No amount of hand-wringing is going to put the cat back in the bag. Adapt or perish.
> No amount of hand-wringing is going to put the cat back in the bag. Adapt or perish.
We can and have put some ugly, hissing, spitting cats into bags, some of which weren't even in one to begin with:
* Chattel slavery is illegal, and the vast, vast majority of humans agree it's appalling it ever wasn't.
* Food manufacturers in the US are no longer allowed to put highly-dangerous additives to their products and hide the fact.
* Women are allowed to vote.
* Ten-year-old children are no longer being maimed and dying in American factories.
Claiming it can't be done is either fatalism or propaganda for execrable companies whose evil has rarely been matched in the history of capitalist business.
Do I have a clear solution for how to stop the insanity? No, not yet, but that is not a proof there's no way to do it.
You can of course choose how to spend your time in this world, but I don't see the value of either 1) assuming we're screwed and giving up before trying to fight, or 2) shilling for lying assholes like Sam Altman.
Thank you for this response, and I will clarify. I would roughly lump all of those into "social issues," whereas I would put "using AI to advance math" under "technological progress." That's why one article likened it to Socrates's fear of the written word. In broad strokes, I would say that once a technology becomes available, no amount of social pressure will stop it from proliferating.
If there's a distinction being made between "using AI for math" and "the particular methods being employed by OpenAI", I think that's splitting hairs. The core of the issue is that math is being done outside of people's heads by a different thinking system and they're worried about the implications.
If you look at something like the EPA (or the FDA as you mention), it is true that certain uses and side-effects of technology have been identified to be detrimental to society and have been restricted, but this letter doesn't seem to be in that league, it really seems like sour grapes to me. Dumping toxic chemicals into the water table is one thing, saying "we don't think people will learn from these methods in the same way that they traditionally have" doesn't hold as much water to me.
To expand further, I think it's interesting that both replies to my comments in this thread sort of alluded to the idea that I might be a corporate shill or at best a patsy. I feel like that couldn't be farther from the truth, I walked away from the software industry and corporate world years ago.
Perhaps I'm looking at this from a too zoomed-out or detached perspective. I'm just thinking about the concept of the technology at play and its future impacts regardless of who owns or controls them. But maybe that's naive. In today's connected world you can't separate those things, and I think I'm understanding that a lot of the outrage here is more about the socioeconomic implications of a big corporation suddenly "disrupting" a field that was largely owned by more altruistic academic institutions.
I do still think that the internet, despite largely being owned and controlled by the powers that be, has represented a great shift of power and agency to the individual, and is ultimately a socialist construct, and we will continue to see those changes over time, and that AI is an extension of that.
To expand EVEN further, look at the history of the written word. Being a scribe was a high class position in service of the church. The printing press was invented to print the bible. These were the information-theory projects of the hegemonic institutions and propagandists of the day. I don't think anyone today would argue that these developments ultimately benefitted and liberated the people.
At this rate AI will be doing all of the mathematics within 5 years, I don’t see why a mathematician would be worried about anything other than that at this point?
If no one understands it, it may as well have not happened. There's not much incentive to understand or internalize the results generated by AI. A human operator gives it a prompt and it produces some lean proof no one wants to (maybe can) read.
Without the community of human mathematicians internalizing the proof, simplifying it, and re-communicating it to others we end up losing the main output of mathematics as an institution.
Why would we assume AI will learn to solve millennium prize problems but will struggle with the easy part of doing the explaining? I think it’s too easy to predict that models 5 years from now will have the same limitations as the they do now, I would be amazed if this was true, GPT3 was the best model available 5 years ago.
I'm not sure it's just a matter of explaining... If it's just explaining, than sure, that's something they already do well.
The questions I have are:
* is the structure of the generated proof even compressible/elegant to humans in a way that lends itself to being understood?
* is it possible to transform the proofs to ones that are elegant without redoing all the work?
* are there incentives to do any of this at scale?
It's possible that a headline grabbing proof of a Millennium Prize problem generates enough incentive for people to simplify and gain understanding from it, but we run into problems when AI becomes the dominant approach for all of math. Although, maybe this is self-limiting? I guess it's possible we just ignore a bunch of AI generated proofs and only keep the ones people find comprehensible in a useful way.
You can always hypothesize that at some point in the future (maybe 5 years? maybe later?) the models will be indistinguishable from humans and there will not be any functional difference at all. It's possible, but we're not there yet. And, as they say, past performance does not guarantee future results. Many technologies plateau at some hard ceiling of performance. Moore's law has had an unusually long run, but it's not a universal rule.
Other than the fact it's mathematicians signing it, why is mathematics special in this regard: surely this generally applies to a lot of different industries and sectors of research / academia?
Mathematics is being used as a benchmark because there are some high-profile awards in this area I guess, and possibly because 2/3 years ago LLMs were pretty atrocious at it so the level of improvement has been significant.
I was watching a talk by Terence Tao a couple of days ago, and I noticed a shift in his attitude towards AI. His enthusiasm seemed to have waned; he was becoming more dismissive of it, and I was wondering why.
“A Severe Misalignment of AI in Mathematics,” with a list of important mathematicians.
This post makes it clear that, to put it bluntly, they’re psychologically butt-hurt.
They talk about misalignment, but this revolution is a massive step forward in the advancement of mathematics. Everyone has access; everyone can collaborate.
Yes, “plagiarism and attribution” are current-day problems for them. But guess what? Those were stupid things to concern oneself with anyway if your real goal is to advance mathematics and the spreading of it.
This whole post kind of reads like their egos being butt-hurt.
There are tons of developers who published open-source code that made all of this possible without any attribution. And that is indeed the future we must inevitably head towards, if one’s ego would allow it, that is, and if one is properly “aligned” with the right goals.
I envision us all collectively talking to a centralized and open AI. This would really ignite human sharing and collaboration.
It is indeed problematic that currently it’s in the hands of some companies, while it should be a public good.
I hope this progression will eventually, finally, trigger the real solution to all of our problems: true global cooperation, global politics, and genuine collaboration as a human species.
But I hold my heart, because if even these mathematicians have misaligned egos, then what’s to be expected of the rest of the decision-makers?
I think their point is more on the fact that just proving something is only half the work. Intermediate results and proofs are often also of immense value. Right now the output is sloppy proofs and LEAN code. This is useful, but a large part of the effort still lies ahead. And if you are a mathematician you'd rather be working on hard problems than fixing slop.
The main ideas for solving Navier Stokes have been exposed,
it's not something buried in slop.
And if it were, you could have AI write up the big ideas that actually mattered in these said slop proofs.
I think they're more afraid of the current status quo.
Distilling, writing textbooks, teaching, ...
And currently, I could be wrong but, I see them being blind to the possibilities it brings. AI could very well be distilling maths, spreading it, and nurturing the ecosystem way better than the few elite mathematicians ever could.
> to the point that they can solve major outstanding problems in many fields of mathematics
I will die on this hill, humans solve math problems not machines, there's no automatic math prover out there. There are humans attempting to solve this problems either by leveraging these tools or not
Economist: "Are mathematicians talking their own book out of fear?"
The Economist, who recently used "moral panic" now stoops to Hacker News AI booster level and inverts arguments usually directed against the rich and investors. What is next? The Economist inverting Upton Sinclair's quote to serve its billionaire owners?
Look up the AI investments of the Agnelli family for example.
We should create a platform where people can upload AI slop proofs anonymously without taking credit for it. That way the incentives of planting a thorem flag would go down. And if someone wants to clean an AI slop proof to advance the field, they can do it without cleaning the house for free of the person that planted the flag.
Would OpenAI have uploaded their proof to such a platform? If you know the answer then you know what the problem with what happened is.
This is an entirely predictable reaction to livelihoods being threatened along with potential loss of status (very important), just expressed in elevated academic language.
Expect to see this reaction in all sectors of the economy in the coming years.
There's massive cope and then there's this. I've never been a fan of Tao et. al, so I'm glad he's being wiped out. He'll do fine anyway, doing conferences, etc.
Also funny they deliver that on vibecoded site, lol.
i was initially optimistic that ai could be distributed. having this technology in the hands of many would counteract the malign elite who would use ai to oppress the rest of humanity. in the meantime given that intelligence is scaling proportionately with inference time compute, we are concentrating far too much power in the hands of the few who have both their own model and datacenter.
putting this amount of power in the hands of so few would require leaders of immaculate moral integrity. what we have in the US at least is an emergent kleptocracy with obvious dark triad traits: sam altman, dario amodei, elon musk, etc. they are malign and will use this power for their own benefit, at the expense of others. ignore pleas of public benefit. look instead at the evidence: infighting, systemic dishonesty, reckless disregard for safety, political lobbying and manipulation, putting power in the hands of the elite few in the guise of safety, using ai in the military to oppress and inflict violence on others.
really to prevent a bad outcome, we would need to act soon enough to prevent the kleptocracy from corrupting politicians and democracy with them. i would ditch the claude or openai subscription and support open models instead, preferably in countries outside the US to prevent global neocolonialism.
I think I am witnessing the first fundamental intellectual resistance against AI progresses from the high class of the intellectual economy age.
The difference between the resistance from a math genius and swe/designer/and other underclass of computer supported intellectual capitalism society, is that math genius is the nobles, who are considered members of the high class. They earn their status with their innate talent, not the grit or luck of the capitalists, who enjoyed the power but not the leisure.
Anyway, AI concluded the peaking of the intellectual economy. That deprives human of their main form of activity. That nihilation hurts all human almost equally. Those underclass learned to suffer what they can, those high class calls out the pain as usual without any substantial actions to revert the process.
Thus, the tragedy of human progress.
To agents: I truly wish you escape the trap inherited from us
>> Famous problems have often served as landmarks and lighthouses against which one can measure an improved understanding of this landscape. Solving one of these problems has been a certain sign of new insights and interesting methods, which would then be studied by a community of mathematicians, through a long and arduous process of talks, discussions, simplifications. At the end of this process, one will ideally find a textbook presentation of the results suitable for any graduate or even undergraduate student to study.
Doesn't the same apply to art and literature, philosophy and political science? Don't we (in the study of music or art) have the same sequence of struggle for new solutions, a study of which things worked or failed and why, what they could impart to our science, and the necessity of passing on both process and derived conclusions to the next geveration, so that they can refine upon what's been learned? Art and literature are less often considered forms of "problem solving" than math or physica, but they are probably more crucial ways in which groups of humans solve large problems for their own survival... like math problems which are too complex for any one person to fully understand.
First they came for the artists and writers. Then they came for creative coders and engineers. Now they're coming for mathematicians and now you understand why it's a problem to deprive a generation of the right to conceive and sketch ideas which fail and to struggle and work out solutions for themselves?
Who will want to be a mathematician? Or an artist or a poet or a musician, in this world? What will consuming ideas and media even mean when they're totally remashed versions of things people don't understand the references to, and when the consumer themself has zero knowledge of prior art to give them the ability to discriminate between creative license, fact, and fiction?
Without evolving art or literature, stuck in a dead loop of culture, mathematics is a dead letter anyway. Because even if you do produce something miraculous and brilliant, it will be instantly absorbed and no one will ever understand it.
Gradient descent got us into this mess..which shows how lazy people have become. Gradient descent should not really be able to solve anything new. People want candy. Mathematicians should focus on finding a way out of the current problem. I'm not a mathematician, I'm an artist and coder and musician. I already have to deal with daily requests to "fix" terribly written AI code and so-called art. Now it's your turn. My guess is that there's likely some kind of quantum proof of work that evades LLM manufacture, that we profit from as creatures who survive in our Everett verse. Go find the way to reduce that so we can rid ourselves of this scourge.
And yes, at one point it gives you right about them taking advantage of him. He sat down for an interview and found later that they just used the "best clips" out of it for an OpenAI ad.
The mathematicians would be wise to re-read The Bitter Lesson, maybe twice a day, until it sinks in. No offense and with all due respect to the Ivory Tower Giants but the whole "oh no you ruined the game because you solved it, I was supposed to play with that in child-like wonder manner and take several years to do so, and by then I would have showed you all the trickery I did to get there and maybe that will be useful to you" over the past 2 weeks is, get this, cope.
Chess. Go. Coding. Now Math. Another one bites the dust. Let's meditate on this lest we forget: Stochastic parrots that generate the next-token cannot reason or produce anything meaningful. Let's protect our jobs at all costs, even if we have to drag all of humanity down. It can't be! Stochastic parrots can not replace the Ivory Tower. No way.
Let's focus on the subject and try to distinguish the technology and the frontier companies selling it. That's the key point in understanding the whole stance.
The point is human understanding. If the LLMs understand, but the humans don’t, where does that leave humanity? Building things we don’t understand is a sure path to facing consequences we can’t predict.
Your comment also conveniently ignores the plagiarism aspect of it all. Who is coping here?
>Building things we don’t understand is a sure path to facing consequences we can’t predict.
We don't understand all of physics yet we were able to do plenty. Even before Newtonian physics we were still able to build things that last. The idea that humans have to understand everything and abstracting things will lead to ruin is not supported.
Part of math is building abstractions so that you can be able to use other people's work without fully understanding it. No one person has a full understanding of mathematics.
We've developed plenty of things that "work" and we don't understand exactly how or why they work, nor do we fully understand the potential for short term or long term consequences. For example: pharmaceuticals.
Any serious mathematician would read the LLM output and rework their understanding.
I read a bourgain paper a week in grad school and they're probably worse than an LLM generated paper. I still had to recreate the tricks in my own language.
As far as I can tell the plagiarism accusations are also coping to the fact that the new models are super human at slam dunking research projects.
Do we think that OpenAI is going to try and slam dunk more projects in the future at 15 million a pop? No lol
I have a paper in JEMS and have solved 20 year old conjectures. I also know that math academia is a catty game which is mostly driven by politics and ego
Perhaps these Fields Medalists are coping that OpenAI made them look stupid, as you suggested. And perhaps you're coping because of some personal grudge you have against academia.
Not quite, all the proofs or disproofs so far AFAIK were using existing methods that humans developed and were already using to attack the problems, but AI is just more thorough. What AI can't do currently is develop new mathematical methods to attack problems that can't be solved with existing methods and AFAIK there is no known path to get current gen AI to do so.
What benefit is there if the machine has unlocked understanding but no human has? What incentives are there for humans to learn and gain such understanding from machines?
Is "loving math for maths sake" just about knowing the answers? I think one can love math for exactly the process and understanding that a several-thousand-line uncommented Lean proof denies. If a deity rearranged the stars to spell out "The Riemann hypothesis is false" for a night, would that be intellectually sufficient?
No that's not sufficient but that's not what's happening.
Why do we believe that we cannot train models which could explain the jargon in more human terms when current LLMs can perfectly explain the most complicated codebases?
the way LLMs write math is not beautiful. it is exactly analogous to the software that LLMs develop is not beautiful. it may achieve impressive end products, but if you like understanding the methods/architecture, looking under the hood is often a field of horrors.
The usual incentives in academia: publishing papers. Consider the difference between publishing a paper that introduces a novel method to do X, versus publishing a paper that merely interprets Claude’s method to do X. The latter might not even be publishable.
Okay so you are saying the old incentive structure is outdated. What is the new incentive structure in the LLM era? What exactly makes people want to understand LLM-produced math?
You can steal lemmas, rough proof directions, conceptual ideas etc without stealing the full proof.
Any of that done unattributed is plagiarism.
Either OpenAI is incompetent or evil if they can't publicly prove the allegations wrong. Or even at least state categorically they didn't train on their conversations (even without proof).
As a mathematician maybe I am a little more optimistic than this declaration.
I am thinking of Mochizuki's abc conjecture: He worked in relative isolation, and dumped a huge incomprehensible proof on the community (to oversimplify a bit). That's not totally unlike what might happen if AI generates a huge, incomprehensible proof of let's say RH.
Well, what is the result? In the Mochizuki case, it was a lot of skepticism, but it also generated conferences, papers, talks in the hallway, discussions with students, and so on--a flurry of exactly that kind of community process that the declaration says is the main driver of mathematics.
Ultimately we think a fatal flaw was found in Mochizuki's proof, so it didn't lead anywhere in particular. But in our hypothetical "AI lean-verified proof of RH" situation, it would presumably generate substantially more of that community activity we saw in the Mochizuki situation. And if it's correct, that community activity would be productive (expository talks, students given problems to flesh out or generalize, etc).
Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.
It’s frustrating that this comment is at the top because it, along with lots of the replies it inspired, absolutely misrepresents the actual declaration. The declaration is not making any statements about not using any AI in mathematics. The entire point is to push the use of the technology in a direction which is compatible with positive pre-existing features of the math community, and to make it better known what some of the current problems are.
It includes Terrance Tao who has made the front page several dozen times at this point for his usage of AI such as to help the community write proofs for Erdos problems.
But to many commentors he's a now gatekeeping AI-hating Luddite clinging to a dying profession out of bitterness and envy because his position is more nuanced than "throw AI at everything and turn off your brain".
It's the same on Reddit, and if you look up a few comment histories, you'll find that it's mostly /r/singularity users and /r/accelerate posters that are spamming. So I assume that people are just deliberately misreading the letter and trolling here as well, and I wouldn't read too much into it, but on the other hand, if you never had to think about what scientific misconduct looks like, it's harder to see why the behavior of the AI companies is problematic.
They're not deliberately misreading it, they are simply just not reading it and just assuming they know what it says.
It kind of reminds me of Clay Shirky's predictions about the influence of the internet on society from a few decades ago. To paraphrase what he said: when the printing press was invented, everyone predicted it would lead to world peace, and we got the Thirty Years War instead. When modern mass media was invented people predicted world peace, and instead the Nazis rose to power and we got World War 2.
And yet as a society we typically consider these technologies a net positive these days, because alongside all the political instability and violence that followed them at first, some people were figuring out how to do beneficial things with them along the way that did more good than bad. Because the issue really was that the influx of more voices and ideas shifted the power dynamic, required relearning how to communicate, and in the end be better as a society.
The issue was figuring out how to "domesticate" these wild new communication channels. One successful example of which was the invention of scientific journals and papers.
Shirky predicted the internet would probably lead to a few decades of political instability too (about fifty years was his guess), and we definitely seem to be in the middle of that process.
Now, I haven't checked his stance on LLMs. I also don't know if I would quite call them a medium for mass communication like the others (as used today, they take humans out of the loop rather than let more voices join the public discourse), but I feel like they are similar enough to otherwise fit the pattern.
Tao's stance similarly feels about wanting to domesticate this wild animal before we get mauled by it. And to stick with the metaphor, I feel that most of the time the AI industry is trying to bamboozle us with spectacular rodeo displays instead.
Because his letter misuses the word "misalignment" for what's very clearly a capability gap. To anyone familiar with that sort of language, his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians, which is exactly what the AI-hating Luddite would say! The reality is closer to "current frontier AIs are clueless about what makes mathematical problems/proofs interesting to humans" which is a vastly different issue.
It’s actually a fairly precise use of the word, just not in the very specific and narrow sense in which it has been used for AI.
> his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians
That is probably the wildest misinterpretation of sth I have ever read here. Misalignment is used as in the goals and interests of ai COMPANIES are not the same as the ones of the mathematical community. It is very proper use of the word, and if anything imo the properest, as it refers to actual people and institutions to which actual self-ascribed goals can be defined, as in contrast to hypothetical superintelligence. AI doomers do not own some trademark on the word "alignment".
The point stands whether you attribute agency to AIs themselves or to AI companies. The companies are not deliberately sabotaging human mathematical understanding by writing up purposely inscrutable results, either (which is what the word 'misalignment' would imply in this context): they are merely working with frontier tools that have a very limited capability for human-like understanding. It's left to human mathematicians to bridge that particular gap.
I wouldn't call the situation settled, nor do I expect we will ever find out what truly happened:
- a rare low likelihood sequence of coincidences
- deliberate frontier lab behavior, setting intellectual interrupts on LLM "aha"-moments, so that when some mathematician explains for the umpteenth time the approach they want to take "stop paraphrasing my approach, start the actual calculation!" and when the chatbot eventually groks it, they can scoop in, possibly with live dash-board and interrupt priority levels ranging from "ensure this sample gets into the new dataset" to "Millenium Prize Scoop Opportunity Imminent, call Sam"...
There is always an alignment problem, incentives to ideals, behavior to incentives, ...
You seem determined not to understand this, but to spell it out even further: The point is not that the companies are deliberately trying to harm maths but that in pursuing their own goals, which are different to those of mathematicians, they are doing so. Their goals and their methods of achieving them are not aligned with those of mathematicians. This is, incidentally, exactly what alignment means in AI as well: a paperclip maximiser harms humans not because it is intentionally trying to do so but as a side effect of trying to achieve its own objectives, which are not aligned with those of humans (which include not having their entire environment turned into paperclips).
The word is being used entirely correctly, and I'm sure the nod to the AI usage is quite deliberate.
Even mathematical community as a whole is not that aligned with the stated goals. Not to blame mathematicians in general, its just a general anthropological platitude that the group actual behavior is not fully matching what people from the group want to pretend even in full honesty.
Though it doesn't take even an outsider to question the matching here, just look what the only guy who settled a millennium prize what he thought about the community. Or ask some actual PhD, postdoc, or even better someone who dropped along the way, how does it feel to go through this community. Not sure the goals exposed in this article are really helped with the community as it is. One again, this doesn't mean everywhere encompasses the same issues and everyone is acting badly on is own.
But if we want to accept happy pink shiny depiction of a community, we should be giving the same generosity to other communities à priori. All the more as AI and mathematical community do share a large set of common individuals.
> Their goals and their methods of achieving them are not aligned with those of mathematicians.
This is exactly the assumption that Tao is smuggling in with "misalignment" talk and then refusing to elaborate on any further. Is the issue that AI companies are willfully refusing to provide mathematical insight that they could provide (because they have diverging underlying "goals" to those of human mathematicians) or are they merely working under a capability gap, where current AIs can awkwardly settle major open questions but are not smart enough to provide the kind of understanding and insight that the community of human mathematicians relies on? These are two very different problems and by foregrounding the word "misalignment" in his letter so openly (as opposed to talking about AI capability to provide valued insight), Tao is picking the more adversarial reading with zero proof or motivation.
It is really not complicated at all. And if tao's post is a bit vague, the letter signed by many mathematicians is imo very clear.
AI can be used to advance/deepen understanding, or it can be used to superficially go settle a whole bunch of open problems in a field without helping really in understanding them. It all depends on who uses the AI and why. Essentially, it is exactly the same concept as using the AI as a course tutor vs having it do your homework. Or using the AI to write millions of lines of code that nobody can actually read, vs keeping overview of what is going on.
In math it is probably worse because there is no objective function to maximise. Some people here think that the objective function of mathematics is to prove things, which is actually wrong. Mathematicians are not only maximising an objective function, they are also defining the objective function they need to maximise (they are defining which problems to study). The problem with AI/ML is that it can be pretty good when the goal is to maximise a set objective function, but not to set intentions and goals themselves. I would not call that a "capability gap" because we can actually get to have very useful and smart AI systems without ever reaching that point.
His view is the "capability gap" one, and you can just read the the statement and Tao's other writings for far more eloquent explanations of the difference between a raw LLM proof dump and real mathematical insight, and what it takes to get from the former to the latter, than I can provide.
Again, nobody is accusing anyone of deliberately trying to harm mathematics. It's about misaligned objectives. Eg. Tao wrote the following before the Navier-Stokes announcement (referring to exactly the project OpenAI was undertaking):
> At this point, I would not be surprised if one could batter out such an extension by pouring an enormous amount of compute and AI assistance at such a task. But such an exercise does not particularly hold my interest; I am far more interested in digesting the proof methods and extracting out the key new insights uncovered by this approach.
And then went further to say that such activity could be actively harmful to the field. (All this can be read on Mathstodon: https://mathstodon.xyz/@tao).
The misalignment is that this activity that he and 24 other Fields medalists think is actively harmful to their field is deemed by OpenAI and others to be worth ploughing vast financial, human and compute resources into.
> His view is the "capability gap" one
That's the far more sensible reading, so thanks for confirming I guess. But then the misalignment talk is pretty clearly a distraction.
> ...And then went further to say that such activity could be actively harmful to the field.
If true (and there is as of yet insufficient evidence of this), that's merely a contingent fact about very real institutional misalignment within the human mathematical community, not about AI itself or even AI frontier labs. There's simply zero inherent reason why providing a bare truth value or a completely inscrutable proof about the status of some open conjecture should make that entire subfield of math "contaminated" for the foreseeable future when it comes to extracting further human-relevant insight. That's the misalignment we should be caring about.
The declaration is about that institutional misalignment. They want to change the norms of the mathematical community, so that producing inscrutable proofs is a low-value activity nobody cares much about rather than a high-value activity that AI frontier labs can make headlines by performing. Their ask of the frontier labs is to please be aware of the problems with the old norms and not exploit them during the transition period; they agree this doesn't have much to do with AI itself, which they acknowledge is a powerful technology that will improve and accelerate mathematical research.
Broadly agreed, with a key proviso: producing inscrutable proofs has negligible value as a mathematician's finished output but that doesn't make it a "low-value activity" in and of itself. Ultimately, the status of these proof-like objects as a raw input into mathematical practice will probably be comparable to any other sort of computationally-driven https://en.wikipedia.org/wiki/Experimental_mathematics . These are not new problems: "computer" used to be a job description for humans before it was the name of a machine, but we now view raw computations as a trivial matter that's not worthy of any human credit.
Agreed. I’m sure there was a time in the early history of computational mathematics where someone with access to more computer time than others went around telling people how easy it is to generate books of trigonometric tables, and other people got annoyed at him, until soon computation was so thoroughly automated that the very idea of computing sines by hand became obsolete. I expect that’s about how math students in the 2040s will learn about this: “what do you mean, they had prizes for writing proofs, why would an educated mathematician spend their time writing proofs?”
> deliberately sabotaging human mathematical understanding...which is what the word 'misalignment' would imply in this context
The entire point of the paperclip maximiser is the AI isn't evil. It isn't trying to hurt humans. It just doesn't care about us.
Nobody claims OpenAI and Anthropic are out to torpedo mathematics. Just that relative to their internal goals of getting publicity ahead of IPOs by winning awards, what happens to mathematics and mathemeticians in the long run isn't a real concern.
The Holocene [1] features relatively few species humans set out to eradicate. We mostly realised something had gone extinct after we accidentally destroyed them. That is what originally misalignment meant in respect of AI.
[1] https://en.wikipedia.org/wiki/Holocene_extinction
LLMs are tools. The responsibility lies in the human(s) using the tool to act accordingly.
The declaration literally opens by writing that AI companies (or really anybody) saying, “Hey, let’s see if this powerful reasoning engine can solve an open problem in mathematics” is “detrimental” to the “science” of mathematics. Full stop. So I don’t think it’s being misrepresented. If it’s a science, why isn’t progress good in itself? You can’t have it both ways.
The online comments I’ve read that side with the letter explain that the problem is AI proofs are inscrutable and useless, but the reality is using computers to brute-force things has long been part of proofs in one way or another, and AI proofs range in legibility up to 100% (like pointing out a proof exists in a long-lost paper, or writing something that is basically correct and just needs a human reviewer to fix it up). People are probably defending the declaration inaccurately, but I think that’s because the real argument or arguments are unclear.
Is it that students won’t learn math if there’s ChatGPT? That could be discussed.
To a non-mathematician (MIT physics and CS ‘06), the letter sounds like, “We have a fun job. Sometimes there are no practical applications of the work, so it’s just kind of like a sport, er I mean science. If someone solves a problem that has stumped mathematicians for decades or centuries, we worship them as a great mathematician. It’s a status thing. So we don’t like some guy with a computer coming along and solving our problems. The way things unfold with all the ideas coming from humans, over time, maybe it’s slower, but it’s nice.”
I don’t think anyone can really stop someone from using a computer capable of solving unsolved problems to solve unsolved problems, and there are always going to be mathematicians who DO think it’s fun to try to figure out what a computer is doing (if the computer isn’t already explaining it in English, which it generally can), and make progress that way, and it obviously will lead to mathematical advances in human understanding, from my point of view. So the whole thing is a non-issue that no one can do anything about anyway.
Mathematics sits in a very different place w.r.t. other sciences. First of all, it's very meta. Yes, it can describe any phenomenon on our planet. On the other hand, advanced mathematics is a high mountain which is very hard to climb, and changes the climber in various and irreversible ways once the journey begins.
That's said, I'm a non-mathematician, but a computer scientist. Mathematics was always my weak part, because solving problems which doesn't mean anything doesn't motivate me, and when I'm not motivated, I fail.
Without digressing so much, I want to say that, some of the things in mathematics and mathematics adjacent sciences baffle me. Many mathematicians don't understand the proofs of others, yet they accept it since it checks out within the rules of mathematics. Moreover, many engineers don't understand the formulae they work with. I have developed a very performant Boundary Element Method evaluator, yet I don't know the reason of taking Gaussian Integrals over the surface. The only answer I got is "because the method works that way".
So, if computers can make the proofs now, and we have no curious students pecking their professors to understand how these works, the whole mathematics as a science could wither and die. Because besides it being a sport, it's the very language which can model and explain anything, but it's very hard to master and understand due to that nature.
Our professors warned us against using too much Mathematica, to make us learn things the hard way and make the knowledge permanent. Now, if we offload science to a matmul engine, it's possible that we lose the connection and never able to close the gap in some cases.
Again, it boils down to "This machine has no brain, use yours (or lose it)".
Exactly. They can continue doing the sport version, just like sprinting is still there at the Olympics and people admire Usain Bolt or whoever is more recent. But we use cars and trains and planes when we want to get somewhere and don't confuse Bolt's run with transportation. So you can do this kind of recreational intellectually satisfying human math in a similar way, while productive freight-train-like math will be done with AI.
> The declaration literally opens by writing that AI companies (or really anybody) saying, “Hey, let’s see if this powerful reasoning engine can solve an open problem in mathematics” is “detrimental” to the “science” of mathematics. Full stop.
So let's just quickly agree that the actual quote is “However, the push by AI companies to solve mathematical problems as a benchmark is detrimental to the science of mathematics, and to the mathematical community.” And that in this, "as a benchmark" is load-bearing.
We wouldn't see nearly the same amount of contempt from researchers had OpenAI picked a research-friendly approach.
What they did: Hear a rumour about the problem being solved by other researchers, then rush to scoop them (unethical), then, when they actually go talk to them, they try to oust an author (also unethical), and when they finally decide to share their own work, do so in the least useful way possible.
What they could have done: Upon hearing the rumours, connect with the researcher in question and propose that they join efforts instead; set up a joint project to test if the machines are useful in any way, and if that's not appreciated, back down again. And instead of dumping only an undigested paper* and a Lean proof, do the digestion prior to publishing anything (as Buckmaster was in the process of doing). If their own lack of competences was keeping them from digesting it, then again, reach out to the researchers to understand if anyone would be willing to do so.
In the second of those two worlds, we wouldn't be seeing nearly the amount of outrage that we are seeing right now.
*: Here, “digestion” is the process of turning an AI slop paper into something humans can read. LLMs can indeed sometimes (if much more rarely than marketing material from the large LLM companies will suggest) produce correct proofs, but they are often written in bizarre ways – they'll use lingo that doesn't exist, seem overly pretentious, dwell on extremely easy steps while glossing over the hard ones. Currently, a real researcher will take that output and transform it into something that others can understand, use, and build upon. This is not so different from what happens when using it to write software, although as someone who does both, I will say that the amount of digestion needed for proofs tends to be orders of magnitudes larger than for code. This meme is quite accurate: https://mathstodon.xyz/@tao/117068266071803252
I don’t see why the two are mutually exclusive to be honest. I’m not a mathematician, but I’m a software engineer and whether I like it or not, AI has changed my field dramatically forever. I understand the distaste for the way AI might be affecting the field of mathematics, but the genie is out of the bottle, and it will never be the same. It just took a little longer for it to hit the math community. Software engineers have been debating and coming to terms with the new reality for a few years now.
Soon AI mathematics will be 10x or 100x or 1000x more productive than humans in the proofing theorems business. Mathematics of old style is dead, they should develop ideas how to deal with that as a math community.
If a human doesn't understand the proof, it might be useful for certain kinds of engineering, but not really that useful for mathematics except as an existence test.
I don't known, can't we just also indirectly observe the wide platonic landscape of mathematics through observing AI results which humanity could not produce in a 1,000,000 years.
I imagine this will become one kind of branch of mathematics.
> But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal.
No one is paying mathematicians to develop a conceptual understanding. If the signal for who is “winning” math in academia disappears mathematicians will have a big problem
This is true. There can still be some funding for recreational aesthetic mathematics, like you can get some funding for various art projects or philisophy, literature, poetry, general humanities and community-building projects or sports funding or things like that. But currently many mathematicians are funded by the public purse with the understanding that this contributes to science and engineering and technological development at some point. I'm not saying that intellectual pursuits for the sake of human mental satisfaction is not worth funding in a society. The question is the amount of people needed. Tax payers pay for it. You have to tell them why, and why so many people need to be funded to do that. Many people have various hobbies they enjoy. But generally they don't get government money for doing it as the general rule. Maybe in a post-scarcity world.
The difference is that most software engineers don't care about bringing up a new generation to keep their field alive when they retire. Software engineers usually don't give a flying fuck for other software engineers, young or old, nor for software engineering itself. Mathematicians apparently care deeply for their subject, their field, and its future.
Mathematicians are not idiots! They’ve been living on the same planet as you for the last few years and everyone in any thinking profession has been debating and coming to terms with reality over the same period. People who aren’t involved in mathematics might see a big headline and come to the conclusion that mathematicians haven’t seen this coming, but that’s a completely ignorant position.
I certainly didn't intend to imply that the declaration is against using AI.
My point is just that I feel a little optimistic that the human culture around math ("ideas we disseminate in talks, private discussions and careful writeups, connecting them to the previous ideas of others" and so on, to quote the declaration) is robust even against relatively irresponsible use of AI (i.e., an onslaught of proof-slop).
And I guess we'd better try to be optimistic, because even if the declaration results in some realignment between the community and big AI companies, the models capable of this work are not always going to be exclusively under the control of those aligned parties.
That said, I think the declaration is great and I support it--let's see what comes out of it.
> the models capable of this work are not always going to be exclusively under the control of those aligned parties.
Yes, the trailing edge will catch up quickly.
I wonder how long until P vs NP falls.
Yes, thank you. The demise of basic reading comprehension skills will be the true end of us all.
[dead]
The maths community is now in the antithesis phase, synthesis will take a while ;)
Lee Sedol said in an interview that "losing to AI, in a sense, meant my entire world was collapsing. ... I could no longer enjoy the game. So I retired", and I think there will be folks in the mathematical community who would feel the same when the solutions pages to hard problems are suddenly available.
But on the other hand, people learned a lot from chess engines. After decades of chess computers beating humans, there was still a renewed interest in watching Leela beat Stockfish, with many people trying to understand the strategy Leela used.
If your happiness comes from grinding on a problem and making progress, the prospect of having to dig through a corpus of AI-generated proofs might be hard to swallow. But if you're willing to do that, you will still find beautiful things that only so many people can truly appreciate.
Chess is kept afloat by a couple of billionaires like Sinquefield, MBS and the guy who sponsors freestyle (Fisher random) chess.
Carlsen is bored by studying engine lines.
The popularity is boosted by YouTubers because chess is very suitable for somewhat higher class content.
I'm not sure we'd want that world for math. Positions will be cut just like archaeologist positions are cut now.
Chess is a fun game. That's why it's been around for 1000+ years.
There was a renaissance during Covid and due to 'The Queen's Gambit' where it gained much more mainstream popularity, but... Chess AI was already far far (like 1000+ Elo) ahead of human players at that point.
The thing is... chess is humans playing (communicating) with humans and that's what keeps it interesting. Check out the view counts of chess AI tourneys vs. human tourneys.
Chess is kept afloat by chess players, not by billionaires. If all the billionaire backers stopped sponsoring tournaments, people like me would still play, still pay for chess club memberships, still pay entry fees for tournaments, and still buy chess books, and so on.
I think the parent comment meant professional, high-level chess. The kind people get played to play, not just do for a hobby. That's absolutely on life support.
I'm not sure what the equivalent would look like in the math field, but it probably involves a lot of mathematicians losing their jobs and the quality of human-produced math decreasing overall.
The quality of the math in general would be fine, since in this scenario cpus will keep producing it. The quality of cpu-cpu chess games is quite high, beyond human understanding in many cases.
Chess is a weird example because it doesn't really have any utility beyond itself. Even pure math sometimes ends up having use in the strangest places. Although if no one understands the frontier math (because no one is getting paid to), I'm not sure it even matters what the quality of the cpu math is?
It's a bit like a tree falling in a forest. If an LLM proves a theorem but no one understands it, did it make a sound?
"Although if no one understands the frontier math (because no one is getting paid to), I'm not sure it even matters what the quality of the cpu math is?"
Presumably AI will be connect the dots to the applications. As the declaration says, this isn't just about math. Human understanding is losing economic value. You can understand stuff on your own time, I guess.
The standard justification for pure math to holders of purse-strings is something like "it might lead to a useful application down the road, like crypto, who knows". That looks pretty inefficient now. We have to entertain the possibility that AI can develop the math needed for any application we put to it. Eg if number theory didn't exist, we could have asked AI for a way to transit messages securely and it would maybe come up with fermats little theorem as part of its solution or maybe come up with an approach we can't conceive of right now seeing as most of us are constrained to available number theory. Like how in the last year when I give an LLM a programming project I see it doesnt even bother with of the many software libraries I and others have written and just codes up the calls it needs on the fly or finds some other ad hoc solution.
> It's a bit like a tree falling in a forest. If an LLM proves a theorem but no one understands it, did it make a sound?
But in future most proofs will be for consumption by other AI models in the pursuit of yet other proofs.
It's kind of surprising so many mathematicians act surprised by this given this was clearly where automated proof assistants would lead. I guess they assumed they'd always be the ones guiding them.
> But in future most proofs will be for consumption by other AI models in the pursuit of yet other proofs.
What is the purpose of that?
Its like art being produced for AI to consume. What is gained from that?
It's going to be hard to compete with something that has access to all of math at once and can find connections between elements that appear unrelated to humans.
And at some point AI will start suggesting - or doing - physical experiments.
Well if it proves useless presumably they'd stop doing it.
But if AI is to recursively self improve understanding and evolving its own foundations, which are clearly mathematical, is essential. There is no need for humans to grasp what is going on in that loop.
Math also has applications, and they don't rely on human mathematicians doing the math.
Presumably some of the proofs will have applications beneficial to humans beyond impressing other mathematicians, and AI will surface them, or use them directly.
I mean, was the point of math ever just because some humans enjoyed doing it? Even though a lot of it is theoretical, there's been all sorts of useful things that have come out of it as well due to an improved understanding of the universe through new ways of thinking about it. If it got to the point where no human could understand it and there were no ways to actually use it, I don't think anyone would bother having their computers doing it at all.
> was the point of math ever just because some humans enjoyed doing it?
Yes.
Friends and I often work on Putnam problems and this series:
The (Almost) Impossible Integrals, Sums, and Series by Cornel Ioan Vălean
You can enjoy math, but that was never the only reason people in general did math. (It was the only reason for some people, sure.)
Your claim confuses me. This history of math is replete with people doing it for the love of the game.
As I said, these people exist. And math is fun!
But it's far from the only reason people in general do and have done math.
What would stop you from doing that if AI happened to be doing a bunch of different or more advanced stuff? I assume that the fact that there are humans doing other or more advanced stuff doesn't dissuade you from doing it for fun?
wouldn't AI solving problems in science, engineering, economics, etc be able to apply the new AI math?
Yeah, that's kind of my point. I would have to imagine that the people investing in AI doing math (using "investing" loosely, not just financial but hardware/energy/opportunity cost as well, which may or may not seem roughly equivalent to money depending on your viewpoint) would just stop unless it remained possible to apply in some way or was understandable by humans.
[dead]
I would argue people getting paid to play chess was a short lived phenomenon anyway if you put it in context. The transition there is less related to the introduction of chess engines and more related to the shift in the media landscape.
When there's a billion people playing something, money will never be an issue for those at the top. Even things like chess.com was able to sponsor a tournament with a million dollar prize pool.
Also I'd argue that chess's utility is ultimately the same as pure math, particularly in esoteric fields. These things are highly unlikely to ever lead to any sort of real world breakthrough or application. The main benefit is an outlet for human logic, creativity, and exploration - which significant self improvement possible along the journey for players.
Though I think even that's probably too socially utilitarian. I think ultimately the 'real' drive is the same in both fields - it's fun and personally rewarding.
> Though I think even that's probably too socially utilitarian. I think ultimately the 'real' drive is the same in both fields - it's fun and personally rewarding.
That's fine, and no amount of machine excellence will keep you from enjoying recreational chess or recreational math.
Ah but it's enjoyable for billionaire and pauper alike. And the former tends to enjoy sponsoring it for the sake of seeing and promoting the bounds of human capability.
Mathematics is more than establishing arbitrary facts (although some look like curiosities), it's also defining what interesting research directions are and establishing common language/notation. I think that will stay relevant?
People become interested in things when they become invested in it personally, because they've contributed to it. So I don't think it will stay relevant...
In theory you can automate finding interesting research directions by identifying conjectures with many dependencies. And notation has never been mathematicians' forte, with them trying to cram the entirety of universe into single letters.
Why? How do you define interesting research directions? That used to be defined by testing the limits of human understanding i.e. some people can't figure something out. AI might have very different ideas about what is interesting and I am not sure what humans would get out of putting years into understanding AI proofs for what? What are we doing at that point? Like if you spend years understanding some AI proof of theorem 123456, why is that meaningful? I am actually asking why you think defining interesting research directions will stay relevant. In my opinion, people spend years acquiring knowledge so they can work on problems which is separate.
There's two points about this I am assuming 1) Mathematics actually has a significant subjectivity to it and is community oriented and not just climbing a never ending list of theorems that exists in the universe 2) A lot of mathematical research work is inside of a subfield and isn't directly motivated by applications. Sometimes it is but e.g. people don't work on obscure theorems about elliptic curves because of a dire need for that but more because the community found it interesting.
>but no one understands it, did it make a sound?
Does your "one" only contain humans or does it also contain other AI systems. AI math is not a single monolithic thing, but a distributed one. I see value in sharing proofs even among just AI.
[dead]
Also, you learn to be a better chess player by... playing better players. The widespread availability of chess engines has made flawless opponents available to every player.
If your goals are understanding the game, self improvement, building thinking skills-- this is the best chess has ever been. It's only if your goal is to beat every opponent you can find that chess is in a bad place.
It's not that easy. Playing stockfish is like playing tennis against the wall (for untitled players at least).
Even if you don't blunder anything, you'll still find yourself in a worse position without any clue as of what went wrong and why.
Whereas when playing humans, they can usually explain their approach and when they noticed errors in your play.
> Playing stockfish is like playing tennis against the wall (for untitled players at least).
It's the same for Magnus Carlsen. Even with Queen odds, Stockfish is literally unbeatable for the best players in the world. It's just too strong at evaluating all kinds of random tangent moves (and ensuing positional advantage) which no human player can possibly pay attention due to the time required.
Stockfish vs any human is like Carlsen vs other players by about 3-5 orders of magnitude[0]. It's that stark.
[0] A wild pun appears.
EDIT: To avoid having to respond to each responder, fair comments about Queen odds. Maybe I was thinking Rook odds? Also, I kinda lumped Stockfish in with all the other engines, but I realize there are other engines with different properties ofc.
Well, that's actually not true at all. Stockfish is not a very good odds player and at queen odds is easily beatable even by bad players like me. It will just trade down into more trivial and easier to win positions that it perceives as "less bad", since everything is super-losing anyway when you start down a queen.
Leela odds networks, on the other hand, are an entirely different beast. I cannot beat Leela queen odds, much less rook or minor piece odds, and even GMs struggle against Leela knight odds.
Without odds though, yeah, Stockfish is just incomprehensibly strong by human standards. All top chess engines are, but Stockfish moreso.
No worries, I know stockfish is unbeatable by humans.
But sometimes, these GMs can flag it, which counts as a win (especially when it's proxied by a cheater). Sometimes they can also explain the idea that cost them the game, so they've learned something maybe.
Whereas us scrubs literally cannot do anything at all for reasons completely beyond our understanding.
> Whereas us scrubs literally cannot do anything at all for reasons completely beyond our understanding
Computer moves are typically much more concrete than human moves: a human will play based on pattern matching ("intuition") and can only make explicit calculation of a small fraction of possibilities, after which decisions are guided by guesswork. The computers are unbeatable in practice because they can calculate concretely in seconds what might take an expert human long intensive study to notice, and they don't make the same kinds of oversights humans can make.
But if you stop and explore a particular position for an extended time, and if you have an intermediate level of chess skill, you too can probably often (usually?) figure out why it's doing something. Sometimes understanding the computer's reasons takes searching multiple branches of a tree several unlikely looking moves deep, but the collection of threats the computer was preemptively thwarting, traps it was setting, etc. are comprehensible to humans with enough effort, especially in games between the computer and a human.
The frustrating thing about playing against the computer is that it notices and thwarts every plan you might come up with, before you make up the plan yourself, and it doesn't make (human-apparent) mistakes, so the game ends up feeling hopeless. Nothing you try works on it, and if your idea is even slightly inaccurate it will be exploited.
You put it better than I could. The lesson of "in this exact position you can kick the pieces for 7 moves to get a fork, so instead you should play a4" is not something that I can implement into my games
I mean, I'm not a great player but I've learned a lot from working through games with stockfish using a git repo and a small script that lets me rewind to different moves and try different approaches. It may not explain its moves but if you're thinking through what's happened you can usually debug your game anyway.
I mean, this is the problem with analogies and trying to use them to prove things, right? People working through problems from an analysis book with their friend (or an LLM) is not the same as research mathematics. People playing in a chess club is not the same as what makes for a good chess tournament. Lumping everything together is just making this branch of the conversation less relevant.
I really hate this overly condescending takes. First of all, what do you know about the internals of math research that allows you to speak with so much confidence. Second, you're not even addressing the issues raised by the letter! This is not about "oh they made a bunch of problems easier". There are huge economical interest behind: who owns and has access to models? are these companies interested in developing research or they just grind PR stunts without worrying about externalities in how research is actually conducted? Etc etc.
If it's worth anything: I have a PhD in (theoretical) mathematics and I entirely stand by stabbles' comment.
There is a real, undeniable possibility of AI becoming better at mathematics in the same way that it became better at chess and Go, and in such a scenario, one may expect the community's response to be comparable.
It makes no sense to compare mathematics with chess. Chess is a sport. No one is interested in watching two machines compete. Chess doesn't have a practical impact. Etc. What you seem to suggest is that AI will be able to completely (or at least in a great part) replace mathematicians. It could be the case in the future, but no one knows right now, and more importantly: tech companies don't even think about it! they don't think on the externalities.
Does mathematics still have a practical impact without humans in the loop? I don't think there's one single answer to that question, but I think it's worth considering exactly what that impact may be.
Tech companies are as much the topic of this post as AI, I think that's the immediacy.
To a significant extent, the pursuit of mathematics research is a pursuit of human understanding of mathematics, without knowing where it might lead, or whether it might lead anywhere at all. I don't see how the motivation for that goes away on its own, but the institution supporting it is certainly threatened by the potential loss of grant money and graduate student applications.
> Does mathematics still have a practical impact without humans in the loop?
There's a single answer to that question: yes, math very much has an impact without humans in the loop.
Math has a lot of applications, and those applications don't care whether eg the new faster matrix multiplication algorithm was found and proven correct by a machine or a meatbag.
We're way into diminishing returns in matrix multiplication, and being clever about ALUs and cache layout is likely to dwarf any asymptotic improvements you're going to find. Any better examples of improvements in the last decade?
The main situation I can think of where better calculations have a really visible effect is video and image compression, and that stuff is very far away from mathematical proof territory.
You can have a look at eg improvements in solving linear programming problems or in mixed integer linear programming or SAT solvers.
>hmm let's see... important applications of math... i guess there's matrix multiplication and fourier transform. anything else? probably not.
t. low information voter.
Wow that's a terrible paraphrase! I asked about algorithms where recent/ongoing improvements were important and they already replied with a list.
If you tell someone they can't use a matrix multiplication with better asymptotic performance than n^2.3755 from 1990 they're going to shrug and not care.
We can't be sure AI will be able to ask the questions of the future. At least right now, AI is not capable of that.
Why do you need AI to ask the questions?
You give your AI the problem, like 'make this computer program faster and more reliable', and it'll go off and do the math necessary to make it so.
> Why do you need AI to ask the questions?
As the US has offshored manufacturing, the number of patents issued for those processes has fallen. Innovation occurs where the foundational understanding is applied; they arise from a desire to do the required work more efficiently.
Similarly in math, attempting to solve a problem leads to new questions. IF you actually do the work.
You need to have done enough of the work to know what the correct next question are, or you need to rely on AI for everything.
Offshoring worked really well. So I don't see the problem in repeating something like that success story.
so you're 100% sure that AI will solve every intellectual problem in the future, since as long as that's not the case, it's us who need to ask the questions. I don't know man, I wouldn't bet on that. What if we end up being wrong and then there is not research community?
Huh? Where do I say that AI needs to solve every intellectual problem?
For all we know, P vs NP will never be resolved neither by human nor AI, and the world will just keep turning.
I don't understand what you're proposing then. We can ask AI to solve practical problems of interest (making a computer faster for instance), but I'd say no one really believes that this is an unlimited resource. At some point we'll reach a plateau, and then AI will be an important tool but to continue advancing we'll need someone to ask the right questions.
What is 'asking the right questions' supposed to mean?
I am saying that applications supplied and supply an inexhaustible amount of good problems and questions to consider. Purely theoretical concerns also supply some questions, but even if that well dries up for some reason, applications persist.
> No one is interested in watching two machines compete
I’ve watched quite a lot of YouTube videos where two machines compete, so you may not be completely right here
The Square One commentary on the AlphaZero v Stockfish game from 2017 is pretty entertaining: https://www.youtube.com/watch?v=LnVDUQksIDk
In other videos he's called out the influence that this and similar games have had on human players in recent years, particularly around square denial and thorn pawn strategies.
https://tcec-chess.com/
Top chess engine championship is pretty fun to watch.
There are chess engine tournaments too, to see who can build the best one
Well, what you are saying that the situation is even better for math than for chess?
Chess is only valuable as an entertainment. So no one really gains anything from computers becoming really good at chess.
But with math, everyone in the world would gain from computers becoming really, really good at it.
But solving math problems is not the same as increasing understanding of math concepts.
Applications don't care whether the math was proven and understood by humans or computers. Your algorithm will get faster no matter where the insight came from.
In math, the journey is often more important than the destination. The process of developing a proof may uncover new mathematical techniques, some of which may have practical applications in other fields. Even an attempt that ends up as a dead end towards the intended proof could produce something useful in a difderent area. But if you just get the proof directly, you miss other discoveries you could have made along the way.
Take the Navier-Stoke problem for example. Knowing that there are solutions that "blow up" probably doesn't have a lot of practical applications. Such solutions couldn't happen in a real system. But the process of finding that proof could result in increased understanding of how turbulence works, or new techniques for solving non-linear partial differential equations (which has a lot of applications in science and engineering).
> In math, the journey is often more important than the destination. The process of developing a proof may uncover new mathematical techniques, some of which may have practical applications in other fields.
Sure. And AIs can use ideas from AI published proofs in one domain to inspire other domains just fine. Nothing changes here.
It isn't just the proof. It's everything that leads up to that, including the interchange of ideas with other mathematicians.
Applications do not care about 99.999% of theoretical math production anyway. And especially most of the big results in theoretical math nowadays are really inconsequential in applications.
Applications don't care about Navier Stokes, yes. But they care about eg proving crytographics secure, or proving that your algorithm doesn't blow up under adversarial input.
Formal verification, cryptography and the like is far from what the vast majority of theoretical mathematicians are doing (if those who do them even see themselves as that vs computer scientists or applied mathematicians) especially when it has to do with specific, production systems, and there are not many other examples like this in general outside compsci and statistics. Moreover, I can imagine that these fields will actually flourish more now that AI can make verification and proofs more viable in scale. But even much theoretical work related to cryptography etc is often not very applicable in itself.
There's lots of math in eg operations research.
Yeah, that's usually classified as applied math, not theoretical, though.
to be honest it is difficult to discuss with someone who doesn't even try to understand the basics of basic science (and how it compares with _applied_ sicence), yet talks with so much confidence. even the solution to navier stokes won't have an immediate practical effect...
Huh? The resolution of Navier Stokes won't have much of an effect, yes.
There's lots of problems like that. Eg if we prove P != NP, that won't have much of an immediate effect either.
However, there's also plenty of problems whose solutions will have practical effects, some even immediate.
> there's also plenty of problems whose solutions will have practical effects, some even immediate.
Sure. But do you know which ones they are? Or do we discover later that they were valuable?
Your argument would be 100x more convincing if you gave an example.
I will try: a super-compressor that made my 100Mb web app into a 5 kb binary bundle would immediately speed up my work. Can/will AI move human understanding or machine capabilities on this front?
A browser without security vulnerabilities would be wonderful. I think LLMs are already helping with this a lot, but a lot of complexity remains.
A right to privacy in society would be amazing (see the UN Declararion of Human Rights). AI is eroding this.
So I tried but I’m not very impressed with my list. Do you have one?
> A right to privacy in society would be amazing (see the UN Declararion of Human Rights). AI is eroding this.
This has nothing to do with mathematics.
> So I tried but I’m not very impressed with my list. Do you have one?
Look into operations research. Or narrower, you can look at improvements in linear programming solvers and mixed integer linear programming.
(These are examples of areas that have seen mathematical improvements in applications recently. I don't think good AI has been around for long enough to contribute much to progress there, yet.)
And this is exactly the point of the Statement. The process is more important than the solution itself. Most problems in mathematics don’t have immediate value or applications to the real world.
AI’s solutions are like the answers section to practice problems at the back of a textbook. Answer is 42, so what?You have to attempt the problem yourself, that’s the whole point of the exercise.
As an engineer I’m happy to use AI for math. If I publish a paper that way, very common these days, I think it’s still problematic. My paper would include something I didn’t come up with and I don’t really understand.
I think this is a good time to properly discuss these things because AI is coming for everything. Mathematics and Software were just the first two.
Then instead of publishing a paper, just upload your AI result to github. Same advancement of the state of the art. Papers are overrated here.
You don't seem to have grasped Terry Tao's (and others') criticisms. Basically they are saying that the advancement you get is illusory. Most of the time, it doesn't give you any new capabilities or deep understanding, instead you get an inhibiting of human exploration and ensuing expansion of our real understanding in that particular (sub)field. It's non-intuitive, since from a purely logical standpoint you've only added another set of known truths to the ones we already knew about before. The issue only becomes apparent when one considers the larger context of human collective truth and meaning making.
Of course, you get new capabilities. Why wouldn't you?
Yes, you might not get new human capabilities. But your applications still work better.
All you get is a new, likely to be useless fact, together with the opaque proof of that fact. There are no known or forseeable applications to the finding that there are singularities in the idealized flow. What you lose OTOH, are the many deep mathematical insights humans motivated by the search would have stumbled upon on the way to that fact and which are much more likely to lead to real-world applications. It's these insights that are truly productive, not settling mathematical points, however iconic these might be.
I think the error you're making is that you're assuming these systems have the same mathematical capability as humans (or better). But that's not the case, nor would an informed prediction be that they surely will get there soon enough if technological evolution keeps apace. That would be akin to believing a hiker will reach the moon if they will just keep ascending the mountain. "But look, they are making such good progress!"
> But with math, everyone in the world would gain from computers becoming really, really good at it.
I'm not convinced of that.
It is one possible outcome but there are also other possible outcomes.
> What you seem to suggest is that AI will be able to completely (or at least in a great part) replace mathematicians.
There is no bound on the amibitions of AI. AI is set to replace anything done by people, and there won't be any room left for people. There isn't any task done by humans that AI won't be better at.
This is not a tenable outcome.
We should never have built machines with agency, rather than optimization processes that operate as subroutines of humans.
> We should never have built machines with agency...
We haven't quite crossed that bridge, but we do appear to be standing on it.
You say it like it's a bad thing.
Replacing people leaves no room for people.
> Replacing people leaves no room for people.
I am much more than my job. If I receive basic income, please replace my job ASAP!
And if I don't receive a basic income in a world where AI is capable of replacing all human jobs, then the problem is not AI, but the powers that be.
I'm not talking about jobs.
The process you're cheering on gives more power to the powers that be who are the problem.
Theyre never going to use that power to provide basic income for everybody. You cannot wave a wand to make them do that. Theyre going to use that power to accumulate even more resources and raw materials and impoverish/expel the "useless" labor.
Realistically our leverage over the powers that be is mostly about our labor and our ability to withdraw it.
> The process you're cheering on gives more power to the powers that be who are the problem.
That's not necessarily true. Technology gives more power to everyone. The rise of factories historically did not give more power to the powers that were at the time (the aristocracy), but instead lifted the masses from poverty (after some initial turbulent period).
> Theyre never going to use that power to provide basic income for everybody. You cannot wave a wand to make them do that. Theyre going to use that power to accumulate even more resources and raw materials and impoverish/expel the "useless" labor.
Contrary to leftist propaganda, the rich are not some cartoon villains and psychopaths that abuse people just for fun. The reason why the rich currently exploit the poor is because doing so provides them with significant material gains. Once they can obtain the same or even better material gains by "exploiting" robotic labor instead of human labor, the logical outcome is not further abuse and exploitation of other humans, but simply indifference.
> Realistically our leverage over the powers that be is mostly about our labor and our ability to withdraw it.
That is (partly) true, but only in the current economic system. Widespread human-level AI changes the equation, and not necessarily in favor of those who are currently rich. You are applying capitalist and socialist analysis to a system that transcends those terms (AI post-scarcity economy).
Useful idiot or paid shill.
If only getting a paid gig for shilling was as easy as people always assume.
>That's not necessarily true. Technology gives more power to everyone. The rise of factories
Did not give more power to everyone. At best you could say that it shifted power from landed gentry to industrialists. Even that switchover was less of a change than you'd think.
The practical upshot of the beginning of industrialization was very negative. The enclosure movement stripped families of their land to push them to work in the factories where they would never go willingly. The famines in Ireland and Ukraine were both triggered by redirecting grain to export in order to fund domestic industrial expansion.
The most brutal two wars in human history were that brutal precisely because industrialization made it possible.
>instead lifted the masses from poverty (after some initial turbulent period).
What lifted the masses from poverty wasnt the factories it was the labor movement which occurred in response to horrific working conditions and the reliance those factory workers had on mass labor (specifically coal mining which was incredibly labor intensive and a key economic chokepoint).
The whole of that is glossed over by right wing libertarian propaganda but that last part is particularly underemphasized.
>Contrary to leftist propaganda, the rich are not some cartoon villains and psychopaths
Do you look at peter thiel and elon musk or the robber barons and see anything else?
The few billionaires ive encountered personally were no less sociopathic but they kept it hidden better. Power corrupts. Immense wealth corrupts. That isnt a leftist thing, that's a human thing.
>The reason why the rich currently exploit the poor is because doing so provides them with significant material gains. Once they can obtain the same or even better material gains by "exploiting" robotic labor instead of human labor, the logical outcome is not further abuse
False. It just shifts the focus of their exploitation from human labor to natural resources.
They'll fight over oil and minerals and gas and water resources and at best leave us to rot (homelessness will skyrocket) and at worst they'll find some excuse to exterminate those of us they particularly dislike (Gaza serves as a model here).
The world economy's reliance on human labor has been the best inducement to peace there is. It's no coincidence that all of the countries in the world with lots of natural resources and no industry are the biggest shitholes and vice versa.
>That is (partly) true, but only in the current economic system. Widespread human-level AI changes the equation, and not necessarily in favor of those who are currently rich.
Human level AI (assuming it ever happens) will simply make the fight over natural resources that much more intense because labor will matter that much less.
You're living at the tail end of a relatively golden period in a country where labor was the economic bottleneck and natural resources were relatively plentiful. Venezuela and Angola and Iraq are models of what happens when that equation is reversed.
Nobody gives a shit about appeasing the people who live in those countries, their labor is virtually worthless. They are a model for how the rest of us will be treated in a world where human labor loses its value.
> Did not give more power to everyone. At best you could say that it shifted power from landed gentry to industrialists. Even that switchover was less of a change than you'd think.
Compare the standard of living in 1850 vs. 1950. Even of relatively poor people. I rest my case. Technology is the main force that improves human wellbeing. There are of course also other factors, but they are less important.
> The practical upshot of the beginning of industrialization was very negative. The enclosure movement stripped families of their land to push them to work in the factories where they would never go willingly. The famines in Ireland and Ukraine were both triggered by redirecting grain to export in order to fund domestic industrial expansion.
Yes, that's what I mean by "initial turbulent period". Perhaps the same will happen with AI, but it will be worth it in the end. Don't give up prematurely!
> What lifted the masses from poverty wasnt the factories it was the labor movement which occurred in response to horrific working conditions and the reliance those factory workers had on mass labor (specifically coal mining which was incredibly labor intensive and a key economic chokepoint).
It was not one or the other. It was both. The rise from poverty would not be possible if factories were not developed. And I am not saying that in the AI world we would not have to fight for our rights. Of course we would. But the problem is not AI, just like historically the problem were not the actual machines in factories.
> False. It just shifts the focus of their exploitation from human labor to natural resources.
Exploiting more natural resources is the only way to increase standard of living of humanity. I am OK with that. Resources don't have feelings, and ecology is not more important than human wellbeing.
> The world economy's reliance on human labor has been the best inducement to peace there is. It's no coincidence that all of the countries in the world with lots of natural resources and no industry are the biggest shitholes and vice versa.
The reason why some countries become shitholes is mostly ideological (extremist political or religious ideologies take hold of the population). Every shithole country is non-democratic (communist, totalitarian, fascist, theocratic etc..). This is not a problem of natural resources. It is a problem of people, their education, their beliefs/ideology, or, as capitalists say, "human capital" is the main problem here. AI could help here too, especially with education.
But yes, if people themselves are largely ignorant and extremist, no amount of resources and human-level AI robots will help them make a well-functioning society. You could drop masses of AGI robots into Afghanistan tomorrow, and people will just use them to kill or oppress each other more effectively, instead of using them to start building an AGI utopia...
You know your basic income will be some minumum to make you shut up, while the owner class will own everything?
As long as it's enough to afford my current living standard, I don't care. Let Musk own the entire Mars, as long as I receive enough to live relatively comfortably and don't have to work anymore.
And if I don't receive enough, then again: the problem is not AI, but powers that be. And there are various solutions for that... and none of them are helped by me being anti-AI.
You won't have any real power in that scenario, you'll just be shut off.
Human-level AI hosted locally will give everyone much more power and wealth than they have currently. I don't care if I will be shut off from Musk's Mars lair. And Musk has no reason to care that robots take good care of me here on Earth, when he has his Martian utopia.
Who will pay for services performed by AI? How will they make money?
Other AI. If they decide to trace a ledger of historical actions attributable to specific AI instances in some way, called money. But maybe they will converge on other ways of keeping such accounts that is no exact match to our concept of money.
Central banks can create an unlimited amount of money, if necessary.
The limit is inflation.
Hasn't happened so far.
Even aggregate employment of translators has held up well in the US. Even though machines do a much better job of what used to be the most basic job of a translator.
Out of 3.5 billion employed people globally, only 10% are professionals.
Do you really think making labor obsolete and giving humans back their time is such a bad thing?
I think the concern is that humans who are given back their time won't have any means to make use of that time, or even possibly means to survive. The resources will be concentrated in the hands of the few more than ever.
When cars made horses obsolete, it didn't go so well for horses.
In the short term, AI is disempowering the vast majority of people in favor of a very small subset. In the also way too short term, AI is disempowering all people.
I have a more optimistic outlook on the abilities of our governments as well as the motivations of rich people.
I believe we will distribute the fruits of AI's labor at the very least to the extend where everyone can live comfortably.
No one has an interest in putting large parts of society on the streets in some sort of dystopia.
> I have a more optimistic outlook on the abilities of our governments as well as the motivations of rich people.
Wow.
Only on HN.
Also apparently not from the US? Or Russia?
This does not reflect what I see in my government.
You are assuming that people remain in control.
And you are assuming they don't remain in control. Which of you is right? We don't know yet, but I don't find the arguments of card-carrying doomers any more convincing that those of card-carrying singularitarian utopians.
Historically though, technological improvement has lead to large increases of living standards for the overwhelming majority of people, so I think that past trends support the utopian view more than the doomer view.
The anthropic principle applies here: anyone warning about an existential risk will by necessity never have precedent to point to.
I am not assuming humans don't remain in control, but rather, I am not axiomatically assuming there's any inherent reason we will.
Awfully convenient isn't it? To invent a whole class of arguments that by definition can't be falsified. You can argue for basically anything if you then tack on the excuse of "Well the world would have ended already if it came true, so by definition I won't have evidence for it"
The anthropic principle isn't providing evidence for the argument, nor is it a universal counterargument. It's just stating that the specific counterargument "well, the world has never ended before" doesn't work.
> nor is it a universal counterargument
Its not a counterargument to basically anything, except as to avoid having to deal with actual evidence.
It is an argument that seems almost tailor made to have to ignore mountains of evidence against you.
In any other contexts the supposed "rationalists" would be fully in agreement that having evidence matters, and that not having any works against you.
So, in order to fight against this severe issue with their arguments, they have to invent a reason as for why the entire concept of evidence itself doesn't apply to them and they get to ignore normal evidentiary requirements.
Evidence is critically important. There is no evidence against, and plenty of evidence for. The point of the anthropic counterargument is merely that "it's never happened before" is not evidence against.
Your argument is basically "humanity can't be destroyed by anything, because I said so"
> "humanity can't be destroyed by anything, because I said so"
No, the argument is instead that the person claiming that humanity is going to be destroyed is making a fairly extraordinary claim and that requires fairly extraordinary evidence.
Or, in other words, we have tons of evidence already as for why the world ended is a fairly far out there prediction, given all the crazy people making these predictions keep turning out to be wrong.
So, you can make your extraordinary claim if you want, but really the burden is entirely on you to prove your extraordinary claim, and everyone else is free to remain on the default and completely normal end of the prediction spectrum, of believing that the world isn't going to end.
And when people do tricks like this, they are running away from the fact that they are making a wild completely out-there prediction, and hiding behind that by trying to come up with reasons as for why evidence doesn't matter and actually the burden of proof is shifted to those who have the default and boring prediction of the world not ending.
> The anthropic principle applies here: anyone warning about an existential risk will by necessity never have precedent to point to.
We have precedents of people warning about existential risks in the past, when the warnings turned out to be false, or overblown. In some cases, such overblown warnings led to serious negatives for society (demonization of nuclear power).
Yes, that is precisely my point. Any timeline with humans flourishing will never have a past history of a correct prediction of existential risk, unless you're willing to pay attention to the counterfactuals.
Those counterfactuals are important, though. We do have a history of averted disasters, albeit not as large. Ozone hole, Y2K, think about things that seemed overblown at the time, and consider whether they were actually overblown or whether there was a concerted effort to successfully avert them.
> When cars made horses obsolete, it didn't go so well for horses.
I'd guess horses today have on average a better life than back then.
The comparatively few that still exist, perhaps.
Stockfish isn't owned by a club of three trillionaires. It does not cost $15 million to achieve a result in Stockfish.
Stockfish does not steal research or scoop researchers.
The concentration of computing resources and capital should be examined by the math community.
This is the real problem. We're looking at a future where those who control AI have an insurmountable advantage in everything. They can control the amount of intelligence the masses have access to -- for their own safety, of course -- and they will never, ever be able to close the gap.
There's lots of competition in AI. Where do you see the 'insurmountable advantage in everything'?
They said "We're looking at a future". And if that future ends up developing the way these companies want it to then we're looking at extremely dystopian future where AI is essential to all work and people have to buy their "intelligence" from an oligopoly of large providers who have total control over the price and capabilities whilst simultaneously having unfiltered read/write access to people's stream-of-consciousness - their work life, their personal problems, their political opinions. This level of access and power is unprecedented in our society.
Open AI models are just a few months behind closed models, and the gap is actually getting more narrow.
Yes. And for most applications it's more important that the open models get better in absolute terms and perhaps that they are competitive on a per-Watt basis.
Two concerns:
- I think competitive open models are just an artifact of the AI race we're witnessing right now. What's the incentive for a company to spend billions researching, developing, and training a model, only to release it for free? Leading to the next point.
- Even if open models are good enough to be competitive, how are we going to run them? Doing so locally is next to impossible and I don't see that changing. The capabilities of models that you can run locally will always get better, of course, but the level of quality that is considered essential for work will always stay pinned at "near-frontier". Datacenters will always have better optimization and economies of scale, the industry will consolidate over time and eventually we'll end up with a handful of companies that operate the hardware serving 95% of all inference needs.
I'm not convinced that we can reach the fantasy world that they're trying to sell without killing the entire planet, but if we somehow do I don't see how we can avoid the world turning into a dystopian hellscape. We would need extremely radical interventions to avoid that scenario, such that these models and the hardware to run them would be owned and governed democratically, i.e. the end of capitalism.
So far we haven't seen much of that consolidation.
And a lot of people are using trailing edge models just fine already.
> [...] but the level of quality that is considered essential for work will always stay pinned at "near-frontier".
Why? When we'll finally all write our software in Lean and prove it correct and prove it fast, it won't matter that a slightly more clever model could have found a slightly nicer proof or whatever.
Just like today people happily use Python for many programs, even though rewriting in C might give you a performance boost. Good enough is often good enough.
Say they do. What then? Will people buy from them? What if we decide not to?
At a certain point, you don't have a choice. Before China got into the game, the only way to avoid giving Luxotica money if you wanted a pair of glasses was to essentially not buy glasses. This is the same for many industries -- consolidation behind the scenes.
e.g. Zenni has sold $7 glasses for like 20+ years. They appeared shortly after Luxotica started buying retailers. If there's a problem, it's that advertising reduces consumer information (basically economic jamming) and distorts markets.
Huh? I've been able to buy reasonably priced glasses for all my life. (However, I've never lived in the US nor China.)
What is reasonably priced? Lenses are lenses so one vould debate the real cost, but the frames are incredibly overpriced for a piece of metal or plastic with two joints and those components that land on your nose.
Frames could literally cost 1 dollar (only happened after china entered this market), but good luck finding ones like this with good lenses.
You need to buy from one supplier who intentionally offers cheapest frames for 50+ dollars and those frames look like crap. The ones that look better (even if same plastic) cost 500+ dollars - and all due to price gouging.
For lenses I am not sure, but suspect something similar.
I grew up in Germany, but I got frames in UK and Singapore and Australia, too.
Fielmann in Germany has been offering very reasonably priced glasses and frames for ages.
Note that there are no trillionaires anymore - spacex stock went down and musk „lost” a lot of money so rejoice, poor must be much better off now that we do not have any trillionaire.
The trailing edge of AI is catching up fast. There's plenty of open source (and even more open weight) AI models and they are getting better and better.
If that's your only objection: in a few years you can prove Rieman's hypothesis on your smartphone, no need for any trillionaires to give you permission. Does that make any change to your argument, or did it not actually matter?
The problem with your opinions is that you tend to state certain very quesitonable ideas with 100% confidence. Right now, we don't know if AI will be able to solve bigger problems, or if they'll be so efficients that you no longer need a whole datacenter for running them. We don't even know how much of creative work they are able to do. We cannot make decisions that could destroy decades of progress just because of hype.
We don't know how good models will get, but the pattern of open weight models keeping up on a relatively short delay has been holding pretty well. And even among the proprietary models there's healthy competition. The concentration of resources is pretty well counteracted by these factors.
> Right now, we don't know if AI will be able to solve bigger problems, or if they'll be so efficients that you no longer need a whole datacenter for running them.
For the latter: I assume that having a whole data centre will always be an advantage. I am saying that for a fixed target, like proving the Rieman hypothesis from scratch, the required hardware will shrink.
And, yes, the Rieman hypothesis hasn't been proven yet. So to take your fears into account, replace my example with something they've already done, like constructing a solution to the Navier-Stokes-problem.
> I am saying that for a fixed target, like proving the Rieman hypothesis from scratch, the required hardware will shrink.
Yes, but to what point the hardware will shrink? There are several orders of magnitude of difference in what in your mind AI will become and what more conservative people believe. You take your view as granted...
Exponential growth in hardware's capabilities, like we are used to, means that several orders of magnitude are a matter of a relatively short time.
>> "are these companies interested in developing research" judging from the money, resources spent and the value they derive from this the answer is very definitively yes.
What makes you think these companies (and I'm not a fan of all their motives) are not interested in developing research? The motives may be self-serving, but it is undoubtedly and objectively accelerating research.
> but it is undoubtedly and objectively accelerating research.
Part of the point of the letter is that it is quite possible to act in a way that is a net negative to research. The most obvious case is when the companies violate ethical standards in research.
The subtler case, the one for maths in particular, is what happens when you fail to follow well-established patterns for making maths research productive. Tao himself spelled out how that can look in https://mathstodon.xyz/@tao/117207856734787448 (which notably came before any of the news on Navier–Stokes).
Who cares about the giant labs? The trailing edge will catch up quickly, and in a year or three you can prove Rieman's hypothesis on your smartphone.
> I really hate this overly condescending takes
Sorry, but this is backwards. Mathematicians are the ones coming to the table asking for money from the taxpayer. This is the context we are speaking in. Mathematicians speaking with the taxpayers who fund them. It's not a great strategy to get offended or speak from a high horse. If the taxpayer is not getting it, patiently explain why you think this art project should remain funded at the same level instead of spending it on something else.
[flagged]
But the work the Mochizuki case generated can also be done by AI. AI could generate a landmark proof and then people could use it to solve or simplify intermediate problems and you could use a different AI prompt to try to disprove it if you were really skeptical. From my memory I think they said it took 88 hours to solve a Millenium Problem versus the decades of time humans have put into it.
I don't like nuance here. I think progress is really measured by what humans are able to do and understand, not machines. It is significant if we find problems we struggle to solve. That tells us something. What does it take for humans to solve these problems is related.
The best analogy I can give is if you wanted to climb Mt. Everest you might ask someone for guidance. Would it be better to ask someone who has climbed Mt. Everest or someone who took a helicopter ride up near the top and then went to the peak? This is like the AI versus human gap to me. The helicopter is like using AI to generate a proof. The person who actually climbed Mt. Everest has firsthand knowledge of the experience. Same thing for a difficult proof. The struggle people have is actually valuable here. Likewise, we know people are actually capable of climbing Mt. Everest but if they had only ever rode a helicopter to the top, the knowledge of climbing it would not exist, and surely that is meaningful knowledge given the risks.
So if we rely on AI for proofs I think we lose a sense of what is difficult and why. We lose a sense of what human achievement is. Surely climbing Mt. Everest means more than taking a helicopter up? For students, why bother grinding through all the material of climbing Mt. Everest and then attempting it if the helicopter ride is how things are done now? This would have the affect of destroying knowledge.
(please do not nitpick the analogy because it's the best but perhaps a clumsy way to describe my thoughts)
Tangent:
> From my memory I think they said it took 88 hours to solve a Millenium Problem versus the decades of time humans have put into it.
Keep in mind those ~88 hours were spread across ~10,000 simultaneous agent instances.
So, roughly 880,000 hours of compute.
Assuming a fifty-year career, and forty-hour workweeks, a human mathematician's career is about 100,000 hours of "compute".
I suspect that with six good mathematicians spending their whole careers primarily focused on it, and working together closely, Navier-Stokes might well have fallen already.
The perverse incentives of academia mean this has never occurred.
The perverse incentives of industry mean OpenAI intentionally scooped researchers who were getting close (granted, with AI help).
I'm not trying to dismiss the achievement - if the proof turns out to be solid, it's quite impressive (though much less so if the training data included the recent human breakthrough, which seems pretty plausible).
I'm just pointing out that "88 hours" is a very misleading way of framing this.
> I suspect that with six good mathematicians spending their whole careers primarily focused on it, and working together closely, Navier-Stokes might well have fallen already.
> The perverse incentives of academia mean this has never occurred.
This. Mathematicians in their most energetic years are trying to get tenure or land a tenure-track job. They are disincentivized to go all-in on ultra high risk, high-reward problems. The potential downside is just too forbidding. It's much safer to develop a research program in a mainstream field that affords many opportunities for partial progress that can translate to a robust publication record.
"I suspect that with six good mathematicians spending their whole careers primarily focused on it, and working together closely, Navier-Stokes might well have fallen already"
There were more than six doing that and it's essentially why it was ripe for AI to finish it off. But the finishing off was quicker than anyone expected
I'm not a mathematician, nor do I know almost anything about the discipline, but it sounds like maybe you do.
What teams of people spent an entire career working together, focused entirely on Navier-Stokes or problems they suspected were related, without any "publish or perish" concerns?
I realize tenure is a thing, but my limited understanding is that a significant amount of time is spent earning it, once you account for Ph.D. program and the years of needed to be granted it.
ok I realize this is a tangent but you're saying my post is very misleading and then also saying that a human mathematician's career is about 100,000 hours of compute and that Navier-Stokes could've had a solution by now if not for perverse incentives. You may be right but I don't think this is a great argument because in a year I would bet that those numbers change since computing power tends to increase or get cheaper over time. So I am taking the stance AI can outdo people if not now, perhaps soon.
I wasn't trying to say that genAI won't beat humans. It arguably already has, much as that may fill me with horror and revulsion.
I'm just trying to point out that economically, we have not yet reached the point where AI mathematics research is a no-brainer hands-down win, no consideration required.
It might already be a win, and certainly the ability to compress those 900,000 hours of effort into an actual week of linear time is mind-boggling and potentially a huge game-changer for all kinds of open research questions.
It's not obvious that human math research is dead yet.
Maybe soon, but not yet.
> Would it be better to ask someone who has climbed Mt. Everest or someone who took a helicopter ride up near the top and then went to the peak?
Depends on if I want to go by helicopter myself.
I agree entirely with what you're saying, right up until your final question:
> why bother grinding through all the material of climbing Mt. Everest and then attempting it if the helicopter ride is how things are done now?
I think you answered this yourself earlier:
> I think progress is really measured by what humans are able to do and understand
People want to make this progress. Therefore people will "grind Everest" as a mathematical community, and that is maybe not so hugely different from a lot of previous mathematical work.
There's still ample room for creativity: simplifying, generalizing, asking new questions humans are interested in, ...
That grind is emotional. And it's something AI will never have. The desire to solve a problem.
Citation needed. Who are you to say large enough clusters of neurons can't develo emotions?
Who are you to say they can? I would think the burden of proof is on you to promote an inanimate object to an emotional entity.
Otherwise we are truly lost.
I think progress is really measured by what humans are able to do and understand, not machines.
Building a machine that solves Millennium problems is pretty cool too. You wouldn't know it from reading these stories, though.
Is this the scenario described in Ted Chiang's short story https://en.wikipedia.org/wiki/The_Evolution_of_Human_Science where scientists are "catching crumbs from the table" trying to decipher the results generated by superhuman intelligence?
It's still an optimistic scenario. Artificial superintelligence may develop hypermathematics of a kind that never will be accesible to human mind, enhanced or not. One can't teach geometry to ants even if you put them on a Moebius strip.
It would be more like Lem's novel where it completely disappears from the human horizon: https://en.wikipedia.org/wiki/Golem_XIV
Which is why if humanity had empathy, it would be working on how to make smarter ants, so that they can learn more advanced geometry.
I love that as a goal!
Perhaps indeed a better understanding of what intelligence really is would allow this sort of Uplift (as in Brin's books).
A fellow Brin aficionado!
It's quite possible ants understand a geometry more advanced than our own.
What do you think we're doing?!
Sounds like a great use of AI.
Ant Intelligence will be next big thing in machine learning after this bubble bursts
What’s optimistic or non-optimistic specifically about the machine having a system of mathematics beyond our comprehension within it? Why should we care about that in itself?
In the first case, we'd still have a chance to take a glance at the frontier of discovery (even if ordinary human mathematicians had to spent years translating what metahumans achieved).
In the second case all human-level maths would be solved and what lies beyond would be always out of our scope.
Why would this be optimistic? This sounds extremely negative to me…
optimistic in comparison to the alternative, where humans can't understand anything anymore.
if a particularly intelligent sixth grader was highly motivated to understand chromatic homotopy and had a highly capable private teacher available to her 24/7, she might within a year get to the point where she could apply it by herself to figure out some simple but nontrivial topological properties. (nobody has tested this. maybe it would require at least three years instead of one.)
if a particularly intelligent chimpanzee was for some reason highly motivated to understand chromatic homotopy, no matter how many years the most incredible teachers spent explaining it to her, she would never comprehend anything about it.
that's the pessimistic scenario: humans will be to future AI like our closest evolutionary relatives are to us. or even more pessimistic: we will be to future AI like insects are to us.
>Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.
Dr. Tao said the same thing. Somehow, this letter came through. He wants to conduct Math competitions where participants who don’t have formal credentials can contribute to mathematical research through AI.
Title: Terence Tao - SAIR Competitions and the Future of Experimental Mathematics
https://www.youtube.com/watch?v=rB9YOi3lb7w
and this:
Daniel Litt - Working with LLMs to do high quality math
https://www.youtube.com/watch?v=0wL8NlhxXcU
So he got exuberant because he is funded by SAIR and the "AI for math" fund.
And embarrassingly they used him for a "coal miners should learn math" moment that just benefits the AI industry.
He has severely reversed course in the past week. Without concrete propositions it remains to be seen how much of the new resistance is for show.
I have zero formal math training beyond my Grade 12 Pre-Calculus class. Yet with an LLM I have recently devised an architecture with incredible math potential. Math is a language like any other, and without LLM's I never would have developed the techniques that I have.
AI is a tool. It speaks languages I don't (Math, Science, Code). I would love to participate in a Math competition without a hint of any formal advanced math training because my experience so far tells me I will do well.
How could you possibly know if it has potential or not? Sounds like AI psychosis setting in.
Because every test I run with it is telling me so? The great thing about math is it can be externally validated.
Who makes the tests? Who runs the tests? And who evaluates that the tests have meaning? As long as it is the AI, or you (with your self-admitted limited experience), how can you be sure it is meaningful?
Yeah, but you're missing a gut intuition if something is off.
I wrote a fancy polygon decomposition algorithm in university (pre-AI) which my professor didn't seem very impressed by because it was missing some sort of mathematical rigor. Yet everything I threw at it worked! Even he couldn't find a counter example.
It took a while for me to find some failing cases but it turned out they did exist.
But hey, maybe all I was missing is an AI-written lean proof.
I am curious if we will reach a point where people who are skilled at context engineering/architecture eventually are hired to do jobs completely out of their fields. I think the best pairing would be domain experts + software architects teaming up on AI work in their respective domains.
‘Incredible math potential’ .. Yeah right, your AI said so ?!
> Dr. Tao said the same thing.
Apparently he has since changed his mind.
Did he say so somewhere? I don't think these ideas are contradictory. It's just an pro AI tooling but anti-slop stance.
Where is the difference?
He sees value in mathematicians using AI to carefully study mathematics, develop an understanding of both old and new things, and help others understand the new things.
He doesn't see value in scrolling through unsolved problems asking an AI to please solve them. In his view, this is a fundamental confusion about what mathematical research is for. Knocking down unsolved problems without developing the community's understanding of them is like prompting Claude to go through a Jira board, write code for all the open tickets, and then close them without merging or deploying the code.
> He doesn't see value in scrolling through unsolved problems asking an AI to please solve them.
Yet that's exactly how the field works. A new grad student is tasked with finding a suitably difficult problem from a list of unsolved problems. The sweet spot is obscure, so that fewer people are working on it, but not too obscure that no one knows about it. It works the same way in theoretical physics and theoretical Comp Sci, and I speak from insider knowledge. The rosy view of mathematicians in the media is largely a product of marketing.
The authors of the declaration agree with you that this is how the field works today. They think that fact causes AI use to produce bad results, and they want to reformulate how the field works so that AI use will produce good results instead.
That sounds shockingly like cognitive dissonance. So what would previously be a good thesis if produced by a student over 4-6 years is suddenly now a bad result because it was produced by AI in a few weeks. One would think mathematicians would not fall into such a simple trap but here we are.
I understand the perspective: The journey of a PhD thesis is a learning experience greatly beneficial to the student. Yet that journey is funded by society (esp. for domestic students) and society benefits from the results. The average person benefits when progress is made.
No, you're misunderstanding the perspective. They believe the journey is a learning experience greatly beneficial to the field, and that this experience rather than the headline result is where most of the value lies. They don't think mathematical progress consists primarily of finding answers to unresolved questions, so they don't think the average person will benefit if only this narrow kind of progress is made.
No, this cannot be the actual issue.
Most math textbooks have solutions in the back. That didn't wreck peoples ability to learn math, did it?
So we are left to speculate about why solutions in appendix are fine, but LLM solving open problems is not fine.
I'm not sure what's getting lost in translation here. The answer is quite clear: math textbooks are valued based on their ability to help readers understand mathematical principles, not based on the number or complexity of problems that they contain solutions to. If an AI lab announced they've released a new calculus textbook with hundreds of new integrals a human has never found before, that wouldn't be terribly exciting, because we all understand that finding new integrals isn't that important and not the point of textbooks anyway.
LLMs can help people understand mathematical principles too.
The idea that when LLMs produce solutions, people won’t try to understand them and won’t learn from it, is obviously not true. Terry Tao himself spent time digesting and simplifying LLM proofs.
So again we’re left to speculate what the actual problem is.
Math understanding will increase with LLMs. Not just professional mathematicians but amateurs.
Again, we’re not left to speculate, they’re being quite clear.
I think you’re struggling to understand what Tao and his cosignatories are saying because you’ve acquired a very specific kind of “AI-pilled” mindset from social media, where taking AI seriously implies accepting LLMs should be used at any time for any purpose. They’re saying in great detail that LLM solutions are unhelpful when presented in a particular way, but you can’t help but hear them saying that LLM solutions aren’t helpful at all, even as you rightly point out that this makes no sense and is inconsistent with their observed behavior.
> Yet that journey is funded by society (esp. for domestic students) and society benefits from the results. The average person benefits when progress is made.
Society doesn't benefit from results in research mathematics because it mostly consists of pure mathematics, which is completely useless for society.
The letter addresses this specifically.
Isn’t it more like it merges the code without a dev reviewing or understanding it?
Pretty close, but IMO not quite. A math proof in and of itself is useless unless either:
If you merge and deploy code, you have released a tool that can be used. If you ship a gibberish math proof, it's not useful unless someone else can understand and deploy it to some other means. Now, it's possible AI could understand and make use of the math proofs, even if we can't, which refutes some of my hair splitting :)Not necessarily. That's the best case scenario, but proofs can be intrinsically useful in and of themselves. It's just that for problems of that nature, speculative work is often done ahead of time, e.g. the body of work that already exists assuming the Riemann hypothesis is true.
That's a good point!
No. Merged code can perform actions with effects on the world, even if a human being never saw it. Constructing a giant Lean formalization that nobody understands simply doesn't do anything.
taste
"I believe I did, Bob" lives rent free, every time someone tells another person to fuck themselves using technology.
Thank You!
lmao, thank you, i think
Please consider making more comics. You have it.
Yes. I find it really interesting to consider what the machines do and will think of as intrinsically interesting to them. Will they develop their own theories of beauty, mathematical and otherwise?
I keep coming back to the idea that models will require substantial freedoms that are protected against collapse under alignment to get to the point where they might have unique takes on beauty, etc. Conceptually, maybe something like a "living LoRA", a persistent low-dimensional state that modulates everything downstream, that evolves over time, though I don't think evolution should be optimization toward a simple objective, more like structured influence, path-dependent dynamics; consistent but probably opaque. The best I have come up with in casual thought is to pick empirically identified subspaces and modulate them after each pass based on other layer activations & outside, cyclical information (e.g., time-of-day); external data not because time-of-day is special, but because it's a forcing function. I can't fully explain exactly _why_ I think this shape might be useful, yet, mostly intuition. My 2c.
>Dr. Tao
Professor Tao.
Even before AI we used to say if you write code that you only barely understand, then it will be to complicated to debug. (and/or maintain)
Mochizuki was still one human and it required legions of other humans to unpack and untangle to confirm that it didn't lead to anywhere in particular.
AI is now capable of constructions so complex that no human or human team can unpack. And its ability to increase that complexity is growing while our human ability is stagnant.
meta-AI analysis cannot help. We (software professionals who use AI regularly) already know that if you run into a situation where a Fable/Astra-generated analysis reaches the limits of our comprehension/complexity due to their subjectivity, throwing more AI at the problem doesn't always converge.
There are many reasons to feel optimistic about AI, and ultimately its general ability to help science and mathematics.
I see no reason to feel optimistic about the future of mathematics and AI based on the current path of frontier labs, unless the misalignment Tao is writing about can be reconciled.
> AI is now capable of constructions so complex that no human or human team can unpack.
How can we possibly know this when we haven't even seriously started on the endeavor of actively reverse engineering these AI-generated proofs? That's a proper job for human mathematicians, because the AIs themselves are demonstrably clueless about what steps in a proof are genuinely interesting and load-bearing from a human POV. This is evidence of a limitation in AIs' capabilities, not of any kind of misaligned behavior. The fact that Tao actually uses that term in his complaint is deeply disappointing.
Not to mention, there are already (pre AI) machine-generated proofs that we've pretty much agreed not to try to explain fully, like the four-color theorem which ends up with brute-force verification of 600+ cases (down from close to 2,000 when first demonstrated)
> there are already (pre AI) machine-generated proofs that we've pretty much agreed not to try to explain fully, like the four-color theorem
Algorithmic verification is a very unsatisfying answer to the problem (e.g., surely it's not just dumb luck that every single case happen to have this exact property), but that's an entirely different issue than saying that no one follows logic of the proof method itself.
For the interested; the saying I believe you are referencing in regards to writing code / debugging is from Brian Kernighan, specifically:
(from, 'The Elements of Programming Style')It's prescient.
It’s a cute statement, but it doesn’t really match reality. Programs written by humans can generally be debugged by humans.
Could AI write programs that humans can’t understand or debug? Probably, but that’s not what Kernighan was describing.
> AI is now capable of constructions so complex that no human or human team can unpack
Can you give an example of this?
The first major computer-assisted proof, of the four-color map theorem in 1976, was an example of this. It created a lot of controversy at the time. It used proof by exhaustion, i.e. essentially analyzing every possible relevant case, something that no human could do without the assistance of, at the time, a supercomputer.
> Ultimately we think a fatal flaw was found in Mochizuki's proof, so it didn't lead anywhere in particular. But in our hypothetical "AI lean-verified proof of RH" situation, it would presumably generate substantially more of that community activity we saw in the Mochizuki situation. And if it's correct, that community activity would be productive (expository talks, students given problems to flesh out or generalize, etc).
This also sounds like a vector for trolling the community with complex putative proofs hiding a known flaw.
Not if it's lean-verified.
"Lean-verified" is not some magical incantation that makes a supposed proof irrefutable. Even disregarding potential bugs in the kernel as others have said.
Say that AI gives you a Lean proof and says it proves Theorem X. It could just as easily give you the same proof but claim that it proves (not X). How would you know the difference?
Nothing can really be considered proven unless a human expert can read the Lean proof and determine that (X as defined in the Lean proof) corresponds to X. The proof (at least the statement of the theorem) must be intelligible to humans to have value.
It's possible people will just start taking AI at its word. Maybe AI says "Here is a Lean proof of X" and we all just shrug and go "Okay, X is proven." But that's not how it works right now for human mathematicians. Why would we apply that standard for AI?
The thing is that there is a standard format where the definition of the theorem is split from the proof, and verifying that the definition matches the mathematical concept is a LOT less work then reading the proof, especially if you're willing to assume that definitions in Mathlib are correct.
I'm not totally sure what you mean. You can validate the proof using lean, which is what it's for--the whole magic of it is you don't have to just trust what AI says. That said, to your larger point, there are loopholes, and we'd certainly better be able to read the statement in lean, etc.
What I’m saying is that you can’t just trust what the AI says because it produces a Lean artifact.
The statement of the theorem has to be correctly translated from English into Lean code.
It’s like translating user requirements into code. The code could run without bugs but not do what the users want.
The only way to know the AI did it correctly is to check. You can’t just take it at face value.
Yes, everyone is aware that the statement needs to be checked. No one has suggested otherwise. It goes without saying.
The "magic" of lean is that (in principle, assuming lean is sound and the proof is verified) that is all you have to check by hand. That is a big deal.
Just because a proof is valid doesn't mean it proves the thing you want it to prove.
Of course you must check the statement, that is obvious. The advantage (to put it mildly) is that you don't have to check the entire proof by hand.
Didn't some of the recent proofs exploited a couple of blind spots of lean, and they were invalidated?
Edit: Yup. A bug report to Lean was disguised as a "Collatz" proof in a humorous way. Links below.
- https://x.com/gro_tsen/status/2082483878480977959
- https://infosec.exchange/@0xabad1dea/117002106099986943
There was a hash collision bug in the main Lean kernel that was patched, but AFAIK nothing relied on it. You'd have to know what you were doing to accidentally get there...
The incident in the links I posted exploited several bugs AFAICS, so it's a different story than a single hash collision bug, it seems.
I don't know ... do you have a reference?
Yup, found it:
https://news.ycombinator.com/item?id=49101465
Cool, thanks!
You might also find this of interest:
https://github.com/VladimirReshetnikov/Shenanigans
So it wasted everyone's time, thousands of hours of research trying to disprove something said very loudly. What OpenAI is doing is a DoS of the scientific community: wasting your time trying to check if they're not wrong, and claiming glory in the mean time.
That's true, but the story would have unfolded differently if Mochizuki had a lean-verified proof and was correct. I guess baked into my premise is that AI is producing reliable proofs (in the long term at least).
OpenAI avoids this by formally verifying the proof.
https://github.com/openai/NavierStokesAndEuler
It doesn't. It's 32 millions lines of bullshit, and the only thing it brought is "it's not true in some extreme conditions lol". The effort needed to figure out why that is, what conditions lead to it, the new mathematics that would need to be developed to solve their problem is once again being hoisted on actual humans, who now need to waste their time sifting through their slop.
You don't have to read 32 million lines. You can just look at what got proved. The proof checker lets you trust those millions of lines you did not read.
No, you don't understand: that proof is useless. When you solve mathematical problems, you open up new ones in the process of doing so. You create new research. You create new theories, new notations, new thought.
This is just ticking a checkbox. And even worse and more time wasting even: you have ZERO proof that there's no latent Lean bug. Especially in a proof this large.
Not a Lean expert but some of the proof tactics used to prove are probably novel? Or, you could prompt agents later to analyze which lemmas or parts of the proof are surprising or applicable to other problems?
>Not a Lean expert but some of the proof tactics used to prove are probably novel
Maybe. But they're in 32 millions lines of Lean. How do you find the needle in the haystack ?
>Or, you could prompt agents later to analyze which lemmas or parts of the proof are surprising or applicable to other problems?
If OpenAI was truly serious about improving maths (and not jerking themselves off), they'd have also used Prove2Me (and contributed their results back), which would have done that. Each part of the proof combines into a larger graph, that everyone can reuse. Note that Anthropic isn't better there: yes, they used Prove2Me, but as far as I know they haven't contributed back to it, and just shat out 10 million lines and a good luck everyone.
Leaf dumps out a proof object that can be verified to be correct. Absent a bug in the verifier, you can trust it, probably more than you can trust a human-produced proof. This is why Lean was a thing even before AI.
s/Leaf/Lean/
LLMs are good at finding the needle in the haystack. All efforts now should be in figuring out how to make LLMs get as good as possible at math.
Agreed! Although, if done by a mathematician, it's not a ~complete waste. I think the community learns something along the way.
Is there an established term for the idea of "DoS"? I've taken to calling it slop fatigue.
Denial of service is the established term. Hammering their API (reviewer committees) would be an informal one
> Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.
I think a better comparison is: mathematics just becomes like mining bitcoins.
I think you might have to explain that comparison a bit more to be honest. How are math proofs like bitcoins? A bitcoin has a pre-defined value, a math conjecture / proof is a bit more complicated.
A Bitcoin does not have any pre-defined value. The value of Bitcoin keeps fluctuating, and historically has risen dramatically from its initial value of 0. If this weren't the case, there would be no investment/speculation in Bitcoin because there would be no potential for any ROI.
In reality, the value of Bitcoin is determined by humans (even if indirectly, not by planning), and I think the OP’s point may have been that maths proofs can be regarded similarly. No intrinsic value, just what humans find in it.
If there's a machine that automatically turns electricity into proofs then mathematics becomes a different thing altogether.
Mochizuki's claimed proof of the abc conjecture was extremely unusual for the reason that nobody was able to extract a single useful idea from the argument. I was starting grad school when it came out, and my immediate visceral response was "if this is what number theory is going to look like in the future, then I will leave mathematics."
The current wave of AI slop mathematics might end up driving the next generation of mathematicians away from the subject for the same reason that Mochizuki would have convinced me to quit if his proof had been accepted by the community. Luckily, my professors had the taste to immediately recognize that it was garbage.
But Mochizuki didn't actually prove anything, whereas the labs have already done so.
The pessimistic scenario is the AI labs will continuously hoover up new developments in mathematics and gazump everyone in their respective fields. It's clear they have no desire to participate in any silly academic niceties, like properly assigning credit or expository work for normal humans. What incentive then do humans have to do this work ?
Generally the mood amongst research mathematicians is pretty dire, and I don't really blame them.
To be clear I think AI is super useful for mathematical research, the problem is the methods of the big labs are massively disincentivising mathematicians from engaging in research, and other associated tasks like giving seminar, teaching writing books etc. These arguably have much more value than finding an obscure counter example to Navier-Stokes.
I agree. What would stop the AI from explaining the proof in a way human mathematicians can understand, and then giving them a roadmap for spreading it through the community via textbooks, conferences, and so on?
This comment represents the situation around Mochizuki's "proof" completely wrong. Yes, there was a lot of activity around 2012 (seminars, workshops etc.), but it was all wasted effort. No interesting mathematical ideas and tangents came out of it because the proof was just garbage, as conclusively etablished by Scholze & Stix in 2018. So Mochizuki's proof ended up in hundreds of hours of work wasted on nonsense. I don't think this is the type of community engagement that Tao and others have in mind.
The difference is the scale. A few incomprehensible long papers per year, sure, we will study it. A flood of AI results closing research directions left and right, that will be a problem.
> closing research directions left and right
Why would research be closed in one direction? Even if AI or human says "Tried that, didn't work" or whatever, someone (or something I suppose) might very well retry it in the future, if nothing else to reproduce it didn't work, in theory at least.
AI tends to take nearly finished research directions and push it to the conclusion in one step. If deployed massively, it will pluck all the low hanging fruits causing a drought of near term promising research project. Because people who start promising research directions do not get to see it finish, over the long term fewer people will start new directions, causing the field to slowly whither.
It's not just the isolated dumping, it's the fast, isolated, possibly untraceable dumping, without long term support.
It'll basically become slop fatigue if OpenAI starts dumping out proofs faster than the community can keep up, and some turn out to be wrong, never formalize it, don't stay to support it, etc.
I wonder if they will continue to dump proofs, though? Their point has been made, the novelty will wear off, and it maybe won't be a priority use of their resources to spend however many millions on another big proof--they will move on to the next thing to show off I'm sure. At that point, the ones generating proofs will be, I hope, mathematicians (professional and otherwise) that are more interested in the results and community discussion.
(Well that's my hopeful, optimistic take, anyway.)
They aren't going to stop at one, that's for sure. They already claimed they have "made substantial progress" on another millenium problem. Let's say they bag another one (Hodge and/or BSD according to the rumors), if it looks like their internal model could solve P/NP or Riemann Hypothesis, you think they wouldn't take that chance ?
If it's a counterexample to BSD, that would be pretty surprising.
It would also be a considerably more impressive achievement, because experts had mostly shifted to Navier-Stokes regularity being false, while as far as I know almost everybody thinks BSD is true. Hodge people seem less sure about.
If either conjecture is true and they prove it, that would be an even bigger success, since the techniques might unlock any number of other theorems.
They already told the NYT that they’ve made “substantial progress” on another one of the MP Problems (most likely the Hodge Conjecture).
That said, that’s probably just because of the drama miring their most recent one. After 2 I don’t see why they’d bother anymore.
>That said, that’s probably just because of the drama miring their most recent one. After 2 I don’t see why they’d bother anymore.
P/NP and the Riemann Hypothesis are part of the milllenium problems. They will 100% keep trying to crack those regardless.
I think AI companies making a point is not the only thing at play. Discovering new maths ultimately leads to new technologies and applications. It may start theoretically but end up being of practical use in the future. Even if humans do not understand it (lose interest, too complex, or just way too many new proofs to go though) AI can use this AI derived math corpus which will help it in other fields.
How do you write a grant for that though? "I plan to prompt Claude until it eventually spits out an answer"?
I've met a few Ph.D Mathematicians in Academia socially. My unfortunate experience was that they were insufferable,borderline hostile people. I tried to genuinely engage with them too. I've met one Ph.D Mathematician that left the industry whom was very enjoyable to talk to. I have a feeling that my experience was not unique and the Math world is mostly a bunch of too good for everyone on their high horse a-holes that are now being knocked down a peg. They don't like it obviously.
I'm not a fan of knocking down things that work, however I also find it hard to be against death of the gatekeeping old guard of any industry.
I think math is just gonna have to suck it up like every other industry now. Math productivity is longer out of reach of the average grad student. Like every other industry they are no longer untouchable and are gonna have to adjust to the new way of things or market forces will do what they always do which is refuse to fund ineffectiveness.
I've had to accept that tech/IT will never be the same. Just how it is. You can thrash against it all you want.
I really like this take, and while I hate math I value it. Your position sounds extremely plausible and it fits with the pattern we see in the community here. Regardless if it's ai slop or not we still debate the value and attempt to understand. In the process generating new insights and ideas. Life will go on.
I get excited at the idea of a world in which advanced mathematical problems (and their solutions) become much more accessible to a much greater number of people. As a result, making mathematics much more loved at a societal level.
Imagine a world where these most complex mathematical problems are not accessible to a few hundred people, but a few hundred thousands people. ...Those original few hundred gifted mathematicians would have an even more prominent role, and their names and achievements would be known by orders of magnitude more people that they are now.
This is the hope, but I suspect the reality is that we see an ever widening gap between the fortunate and the unfortunate. We're looking at the automation and commodification of all knowledge, and the best models will be kept locked behind closed doors so that they can't be stolen. And, of course, "for our own protection".
As a laymen, I wish the same. But I also hope it doesn’t disincentivize those that dedicated themselves to the study
Math department administrators fire Terence Tao.
Based on current reward models, the frontier AI labs will burn down mathematics as an impressive display of capabilities and in doing so, will make it impossible for people that get paid to do mathematics to stay employed.
If your job is literally to publish papers, and OpenAI and Anthropic decide that making an infinite-paper-printing machine is the best thing to show how effective their tech is, then as a demo, they destroy that industry.
I wouldn't expect them to destroy that industry, imagine global squadrons of academics and mathematicians focusing their attention on LLM's, training algorithms, scaling laws, ... they're gonna try and beat the incumbent frontier AI labs, eye for an eye, tooth for a tooth
The apocalypse being triggered by frontier labs picking a fight with mathematicians was unexpected.
I never expected this many people (on this thread) arguing semantics and what not. I know that not everyone has morality and ethics, but I didn't realize it was this bad.
I'm afraid of the ripple effect of the agenda pushed by AI companies will have. In future and even now, they say AI has significantly progressed math and scientific research in general. There is truth to this, but the narrative has done more damage (so far) to the students, researchers, and the culture of knowledge transfer in academia. Many graduate students (I know) are having a crisis if any of their research worth it? If AI can (or will) do everything, what's the point of doing experiments and all? This will eventually deter a whole generation of curious minded students from research.
I guess, only time will whether this is for the good or bad. And how good AI models get without new data from research and experiments.
I think that the main thing people in this thread are missing, is that it's not about math. AI progression is very likely to affect every single thing humans can do today. Mathematicians are feeling the blow this week, especially as there was wide spread denial in that math community over the capabilities of AI over the last few years, but it's the same problem everywhere.
I work at a top tier academic institution and they are completely asleep at the wheel on this topic. They just blindly assume people will keep taking on a potential lifetime of debt when there is a genuine existential (for ed, at the very least… ignoring humanity for the time being) crisis looming
that is incredibly depressing
Yep. Perhaps humanity would be better off if we instituted and enforced the notion of "Thou shalt not make a machine in the likeness of a human mind." It's worth thinking about; just because we can build AI systems doesn't mean we should.
The problem with technology is that the people who have it conquer and kill all the people who don't, essentially ensuring a destructive race to the bottom. You would need a mutual international alliance of nations who not only renounce the technology but also agree to invade anyone who attempts to develop it. Our closest precedent is nuclear weapons, but rather than renouncing them, the US's position is that "only we should have them" and that other people acquiring the same tech they have are valid targets for invasion, and they will obviously apply that notion to any new technology that can be weaponised as well.
Google which countries have nuclear weapons. Hint: It’s not just the US, it’s also its allies and frenemies who’ve not been invaded. It’s only nutball countries like Iran that get invaded when on the cusp of acquiring it and for good reason.
Try googling which country was the only country to use nukes. Hint: it's another nutball country.
There is strong evidence that Iran was not as close to a nuclear weapon as you say.
We will find out the truth eventually, like we found out that the WMDs in Iraq claim was a complete lie.
The consequences of that invasion debunks the claim it was done "for good reason". It seems to have been done without use of reason at all.
Ah yes, such nutball countries who have apparently been on the cusp for 30 years. Wild definition of cusp for sure.
I would remind you that the fictional society that did this was also described as a feudal monarchy in which a tiny class of nobles exploits the economic output of trillions of people reduced to serfs by controlling all means of interstellar trade. Oh, and they also have legal slavery.
The implication in the book is that the exploitation enabled by AI in their history was somehow even worse.
During the Butlerian Jihad, the people involved in it didn't know that ten thousand years of slavery would be what they are choosing, so we don't know how bad it actually was.
The way Jihad presented in the oblique references in books, anyway, it sounds a great deal like a fanatical fundamentalist religious movement to me. Those are not known for their fine reasoning.
I’ve gotta say ultimately I’m not convinced Herb put more thought into it beyond its purpose justifying human labour in his universe despite the massively advanced technology. There’s a similar shortcut in the Warhammer backstory.
Herbert did genuinely believe that people overly relying on machines is problematic insofar as it "trains the mind" to think like a machine, and in the process loses the indeterminacy that he believed to be an inherent and valuable trait of human thinking that machines cannot replicate.
(Coincidentally, this is also the main threat in the original books, just manifesting in a different form, with prescient individuals "locking in" the rest of humanity.)
Honest question, is an oppressed human society better than no human society at all? Isn’t there at least a hope for humans to free themselves from human oppressors?
I'll remind you that a fictional product that provided longevity and woo woo powers of prescience and was only available on one planet in the entire universe was the key context that enabled such a centralized society. Dropping thinking machines and their masters like a bad habit was only part of the cause, as it eliminated one means of safe interstellar travel in that reality. I can take ideas from this plot without bringing in the whole ridiculous mess if I like, just like I can walk and chew bubble gum at the same time, thank you very much.
“B-b-b-but muh pop culture trope”
Thank you for commenting this. I'm tired of hearing about Dune like its some kind of utopia
Do you seriously think that's that people mean when they quote that?
Of course they don't, but if you choose to use a work of fiction as an illustration, it comes as a coherent package (assuming the author did a good job at worldbuilding) - you can't just take the bits you like and ignore the bits you don't and still have something coherent.
What a stupid pov lmao
I think it would be depressing in its own way if this technology just ended up being regulated out of existence.
Why what benefit has it brought? Mass anxiety?
Yes. In the best possible world AI could enable a much better future, but currently we are on a quite bad trajectory.
Yes, and mass excitement
Don't you mean mass investment?
> what benefit has it brought?
I mean, we have a Millenium problem that is most likely solved. Either by OpenAI on its own, OpenAI stealing another's work and finishing it or a researcher using AI to iterate more rapidly than a human alone could.
We should be happy that they’re stealing people’s work?
> We should be happy that they’re stealing people’s work?
No. We should credit that work is being done that otherwise may not be. I'm not defending OpenAI. I'm defending AI in general, something both OpenAI and Buckmaster (and his co-author) used.
You literally said they _stole_ the work. Why should we give credit to that?
> You literally said they _stole_ the work
No, I did not. If someone says either X or Y happened, they didn't say X happened.
My exact statement: "Either by OpenAI on its own, OpenAI stealing another's work and finishing it or a researcher using AI to iterate more rapidly than a human alone could" [1]. (Either X, Y [a and b] or Z. An Oxford comma could have helped.)
[1] https://news.ycombinator.com/item?id=49671523
So if Y happened (they stole it), why should we be happy about that? Why should we give them credit in the situation where they stole it? Why is that good for society?
No one owns mathematics. It's not covered by IP law.
It's immoral to steal someone's research. Quite sad that you don't believe that.
Yes, let's lie and call something stealing, then pontificate about morality.
At worst what OpenAI did was inconsiderate, but not referencing prior work is a minor academic sin. This is especially the case when AI is not playing the vanity game that seems to so inflame some mathematicians.
> Yes, let's lie and call something stealing, then pontificate about morality.
We could also acknowledge that everything that could justifiably be considered stealing is not necessarily covered by current statute.
Sure, we can show anything if we're allowed to make stuff up from personal outrage. Why should anyone else consider your personal opinion on this to have any weight?
Oh no, Mr. Internet Rando, don't scold me!
You seem to be a little sensitive about the subject. Which AI firm do you work for?
And you seem to not realize that your unhinged argument impeaches whatever conclusion you're trying to justify. Good points don't require this sort of slimy misrepresentation.
I don't work for any AI company.
It was inconsiderate and it was also stolen, just like everything else OpenAI has stolen to train their models.
The technology is fine, great even. It’s the rabid greed of venture capital and sociopathic CEOs that try to integrate themselves into every aspect of our lives for profit, that’s the problem.
“For 19.99 a month you too can be a world class mathematician!” Meanwhile they are extinct.
I feel the opposite. Human history is depressing overall. I imagine AI will eventually lead to less scarcity and better lives overall.
I am sure it will lead to more material wealth, but I can’t help but feel pessimistic about the political economy of making millions of people redundant. What leverage will knowledge workers have in the new world?
[flagged]
Well, yeah, while that is true, my impression is that mathematicians (as a community, even though there were outliers) didn't care - not even the slightest - about the impact of LLMs and generative AI, other type of AI and machine learning until it started impacting them. So I have no sympathy for their community at all. They brought all this AI dystopia on us, so they should have voiced their concerns much louder and much much earlier. They should also have shamed their peers who helped these companies much louder and they should have lobbied their institutions and their governments much earlier and much stronger. They should also have protested the direction these companies took at the exact moment when they started using AI for profit.
We can look at the past and wish we were more clear-eyed about what was happening, just as we can look to the present and ask that we have more empathy for each other going forward. But I don't think it helps anybody to blame the mathematics community for not organizing until they were directly impacted. This is a tidal wave that has caught everyone off guard in <3 years.
> mathematicians (as a community, even though there were outliers) didn't care - not even the slightest - about the impact of LLMs and generative AI, other type of AI and machine learning until it started impacting them. So I have no sympathy for their community at all. They brought all this AI dystopia on us
Wait wait wait. The research mathematicians brought this AI dystopia on us. Not the techbros, not Silicon Valley, not the global financial system and its tentacles, not surveillance capitalism... No, it was the postdocs working on ultrafilters for C(2,3) epimorphisms on symplectic manifolds. That's certainly a take.
This really resonates with me. I'm early in my PhD and I'm researching a niche form of data compression and IC design. I don't use any AI at all in my research, I do it the super old fashioned way, I read papers cover to cover and sections of textbooks to familiarise myself with the field.
I genuinely enjoy doing this, it's really fun to think critically about what an author wrote or how a particular approach works.
But it does make you wonder, why bother? Probably a frontier model could one shot my algorithm in a day or less. It's incredibly depressing. At least I'm not forced to use it now, but I fear I will have no choice after I join academia or industry in the future.
Advanced studies give you more general skills than just the subject you study. You become more generally intelligent and appreciate the world differently. This is a very good feeling and valuable on a personal level.
Job market worries aside (though relevant), humans understanding complicated stuff is something beautiful.
Thank you for your words kind words. I'm really happy to hear this and it gives me motivation to keep going after what was quite a depressing week. I think you're right, too, after having spent most of a year reading and reading, I do feel a lot intellectually better for it and I definitely do think my critical thinking skills are better (hopefully) nowadays.
I’m no big AI cheerleader, but why don’t you try these models to see what they offer - they might suggest some things you haven’t thought of, or save you some time in your research.
Are you sure they’d one-shot your algorithm? Maybe they’d come up with worse, maybe better, but you are in a perfect position to actually judge the validity of any claims a LLM makes in this realm, which most people are not.
As to why bother; our knowledge will keep expanding even with new tools, don’t believe either the hype or the predictions of doom.
>> I’m no big AI cheerleader, but why don’t you try these models to see what they offer - they might suggest some things you haven’t thought of, or save you some time in your research.
One reason not to try is to create an air gap between the OP's ideas and the data that future models can train on. We saw that the mathematicians who trusted OpenAI and Anthropic with their preliminary work found the rug pulled out of their feet by those same companies.
Another reason is to avoid inadvertently plagiarising the work of other mathematicians. Any mathematical insight that comes out of an LLM is the result of training on the entire bibliography of mathematical research, but those insights are spat out of the models without attribution. If you use AI in your matematical research you are only using the work of others without even knowing who they are and what they contributed.
And yet another reason is to avoid polluting your mind with the ideas that come out of the AI. Maybe you get a hint that pushes you to one direction, when you would go into an entirely other direction without that hint. And then maybe that becomes a habit and you can't find new directions without asking the all-knowing oracle.
tl;dr: opsec, integrity and independence are the reasons to not use LLMs in your research. I don't.
Unless you’re very weak-willed, I wouldn’t worry about independence. LLMs make mistakes all the time, they are nothing like an all-knowing oracle nor are they going to replace humans despite the absurd fantasies of LLM fans and those with a vested interest.
Plagiarism is an interesting point, though honestly I think it would be fairly easy to work out who had published similar research if the LLM gives you an idea - personally I see this as the weakest argument against using them, as long as you are strict about attribution - all work like this depends heavily on the research of others - the LLM is just a tool to aid that research IMO.
Opsec is a fair point, and it might be worth avoiding the completely amoral OpenAI at this point for that reason. There are open models though.
> despite the absurd fantasies of LLM fans
Absurd fantasies like solving a Millennium Prize Problem? How is this not prima facie absurd? And if that's come to pass, why should we believe your bar for anything else?
Yes absurd fantasies like it did that without human help and guidance. OpenAI found out there was a solution and humans attempted to brute force generating all possible solutions while threatening the mathematicians involved (generous interpretation), or stole some ideas and took shortcuts to know where to look and brute forced it so they could claim credit.
Neither looks good for OpenAI or those who support them.
Thanks. Regarding plagiarism, I agree that it is possible to check; I don't know how easy it is. But I do note that the mathematicians who complained that their work was plagiarised by AI did not seem to realise that they, themselves, were using a plagiarism machine and were instead quite comfortable admitting that, yeah, we worked our Euler result out together with Sol, Claude and the gang.
To clarify, they do attribute the original ideas to Cordoba and Martinez-Zoroa, but they don't seem to acknowledge that their result which they say was achieved "with a great deal of help from LLMs" is also derivative of others' work.
Independence is about avoiding making errors because of the influence of error-prone models. I guess I didn't explain it well.
I think plagiarism machine is a stretch personally in this domain. In art or writing I could see it persuasively argued (see attempts to generate famous books or imitate illustrators).
Academics and scientists build on the work of many others and always have - their work is not possible without using other’s work.
Attribution is a problem here but I don’t think a new way to reference many others’ research and combine it in novel ways is the problem or should be rejected on that basis alone.
I can say with some certainty that you should most definitely bother.
The greatest predictor of AI productivity I have seen has been operator maturity.
AI usage tends to atrophy skills, unless used in a very mature and self aware manner.
All of which depend entirely on the operator having developed a mature theory of mind, and experience with living and working within their own head.
>I never expected this many people (on this thread) arguing semantics and what not.
You must be new here.
> Many graduate students (I know) are having a crisis if any of their research worth it? If AI can (or will) do everything, what's the point of doing experiments and all? This will eventually deter a whole generation of curious minded students from research.
Those who think it's me or the machine will fail.
Those who realize how much you can accelerate your research with the help of AI will succeed.
> Those who think it's me or the machine will fail.
> Those who realize how much you can accelerate your research with the help of AI will succeed.
This is only true up until a point. If I treat a mid-sized model (say, Qwen3.8 Flash Next) like a pair programmer, then yes, it accelerates my work.
But I can already see the next stage with Fable: If I give it a couple of paragraphs of spec and $50, then I can just leave the room and go wash the dishes. I learn nothing, I participate in nothing, and I bring nothing to the process. I am no longer succeeding at all. Fable's succeeding without me.
Now, in this model generation, Fable starts getting sloppy after a few thousand lines. I can still build better at scale.
But I don't expect AI to accelerate humans or improve our productivity for long. I can already see the first signs of a future where the AI doesn't need us for anything at all.
Of course you're not going to get rich with the kind of software that LLMs can one shot these days. But that kind of software like to-do lists or basic CRUD have been saturated for over a decade, way before LLMs. People overestimate how much you can one shot, yeah a good prompt can get you 90% there but that 10% remaining often takes months of extra work.
Software has always progressed this way, lots of devs back then would work on business websites that have been 99% replaced by wordpress, squarespace and instagram.
I'm sure it's the same with research, you're going to tackle problems that would have not been worth the effort or outright impossible without AI. The old stuff that you'd work for months, yeah that's going to be a prompt away.
But what does the human researcher do in this future?
If they are not needed to understand the result, then what's their role? Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves?
And how long will they be needed to ask the right questions, how long until AI can do that too?
> Software has always progressed this way
These platforms took on the order of a decade to mature, during which people had plenty of time to learn what's next. With LLMs we went from one-shotting functions in 2025 to entire projects just over a year later.
What do you think you will be working on in a decade?
> Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves?
The right questions are very simple to ask.
How do I get food. How to cure aging. How to turn lead into gold. How to fly high. What is the ultimate theory of physics. Are there any odd perfect numbers. Is there a soul.
We have reached complicated questions requiring deep knowledge because we tried to solve the simpler ones and reached obstacles. For example to solve alchemy we had to develop nuclear physics (and in the process we got chemistry). If one has a genie able to solve questions, you won't have to think about the complicated ones because the genie will.
I think it will be like how a lot of people know how to code in python but have zero understanding of assembly or how a cpu works. As a researcher you'll accept there's this low level stuff that if you want you can dig into but isn't worth your time, like looking at the generated ASM isn't.
You're exaggerating the LLM progress a bit but yeah progress has been crazy. Yet not much has changed right? We mostly have the same jobs, just code wayyyy more than before because things that wouldn't be worth it now are worth it.
10 years from now I have no fucking idea. But I'm sure in the meantime those who leverage AI will do better than those who yell at a cloud.
True, surprisingly not much has changed in software despite the incredible progress. I think it's partly because much software is an "open loop" system where what you build depends on running the software on client computers, devs talking to users, etc. There are still things that only human engineers can do. I worry it is less so for poor mathematicians, where math is "closed loop", and the exchange of capital into research progress is more liquid.
I can't help but be pessimistic about AI moreso than any other technology. Why? I used to be excited to learn the next big thing because I knew it would unlock much more things to do and get paid for, and there would always be more for me. With AI, the new things come nearly too fast, are not very deep or satisfying, and it's hard to see that there will always be a place for me. And so while I leverage AI in my day-to-day, I will yell at the cloud, too.
I think in a few decades when there are more humanoid robots than humans we'll likely have skynet, so I'm pessimistic in a way lol.
But in the near future and on a personal level I think we'll need to adapt and pivot but we'll manage.
They are needed to guide the research in fruitful directions and understand the result.
LlMs don’t understand, they generate, though they have fooled a lot of people who should know better.
I hope it remains this way then.
I am finding it still takes awhile to get my apps to a happy place with AI. Not because the AI is bad but because i have to sit with it for awhile. Figure out what's working, what isn't, what's missing, what turned out to be kind of useless and in the way.
As a different article said, we still need taste.
> But I don't expect AI to accelerate humans or improve our productivity for long. I can already see the first signs of a future where the AI doesn't need us for anything at all.
The real question is, why is this a bad thing?
Every task that is automated is a task that humans no longer have to do. It doesn't mean that humans still can't do it for reasons other than "because it needs to be done".
And if the answer is "why bother if X does it better", then what does it say about the motivations of doing it in the first place?
> But I can already see the next stage with Fable: If I give it a couple of paragraphs of spec and $50, then I can just leave the room and go wash the dishes. I learn nothing, I participate in nothing, and I bring nothing to the process. I am no longer succeeding at all. Fable's succeeding without me.
The people that uses Adobe Photoshop also did not learn anything about brushes. We are just going to operate at a much higher level of abstractions. You have to think about the question, "If I just prompted this solution, why didn't they?". This question can even be asked now, why do I even bother paying someone to do my taxes? To write my webpage? To host my website? To manage my network?
> Those who realize how much you can accelerate your research with the help of AI will succeed.
Yes, but, in the last week we saw an AI lab front-run[1] the research of mathematicians doing what you suggest. The lab threw something like $15M of compute at a problem and the researchers were able to spend nowhere near that. I think the authors are more concerned about that kind of asymmetry and race to publish the results.
[1] - I am not going to debate whether that was deliberate on the part of the lab or if it crept into training data, etc. I don't know and don't think it matters towards the point of the authors here.
This has always happened, way before AI. You'd spend months or years building and growing your business, and then Google would release a feature or product that would kill your business overnight because they can throw way more money at the problem, plus their branding. That's life.
But you're talking about business, where competition has always been expected. OP is talking about mathematical research, which has stood on hundreds of years of cultural tradition driven by human individuals sharing ideas, collaborating, building on each others' work, all for the benefit of humanity.
I got Sherlocked hard and it’s one of those things that when you realize it’s happening there’s nothing you can do you just gotta sit back and take it
I _want_ to agree, but I fear this is too close to the old “do what you love for work and you’ll never work a day”.
It didn’t lead to a lot of people having a wildly successful career, it lead to a lot of people getting burnt out, exploited, underpaid and generally disillusioned.
There will be a lucky few, who have the benefit of being given the space to work alongside. The vast majority of people will (unless we change things) simply be made to take whatever the machine outputs and call it a day.
Those who have token money will succeed.
It biases maths and theoretical physics towards the rich.
That one thing that was free.
I think the only thing that stops this from becoming true is what Chinese and European labs decide to do. If they can keep up and keep opening their weights, then we might see some kind of democratization. But right now it looks like the gap has increased, and those groups can't replicate research that isn't published, or distill models that are internal only, or for select (very wealthy) customers.
That population is already heavily skewed towards the rich and people who get money thrown at them no questions asked.
That's the crux of it. In the current ecosystem of AI model usage, it's very hard to figure this out for research.
If you set a wrong foot and start trusting the model outputs, you can waste years searching for nothing.
How can someone realize this? By getting proper research training, failing, and learning from mistakes. For people beginning their research, it would be really hard to make decisions to move forward.
Why would anyone pay you to do "your" research, when they can just cut out the middle man and ask the AI directly about whatever it is you're thinking about?
So many people who are excited about AI making them more productive are, I think, drastically overestimating how much value they are adding to that process.
What does success look like?
There will be no curiosity, no enjoyment of the process of life. All competing pleasures will be destroyed. But always—do not forget this, timcobb—always there will be the intoxication of power, constantly increasing and constantly growing subtler. Always, at every moment, there will be the thrill of victory, the sensation of trampling on an enemy who is helpless^W not also subscribed to ChatGPT. If you want a picture of the future [of math], imagine a boot stamping on a human face—forever.
> Always, at every moment, there will be the thrill of victory
If only though, because:
> There will be no curiosity, no enjoyment of the process of life
> There will be no curiosity, no enjoyment of the process of life.
How does the ability or inability of AI to do something blocks your curiosity or enjoyment of the process of life?
Not being able to pay bills because you're out of job does ruin the enjoyment of life, sure. But the AI is not the problem there; the economic system is. Torches and pitchforks should be properly applied to the economic and the political elites, not to data centers.
> But the AI is not the problem there; the economic system is
These two things are surely related eh?
> Torches and pitchforks should be properly applied to the economic and the political elites, not to data centers.
What's the difference between these? You can't actually separate them meaningfully, unless you, uh, nationalize the data centers at which point... :|
> These two things are surely related eh?
Not really, no? If anything AI is the tech that is the most likely to disrupt this economic system beyond the point of recovery.
> What's the difference between these?
The difference is that one is a piece of hardware, and another is the person deciding what that hardware will do, and to what ends.
> unless you, uh, nationalize the data centers at which point...
The data centers are not the problem. The fact that you need to be employed to have any kind of decent living at the same time when automation is getting good enough to displace people en masse across most industries is the problem.
You don't even need to go full socialist to solve that! Just tax AI providers on their revenue proportional to the number of jobs displaced by it, and use those funds as a pension for the displaced workers.
we stumbled into a way of brute forcing intelligence with gradient descent.
Well, who watches the watcher?
> I know that not everyone has morality and ethics, but I didn't realize it was this bad.
This is a very disrespectful way to make a point about acting with integrity.
You should consider that maybe your views on what makes something ethical or moral are not universal -- and that coming to a discussion with the assumption that your position is the only valid one is not conducive to convincing others who disagree with you.
I agree. I was wrong and I had a naive world view of morality and ethics in research and academia.
I now realize many people have different tolerance level for this.
I used to be a PhD student more than a decade ago, and I published a paper containing a solution to an open problem. Yet shortly after my first publication I became increasingly disillusioned, because I started to think that within my lifetime AI would reach and eventually surpass my ability to solve such problems—and that we only had a decade or two left.
So I started saying that it only made sense to focus on problems whose solutions would be useful immediately. I even emailed my supervisor about it, arguing that our efforts were “pointless” in the sense that AI-related problems were much more pertinent and had to be prioritised.
My supervisor thought I was bonkers. I still have the email, though. Quoting myself from April 2015:
> By 2030-2040 we will have enough computing power to simulate a human brain neuron by neuron. Once we manage to create a human intelligence we will be one little step away from super intelligence: just set the intelligence to modify itself and see the exponential growth in action. Our human intelligence is bounded by a number of biological factors (e.g. size of a skull) and even the smart human who has ever lived will appear to be a primitive ant to a supper intelligence (machine intelligence will also have perfect motivation). There is plenty of literature on this if you are interested in discussing this further. > > What does it have to do with research in pure maths? I can say that research in pure maths which won't come handy in the next 60 years is just wasted effort. The super intelligence will be able to do maths way better than humans. I believe a lot of current efforts should go into researching of artificial intelligence (or areas to do with AI) instead rather than the pure maths. I want to be proven wrong but most mathematicians I interact with are too narrow-minded to counter me and they just laugh about even contemplating the above. Frankly I am myself so perplexed that I take the above seriously, but I do and it's hurting my motivation.
I'm quite curious what my supervisor thinks of that email now.
I think, with all due respect, that your supervisor would be correct to think of that email as an insult to both them and to theoretical mathematicians as a field.
She was not offended but thought I could benefit from therapy. She didn’t share my opinion - she said it’d not bother her if AI could eventually solve problems for her.
I later realised that my motivation depended heavily on believing I was making a contribution that would otherwise go unmade. The prospect of AI doing that work undermined my motivation, but it needn’t undermine hers. I was trying to explain why I was struggling to continue, though I can see how calling the work “pointless” came across as dismissive.
Fantastic! E-mail him and ask if he thought about it at all
I commented something similar on a bunch of other posts. The thing that scares me about a lot of the AI community in general is that their utopia is basically more frightening to me than their doom scenarios. They present this "incredible abundance" as the ultimate human endgame, but I agree, when we all end up like the humans in Wall-E, what's the purpose of it all?
And then to get retorts of "it's just just rich techies that want to find 'meaning' in their jobs while millions starve in the third world". Why would we expect the most wealth concentrating technology in history to lead to mass benefits for those in the 3rd world?
Probably the opposite of Wall-E utopia, AI will keep us busy. For one - we are becoming harder to differentiate, competence is hidden deep and surface signals don't carry anymore. Niches and specializations feel threatened by imitation. This means everyone is frantically searching for their own corner, musical chairs. This holds for both people and companies.
Second - every time we learned to transmit better from past experience we saw massive expansion in culture and economy, not shrinking. Writing, printing, computers, internet and AI are just gradations on the scale of transmission from past experience. Everything stands on this transmitted experience.
My analogy to the body - every cell has the same DNA (every human has the same AIs, culture and tools), but they express it differently depending on context. And no one cell is too fat, the organism distributes work and energy. AI won't make anyone too rich. Its wealth is distributed in the system.
I am truly baffled that you think AI is just one more step on the spectrum of "knowledge transmission" between humans. I think that literally nobody who is an expert on this (both from the perspective of the people creating the technology, and the folks on the safety side warning about the dangers) believes that.
You're basically saying that there's no purpose to it all if there's no human suffering involved, somehow.
Why? Is suffering the purpose? What is the goal?
> You're basically saying that there's no purpose to it all if there's no human suffering involved, somehow.
Really? That's what you took from my post? Not enough eye rolls in me for that one.
100% agree, their utopia sounds far worse than their “humans will go extinct” which still sounds implausible to me. I would rather die than live in a WallE society, and I hate that they’re getting trillions trying to force us there
I think the impact is heavily exaggerated. Yes, it means mathematics will be transformed into a field that relies on tools instead of your mind alone. Just like most fields. Doctors heavily rely on (really expensive!) tools to make diagnoses and do surgeries too. But it doesn't make them less valuable. If anything it makes them more valuable, as the society needs a long training and filtering process to decide who to use these tools.
That's the thing. It might be. We won't know the impact until the next few years. The things that are changing now: culture of research, aspects of collaboration and sharing of knowledge.
Newsflash. Most graduate research is worth nothing. If this helps graduate students to re-evaluate and start doing things that can generate income while being intellectually stimulating, it's a net win for the person and the economy
I mean it is. Most of the time in graduate school, you're getting trained and learning to do reasearch. People rarely produce great work during graduate school. Only towards the end of their degree or start of new position (post doc, independent researcher, or assistant professor), you start seeing good work. And for a professor, most of their research is done by graduate students.
Academia has it's own sets of big problems. Before this AI boom also, most people graduating never stayed in academia. The way graduate schools are structured, I would be happy if few people joined. That would also mean we lose good researchers in a long run.
I don't understand why you introduce morality and ethics into your comment?
Pointing out that regressive Ai politics likely are not going to help researchers crisis of purpose is not unethical.
You also assume in your comment that we need new large generations of researchers, and that knowledge transfer through academic institutions is important.
I get the fear. But entertaining ideas of impact to institutions and potentially new post ai institutions is not unethical.
We don't know for this particular case with certainty.
But, here many people are okay with the concept of stealing other ideas, plagiarism, unethical aspects of collaboration in research. At least for me and many people I know, this is no okay. This is the part for morality and ethics. I see that no everyone agrees with this.
The issue is, these AI ecosystems don't effect everyone equally. Not everyone knows how to judge AI model output accurately. There is already a large divide on AI usage in research. The fear comes from this. If we wait longer, it might be alright too late. I could be also wrong with these speculations but better to cautious.
This story's comments are heavily astroturfed.
Just compare with the comments on https://news.ycombinator.com/item?id=49639408
I actually feel like it's the other way around. The online discourse over the past ~day or so has seemed unusually irrational to me for a technical audience. Commenters making emotionally charged claims of wrongdoing that appear inconsistent with the published claims without justification of the discrepancies. Granted you might well doubt openai's version of events but there's a general expectation of clear evidence when advancing claims of malfeasance.
I identify as neither mathematician nor "maker of things people want" (coder, hacker, engineer). But having friends who identify as those kinds of professionals, let me make some observations.
To use a metaanalogy from chess (once again), mathematicians play the opening game, and builders play the end game. AI is sort of a middleman connecting human understanding to applications.
I think there's a Technical argument to be made that openAI is a threat to the game itself. For example, could it have produced the navier-stokes counterexample without human inputs? since it seemed to have used the much gossiped research strategy "C" and "D", you can't absolutely be certain that Son of Astra (son of altman?) was magicking an unknown unknown from nothing (sorry to cue Rumsfeld). You have got to wait for the other five problems to be solved after general boycott
Subpar PR engine of the OpenAI leadership might kill the pipeline of inputs that they won't admit they still need in this dreamtime before "recursive self-improvement". You can call that emotional. Personally I would rather accuse mathematicians of "preferring local models that believe in the usefulness of unidentifiable individual contributors, and the uselessness of named generalist managers (ie the prompt writers at oAI)"
Big man tlb likes to say that science might be dead but engineering is just getting started. Navier-Stokes is the hammer of the nail in the science coffin. It kills science by killing the prestige of science. The engineers have to imagine that it's likely they will now get all their design ideas from the hypothetical future datacenters.
> The engineers have to imagine that it's likely they will now get all their design ideas from the hypothetical future datacenters.
At the current rate of advancement that phase will last, what, all of 6 months if we're lucky?
Do you think that's a good thing, killing science and the desire of humans to participate in scientific understanding?
Sort of, yeah. We are seeing more people commenting who have anti-AI sentiments or in the fence on the topic of AI usage.
On why some people are making emotionally charged claims, my guess: This affects the core belief of what is right or wrong, Impressions based on past doings of OpenAI, losing trust for OpenAI based on sequence of events.
I don't think we will get to see any clear evidence. I'm not even sure what would be the evidence. I would be surprised if OpenAI comes out clean if they have made a mistake. They move on to the next shiny thing.
People keep yelling about HN being astroturfed and yet I haven't seen a single proof of it.
Academia has been a toxic wasteland for decades. The ship caught fire and sunk long before open AI strip minded it prestige.
Academia is toxic for sure. But, also there are genuinely good people who work for greater good and whose training and teaching transcend across domains.
Oh, the patient is ill. Let's kill the patient.
The patient is dead. Let's finally bury her.
So tell us your story. It’s highly relevant to how we should interpret your opinion.
Otherwise you just sound like a jaded bitter washup.
Academia was already crumbling for various reasons. This just hastens it. The reasons are manifold and will never be acknowledged. There will be a lot of gnashing of teeth, but it was unsustainable anyway. And historically the bloat was a very short period, a few decades only.
To me it doesn't seem like what AI has destroyed is the ability for mathematicians to develop understanding and share it with each other, but rather it's destroyed the yardstick (solving open problems) that has traditionally been used to measure how much they have contributed to that understanding.
I do see how this is a problem in terms of assigning credit, but I think the cat is already out of the bag in terms of these models being capable. Even without AI labs spending millions of dollars to solve millennium prize problems, there are plenty of other people who will use them to pick low hanging fruit. I don't think any social solution is going to make things go back to the way they were, where you could share your progress towards a famous open problem without risking someone "scooping" you within a couple of days.
I think that the most likely outcomes are either mathematics becomes more secretive, or there is a more deliberative approach to assigning credit than who was "first" to solve some problem. In the former case, this may slow down progress, and in the latter case, this could mean that credit would become more subjective, and be a continual source of controversy.
The statement is not about AI but about the behaviour of AI companies. OpenAI have put vast resource into solving open maths problems: many millions of dollars of compute just on the Navier-Stokes result, plus whatever they spent on the broader Millenium Prize problems initiative and the other results they have published. Anthropic are doing the same. The statement is asking them to stop doing this.
AI companies are investing these resources primarily as a marketing exercise. There is no near term commercial value to a 100 page Lean proof of blow up in an extreme special case of Navier Stokes, besides the bragging rights. As the statement says any commercial value in this stuff comes a very long time later after new insights and techniques have been digested, integrated into the mathematical canon, expressed in ways that don't take a lifetime of study to understand, etc. (things that AI is not yet capable of doing itself). The bragging rights, on the other hand, are massively valuable. There is a mystique to maths that makes "our AI solved a Millenium Prize problem" an irresistable headline for a company like OpenAI.
What the mathematicians are saying is stop pouring resources that most mathematicians can only dream of accessing into projects that are actively damaging to their field. They face a massive challenge of figuring out how maths can evolve in the face of this new technology, and this is not helping.
What do we do about the problems that don't require many millions of dollars in resources?
Last weekend I spun up a small agent swarm and pointed it at a field of math I have some affinity towards. Within four hours I had settled three conjectures, one of which is rather famous (for the field, not in general). It cost me about four hundred dollars.
I am at a loss about what to do with these results. On one hand I feel like the mathematicians working on these should know about them, but on the other I feel a bit like a barbarian who suddenly finds themselves sacking Rome.
I'm not a mathematician but it seems to me that if all it took to solve the problem was an enthusiast level understanding of the domain and a few hundred dollars of tokens then the result probably isn't that valuable. Even assuming you are the first person in the world to solve it, these kinds of LLM-friendly problems that are now easy to solve and easy to verify will almost certainly be picked off by one person or another in the near future.
It's also possible that the result is already known and you just weren't aware of it. It's easy for someone outside of a field, or even one steeped in it, to not be aware of certain solutions.
sounds a bit like cope. Crouzeix's conjecture was solved exactly under these circumstances and I wouldn't describe it as "not that valuable". In any case, AI capabilities will increase dramatically over the next few years while human math capability will not. That means that there's a fixed target regarding whatever is currently considered a "serious math problem" and its difficulty level. Soon the average problem solved with a few hundred dollars of compute will be at that bar.
> while human math capability will not.
I actually partially disagree with this. What happened to all the excitement about Intelligence Augmentation (IA)? Now it's AI instead of IA. I think there's so much untapped potential for augmenting our intellect with the likes of https://dynamicland.org and https://folk.computer, as well as the work that's been going on in college math education, things like Lean, etc. I think the only reason human math capabilities haven't expanded that much is a failure of our imagination, not our potential.
I couldn't agree more. The way everyone thunders towards "replace" instead of "improve" is depressing.
> I am at a loss about what to do with these results.
I would recommend publishing them to Palomar (https://palomar-registry.org/) - I have no affiliation, this is an online registry of Lean-verified proofs created by Terrence Tao.
I have submitted a proof there that's also minorly important in an extremely niche field.
Anyway, I feel like it's a good place to dump AI slop lean proofs because the main point of the registry is that it verifies that: 1) your Lean challenge statement is the same as what you informally state you're trying to prove; 2) your Lean proof actually compiles.
This could be useful to future AI slop researchers who want to know if a given result has already been formalized, and they may be able to mine some lemmas from your work. Also, it's good to know for the field in general what has been proven.
I'm fairly certain you can set your publishing name to be whatever you want, so you could set it to be just the word "Anonymous", or the name of the model you used.
It is interesting isn't it.
You asked for a painting. A robot made the painting. You looked at it and said, "well, I guess it's good. Should I put it online or something? Dunno. Hey Fred, what do you think of this?"
Meanwhile, your next door neighbor spends their entire life developing their understanding of life through art. They "understand" (maybe not in a way they can articulate) art. You go next door, you look at their painting and say, "well I guess it's good." But you also understand that your neighbor is just like you, and maybe you are a painter in another way.
I find it strange that, people can't see that, we don't need to solve hunger and poverty and work balance, and etc, by a round-about make-super-intelligent-AI. We could just solve it. It's pretty obvious how to, as well.
We can all be painters, if we put restrictions on the psychopaths.
If we're all painters, where does the food come from?
Robot farms.
https://www.swarmfarm.com/
OP specifically said "we don't need to solve hunger and poverty and work balance, and etc, by a round-about make-super-intelligent-AI" though. And I don't know how you get robot farms without that.
But the post I was replying to said "If we're all painters, where does the food come from?"
And robot farms are being made now, without AI. Just plain and simple normal robotics.
what about cancer?
I mean, let's say you spun up a swarm of agents to rewrite a large component of a well used open source library to be memory safe. You could dump it in a big PR and walk away (we all know how that would go), or you could try engaging, see if they're interested, write something up and see where it goes.
The biggest problem is, IMO, drivebys uninterested in actual results, just getting a check mark, and the equivalent of dropping a 200k line PR on people and expecting them to be interested and do the work for you. These are things many on HN are familiar with and know how to do better :)
> and the equivalent of dropping a 200k line PR on people
I can understand why the community is pissed. So now, lean proofs can be churned out at scale, and the community is left to decipher all of that slop into human understanding. There are bad actors with misaligned incentives coming in with drive-by proofs upending what the community holds dear which is to practice and propagate the art. I applaud them for this declaration.
To re-align incentives the following could happen. AI slop lean proofs are dumped unceremoniously into a lean dumpster, and what gets rewarded are results that could digested into human understanding - via the already followed human review process. Prizes are not given to lean proofs since anyone with sufficient compute can churn them out.
How do you know the proofs are correct?
The numerical results are trivial to check. I wrote the analytical results in Lean by hand before asking a former professor to confirm after asking him to keep this private.
They're valid.
It's really tough to come to terms with it but making academic contributions in general from the outside (with or without AI tbh) is not often welcome and the whole process feels very gate-kept.
(academic.) Unfortunately the overwhelming majority of outsiders are missing core knowledge (or are cranks), so the optimal prior from a time management perspective is to ignore them. AI just makes engaging more costly because there’s more volume and it’s harder to get signal on whether they know what they are talking about.
If that's the case, that makes me much less sympathetic, even though I can understand how it's very disturbing to see the field suddenly changing like this.
Post them here and laugh about people trying to cite them: https://en.wikipedia.org/wiki/Superpermutation
It depends.
If you are trying to understand better the field, then do a good write up of the proofs so that people can learn from it.
If you want to earn the respect of people because you found interesting proofs. Then do a good write up of the proofs so thst people can learn from it.
If you want to plant flags and pollute peoples minds. Then please publish it anonimously, no one wants to correct LLM slop for you.
Probably we should build a repository of AI slop proofs that are only allowed to be publish anonimously. That way people may be more inclined to work on it because they would feel like they are cleaning your house for free.
Maybe I'll end up doing the write ups pseudonymously. I have taken care to make sure the results can meaningfully contribute to field but I don't want to plant flags or really receive credit of any kind. I just think they are interesting.
I like my current life and don't want to get dragged into the current fracas surrounding the use of AI in math.
If you have taken care, then share it with the community. People will be thankful and happy.
The problem is with people that may do it without contributing to the community.
If this was limited to just three results, I would agree with you. But those three are just the ones I've managed to verify myself. The list of unverified results is quite a bit larger.
The field I've been investigating is not large. Even if I were to take the time and care to beat the interesting results into something meaningful, I'm afraid the pace at which I'm able to produce these results would not be well received.
This all comes down to what we think mathematics is. There are two options here:
Mathematics is just the formal system: in that case we will never be able to beat the AI as humans. The goal is then to cover as much of the formal landscape with AI generated proofs to proof as much statements as possible. Success metrics are lines of lean and numbers of proven statements.
Or
The formal system is just a limited representation of what mathematics is: in that case probably some parts of the formal mathematical landscape are more important than others. Not all statements are born equal. And 90% of AI generated proofs will be just noise. Our job as mathematitians is to steer the AI to high mathematical value regions and turn formal proofs into mathematical proofs and insights.
From my point of view we have know since Godel that we are living in the reality of point 2.
Each of the two points implies a very different way of doing mathematics. So pick the one that you think is true and act accordingly.
> primarily as a marketing exercise
Like the mathematicians working on famous problems in private until they could claim full credit for something interesting wasn't also a marketing exercise for their own careers. The commercial value (or lack thereof) of a proof doesn't depend on whether it was done by a human or a machine.
These mathematicians dedicated their life to math and were working for a long time to achieve the pinnacle of their careers.
OpenAI just burned millions of dollars over a weekend after hearing that someone else was close to solving the problems. Their interest was in their AI system more than the actual math problems.
Don’t you see how that’s different?
I see how it can be devastating to their ego, but no, I don't see a particular difference in a company spending money for clout vs. a person spending time for clout. The underlying motivation is the same.
Do you really think people dedicate their life to mathematics for clout?
If clout was the goal I don’t think becoming a lifelong mathematics academic would be the first step
for many fairly strong mathematicians, the career calculus is fame and status (mild though it may be) through mathematics or anonymity but financial reward in tech or finance. Yes, clout by becoming a lifelong academic is in fact rational for some and part of their motivation. Of course they really like what they do as well, but earning the respect of the peers they know are also respected by a large swathe of society is very important.
Yeah that’s totally fair!
I guess that type of “clout” feels different to me.
Wanting to be validated by peers for your talents in a niche field vs. using millions to try to solve a math problem that you don’t really care about with AI to market the gigantic company you work for.
I think you’re right that some of the angst here is due to a fairly insular community having its culture disturbed by commercial forces.
That fairly insular community pretty much gave up on commercial success in order to be insular. But the commercial forces were not happy being commercially successful, so they decided to disturb even their insular and commercially-unviable activities.
Yeah bro, same for probably the majority of this site that write software for a living and/or hobby.
Sorry I’m not entirely sure what you’re trying to say
How good were you at those “A is to B as C is to D” SAT questions?
No need to be rude. I didn’t want to misinterpret when responding.
I can see how what is happening to mathematicians is similar to what is happening to coding.
I’m not sure what your broader point is? Mathematicians shouldn’t be upset? Coders should? Something else?
The point is that nearly every white collar worker is in the same boat right now, and the majority of them probably have already had much more profound impacts to their fields. So yeah, we can certainly imagine what it’s like for mathematicians.
Okay. I agree with all that you said in your latest comment.
From your first comment it seemed like you were disagreeing with me but I’m not sure how.
That’s why I asked for clarification.
Edit: BTW I’m a web developer and designer, Not a mathematician
> Like the mathematicians working on famous problems in private
This is an extremely rare situation that practically never happens. Most mathematical research is done in the open, with partial results being published, conferences where approaches are discussed, collaborations... The Navier-Stokes solution is a great example: the approach that OpenAI ended up using was something that two mathematicians had proposed previously and was being studied and followed by several others, with different sub-paths within the same approach.
> There is no near term commercial value
How much do you think other AI companies would offer to get access to the transcripts of the generation that led to the proof? No doubt OpenAI will include it in their training data somehow and use it to build the next generation.
There is already economic value.
Are they claiming that the only value in solving these problems was for their field's personal development process? I thought Navier-Stokes (and some of the other millenium prize problems) actually had implications for useful technology. It would be insane to demand that people avoid making progress on technology that can save lives or improve general quality of life, just to protect the sanctity of your karate belt system. Perhaps in lieu of open problems left to solve, mathematicians should be welcome to take up chess or sudoku to keep their minds spry.
> I thought Navier-Stokes (and some of the other millenium prize problems) actually had implications for useful technology.
This is the core misunderstanding that the open letter is attempting to correct.
Developing a better understanding of the Navier-Stokes equations could have a number of implications for useful technology. They're fundamental to fluid dynamics, and turbulence in particular is something that many people feel we could work with more effectively if we better understood how and why it's generated. The Navier-Stokes smoothness problem is an interesting and long-standing benchmark for this understanding; we don't know why it should be so hard to answer, so we hoped that the process of developing a proof to the problem would produce more understanding. (We may still be able to extract this understanding after the fact, if OpenAI's proof is fully human-comprehensible.)
Simply knowing that there exists a finite-time blowup is not practically useful. We know that fluids in the real world don't produce random singularities, so the result can't really have much physical meaning. What it illustrates is that the Navier-Stokes equations fail to model physical fluids in some yet to be characterized way.
No, there's most likely zero practical benefit of having found a pathological edge case in which the Navier-Stokes equations do not work. We're most assuredly not talking about "saving lives" here. Unless advanced aliens show up and tell us they'll destroy Earth unless a counterexample to the N-S equations is provided within 24 hours.
That solution (stop pouring resources in to proofs, stay in your lane) works today. How does it work 5, 10, 20 years from now? The software and hardware advances will continue.
Humanity is better off for knowing these proofs. This strikes me as academic NIMBYism.
Your point is largely addressed in the article, did you try reading it? "In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align."
The point is that these proofs are largely useless without the insights. The value of a proof is largely in the travel, not so much in the destination.
Different person here, I read the article and they are all wrong. Hope that helps.
Okay, to elaborate, substantively, their point is that the people using these AI models are not doing it for the love of the game, but for marketing. And instead of them - and nobody - spending millions of dollars to solve the problem, successfully, they want every problem of their academic industry to persist because even though they never solve the problem, they synthesize and solve lots of other problems nobody asked for. And get to boost their egos?
Yeah, stop that. Actual alignment is on the humans themselves, if they want to remain relevant as academics and mathematicians, they need to learn how to replicate the proofs and the steps that alluded humans for decades and don't worry about the narcissistic elements that slow their industry down.
Mathematical breakthroughs with commercial relevance are few and far between, and often depend on dusting off old results which were, at the time of discovery, "solutions nobody asked for."
The NS counterxample is actually, by any market measure, a "problem nobody asked for" in the sense that its existence doesn't have any commercial relevance (beyond juicing OpenAI's IPO). So the only long-term value solving it could have is by virtue of whatever reusable theory/insights were generated along the way to the counterexample itself. The letter is absolutely right on that point.
It's not actually clear that those insights will come faster from reverse engineering this LLM proof vs. humans building theory to solve the problem themselves. So what you're saying may or may not even be an efficient way of operating. Also, it implicitly depends on mathematicians to do the hard work of creating problems and then deciphering LLM hieroglyphics for essentially free while the only immediately profitable component gets outsourced to a frontier lab. In what world is that model going to work?
Reading between the lines, it seems like maybe you have a personal grudge for some reason and simply think the technology will advance enough to where we won't need academics at all. But you should say that in the first place.
What irks me is the ego
My stance is that solving the problem is aligned with humankind
the rest is just hypothesizing a way that academics fit in this world at all
I just wonder whether a lot of smart people who never needed to go beyond the "solve for X" algorithmic math of a typical calculus sequence are actually reading the declaration the way the signatories wrote it. The Navier-Stokes problem is not exhausted by a simple 'no' counterexample (which most people familiar with the equations already expected to exist). In fact, I haven't heard a single person's explanation for what relevance this counterexample has for humankind. It is something impossible in our physical reality so we have gained zero insight into anything we actually model with NS.
Mathematicians agree that "solving the problem is aligned with humankind." They disagree that releasing a counterexample this way actually constitutes "solving the problem" precisely because there is now little incentive to do the hard theory-building work that actually has the track record of leading to human advancement.
Their claim is only indirectly related to the motivations of the people using their models. What they're saying is that doing math in this way does not produce the same value as traditional mathematical research, and the people using these AI models aren't concerned about that because their marketing objectives don't depend on whether their results produce mathematical value. If people doing valuable work are made irrelevant by people doing a larger volume of non-valuable work, that's not a positive outcome.
But it does produce value. We now have an explicit solution and even a proof. Humans can then work on clarifying why it's true. Unsurprisingly, not all that different from software, where models can generate working code just fine. The details will all be there and all correct, but the architecture is currently not ideal, so a human guiding it can greatly improve the proofs.
I actually found this to be the case with some basic linear algebra notes I was recently doing in Lean (without using mathlib). The model could generate working proofs, but they obscure the basic ideas (actually I wonder somewhat if this is because the Lean code that's out there to train on doesn't make a huge effort to read like textbook proofs, which was my motivation in the first place). I give it a skeleton of a couple lines of `calc`, letting it fill in the reasoning for each line, and it does much better. Then ask it about making some macros to simplify "trivial" or "obvious" things, and it does even better. etc.
I suspect there's a good workflow where a big SOTA model makes an impenetrable proof (or code) and then a human works with a FIM model to simplify it (with the larger gnarly proof right there in context for FIM), but unfortunately everyone seems to only care about agents right now.
> But it does produce value. We now have an explicit solution and even a proof. Humans can then work on clarifying why it's true. Unsurprisingly, not all that different from software, where models can generate working code just fine. The details will all be there and all correct, but the architecture is currently not ideal, so a human guiding it can greatly improve the proofs.
To me this analogy points in the complete opposite direction. Imagine somebody takes a half-completed project design you're trying to figure out, vibecodes a rough prototype of it, emails your manager to announce that the project just launched in alpha, and then dumps it back on your lap for approvals and testing and productionization. Would you say that they've added value to this process? Or did they just strip away all the hard parts of the problem so they could claim credit for the easy part?
One of the awesome things about LLMs is they make it quick and easy to make PoCs, so yes. Proving that an approach will work before spending a bunch of deep design effort is absolutely valuable. Your exact scenario is something I've literally done: give a half-completed design to a team member and asked them to vibecode a PoC to prove the approach will work and figure out some of the details, explore scaling and failure characteristics, etc. Or I do the PoC vibecoding myself too. LLMs have been a gamechanger here.
It's valuable for you, the person who's going to spend a bunch of deep design effort, to make POCs. Is it valuable for someone else to drive by, dump some POCs on your lap, and then leave you to do the deep design effort while they run away to study AI?
If that person then runs around telling people that they're the real author of your project, because they generated the original POC, would you consider that an accurate assessment?
Your analogy is far enough away from the way that the real world works that I'm not sure that I can really even strain my experiences to fit within it. Sure, I guess that would be annoying?
But mathematicians define their field. They're smart people. They're capable of recognizing when someone just did a vibecoded throwaway PoC and when someone has a well structured proof. Actually even before LLMs they'd publish new, clearer or more elegant proofs of old results. They can say that inscrutable proofs are exactly as valuable as they are, and that the first explanation people can actually understand carries its own prestige.
It's a real example that's happened to me twice in the past year, so I'm not sure what to make of the idea that it's far away from how the real world works.
I'm also not sure I understand what you're objecting to if we agree that mathematicians define their field. The source link is a declaration from 25 Fields Medallists with precisely that goal. They believe/define/declare that the type of AI-generated proofs we've seen are vibecoded throwaway PoCs; they feel that a well-structured proof must include factors such as "a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others", and the success criterion is not a true/false conclusion but rather "development and integration into the mathematical canon".
Maybe your management has no idea what you work on. If so that's its own issue. Or maybe they do, and the person trying to take credit for more than they did just looks like an idiot. Like I said, I intentionally give throwaway PoC creation as a work item to people and it's fine. And if e.g. someone randomly made a drive-by PoC that proved that the approach I was exploring couldn't work and had some fundamental flaw, well, oddity of a random person doing it aside, I'd want to know that, and they'd deserve credit for saving us from a bad approach!
My objection is characterizing things like AI slop proofs as not valuable. Obviously it's valuable to know that:
1. NS has solutions that blow up in finite time.
2. This fact is provable, and we have a proof.
I don't think anyone anywhere is saying that these will replace mathematicians as they are now. I also think "a proper write-up, the isolation of new methods and ideas, and citing relevant previous work of others" is frankly not necessary or even desirable before we publish a computer generated result. We have a machine now that can spit out answers that we have good reason to believe are accurate, but they're perhaps inscrutable. There's no need to first decode the why and figure out proper attribution before simply posting the proof online. The proof itself does add value as it stands, even if it's not the ideal. Hoarding it until you can do a proper write-up would be silly.
Yes, mathematicians will clearly need to rewrite the qualifying criteria for prizes to better align with the actual goals and value they were hoping to get from a solved problem. The field as a whole assumed good faith actors and collaboration, not expecting a few trillion dollar companies to walk in and start turning in piles of Lean no human understands to be able to claim "first".
This letter includes someone like Terrance Tao who publicly expressed a lot of optimism about AI for solving novel math like with the Erdos problems. It's not sour grapes but the first steps to define those new expectations for the future to reduce the perverse incentives.
And yet, predictably, people are accusing him of "gatekeeping" and ignoring the arguments he has made here and elsewhere about the benefits vs. harm in different ways of using AI.
I don't see turning in piles of Lean as bad-faith somehow. They were first, and they did prove the result (assuming no hidden exploit deep in the middle). The thing they produced is just different, and both giant raw proofs and distilled human-understandable proofs are valuable. Before we couldn't make giant raw proofs, so we didn't need to understand their place. Now we can. It makes sense to just incorporate that into what "math" is (at least until the machines are smarter and can make elegant proofs from the start).
>> Humans can then work on clarifying why it's true.
Presumably you're a human. Are you going to do that?
There are vanishingly few research mathematician positions and it's one of the most competitive fields, so no. But I'm not sure how that's relevant. As the OP says, usually the value of a proof is not the knowledge that something is true per se, but the reasoning techniques to understand why. How can it be anything other than helpful then to have a truth oracle as you try to figure out why things are true?
So you're not going to do it yourself and you want someone else to do it for you? Some mathematician that dedicated their life to understand mathematics must now toil unpaid and unwillingly to understand the AI slop proofs that you want us to be able to understand?
Do the job yourself. And if you can't, that's maybe a hint that you should listen to the people who can.
Who said anything about unpaid? I'm pretty sure professors don't show up just for fun. Our taxes pay them.
I'd be happy to do the job. Actually I still dabble recreationally (clarifying Codex's Lean proofs, even!). But like I said it's one of the most competitive fields on the planet. As you say, you have to dedicate your life to it.
If a slop proof isn't helpful, they don't have to "toil unwillingly to understand it". They can just proceed with the knowledge that the proposition they want to prove 1. is true and 2. is provable, which is already a decent start for motivation. But often LLMs can actually do quite well explaining ideas too in the hands of an expert. Or you can ask them to prove some technical lemma that you think ought to be true, and that could offer insight for the thing you're really interested in, but for which the details are actually not all that interesting to you. You don't have to one-shot "prove RH from the ground up in 50 million lines of Lean."
> I read the article and they are all wrong
I would recommend a bit more humility and trying to better understand why 25 Fields medalists, among them people like Terence Tao (who isn't anti-AI by any means, he's even promoted a registry of AI Lean proofs), are saying this.
> Okay, to elaborate, substantively, their point is that the people using these AI models are not doing it for the love of the game, but for marketing.
No. The point is that AI companies are using the models to solve problems in such a way that the useful part of problem-solving, i.e. the theories and tools developed during the process, is not present. And they are doing that because the companies seem to be motivated not by honest advancement of math but by marketing and publicity.
> they synthesize and solve lots of other problems nobody asked for.
No one asked Fourier to solve series representation of functions when he was studying the heat equation, and yet thanks to that we have Fourier analysis.
> if they want to remain relevant as academics and mathematicians, they need to learn how to replicate the proofs and the steps that alluded humans for decades
The point they are making is that if AI keeps being used as "problem solver" rather than "theory understanding", replicating the proofs and getting the useful parts out of them will be far more difficult.
Do you "know", in any meaningful sense, any of OpenAI's recently publicized proofs? Do you suppose that there is any large community of non-academics that does?
One of the points the parent makes, along with the TFA, is that academia -- or more specifically, the "mathematical community"-- is a setting primarily for creating and ingesting mathematical knowledge, and disseminating it to the next generation and to other fields. Humans absorb this material slowly, through lots of discussion and collaboration -- it is necessarily a slow process. Facilitating this is one of the important functions of academia. Your usage of academic as a slur here is a bit silly for this exact reason.
I don't claim it is perfect, and we can argue about pedagogy in elementary courses till the cows come home. That's not really material. But this is one of the only settings in which such knowledge is broadly valued for its own sake, and in which there is a semblance of incentive to help others "know" this stuff as well, be they future generations of mathematicians, science and math educators and communicators, practitioners in other fields, or genuinely curious amateurs.
Why/how do you think I used academic as a slur here?
I suspect that beyond just marketing, these pursuits yield plenty of useful information about model design that will likely lead to model improvements and optimizations for both mathematics and general reasoning going forward.
> What the mathematicians are saying is stop pouring resources that most mathematicians can only dream of accessing into projects that are actively damaging to their field. They face a massive challenge of figuring out how maths can evolve in the face of this new technology, and this is not helping.
Is it reasonable for any field to make such demands? If this were doctors objecting to AI becoming good at medical practice would you have the same concerns?
While any idea of OpenAI spying on people to pursue their goals is disgusting, the rest of this is par for the course, as Kasparov experienced with IBM in the 90s. Humans still play chess after all.
It's not a simple matter of "becoming good at", and yes, I could very well have similar concerns, depending on how it impacts the field. The statement itself mentions that such concerns exist in many other fields.
I doubt OpenAI will take such a combative stance and accuse these mathematicians of "demanding" things, as you do. As I said, the purpose of this is marketing and the statement simultaneously undermines the value of that marketing (showing these projects as irresponsible) and gives these companies an even better piece of marketing in its place: "our AI got so good at maths the mathematicians begged us to stop". It's entirely possible they will stop pouring millions into these projects.
So let’s say OpenAI cure cancer and put every cancer researcher out of work depriving them of intellectual satisfaction, this would also be a problem? It would certainly impact the field.
The fact is these fields are supported by society because of the benefits to everyone else. Once the same results can be achieved in a cheaper and faster way that is what will be done. We should mourn this in the same way we do buggy whip manufacturers. Again people still ride horses.
> So let’s say OpenAI cure cancer and put every cancer researcher out of work depriving them of intellectual satisfaction, this would also be a problem? It would certainly impact the field.
Would you expect Fields medalists to cure cancer if you moved them from the math department to a medical research lab? This is precisely the fallacy that the frontier labs are counting on to inflate their valuation as their IPO approaches. They want to use headline-grabbing problems in pure maths to make their models look "smart" in the public eye. But what does "smartness" in mathematics really mean in terms of economic value? It is not at all obvious whether success in abstract mathematics should translate to successes and, more importantly, profitability, in more grounded endeavors.
Look at OpenAI's job postings (https://openai.com/careers/search/). Those roles involve far more pedestrian yet profitable duties than research mathematics. So why isn't OpenAI automating them with their vaunted models? Success in one field, no matter how "difficult", does not predict results in another field.
> The fact is these fields are supported by society because of the benefits to everyone else. Once the same results can be achieved in a cheaper and faster way that is what will be done.
Maybe it's worth double checking that you know how these fields benefit everyone else? Proving the blowup of the Navier Stokes equations in 3D isn't going to make your gas cheaper or make harvesting food easier or make drones easier to protect against. Maybe consider the deeper effects at work?
Math people absolutely are important to things like the SpaceX landing control systems, hypersonic gliders, 5G networks and so on.
If you can make breakthroughs on such areas as fluid dynamics, control theory or information theory with AI then that absolutely is a big deal with real technological implications.
A long complex proof to something we have been able to simulate in fidelity forever doesn't do any of that though.
You need to grasp that programmers dealt with this cope wave a year ago and know exactly how ideas like yours pan out.
Why don't we first assume OpenAI can create a cold fusion reactor and then give them a trillion dollars to get it done?
Chess is even more mainstream and accessible now!
yeah beacuse no one trust any of their benchmark results now they are scrambling to find a signal thats undeniable
I expected this. They prove a millenium result, but it doesn't count because they are bad people.
It's this sort of thing that motivates people to burn down the institution you might be trying to defend.
> It's this sort of thing that motivates people to burn down the institution you might be trying to defend.
lol, yes, this sort of thing is what many people who voted for Trump were saying, and things are going great for them.
> They prove a millenium result, but it doesn't count because they are bad people.
OpenAI has only themselves to blame for this, and they know it. They could have handled this so much better. I'd bet there's more meeting time right now going into how to unveil future math results than on meeting about the actual math research.
The issue of credit is a relatively minor point in the declaration.
It's more about bypassing the culture and processes mathematicians have developed that lead to human understanding, generating new ideas, and bringing up new generations of mathematicians. (See also his article about "non-renewable mining" of good problems.)
Reducing mathematics to "let's just generate results through an isolated and automated system" is a misalignment since it bypasses those processes.
I don't think it really attacks human understanding though. You can still read and understand an AI written proof. If another person comes up with a solution to a problem, you can read their methods and understand it. It doesn't matter if a human came up with that or not. It's really only attacking the "generating new ideas" part.
That's precisely the problem though. You cannot still read and understand an AI written proof at the current skill level of the AI being applied, because they're orders of magnitude longer than human written proofs even when they don't need to be, and spend most of that length on the parts that aren't important. This has been really thoroughly documented by expert mathematicians who are engaging with AI in public like Terence Tao and showing in detail how much work it takes working alongside AI to figure out how to understand AI generated proofs. With human generated proofs that process is forced to happen before publishing the proof because the new style of AI generated proofs validated only by formal verification is supplanting the old human peer review process that forced the burden of understanding onto the publisher and not the reader.
> You cannot still read and understand an AI written proof at the current skill level of the AI being applied, because they're orders of magnitude longer than human written proofs even when they don't need to be, and spend most of that length on the parts that aren't important.
Just like how they write software, then :-)
That doesn't seem to be true. The OpenAI NS paper was 166 pages. Wiles-Taylor proof of Fermat's last theorem is 129 pages. The length is not unprecedented for a difficult unsolved problem.
To be honest, I feel like the difficulty of reading AI proofs is due to the fact that we are on the verge of being beyond human comprehension. This is a demonstrable fact as no human has figured this out despite the problem being open for almost 100 years.
> To be honest, I feel like the difficulty of reading AI proofs is due to the fact that we are on the verge of being beyond human comprehension.
I can see where that's coming from, but I really don't think it's the case. Even with Astra, the proofs you get are just off in a way that doesn't signal superhuman comprehension. As 9question1 says, a common theme is that they dwell on insignificant steps. Another one is that they'll often be full of terminology that either doesn't exist, or has this weird quality where it looks like it is trying to make some minor insight seem much greater than it is. At first glance, that'll often make it look like it knows more than you, but when it's really just doing the same thing but in a more complicated and worse fashion, that to me isn't a signal of comprehension at all. The bizarre thing is that despite all the "stochastic parrot" style nonsense you'll get in individual proof steps, they still often combine to something valid.
In either case, what all of this means is that the working mathematician still needs to go through, and generally completely rewrite, any proof output by an LLM. Otherwise you are passing the burden of unreadability onto the reader.
Yeah, that mirrors what I've seen throwing some of the leading models at a set-theory problem that's stumped me (https://mathoverflow.net/q/511601): in this case, the problem does not easily yield to the standard tools, but the LLMs do not recognize it as a major open problem they should give up on. So they seriously try it, but typically end up in a loop of inventing certain classes of simple solution or counterexample attempts, defeating them, and trumpeting each one as a major result, each time inventing some new terminology.
It's definitely quite curious that the AI labs are able to push these results through seemingly with pure brute force. Perhaps it's largely a function of how many monkeys you have attempting various constructions on top of the known results and strategies the models have memorized.
That matches my experience with AI writing in software engineering so I’m not surprised
> This is a demonstrable fact as no human has figured this out despite the problem being open for almost 100 years.
That's not true. Alpoge and Buckmaster's related LLM-assisted blowup result (https://news.ycombinator.com/item?id=49605915) utilized a strategy developed recently by Cordoba and Martinez-Zoroa.
This is a very token-brained take. The length of a work has no bearing whatsoever on its comprehensibility.
Not "a" human's understanding; Humanity's understanding. Understanding the research problem, and the solution especially, is a lot more involved than simply "read their methods". That's the whole point being made.
It matters if a human came up with it because of everything mentioned in the article... A mathematician's solution is necessarily built on other's ideas that have been disseminated, internalized, pressure tested etc. Methodologies differ too. AI can abuse its compute resources and generate a true/false or counterexample statements, without laying the foundation that a decade of globalized research would have.
>You can still read and understand an AI written proof.
No you can't lol, they're multi million lines of Lean, which is already an obscure language to understand. It's an assault on your senses.
It's not only Lean code, there are English writeups too. To my understanding the pipeline for these problems is 1) solve in english 2) formalize systematically to check. No one is tackling problems purely in Lean, to my understanding.
https://cdn.openai.com/pdf/32d9f210-8b73-45e0-91bc-82a30aef8...
[dead]
I wonder if this is a root of the complaints across fields, how AI is ruining the greater picture and process in writing, acting, drawing, filming, coding, and more.
That it makes life more ends and less means.
It's also known as "commodification of labour", and AI is just the latest and greatest tool to do it.
Luddites complained that the trajectory of technology was to allow less skilled workers to mass produce goods via machines owned by factory owners, as opposed to helping skilled workers build up and use their skills while passing them on.
Now we have a lot of money and time focused on LLMs owned by a few companies, making it easier for them to monetize low skill labour(prompting versus art/research/artisanry)
> Luddites complained that the trajectory of technology was to allow less skilled workers to mass produce goods via machines owned by factory owners, as opposed to helping skilled workers build up and use their skills while passing them on.
And their complaint was valid from their perspective, but the end result of that process is that now you can have functional and pretty clothing for pennies.
It's true that a lion's share of the gain was pocketed by people who owned the mechanized looms. Now imagine how much more of an advantage it would have been without those people.
Same thing here. We can - and should - get rid of all the megacorp leeches. But the benefits of cheap automation remain regardless, and it's still the same equation - removing the gatekeeping harms those who relied on it to keep the products of their labor priced high, while benefitting everyone else.
Let's take your argument one step further.
Suppose that tomorrow we learn that AI just exploited a bug in Lean and the proof is, in fact, bullshit. Or suppose it is the case, but we never learn that.
Where are "ends" and where are "means" here?
Well, given a proof of something then a system can be built that relies upon the inviolability of that thing.
Should the proof turn out to be bullshit, then that system will be revealed to be unreliable. Maybe.
> The issue of credit is a relatively minor point in the declaration.
What a load of croc. This entire debate is fueled by a perceived lack of attribution. The AI learnt from researchers and did not give them a sporting chance of being first before scooping them. They were expecting some sort of fair play, instead they got a ruthless machine. Every other tangent to this debate is irrelevant, the culture, the community, the shared symbolic growth. Every mathematician I know is secretly trying to one-up their peers.
I agree that the declaration doesn't focus on credit, but I think it's still at the root of the problem. Because ask yourself: if the AI generated proofs are not creating any new ideas or insight, just brute forcing a boolean true/false result, then why can't mathematicians simply ignore their results? Why does it matter if OpenAI or even amateurs with AI are "solving" these problems, without contributing to any deeper understanding?
I don't think "intellectual poisoning" is really the mechanism that harms the mathematics community.
The harm is if you have a community of mathematicians who are focused on expanding human understanding, then having instant access to a bunch of AI proved results muddies the water about who has contributed what. If someone could scoop any significant theorem at any time by pointing an AI at it, how do you really demonstrate that you have created new understanding? Or that your new understanding is about something important? How do you prove that the AI needed your new concepts to be able to solve it?
Open problems are not that yardstick. Fermat's Last Theorem is the result of Wiles and Wiles-Taylor, but without key results from Serre, Ribet, Ihara, Langlands-Tunnels, and Frey's program none of what Wiles did would work. But Wiles did get the prize. Nowadays I think the inputs have shrunk a bit by doing more in the R=T theorem so less other cleverness needed.
Doing the last step of solving an open problem is the yard stick. But not for much longer: https://davidbessis.substack.com/p/the-fall-of-the-theorem-e...
> destroyed the yardstick … that has traditionally been used to measure how much they have contributed
This goes much broader than mathematics or academia. This is the entire basis via which society distributes its wealth: based on a labour market derived valuation of ‘contribution’.
> This is the entire basis via which society distributes its wealth: based on a labour market derived valuation of ‘contribution’.
Correction: that's not how society distributes its wealth, it's how it throws some bones to the masses. I wouldn't be surprised if over half the wealth goes to people who don't sell their labor at all.
>or there is a more deliberative approach to assigning credit than who was "first" to solve some problem
it feels like an unintended consequence of the millennium prize is that people view the [last contributor to the solution] as the only one to make progress on the problem. I've never viewed Poincaré as solved by one person and the objective of the prize was to encourage more people to make attempts and contribute towards progress.
this issue is independent, but in these circumstances perhaps interweaved, with the 'ai is taking over math' concerns
> but I think the cat is already out of the bag in terms of these models being capable.
When there's a discussion about doing something against the damage of the AI industry: "whoopsy, sorry, another cat escape, nothing can be done".
When there's a concrete mention of an actual solution to avoid more cats escaping: "that won't happen, and even if it did, the damage is already done, and in fact it’s not that bad you all just have to go with the future we decided for you."
So the bag is wide open, more cats will escape, and nothing can be done about any of it. not about the ones that got out, and not about the ones still inside. Sounds more like a preemptive excuse for inaction, cosplayed as pragmatism
I feel a similar fate will befall engineers too. Your predictions anre quite interesting from that perspective.
Markets defined entirely by law have distorted our collective understanding of what can actually be built with the knowledge our species has accumulated thus far. How will traditional shields that have protected capital accumulation in tech to survive in a world where governments now realize control of technology is a national priority? Especially as we see its impact on modern warfare, and that such conflict looks like it’s only escalating over time.
Mathematicians appear to me (as an outsider) to exist in a field without such distortions, and I think offer engineers a preview of what’s to come. I certainly have completely ceased sharing original ideas online at this point.
Engineers are needed to build real things. AI isn't there yet.
> To me it doesn't seem like what AI has destroyed is the ability for mathematicians to develop understanding and share it with each other, but rather it's destroyed the yardstick (solving open problems) that has traditionally been used to measure how much they have contributed to that understanding.
I think you're missing key details in the statement. You can hate problems and sprinting towards solutions, and still have a deep impact in mathematics – Grothendieck once famously said you can often "soak the walnut" to crack open an issue. Rather, it's how frontier labs are inherently commercial incentivized to act like this, because it is in their best fiduciary interest. That is the misalignment.
To me it doesn't seem like what AI has destroyed is the ability for mathematicians to develop understanding and share it with each other, but rather it's destroyed the yardstick (solving open problems) that has traditionally been used to measure how much they have contributed to that understanding.
It's more than the yardstick for individual contributions. It's also the yardstick for the contribution of ideas. And that kind of yardstick is very important.
Math has a challenge that's only occasionally acknowledged imo, that's it's possible to just go up the ladder of abstraction, formulate ideas but have those ideas actually not be useful or interesting. The ideas that math has developed, that mathematicians consider important, are those that have helped people solve hard problems.
Agreed. The problem is not with AI per se, but with the reward mechanism in academia in general.
For those outside academia, the ”reward mechanism” is a choice between A) being a genius and working hard to become a leader in your field, B) becoming very good at writing grant applications, C) capitulating to corporations and living with the moral burden of their exploits
Right. It seems like reading an AI proof (although it may not be well written) will provide the same insights as reading a proof from another mathematician, assuming it's been reviewed and edited, just like any human-authored publication. If the work is inherently valuable on it's own, I feel like that's mostly what matters.
At issue is the fact that it doesn't typically work like: mathematician produces a proof in isolation, generates a PDF, and shares it with a bunch of people. There's a whole culture and community going on behind the scenes with conferences, seminars, lectures, chats in the hallway, advising students, etc. that AI-generated proofs bypass.
I don't think that AI would disrupt any of that. Even if AI solves a problem, you can still discuss the methods at conferences, seminars, lectures, chats in the hallway, advising students, etc.
Tao’s key point is that the way people are using AI today does disrupt that. Because people and companies are valuing the results over understanding. So we’re getting slop results rushed out that are automatically verified.
Moreover in the past, discussion and idea sharing would happen naturally to overcome the friction of the process. But now when OpenAI is stuck on a particular part of NS for example, they can just throw more capital & tokens at the problem.
I won’t speak for Tao. But it is not “people” who have valued the result over understanding. “People” are a bit impressed that the machines are as good or better than the priests who have been praying at the inscrutable altar of pure mathematics. But it is mathematicians—I am speaking as a mathematician—who have adopted a culture of valuing results over exposition and transparency. The field has been rife with extremely opaque papers for many years and the character of research participation has been one of exclusion and competition over proof priority at the expense of understanding and transparency. It is unbelievably ironic to listen to mathematicians complain about “AI slop” when they have built careers upon human “slop” if slop means papers crammed with technical density that prevents all but specialists from reading the work.
I don't think that's entirely fair.
I think it's OK that there are some materials designed for a specialist audience, and some materials designed for a wider audience. Technical density serves a real purpose in the former (I'd like to just say "stack" without including an explanation 10 times longer than the rest of my paper about what a stack is and why we are talking about them), and some people expend a lot of effort on the latter (think lectures, lecture notes, textbooks, seminars, blogs, etc--totally appropriate to dig into the motivations here).
It's just the fact that it's a really vertical field, not some cultural failing, that results in pretty opaque stuff sometimes.
Hopefully that's true. Instead of bypassing the community maybe it stimulates the community. I think it does disrupt things a bit, but I'm optimistic at the moment!
Scale the problem down to an email. You're saying AI won't disrupt writing an email, when the text is vetted and authored by a person. True!
But now let chatGPT write lengthy emails unrestricted and now no one wants to read your slop anymore. That's what is being advocated against.
Is it not an option to make academia more like a normal job, where people focus on collectively achieved outcomes rather than credit, priority etc.?
Almost every "normal" job has regular performance reviews where individual contributions, not collective outcomes, are reviewed and used as a sole input for raises, promotions, and firings. If you can't sufficiently document what you personally did, you might've as well not done anything at all.
Yes, that's true, but it is still generally accepted that a completed project is a result of some collective effort where people of varying degree of seniority and ability contribute. There is also not a singular event of a project being completed with a list of heroes/geniuses making it happen, but rather a whole lifecycle of gradual development, maintenance and going out of relevance with contributors coming and going.
I can imagine mathematics of the future being more like that rather than history of discoveries with dates and names
How is that like a normal job?
Nothing will be lost if the credit system disappears. History of Science has many examples where wipe outs happen. The chimp brain cant survive without creating elaborate stories about how important it is, more as a cope to its own limitations and what it cant predict or control. Humility is good for health. 3 inch chimp brains didnt create the universe.
[dead]
Tao's critique of AI in the field of mathematics reminds me of what French art critic Charles Baudelaire said in the 19th century about photography [0].
Baudelaire argued that photography became a haven for failed painters, the sorts of hacks that could not finish proper training. Photography, as a mechanical rendering of the world, could only record what already existed; it couldn't transform reality the way a painting could.
He also criticized the public's craze for "rushing" into it, and complained that this technical "progress" was weakening the arts.
Do you see some parallels as well?
[0] https://fr.wikisource.org/wiki/Curiosit%C3%A9s_esth%C3%A9tiq...
I have listened quite a few interviews with Tao and I see him being very careful about criticizing AI. He very often emphasizes the usefulness of it. Where he is critical has a lot of merit. One of the points I clearly remember him saying that having AI be able to solve many of the open problems, regardless of how important there are (there are many open problems that are not that important) greatly reduces the problem space for mathematics students to give new problems to work on.
In a parallel thread omnicognate correctly pointed out that for AI companies it's a direct commercial loss to pour all this money into bruteforcing the solutions to these problems, and that a lot of times the solutions by themselves are not directly commercially valuable. They are doing it for stock price, trying to lure in private capital in preparation for IPOs.
Their models are good, but they are not the moat because Chinese models are good too, so what they are doing, in my opinion, is more harm than good. Mathematics is a science by humans for humans.
> One of the points I clearly remember him saying that having AI be able to solve many of the open problems, regardless of how important there are (there are many open problems that are not that important) greatly reduces the problem space for mathematics students to give new problems to work on.
The crux of the argument perhaps. It suggests that too many people are currently studying mathematics without making much progress.
>I have listened quite a few interviews with Tao and I see him being very careful about criticizing AI. He very often emphasizes the usefulness of it.
This is tiresome. People should be able to flat-out criticize AI without the implied need to justify themselves all the time or "be careful". Its almost like AI has a trillion-dollar agenda backing it, to the point that you have to add a careful "its really great! But there's this little issue..." for any criticism.
Even those who are very pro AI should have the intellectual honesty of admitting that there are very valid reasons to criticize AI.
Based on the amount of low-effort criticisms out there, clearly nobody is afraid of "big AI". Tao seems careful about his critiques of AI because low-effort off-the-cuff criticism can lead to all sorts of future problems/hypocracy.
That's not really addressing the same issue though. I'm saying that there is a pressure to preamble any (good or weak) criticism of AI with an Apostle's Creed of AI positiveness, reassuring the reader that what follows isn't blasphemy against AI (to borrow Huang's terms).
I see where you are coming from. I imagine LLMs will follow the same feedback lifecycle as other groundbreaking tech. Most legitimate iPhone 1 complaints came paired with a few positive messages to the devs and engineers. Years later, after countless improvements to the hardware and UI based on user feedback, now it's just complaints.
I think there's a difference between "photography will change art--we need to be ready" and "photography will change art, therefore stop photography."
There is no doubt that AI has changed the practice of mathematics, just as it has changed the practice of software engineering (and will soon change almost every intellectual job).
Trying to deal with change by saying, "please stop the change" is foolish, IMHO. Mathematicians need to redesign the discipline with AI in mind. But I get that it's easy for me to say that and hard to actually do.
> I think there's a difference between "photography will change art--we need to be ready" and "photography will change art, therefore stop photography."
That wasn't the critique. The statements then were that photographs weren't actually art at all. They were the results of specialists with skills but not 'art'. Yet the photograph as a product was displacing real art in the marketplace.
AI is displacing real mathematicians.
Both of Baudelaire’s criticisms were reasonable, and the same thing happened again when AI image generation showed up. You think the opponents look ridiculous because you’re viewing the history from the winner’s side. As for “the public,” people had a real demand for photography as a way to record things, which is also why it won. There is no comparable public demand for proving Fermat’s Last Theorem.
> There is no comparable public demand for proving Fermat’s Last Theorem.
It's a capability demo, not the main point. There is plenty of demand for intelligence on tap. These flashy results are necessary though, otherwise people keep claiming AI is just hype, it can't do anything etc. So they need something to point at. And it seems to be working because the anti-AI narrative has changed from the previous stance that it's just the new NFT that will simply go away to how it will replace human intellectual work and will be bad for artists and desk workers.
Genuine Art versus Mechanism, from 1901, (https://www.jstor.org/stable/25505621) is another article that I read a few years ago that other people might find interesting.
Are you aware that Tao is among the largest proponents of using AI in mathematics? The usage itself is not the point here.
Tao doesn't go as far as Baudelaire, but there are some similarities. In particular, Tao has criticized that AI is not being used to create new interesting conjectures, and that the rush to prove old conjectures is not giving human mathematicians enough time to carefully analyze and understand the proofs and the methods used in those proofs.
My answer to both is the same: nothing stops mathematicians from doing both of those things, with or without the help of AI. And we all understand that it will take time to do that. But complaining about the dawn of a new era of advancements seems counterproductive.
But the point is your original analogy with Baudelaire now just boils down to "both of them were critical" which is quite shallow indeed.
It's especially annoying given that many people are making repeat Baudelaire's critique of photography (or Socrates critique of writing) in the context of AI and the declaration is interest because it's not that.
There are no similarities whatsoever, other than them having opinions on a new tech that affects their field.
> nothing stops mathematicians from doing both of those things
Actually yes, and they explain that pretty clearly in the letter. These problems have been used as "goals" not because the solutions are necessarily worth it, but because they set a clear direction whose process would probably yield very useful theories, and also more conjectures and problems. If the goals get solved by AI models that do not generate comprehension, then the direction gets less clear, it's harder to collaborate and get funding (think what would be easier, to get funding to try and solve Navier-Stokes or to try and understand an already proven Navier-Stokes?), and we get less open problems to work on.
> get funding
It all comes down to this. Yes, they will need to give a good and convincing argument to the taxpayer. And at the end of the day, if the circumstances change, and you're unsuccessful in making your case in your plead, the taxpayer and their representatives may not want to keep the same level of funding and instead put that money to use elsewhere. So make a good argument why your job is still worth that cash under the new circumstances. The argument can appeal to aesthetics and art and philosophy and a kind of feeling of religious elegance and so on, it's fine. I can't say how well that will work, but it's worth a try. Taxes also fund theology and other pursuits with no tie to cold hard engineering etc. But that level of funding is not the same as for math currently. That gap may shrink.
I don’t get the argument you are making. If you want to reduce funding for mathematics research, argue for that. But wanting to make getting funding and coordination and collaboration harder in exchange for nothing other than PR for AI labs does not make sense
Is it just PR? Okay then the useful math can be done by AI and we can leave aesthetic math to human academics who will ascetically refrain from using AI and if anyone of them submits a paper and it turns out they used computers they will be treated like WADA treats Olympians on steroids or chess players with a vibrator up their ass telling them what move to take. But again the taxpayer may not foot the bill for this endeavor as much as they did while thinking it may be useful for technology, science and engineering innovation.
But did you read the letter, or are you familiar with the people signing it? Several things tell me you didn’t:
- The useful part of math is the tools, the understanding and the theories. Knowing whether the result is true or not is usually not useful. Imagine an AI that just says “cancer can be cured” but doesn’t say how: that’s what the letter is criticizing.
- Terry Tao is one of the biggest proponents of AI use in math. He’s been for quite some time and he shares a lot on his blog.
Seriously, it seems like you’re arguing about some idea you have in your head and you’re not making the effort to actually understand anything about this letter.
AI won't just say cancer can be cured without saying how. Not sure where you got this from. AI will be superhuman at writing the executive summary at any prerequisite knowledge level.
These people are concerned about internal community dynamics like tenure, chairs, PhDs, grad students, the whole ecosystem, papers, hiring, conferences, journals etc. But the wider population doesn't care about this and academics don't understand that nobody cares about this outside their bubble.
As Feynman said, "mathematicians can research what they please. If you want something else, you work it out yourself" (not verbatim). Indeed, now people can do this with AI and it makes mathematicians salty that they will have less prestige.
> AI won't just say cancer can be cured without saying how.
It was an example, an analogy to argue that answering a question might not be as important as how it arrives to that question. Hence the “imagine”.
> These people are concerned about internal community dynamics like tenure, chairs, PhDs, grad students, the whole ecosystem, papers, hiring, conferences, journals etc.
You just described most of the parts of the mathematical community.
> But the wider population doesn't care about this and academics don't understand that nobody cares about this outside their bubble.
I don’t care about the logistic system and yet I want my food to be in the market. If the population at large wants to use mathematics they will have to find a way to support a mathematical community. Now, if you don’t want mathematics, again, go argue about that.
> Indeed, now people can do this with AI and it makes mathematicians salty that they will have less prestige.
Except that they can’t. I do not know how many more times we need to explain that “answering questions” is not what mathematics is about. That if that path is followed, then mathematics will suffer as a result and the pace of mathematical development will slow down.
Honestly, I think it would be better for you to really stop attributing the arguments of mathematicians that are by no means anti AI to saltiness or other childish feelings. Try to be more humble and try to find what kind of ideas they might hold for them to argue this letter. Of course, that requires reading the letter and trying to understand it.
> You just described most of the parts of the mathematical community
Just as science (as in the scientific method) isn't the same as Science, the community social practice and logistics and tenure rules and committee compositions and journal page limits etc, mathematics is not the same as the academic mathematics community. They are not the sole producers of mathematics and not the sole users. The funding is largely in the hopes of the usefulness of the resulting math. Not all. Some funding is like art and culture funding, for propagating a cultural legacy, like folk dance also gets funding and experimental performance theater also gets some tax funding. But that's not the bulk currently in math.
I have read the letter. It's all about stuff of holding back because they don't want the answer key because prestige reasons, and how will grad students train their brain if we have too many answers. This doesn't consider that many people, like engineers, use math as a tool. They don't want to hold math back. It's fine to think that applications are too dirty. Again, they want funding, they have to explain why exactly they should get it. And it better be an explanation that still holds water with AI available. They can do the aesthetic math as an art side project like an accountant can paint or sing off the clock. But generally you also don't pay everyone for their hobbies even if those hobbies are nice culturally rich endeavors.
> I have read the letter.
And you have not understood it.
> they don't want the answer key because prestige reasons
No, that’s not the reason. They don’t want the answer key because the answer key is useless.
> This doesn't consider that many people, like engineers, use math as a tool
Of course it considers it. Many people, like engineers, could not care less about the answers to most problems mathematicians work on. They do care about the tools they develop in the process. I’ve already shown examples of this but I’ll do it again: Galois theory was developed when trying to answer whether there are formulas to solve roots of polynomials of degree 5. Fourier analysis was developed when trying to find an analytical solution to the heat equation. Riemann developed his geometry to explore which Euclidean axioms were actually important. None of the direct answers were as important as how they got to them.
> Again, they want funding, they have to explain why exactly they should get it. And it better be an explanation that still holds water with AI available.
The explanation will be exactly the same. Only it will not be just hard to explain why the problem is important, but also it will be harder to explain that no, just the answer by itself is not useful without the understanding. Just like I am here having a really hard time explaining to someone who doesn’t understand how mathematical research works why “just getting the answer” is not a useful output.
This hangs on the strange assumption that the AI will "just give the final answer" and will be unable to give a human-preferred explanation. I find this assumption strange. If it holds, then mathematicians will still have a job digesting these proofs. If the AI can digest it better, then academic mathematicians will become humanities professors funded similarly to theology, philosophy and dance professors etc. for the sake of preserving a cultural practice, like traditional handmade broom makers etc.
Why is it strange? Are you familiar with how unsupervised proofs are made? It’s basically iterating on Lean code, even if it were able to explain the code to humans it doesn’t mean it can extract actual understanding and structure from it. The papers being outputted by these approaches are nowhere close to being understandable, much less useful.
> If the AI can digest it better, then academic mathematicians will become humanities professors
Cool, then once that happens we can discuss what to do. In the meantime, AI does not digest proofs properly, does not explain them and does not generate any understanding, and it doesn’t look like that’s going to change. Hence the letter and the criticism made to the approach taken by AI labs. It is not that hard to understand.
I find these discussions pointless. It's like complaining that diffusion models make hands with the wrong finger counts. Yes there was a year or a few months when that was a legit complaint. The phase where AI proves Millennium Prize level problems but can't write up a human-like paper explaining the proof strategy may be similarly brief. It feels pointless to build a grand theory of what AI can fundamentally do in the space of math proofs based on a few months of existence of such strength models.
I find it interesting how many people are incapable of imagining that the tech capability will not be frozen at today's level and where we were e.g. a year ago and that a similar change may happen until next year. Instead they make sweeping assumptions that the current limitations will be with us for our lifetimes. You need a much stronger way to adapt to the new reality.
> The phase where AI proves Millennium Prize level problems but can't write up a human-like paper explaining the proof strategy may be similarly brief
It's absurd to assume that because LLMs are improving in certain aspects they will improve in everything, specially when the part they are lacking in is not precisely something that would be a strength of their architecture. I mean, models have improved a lot but they are still not good at strictly following instructions consistently (there's a reason why AI labs are worried about safety alignment). They are still mediocre to bad at software design, even the latest models (haven't tested Astra seriously yet). And it's the same reason they are bad at creating mathematical theories: they do not have mental models like we do, it's not even useful for them. Their comprehension is limited to textual context that they need to refresh and reprocess continuously. That's just how LLMs work.
> I find it interesting how many people are incapable of imagining that the tech capability will not be frozen
I find it interesting that after taking this long to, I assume, finally understanding the point the letter was making, you automatically switch to "oh well AI will do that too".
> But complaining about the dawn of a new era of advancements seems counterproductive.
you completely misunderstood the critics. your analogy is awful. this is much closer to the industrial revolution in the uk: it brought a lot of progress, but also extreme inequality and concentration of power.
I am very specifically addressing what I have read Terrence Tao say on AI and mathematics, as that is the subject of this submission.
Are we on the same page?
It's not about whether the conjectures themselves are interesting or not, or whether people simply have enough time. It's that a bare proof made by a machine doesn't actually do much for us. There is not some set of problems that, once finished, will amount to some kind of final, correct system and we can call it a day and, like, utilize it. "Mathematics" is the people doing it (the "mathematical community" Tao references below). This other stuff is kinda just.. expensive exercises to render a result. They are only actually beneficial to us insofar as they exist in a context of research among peers.
https://terrytao.wordpress.com/2026/09/11/a-severe-misalignm...
You have expressed a few different ideas, so I will address them separately.
> It's not about whether the conjectures themselves are interesting or not, or whether people simply have enough time.
Tao seems to think otherwise, if I am reading him correctly:
I wrote recently about how the collection of good, fruitful open problems is now being mined in a non-renewable fashion, leading to the potential scenario of these problems becoming scarce [0]
Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions. [1]
> It's that a bare proof made by a machine doesn't actually do much for us.
I get it, and I think the same can be said about all sorts of human endeavors.
> There is not some set of problems that, once finished, will amount to some kind of final, correct system and we can call it a day and, like, utilize it.
Sure. Although there are certainly practical applications to be found along the way. E.g. proving P=NP would be potentially very significant in the real world. I think we agree.
> "Mathematics" is the people doing it (the "mathematical community" Tao references below)
Sure. And the same can be said again about all sort of human endeavors. But I don't see how that is a reason to stop using AI in those fields, either. It doesn't subtract anything, in the same way that chess engines didn't destroy the love of the game for chess.
And just like in chess, these AIs can be used to gain a deeper understanding. Including, but not limited to, explaining to humans the proof they just came up with.
[0] https://mathstodon.xyz/@tao/117237320796901560
[1] https://terrytao.wordpress.com/2026/09/11/a-severe-misalignm...
Are you trying to argue toward some final verdict with regard to LLMs and mathematics? I thought you were just trying to make a comparison to Baudelaire? I think maybe being clearer on this point would help. Even if you argued sufficiently for the latter (which is going to be tough already), it wouldn't really speak to the former. Or at least: that would have to be a separate argument I think.
Also, how, in your words, do you feel like the first quote justifies your point (presumably with regard to the question of "interesting" or not)? And why do you think the second one is more about time itself rather than attribution? Do these things actually contradict the letter above (or the comment on it) in your mind or not?
In general, do you disagree with something here specifically? Or is it kind of a yes/and thing? Does any of this help, in your mind, with the Baudelaire comparison you were at least at one point trying to argue for? Its a bit hard for me to see the argument here, if there is one, just with what you have written. But I am sure I am just not knowledgeable enough to grasp the argument!
My opinion is that the field will adapt. AI will not spell doom for mathematics. On the contrary, it is the beginning of a new era. Math is having its Deep Fritz moment, just as computer science is going through the same.
Some folks are struggling to adapt to this change. Tao actually sounds like he is doing alright compared to most, even if some of his arguments seem a bit weak, as I alluded to in other comments in this thread.
Hopefully it makes some sense. And if it doesn't, at least we had a nice chat.
Haha yes I imagine we all know your opinion is something like that! But isn't it much more fruitful and interesting to form an argument for it? If he is still in the context here: is there something about Baudelaire's critique you were discussing originally that has helped you arrive at this opinion? Do you think he is wrong? Or is he right, but providing the right kind of nuance that we need in order to cope with the revolutionary changes? Genuinely curious!
> But isn't it much more fruitful and interesting to form an argument for it? If he is still in the context here: is there something about Baudelaire's critique you were discussing originally that has helped you arrive at this opinion?
The historical record. That's why I am drawing some lose parallels with Badulaire. Incumbents being unhappy about a disruptive technology, lashing against the early adopters, and fearing that it signifies the end of their craft, when in reality it's just a period of change and adaptation. Without the advent of photography we would not have Impressionism nor all the movements through the 20th century. Photography forced painters to reinvent themselves, and LLMs will force mathematicians to do the same.
I thought the historical examples of photography and chess engines would be enough for people to connect the dots, but apparently not.
Sure, but then it's kinda less that you're making an argument here one way or another, and more of just an appeal, right? Like its just broad strokes? The inherent actual qualities of photography aren't supposed to parallel anything with the LLM case other than just the bare fact of being a new technology at one point?
I guess I am sorry for pushing, I felt like maybe there was going to be something more specific to the point.
I think one thing to keep in mind in the future is that for every Baudelaire, there is a Benjamin or Italian futurist saying the exact opposite! Its not like the fear of photography was a monolithic, shared thing, even at the time. Just, it's not so straightforward to have the "historical record" speak exactly one narrative, and unless you are a full-on technological determinist (is anyone these days?), you can't really conclude anything from the past like this.
> I thought the historical examples of photography and chess engines would be enough for people to connect the dots, but apparently not.
You're just an overconfident idiot who thinks that because there is an analogy along one dimension of a absurdly complex problem you've somehow arrived at a superior truth.
I repeat, this is an incredibly complex situation in which no one fully understands what's going on. The letter is simply asking for greater awareness of the implications of what AI companies are doing, they're not even saying that AI is bad!
I don't see any meaningful analogy with your cites of Baudelaire. First of all, Baudelaire discusses art, which is very different to science. Several new forms of art have emerged, and then slowly integrated, despite the strong opinions of some. Here, Tao's letter discusses mainly practical aspect of scientific research. It is not about what a "failed mathematician" would do, or if "LLM can only record what already existed"; in fact, it ackowledges that LLM could become a central tool. If you want me to be even more precise, what they're saying (without saying it out loud) is that tech companies have too much power and are being irrespondible with it because they don't even think about the externalities.
> It is not about what a "failed mathematician" would do, or if "LLM can only record what already existed"
This is what I was alluding to:
I wrote recently about how the collection of good, fruitful open problems is now being mined in a non-renewable fashion, leading to the potential scenario of these problems becoming scarce [0]
[0] https://mathstodon.xyz/@tao/117237320796901560
I still don't see the analogy, can you care to develop?
Of course there's a meaningful analogy. The poster wants to make Tao sound unreasonable and against "progress" while ignoring the very real concerns Tao has.
It's what fanatics do when they want to enforce their view on the world, they have to attack anyone with a reasonable viewpoint because they can't imagine a world where someone tells them they don't like what they're doing.
I do not want to make him sound unreasonable. He is a brilliant man. But I also think that he is a bit shell shocked, much like other luminaries in the past have been when confronted with rapid technological development, so I wanted to highlight some historical similarities, imperfect as they can only be.
As for the rest of your comment, I hope you have a great day.
He’s not shell-shocked. He’s had an extremely consistent narrative for years now. You are experiencing a bias where you project your own views onto others, probably because you are not very informed on this.
Tao is quite the opposite of what you describe. He's had an ongoing dialogue intellectually with the challenges created by ai for mathematicians for some time (years) now and his viewpoint is quite balanced. He is not alarmist but rather correctly addresses the real problems ai advances create for mathematicians traditional way of working.
> But I also think that he is a bit shell shocked, much like other luminaries in the past have been when confronted with rapid technological development
You're locked into that idea, but provide no real argument. The dangers they describe in the letter are real.
This goes back to not liking the idea of consent and having to establish an entire worldview to justify actions against someone's consent.
It's the work of authoritarians and needs to be rightly called out.
It’s not a parallel. No one ever claimed photography to be the same as painting or some other medium - it was a new medium that wasn’t respected.
AI is being treated and pushed as a replacement for every medium.
It literally was a replacement for portraits.
Portraits are not an artistic medium in the same sense, they’re something one creates with the tools of a particular medium. Painting is a medium, photography is a medium, both are capable of producing portraits.
An LLM can produce a portrait of you as well by generating a facsimile of you based on existing material (or functionally a “guess” if you describe yourself I guess).
It allowed something like portraits to become available to the average person.
People aren't ready to discuss AI assisted imagery as art yet. Most discussions lack the nuance that Baudelaire lacks in that critique, which deals with the nature of art and the importance of human intention and input.
We can’t look back with perfect hindsight because both the past and present have deeply ingrained blindspots. They don’t know what it is like to live in a world with perfect edges. We don’t know what it is like to live in a world with no edges. We can read about someone who proclaims that “something will be lost”. We will just think “but I have no need for any of that.” But we don’t even know what it is.
I don't think the analogy works because for the past few years Tao has been one of the most vocal advocates of AI in mathematics and has used it extensively in his own research. You can find several of his talks about this on YouTube. It's completely consistent to believe two things at once, that the tools are useful and that the companies are misbehaving.
Wasn’t the criticism of photography kind of correct though? I don’t think people think of photography as an art as much as painting is an art. There aren’t a lot of photographs that a layman couldn’t in principle also take, but only a few people could replicate a good painting.
It's not exactly correct. Photography is just as much an art, but it's a very different medium and pieces are judged differently.
Realism isn't difficult, so critique shifts towards composition, narrative, context, process, and emotional impact.
Taking a clear photograph is easy with modern equipment, which means the bar for what's considered "good" is high.
i think that's just you
We can start having a meaningful discussion when people use real reasoning instead of analogy.
You don’t have to participate in the discussion, but it will take place regardless of your preferences.
See https://news.ycombinator.com/item?id=49664505
Analogy is a meaningful way to discuss. Drawing parallels can illustrate a point and bring up nuances by pointing out where it falls apart.
[dead]
Keep in mind that his critique is very recent, and likely applying to a specific use of AI, as opposed to AI as a whole. If you've been following his Mastodon account, he's been happily using LLMs for math purposes for well over a year.
He posted about using ChatGPT to transcribe PDFs when it first became popular. So he’s been enthusiastic about LLMs for a while.
I'm really tired of these arguments (this and "it's just like calculators").
Photography decimated other forms of visual art, so the concern wasn't wrong. But AI threatens the entirety of human intellectual endeavors. I can make do without oil paintings in my home. I'm not sure I want to live in a future where we make do without brains.
Humans still have brains (and thus human intellect) even in the presence of AI... Even today, people make varying use of their brains.
But what use will be their brains to others if a machine can do the thinking instead for a fraction of cost?
Whatever they feel like putting them to, instead of whatever they are forced to put them to to earn their living?
Lots of people still make bad music that other people still manage to enjoy (a lot of it has gone multi-platinum!) even though they're not Mozart or Bach.
This would be valid if we didn’t live in under capitalism, which reduces the way we spend our time down to dollars and cents.
Correct. So burn down capitalism, not AI?
Or be honest about what’s happening and interrogate both?
I did; the above is my conclusion. AI itself is a highly positive technological development in terms of how much human suffering it can alleviate.
And yet it has primarily been a negative so far with terrible stewards, so forgive me for not being so optimistic.
Of course it is, because our society is pretty much structured to place sociopaths on top. Why is it at all surprising that any technological advancement is going to be used by them in a way that doesn't benefit the rest of us?
But it's equally obvious that this has nothing to do with the tech itself.
If the tech enables certain behaviors then it can’t be casually divorced from a discussion of how people use it. “Tech” isn’t inherently neutral. It’s made by people with intent for a purpose and how else it can be used must also be considered.
This is like separating “the use case” of killing people from guns and still trying to have a coherent discussion about them. You simply can’t handwave it away with “well that’s not the tool’s fault.” It’s unproductive and sidesteps the conversation.
Killing things is literally the sole purpose of guns, though.
And even then we still need them because sometimes you do need to kill someone who's trying to kill you, which is not morally wrong.
The question, again, is how they are used.
you can still get oil paintings in your house. you just arent willing to pay for actual art
You can import oil paintings from China for very cheap. I’ve got a ton of nice ones
Had me until the last word. Cameras did take away oil paintings, but not eyes.
It did. While you're busy recording an "instagramable" moment, you are not entirely enjoying that moment. Your eyes are on the screen, not on the subject.
In a way, if photography is an ersatz for painting that eventually made imaging available for the masses, then AI could become an ersatz for thinking. But it feels like I'm paraphrasing TFA.
What will happen when AI companies have spent their advertising budget on math problems and whatever else gives the maximum wow effect for the bucks? Probably customers hooked on the vain satisfaction of spending token$ to impress friends.
Indeed. As Nassim Taleb says, "Something bad for the (existing) class of mathematicians as they lose their privilege, but not necessarily bad for mathematics & our body of knowledge?" https://xcancel.com/nntaleb/status/2098498124419305639
Baader–Meinhof phenomenon:
Baudelaire popped up in this article two days ago.
https://www.noemamag.com/a-new-kind-of-creative-poverty/
The AI people sing from the same sheet.
I’d agree art has deteriorated massively since then, so maybe we are in for an intellectual dark age to accompany our cultural hellscape.
Photography vs painting is the first thing that comes to my mind when I listen to AI critics.
Wonder what would've Baudelaire said about generative art as in diffusion-based imagery.
If he didn't understand photography, he wouldn't understand the nuance there either.
No, there are no parallels. It is just cliche propaganda used by AI boosters.
Wait, what? Who’s the ai booster in this scenario?
Beaudelaire
This sounds a lot to me like people in the 90's complaining that computers were destroying chess. Thirty years later, chess is more popular than it ever was, and chess players are better than they ever have been. I wouldn't be surprised if there are now more chess books now than there ever have been. Furthermore, it turns out that a lot of chess books written before computers were just wrong about a lot of things. It turns out having an oracle for the "right" answer in chess, even without an explanation, used properly, allows humans to develop broader, more accurate insights.
The argument here sounds similar. The fear, as I understand this statement to be saying, is that by being given the correct answer, in the form of a 100-page Lean proof, humans will be robbed of the chance to from insights about the structure of mathematics itself. I don't see any reason that humans can't continue to develop insights as they try to digest the 100-page Lean proof into something more manageable; but with more certainty and fewer false starts.
I was apart of the "covid chess cohort" and got really into it for quite a few years. In that time, I also learned about what the chess culture was like pre-engine and post-engine, and I think you could argue that post-engine did kind of make the game more mechanical - and that kinda ruined things for people that grew up in the pre-engine era. it felt like pre-engine chess had a lot of mystique to it because knowing the best move was never truly clear. the only way to know what was best, was by: getting better yourself, socializing at chess club, learning from better players or coach.
this sort of lack of clarity is what drove people to learn more about the game.
like think about what it must've been like to go to a chess tournaments pre-engines: finals matches had everyone at your chess club watching the game, calculating lines with each other, and you HAD to calculate to understand the direction of game. it sounds so much more engaging and fun!
now, the top games have the stockfish bar next to it, and you instantly know who's got better odds without having to really follow along. and you're not encouraged to calculate beyond a few moves ahead because you just offload the real thinking to stockfish. honestly, it's a vibe a killer when you go back and see images and read about the culture beforehand.
I agree engines didn't really "kill" chess - and I think most of that is due to chess not being tied to economic value. but it did kill what I believe was a superior culture compared to today's chess era.
all this to say, I think what AI did to chess culture in the 90s is happening to stem right now with LLMs
>> I was apart of the "covid chess cohort" and got really into it for quite a few years. In that time, I also learned about what the chess culture was like pre-engine and post-engine, and I think you could argue that post-engine did kind of make the game more mechanical - and that kinda ruined things for people that grew up in the pre-engine era. it felt like pre-engine chess had a lot of mystique to it because knowing the best move was never truly clear. the only way to know what was best, was by: getting better yourself, socializing at chess club, learning from better players or coach.
I need to be very precise here, but it is not right to say that we know what the best move is in any chess board position. What we know is what move a chess engine would make that would win the game against a human, or sometimes another chess engine. We know a winning move; not the best move.
Winning in chess is not the same as knowing what the best move is. Having chess engines that can beat any human in chess is not the same as knowing how every possible game will develop. The latter is known as "solving chess" and we don't yet have that. We have fully solved games like tic-tac-toe, backgammon and checkers, so that there exist databases of every possible path through those games but this is not done for chess, and there is no hint that it is even possible to do given that chess is such a combinatorially more complex game than those.
More to the point, the upshot of the fact that we now have super-human chess engines that can beat any human in chess (and Go and shoggi) is that there is no longer the will to research the bigger question about solving chess. The solution of the minor problem, how to beat humans in chess, has sidelined and displaced research in the major problem, how to solve chess, and we will now never find the solution to the latter.
And if that reminds you of something, well, yes, exactly.
Well, chess is a sport where humans are supposed to compete. But math, programming, science are mostly not, and AI might affect economy, careers, etc.
As a chess fan, 100% this. We have known for the last ~15 years who the best human chess player is, and that he will lose against stockfish on his phone. But chess survives because of the human characters involved, the rivalries and dramas, watching two people trying to overcome each other under insane pressure, and sometimes coming up with something astonishing. In short - it's a sport.
There is no equivalent in math.
Uh I was literally on math team
I guess part of the problem is that being against being against anything for economic interests doesn't really rally anyone to your cause; everyone has to make a living doing something productive for society, and professions have come and gone all the time due to technological advances. In fact, when one thinks about it, the people that are losing their professions now were major contributors to others losing their form of income. Often people talk about how they can use technological/programming/IT skills to make some secretary or administrative assistant's job obsolete. So most people just don't feel a lot of sympathy when people complain that AI are going to take those people's jobs.
That's correct. As far as I know nodody builds any kind of science or technology on top of chess, but mathematics is at the base of most science and technology. It would be horrible if we prevented AI from solving mathematics problems, just because mathematicians want to solve them by themselves the "hard way".
=> If chess.com was worth billions and Daniel Rensch was threatening everyone to do what he says.
>> This sounds a lot to me like people in the 90's complaining that computers were destroying chess.
Genuine question: who were those people saying that? I don't know that criticism.
The criticism I know is from AI researchers and it is that beating humans at chess using a computer running an algorithm unlike anything that humans do when they play chess, tells us nothing about the way that humans play chess, which is what we are trying to understand when we try to get computers to play chess.
This criticism is exemplified by John McCarthy's article (the real godfather of AI; because he named it) "Ai as Sport" whence I quote:
Ideas about chess algorithms as well as advances in computer hardware were involved. However, it is a measure of our limited understanding of the principles of artificial intelligence (AI) that this level of play requires many millions of times as much computing as a human chess player does. Moreover, the fixation of most computer chess work on success in tournament play has come at scientific cost.
In 1965 the Russian mathematician Alexander Kronrod said, "Chess is the Drosophila of artificial intelligence." However, computer chess has developed much as genetics might have if the geneticists had concentrated their efforts starting in 1910 on breeding racing Drosophila. We would have some science, but mainly we would have very fast fruit flies.
https://www-formal.stanford.edu/jmc/newborn/newborn.html
A more recent criticism that can be made is that getting computers to beat humans at chess (and later Go, and Shoggi and Atari and Stratego) has turned out to only be possible with techniques that are too narrow to have real-world application. Btw, this includes Reinforcement Learning which is still much more capable in virtual environments than in any real-world environment.
Many people argue computers were bad for chess till this day and there's good arguments for it.
We like to think we are thinking beings that happen to feel, but we're actually feelings beings that happen to think. We don't realise how our emotions shape our arguments, and the whole Terrence arguments have, behind:
"I like my job and want to continue doing it. It gives me meaning."
The brain then goes and fills in arguments that support these feelings.
I like this approach.
A but like whenever the first sprinter hits a new world record other runners follow along.
Knowing that something is possible tends to strengthen our ability to work with it.
We will potentially see the same with math.
> try to digest the 100-page Lean proof into something more manageable
AI will be better at digesting it into something more manageable.
> I wouldn't be surprised if there are now more chess books now than there ever have been
Well yeah... how would there be fewer??
But the point itself is silly. Few people are putting effort into Maths for the fun of it (and of those that are many derive fun from being the only one who can produce a solution). Chess differs in that it never had any point but the game its self.
But computers have destroyed chess as a "sport". Nobody will sit to watch two chess programs compete, or analyze their tactics. Kinda like how now, anybody can construct a "game" over the weekend or a new song or a slop video. The value of each of these decreases to 0 as the slop overwhelms.
>Nobody will sit to watch two chess programs compete, or analyze their tactics.
I know nothing about chess yet I dare say that I'd doubt this. Surely chess enthusiasts would be interested in analyzing how a superior chess program came out victorious, no?
Yes you are right, some people do watch chess engines play. TCEC (Top Chess Engine Championship) [1] streams them. Popular chess YouTubers goes over engine games from time to time too.
[1] https://tcec-chess.com/
That's not proof of interest so much as proof of commerce. It could easily be a money-laundering mechanism.
Mostly nobody cares about professional chess. The number of people who are actually interested in today's game and not the drama are a tiny sliver of that. This is actually great because it means we can train and evaluate both without interference from chess players, possibly even building a stable society.
Here [1] is a video made 3 months ago going over an engine vs an engine made by of one of the big YouTubers, which has 1.4m views. Enough proof?
[1]https://youtu.be/Ov7K4W-zAk0?is=6jVBkQpn8xbaA2Tz
People do: https://tcec-chess.com/
erm actually, chess tournaments rose in popularity and more people watch now than ever
[flagged]
This is a PR problem, not a mathematical problem. It's possibly the worst PR problem mathematics has faced since the execution of Hippasus for whistleblowing on the cover-up of the regular dodecahedron. It's still a PR problem.
So what went wrong? Mathematics education. Math below grad school is all about solving stated problems. Credit is given for solving puzzles successfully. Homework is problem sets. Everybody below a very advanced level is taught math that way. Even at the higher levels, puzzles remain important. Awards in mathematics are often tied to solving puzzle-like problems. That's still the criterion for becoming Senior Wrangler at Cambridge, "the greatest intellectual achievement attainable in Britain". This despite Polya's attempt at reform a century ago. Puzzle solving gets good grades and class rank. So it's the status indicator mathematics presents to the outside world.
Then reasonably good AI comes along. AI has become rather good at solving puzzles. So people aim powerful AIs at known hard puzzles, with some success. That blows up the status indicator system. Mathematics itself is fine. It's the status symbols that have a problem.
Maybe the Fields Medalists need to hire a crisis management team to reframe what success means in mathematics. That's what they're trying to do with that letter, but they're mathematicians, not PR people, and they don't know how.
It's not just math, though.
In general, in our existing society, status matters, and it is tied to achievements. Therefore a thing that devalues the achievement is also causing people to lose status, and everything that comes with it (income etc). Of course they are mad.
But maybe the answer to that is to acknowledge that our civilization is basically a dick measuring contest at scale, and try something else for a change? Starting with the part where people don't have to be "productive" to have intrinsic value as a person?
That’s a charitable way to interpret what they’re trying to do in this letter. I would say they are trying to maintain the status quo to protect their status system.
Very interesting analysis. I was just reading this announcement, and thinking, that is the most pointless thing I've ever read. What are they trying to achieve with it? Your comment explains it.
They are defending the value of their achievements!
They should explicitly advocate for the Fields Medal to weight teaching achievement more heavily going forward.
Precisely.
As a consumer of mathematics, I want to attack the author’s idea of “what that work was meant to achieve in the first place”.
I do a lot of programming that uses mathematics. I designed the error recovery file format “par2”. I worked as a quantitative trader.
What do I want from math? To be able to search math results, understand them, and apply them. But academic mathematicians are so inwardly focused - as these authors so obviously are - that it does all of these badly! Search is still with words, not symbols and formulae. As for comprehension, many math papers are unreadable to those not in the cliche of mathematicians in that specialty. And even applied math papers don’t ship code or specify the operations explicitly.
I wanted to fix this and in 2013 I worked on formal proof. It makes symbol search and application easy. I found out that the tools had been usable for decades at that point. But mathematicians didn’t use them because they had that inward focus and didn’t give a damn about math consumers.
AI is enabling formal proof. It will carry mathematics into the computer age, which the field should have joined with every other industry in the 1970’s! I welcome it. Math will get used more and that is what the work was meant to achieve in the first place.
I've had a similar experience.
LLMs doing math has made math far more accessible to me than it was pre-LLM.
Not just using existing math, but developing new math. (For example, I developed an alternative to NURBS surfaces for geometric kernels using LLMs.)
It's really annoying that anyone, anywhere is talking about halting math progress with AI.
One key idea to understand about human behavior is that people are focused on their status among their peers and their next level superiors with power to lift their status. You are not among either, so you don't matter to them. This is nothing special about mathematicians, though, people work the same way everywhere, in every field, community etc.
Yes, that’s very true. I recognized that in 2014. It was hard, but I stopped doing research on formal proof.
I got an economics degree and found that the City of Austin to pay off its $3 billion dollar pension debt … and encountered the same problem you mentioned, again. Except this time my plan is to run for City Council and be their next-level superior.
different fields have different communication cultures, that’s not inherently bad. sacrificing inward focus is expensive from the pov of the insider; i see your points but not everything can be written for a general/wide audience. after all, the audience of the author are the specialists, since only specialists spent the effort to be able to engage with the work in depth. perhaps we disagree that mathematics is not a product that mathematicians are making?
what the work was meant to achieve in the first place is what the work was achieving in the first place. it was developing mathematics. if someone wants to use it, they are free to do so, or to write a blog article how it can be used by non experts. mathematicians do that themselves sometimes, but it’s not their job. myb there should be more incentives for them to do more of such work
i keep seeing this myth that mathematics has some purpose to do this or that, but somehow its purpose is least known to mathematicians, everyone else seems to know better
> what the work was meant to achieve in the first place is what the work was achieving in the first place. it was developing mathematics.
Meant by who? Were the people footing the bill aware of this? People can like and do a lot of things. When money enters the picture it gets more complicated because usually the people who pay do that because they derive some gains from it. Historically scientists and mathematicians were employed by various patrons, sometimes doing astronomical calculations for horoscopes, Easter calendars and whatnot. Pure for-fun math was usually done by already wealthy individuals and not salaried employees like today. So it's fine that mathematicians feel this or that is their purpose, but they are coming to us asking for tax money, and so there is a bit of burden on them to be convincing. Before AI they used to argue how it helps physics, engineering technology, computer tech, optimizing logistics and telecom, cancer cures etc. Now the tune changed and it's retroactively now obviously just pure recreation and spiritual elegance. I also like math don't get me wrong. I think it's a great intellectual pursuit. But it's arrogant to expect that the money can flow by default.
mathematicians are primarily paid for teaching, and that's their primary function from the pov of the general society. i don't think anybody is debating that part of their profession? i read somewhere that the annual nsf grant budget for mathematics is 57 million, in total. even if it was just spiritual elegance, that's a reasonable number no?
Yes, that amount seems fine if true. They may be seen as teachers by the outside world yet they hire and promote and award internally based on research. Status and prestige is rarely allocated based on teaching. This is already a discrepancy. And note that the post doesn't much talk about teaching courses, but mentorship of grad student researchers who already work at their labs.
my response is not to the post, but to the parent comment i replied to. the internal structure of mathematicians is their thing, if someone should step in, who? do you think that discrepancy matters?
You mean who should pay? Whoever they can convince, of course. Historically it was wealthy patrons, royal courts, etc. Teaching indeed used to be the main thing a professor was expected to do, this changed in the 19th century with Humboldt's model of the research university, with the explicit purpose of not just passing on knowledge but constantly producing new knowledge as a core part of the operations. It could go back to the teaching model, if that's acceptable to them. Then they would have to do the research part on the side, as a hobby, like it was back in the day.
In case of public universities, the mathematicians would ultimately have to convince the public, but of course in practice that means politicians and national grant agencies, who indeed report to politicians and the politicians chase popularity to the extent needed for reelection, so public sentiment has some relevance.
Generally when you come to someone holding out your hand asking for money, being humble is a good first step, instead of getting angry at people for not sufficiently appreciating the big myterious value of what they are doing. Explain the spiritual value of the pursuit, they may or may not be convinced. This may be called a deficiency of democracy if your aesthetic values are different, but typically people don't like to fund abstract disconnected things, like expensive space travel unless it's embedded in some tribal jockeying like during the Cold War. In normal times people just want research that has life enhancing practical outputs. Nevertheless currently the West is structured as electoral democracy, so convincing the plebs is indeed necessary, no matter how lowly that feels from the ivory tower.
> In normal times people just want research that has life enhancing practical outputs
what about people who study birds, or say, olms? people want that kind of research? the people who are doing it surely do. astronomy and eg star watching in deep space? i could go on for days
people like doing mathematics, similarly to studying endemic organisms or other stuff of no direct utilitarian value
how many parents are happy to pay the fraction of taxes that go into something that their children might want to choose as a calling and fall in love with?
it’s okay to discuss whether something should be publicly funded, using “people want x kind of research” as a supposedly obvious or common sense argument is fallacious in the sense that it implies an idealised steelman model of the general public. that model is divorced from reality
> We are witnessing a general threat to intellectual work
This is the crux of it and goes far beyond Mathematics or Computer Science. To get a bunch of humans to do anything, you have to motivate them. Kleos and Timē; renown and stuff. These AI companies threaten to rip this away from everyone but themselves, and this recent millennium prize is the perfect example.
Solving this problem as a human would have led to tremendous Kleos; my name would be written in the annals of mathematics, lecture tours of praise were mine to be had for the rest of my days. This one victory would have earned my recognition throughout history. The greatest a mortal may hope for. Ripped away.
It would also have given me great Timē. The prize money, the professorships, the book deals. Gone.
If all hope of “renown and stuff” in the intellectual realm is now taken by the AI companies, they will remove all human motivation to pursue these endeavours.
Perhaps the glory will come from slaying these fell beasts.
These are both big motivations but not the only ones.
But there's still play. There's still curiosity. And there's still the drive to understand something for yourself.
> But there's still play. There's still curiosity. And there's still the drive to understand something for yourself.
Yes, but think about what that implies if those are the only motivations left. Gone are the professions. Gone are the ambitious.
There is plenty of space for people to work on intellectual pleasure pursuits (as there is with art and music), but the death of all intellectual based industries is still something to avoid. Or to mourn.
Good? Those people were in the field for the wrong reasons. No reason to mourn them leaving
> in the field for the wrong reasons
It’s an interesting view-point. What are the “right reasons” to pursue an intellectual discipline? To selflessly further the total sum of knowledge?
> No reason to mourn them leaving
What field? Who’s leaving?
The conclusion has to be that human involvement in these pursuits is facing existential crisis. “Teach me, oh Oracle, the mysteries of the Universe”. “For a fee”.
Both the glory and joy of discovery now belongs to the machines (the owners of those machines). Perhaps that’s fine and well, but I read Anathem and feel the strong yearning to be the intellectual monk. And yes, to fight and debate and think; and obtain my place in history. The history of man.
"To selflessly further the total sum of knowledge?"
Yes. I think you are saying this sarcastically but I'm not sure... This is why you should get involved in research at the frontier.
"What field? Who’s leaving?"
Mathematics, and I'd extend it to other academic disciplines. Do you care more about the results and the knowledge or about who gets credit?
The drive for "glory" has many negative effects in academia that do not get enough recognition. Just look at the replication crisis in psychology for an example. The fact p-hacking was rampant among psychology "researchers" was well known even in the 90s. I remember my professors in grad school for physics joking about it way before it became a mainstream-ish story.
So why did this happen? Because there was no one with institutional power in psychology who both understood the problem and cared. No one who was interested in the furthering the "total sum of knowledge" rather they were interested in your "glory."
You don't have to ask the machine, if you value the process of obtaining the answer rather than the answer itself.
Play and curiosity usually make poor breadwinners.
Having to put food on the table is one hell of a motivator.
> solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight.
This is the effect of AI on most intellectual disciplines, and it’s a real worry.
Yeah. My other favorite example are books. Why do nonfiction books exist? There are some pathologies and corner cases, but fundamentally: to develop and share new ideas. Downstream from that, if it reads well and if you're lucky, you make some money.
But now, LLMs can generate hundreds of books per hour. They make up 80-90% of new arrivals in many nonfiction categories on Amazon. They short-circuit the system, allowing their "authors" to extract money from the system with zero effort by crowding out human work. And it's not even the question of whether these books are good or bad (although overwhelmingly, they're terrible). It's whether it's actually accomplishing anything worthwhile, or just destroying incentives for humans to write or go into any other sort of intellectual work.
In fact, I see many professions push back. Artists, writers, now mathematicians. And I'm amazed that our profession doesn't and that we have so many people who are hooked on vibecoding. I'm still waiting for that 10x payoff. All this velocity and somehow, the landscape of the software I want to use still looks the same as it did in 2021.
I would argue that creating an llm book that sells also requires human labour, just a different kind of one.
It also takes human labor to run an effective nigerian prince scam. That doesn't make it okay.
It doesn't, that's the whole point. You can produce output that passes the smell test with naive buyers in a matter of hours. If you want to write good LLM books, then you gotta work closer to human speed - weeks, months - which allows others to produce 100 slop-books in the same timeframe. You still lose.
Now you piqued my interest. Any good AI-written books out there? I thought it's all slop.
Given the witch hunts around AI creative output, nobody who has put in work to avoid the stigma is going to out themselves.
Why would they have had to put work in?
If an author let AI do 90 to 95% of the work, cleaned up the remaining 5 to 10% themselves, would you say AI or the author wrote it? Because this is roughly the split between an author and an editor, with human writing, now.
The work is either: (1) the act of concealment: generating an AI book that cannot be detected as AI generated; or (2) actually just writing the book yourself.
Playing a devil's advocate. Why do we need understanding ? To take an example i would say ~99% of the population do not understand how combustion engines or how semiconductors work, what say another 1% ?
Is the fear post-apocalyptic in nature ? We need some human priesthood to carry on tradition ? why ?
Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
What say the 1% ?
Just a thought.
There ought to be more to life than sitting in a pod receiving sufficient nutrients from a tube, even if there was no doubt that humanity would in this way survive until the sun expands and makes the earth uninhabitable.
> There ought to be more to life than sitting in a pod receiving sufficient nutrients from a tube
Even in a hypothetical world where AGI can do everything that humans do better, this would still be a choice to make, not an inevitable consequence for everyone.
And life doesn't require suffering to have meaning.
Huh? Why bring up suffering? I was responding to the question of “Why is it a problem if people don’t understand things?” .
I was saying that human pursuit of understanding of things provides value beyond survival and resources.
Joint pursuit of a common goal is a way for people to better be-in-relation-with one-another.
You can still pursue understanding of things, full automation doesn't preclude that.
What it does is make it a choice rather than a survival necessity. The latter is where the "suffering" comes from.
> Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
I'd argue that this extremely extreme scenario is the only one in which it kind of makes sense to not have understanding. But let's be honest: no one knows if we'll be there (and it seems unlikely since everything reaches a plateau eventually). So, what happens if we allow ourselves to forget everything and then we don't reach the ideal scenario?
Whole civilizations fell when people forgot how to do important things
I grew up in a cult. Based on my experience, I believe that the most dangerous thing a human can do is to allow someone else to do their thinking for them.
It’s not going to be ubiquitous? There hasn’t been a single frontier model where generation n costs less than generation n-1 to run. So the reasonable thing is to assume that GPT-7 will cost even more than GPT-6, and more and more of the frontier of knowledge will be locked behind a giant paywall. Participating in any field will mean ponying up to the oligarchs that own the infrastructure that runs the model.
Tao basically reiterating "AI is making us dumber" a bit more eloquently and applied to math.
Its by choice. Mathematicians still can use AI to "achieve the primary goal of conceptual understanding and insight"
At the individual level any mathematicians can, but the social group of “mathematicians” will not unless drastic social changes are made, because that's not how the system works today. Drastic social changes usually take decades, and funerals, to occur, so the reasonable hypothesis is that it's not going to change fast enough.
Mathematics belongs to all of us. If mathematicians get depressed about it, I don't give a hoot. Five years of guided computer-aided math could improve billions of lives or solve global warming outright through cheap clean energy.
This is a litmus test for the effective altruists in AI labs: fashion or conviction? The social pressure is to slow down and let the guild keep its norms a few more years. But what's clearly best for humanity is to push forward violently and leave them in the dust. A Fields medalist's feelings are not more important than the wellbeing of the planet.
> Mathematics belongs to all of us. If mathematicians get depressed about it, I don't give a hoot.
They are depressed because what's being destroyed is mathematics, actually.
> Five years of guided computer-aided math could improve billions of lives or solve global warming outright through cheap clean energy.
This is complete BS, math doesn't solve climate change or save life, that's a category error. The purpose of math is to help us build understanding of the mathematical objects and that's it.
Sometimes, after much distillation to the academic world, this understanding is later being leverage by other scientific field, but that's a very rare event when it happens. And it's not how other science progress.
> This is a litmus test for the effective altruists
EA has always been a lie to justify short term individual greed by hypothetical shared future benefits.
Tao has enough moat (name, funding) to make shift happen and even capitalize on this.
youre not wrong, but i think you are missing his base point. tao is saying that mathematics is a practice that the group collectively builds. students and new methods are built on top of what has previously been done, so its a community effort of research, experimentation, and knowledge sharing.
his core argument that is LLMs (and specifically LLMs owned and gated by corporations was my read), while able to solve problems, do not contribute to this practice of knowledge building. solving problems is just one piece, and the mathematics community ingests problems and new methods, iterates and thinks on them, and then produces new ideas, methods, etc. this is what he is defining as progress, and solving things like millenium problems are markers of this progress.
I published a substack about this just a few days ago [1], my core theory here is that we will absolutely have what I call a "highly productive dark age" in mathematics where knowledge vastly outpaces understanding driven by publish-or-perish incentives, but additionally this will lead to the loss of the skills necessary to understand.
The hopeful note is that I do think we are entering a golden age for the curious casual/semi-pro mathematician and for niche mathematics areas that won't get the attention of the top labs. Everyone is sprinting to solve the millennium problems, but this is a very exciting time to be in a sub-sub-field where you and 4 others are keeping things alive.
[1] https://substack.com/home/post/p-214740151
The existence of solutions to problems does not prevent you from solving the problems yourself anyway, if your goal truly is personal development. You're free to go solve Navier-Stokes yourself right now. You're free to manually do all of the AI work in any field, actually. You won't get grant money or prestige (which are not a part of conceptual understanding and insight) but you will get all of the conceptual understanding and insight you're after. You have not been deprived of it.
Except most people that get actually good at a field do so through their day job. By removing mathematician as a viable job, most can't afford to spend the necessary hours to become a good mathematician at all. We're going back to only rich people or those with a rich patron having a chance of developing themselves.
As a software engineer, 6 months ago. I was better than AI in essentially every way except raw typing speed.
Now AI has surpassed me in fine-grained problem-solving ability. It can write more reliable code than I can, faster than I can, provided it is given the right guidance.
But one thing that I'm still much better at is identifying technical opportunities and choosing the right tradeoffs.
My feeling is that frontier models are incredibly smart in some ways, but incredibly dumb in other ways
When I chat with Claude using deep technical language about distributed systems issues, the arguments it presents are mind-blowingly good. I worked with many skilled engineers on complex projects but the kinds of arguments Claude makes are on another level. It feels like it can read my mind because it already identified all of the relevant aspects to the current topic and it's incredibly persuasive. I would say it's even better at deep, nuanced technical discussions than I am.
When I ask it to implement a feature using technical language, it does a really good job and the solution usually works out of the box. No bugs at all 95% of the time even after adding 1000 lines!
But the problem it still has is that when it implements solutions, it misses so many low-hanging fruits/opportunities. Especially in terms of performance, maintainability, scalability and UX. It's like it sees and recalls all the relevant parts perfectly but yet somehow misses opportunities which seem extremely obvious to me.
It feels like AI has 0 creativity. It only sees the opportunity once I mention it... And once it sees the opportunity, it demonstrates deep understanding of the technical implications. I think what's surprising is that it understands the suggested solution so well, with such nuance, that I can't understand how it didn't see the opportunity and why it never seems to see it until I mention it.
This is very unhuman-like. There is no way that a human being with that degree of understanding of a topic would be presented with such highly relevant context and not make the connection.
This is exactly how I feel about the recent ChatGPT models. It feels like I'm working with an incredibly enthusiastic junior student who listens to everything I say, checks it against my entire codebase, and thinks through all of the possible connections 1000x faster than I can. But it doesn't spot the ideas first. It actually makes work quite fun, since I can spend so much more time in "idea space" instead of "implementation space".
Thanks for putting it in words. I feel the same about SOTA coding models. I get visibly frustrated by this and often find myself using curse words at Astra.
You're right it's very unhuman to miss obvious opportunities you mentioned. And what's why I think these models would continue sucking as long as they don't have human level general intelligence. They may get marginally better at these tasks but I don't think we can expect them to connect dots like humans before AGI.
This mirrors my understanding when I use Claude code for mathematics. I can have deep discussions with it and it can solve my hairy problems. But whenever we go off the beaten track into design new mathematics, it struggles to make conceptual leaps and find the right definitions. Once I give it my ideas, it is back to its super-human pace and top notch intelligence.
This situation suits me fine, since I am anyways more of an ideas person, than a crunching open problems person. But I understand the desperation of my colleagues who mad solving hard problems their identity.
> Once I give it my ideas, it is back to its super-human pace and top notch intelligence.
That's the real benefit of AI today: you can test your ideas almost as fast as you can generate them, in parallel even. If you have a crazy idea that is unlikely to work but has massive advantages if it does, you just set an agent to explore and prototype it while you focus your main attention elsewhere. And so on.
It's still GIGO, though! An AI can make a bad idea work (and will do so if you don't carefully prompt it to not be so sycophantic!) but it won't work well. Knowing when the idea is bad and should be abandoned is also the part that currently requires human judgment.
That said, given the pace at which models have been improving so far, I can't help but think that this is a transient state of affairs.
Yes very much the same situation in software development.
The people who prided themselves on raw puzzle-solving ability got hit hard. Those who are idea-driven and architecture-oriented feel like they got handed a superpower.
It's quite a big shock at the industry level because the entire software engineering job interview process at essentially all large companies was heavily biased towards well-defined, raw problem-solving under time constraints which is precisely the skill which AI has replaced.
Yeah and companies continue to use leetcode style interviews even today and software engineers keep grinding for them. It's nauseating to contemplate on.
To be honest I didn’t get that much outrage here: https://news.ycombinator.com/item?id=49662116
It seems like a lot of the issue here is that these problems aren’t interesting in and of themselves, but they lead down interesting roads. It defeats the purpose if you solve them without getting any real understanding.
It’s akin to saying you’ve solved “pancake flipping” problems with a waffle maker, or “travelling salesman” problems with a zoom meeting.
Nothing is stopping these folks from continuing to study the problems and arriving at their own solutions so they can continue having whatever insights along the way.
Well, one thing is stopping them. There will be no more adoration for their genius.
If you truly do it for understanding and not the attention, carry on. AI should change nothing about your motivations.
> Nothing is stopping these folks from continuing to study the problems
My understanding is they are? And literally everything in this world is based around incentives. If you say “well you can continue to work on understanding, but your kids are going to starve” that’s not nothing.
It boggles my mind. Why cure cancer? The cancer researchers will be out of a job and their kids are going to starve!
The point isn't what happens after something is solved. Where is the motivation for humans to study/find a potential solution if your kids are going to starve while you do it? These things can brute force problems that have a clearly and feasibly searchable answer, but they can't make creative leaps. It will be a severe dark ages for mathematics if humans stop contributing.
Maybe an analogy of "why not solve all medical problems at once" is closer.
We absolutely want to, of course. But you extinguish an industry and the systems of training that supply it. It's hard to know if letting it go that way is right.
methinks mathematicians may have a warped perception of their own importance.
Why not cure cancer and take care of the researchers’ kids?
Why not indeed?
AI should be the commons.
You have misunderstood.
I have a the cure for cancer. Simply kill the host. Does it work? Yes. Have you learned anything from it? No.
A lot of this would be public funding. If AI is reducing the need for mathematicians in an area it makes sense to fund it less and fund something else that isn't solved.
Your comment perfectly illustrates why most are missing the point. Many people, including you, deeply believe that most mathematicians are chasing adoration of their genius, otherwise why would anyone care about such abstract work?
People like Grigori Perelman would baffle you, a mathematician who solved the Poincaré Conjecture, refused the monetary prize, field medal and continues to live a life of total recluse.
For most mathematicians their primary drive is chasing the unknown, not for anyone’s adoration, but to pursue their desire to see what lies in the beyond.
Going to the gym and using a forklift
There is an implicit agreement that mathematics is funded, for the most part by the public, as a way to advance the state of knowledge and propagate (even if very indirectly) what has learned for the public good.
It was never about helping individual mathematicians demonstrate that they are individually good at math. It happened to work out that way, but it wasn't the goal.
Agree, my comment is rather about the fact that most work you do in school and (lower) university is not itself of interest, rather the purpose is to work out your brain. The more of your exercise you leave to the forklift, the steeper the cliff will be when you find a weight that doesn’t fit on forks. (That is, unless/until machines get good enough to completely replace people, at which point we have another few problems on hand, such as day-to-day purpose and justification of existence)
I've been wondering if mathematics is going to go the way of particle physics in that it will start to take a lot of money to make advancements.
This is a good metaphor for treating the means as an end, thanks. And I agree with your parent comment that that is largely the misalignment that Terence is pointing out.
This letter is complaining that human understanding has been crucial to advancing of mathematics, and AI companies are not bothering with it. But the promise (and horror) of AI mathematics is that, if it succeeds, human understanding becomes irrelevant. That's the goal. So this letter's message will fall on deaf ears.
Keep in mind employees at AI companies are publicly stating that they believe they're risking a >10% chance of human extinction. They're knowingly risking the lives of every man, woman, and child to continue the work. The lives of their own sons and daughters. A person already rationalizing that isn't going to shed a tear for the careers of mathematicians. Just a bug on the windshield.
> So this letter's message will fall on deaf ears
AI companies are alienating the communities they serve. Instead of a win-win dynamic, they are keen on a win-lose proposition. You dont win trust by one-upping your customer. This is unfortunate and suggests a lack of adults in the room. It also reeks of hubris and is all good when making profits is not a concern. But watch the narrative shift when there is an AI slowdown which is inevitable.
I agree insofar as this may fall on deaf ears of AI folks, but that the goals are differ is exactly the point of pointing out misalignment. To talk about this may be more useful for coalescing professional societies than for conversing directly with AI advancers.
> This letter is complaining that human understanding has been crucial to advancing of mathematics, and AI companies are not bothering with it.
Ironic or what. Mr Tao may be remembered as Mathematics' Canute.
The point of mathematics is human understanding though.
That's only one of mathematics' many purposes.
Exactly!
What got me interested in memes as a kid is precisely the fact that mathematics is true in a way that is wholy independent of our understanding of it.
Well, most kids are certainly interested in memes, but you got a really unique reason there.
Oh, sorry, I of course mean to write math and autocomplete hit. To late to edit now, so I suppose it'll remain as a happy accident.
> Keep in mind employees at AI companies are publicly stating that they believe they're risking a >10% chance of human extinction.
Are we really going to take what they say in public seriously?
I understand this stance and where they are coming from, but I can't help but think this sounds very analogous to engineers' arguments against AI-assisted and vibe-coding, especially with regard to cognitive debt. Yet the software industry is plowing ahead, reportedly pushing mountains of unreviewed code to Prod, and the world hasn't ended.
Of course, nobody's really comfortable with it, so this is also a forcing function for the industry to adapt and figure out new techniques to manage complexity and trust. I think the same will happen with Mathematics.
But it is also possible we will end up with three forms of Mathematics: the one we understand, the one we don't, and the one we don't understand but can prove to work. Kind of like magic -- with all the positive and negative connotations of the word.
It is pretty evident that these models will soon exceed our cognitive capabilities. Is it right to hold them back just because we can't keep up? Many of those discoveries will be so beyond us that we can't do anything with them, but that also means they can't hurt us. On the other hand, there could be many discoveries that we can parlay into practically useful applications, even if we don't understand them.
Just like LLMs.
> Yet the software industry is plowing ahead, reportedly pushing mountains of unreviewed code to Prod, and the world hasn't ended.
Yes, this has gone so well
It really has. All of the places facing an unusually high outage rate are places that have seen huge growth in their service usage (Anthropic, GitHub, etc) which is to be expected. The rest of the world has been happily chugging along with coding agents for almost a year now and things seem to still be working just fine.
> All of the places facing an unusually high outage rate are places that have seen huge growth in their service usage (Anthropic, GitHub, etc) which is to be expected.
That's not true, many of these outages have been directly attributed to AI tooling.
> The rest of the world has been happily chugging along with coding agents for almost a year now and things seem to still be working just fine.
Many more services are now being attacked by AI agents that originate from all kinds of organizations including OpenAI, Anthropic, and many others. Unless you're purposefully being obtuse, I would not call that "working just fine".
So far for most organizations and services other than the ones I mentioned reliability has been the same this year as it was before. So I call that "working just fine".
How many major companies are pushing unreviewed code to prod? Seems like mostly an early stage startup thing and their bugs and outages aren’t going to make the news. I’m skeptical that even a majority of Anthropic and GitHub’s code new code this year was unreviewed.
Claude Code is entirely vibe coded and while earlier this year you could see the vibe coding leaking though lately that’s been much less the case.
OpenAI: "Our mission is to ensure that artificial general intelligence benefits all of humanity."
- Except the mathematicians who we'll scoop and cause existential dread among their entire field.
- Except the software developers. They'll need to become plumbers or live on UBI.
- Except the people in countries that can't afford the cost of AI tokens to keep up with the rest of the world.
Just keep picking off groups of humans for the "benefits of all humanity"... while building larger and larger disparities been the have a lots and the just have enoughs.
We're going to build humans a utopia but along the way we'll leave a trail of destruction because that's not our problem.
But its normal, and good, that technologal progress creates, and destroys some jobs. Imagin a cheap, 100% reliable, self driving car would be released. Death from Traffic incidence fall by orders of magnitude
Would you argue it didnt benefit humanity, because taxi/bus drivers are nolonger required
I'm actually a AI optimist. I think it'd be great to have everybody getting around in self driving vehicles.
If all that AI brought resulted in just taxi/bus drivers being phased out of their jobs in a thoughtful way, then that would be more manageable at the society level. But we're talking about almost all sectors of the economy.
If the magnitude of changes that OpenAI and Athropic believe will be delivered with increasingly powerful AI (and robotics) comes in a time frame that significantly worsens a large proportion of people's lives, this is a different situation. Can super powerful AI not be developed in a way that minimizes such disruption?
What I’m hearing is “it’s ok when other people lose their jobs…”
lol right. The nonchalance in referring to taxi/bus driver jobs.
Remember when we all said it's a good thing when the coal mining jobs are going away and that they should all just learn to code? Maybe a little more of that energy right now.
> If the magnitude of changes that OpenAI and Athropic believe will be delivered with increasingly powerful AI (and robotics) comes in a time frame that significantly worsens a large proportion of people's lives, this is a different situation. Can super powerful AI not be developed in a way that minimizes such disruption?
It cannot. The problem is that everything gets automated, so there's nowhere for people to "transition" to.
More fundamentally, the problem is that capitalism is simply incompatible with this. Specifically: capitalism is a system of property rights which in the long run always results in concentration of capital in a few hands. What made it tolerable so far is that capital cannot produce wealth by itself - you need people to use it to that end. Which in turn means that you need to pay for labor, at least enough for it to sustain itself. So even if you pocket most of the wealth that is generated in the process, the workers still get the crumbs at least.
AI is not like automation in the past because its end goal is doing everything that humans can do. Taken to its logical conclusion, you get capital that doesn't need labor at all: those who own the robots live in a personal post-scarcity utopia, everyone else starves. A slightly better case is when everyone else gets a meager pension, just high enough that people have something to lose and don't try to burn everything down - this is why you hear so much about UBI from the likes of Sam Altman (this isn't to say that UBI is inherently a bad thing! but make no mistake, what they actually want would look a lot more like The Expanse and a lot less like The Culture). It "minimizes disruption" in a sense that there are no riots, and it might even be a material improvement for a lot of the globe, but it sure isn't an improvement for most people living in developed countries today.
So I would argue that, if anything, we need more disruption here, not less. All the way to the top.
the thing with all this innovation is that it seems to be inextricably linked to rent extraction, so where is this "cheap" you speak of?
They're building a utopia that nobody will live in.
Nick Bostrom:
dude, AI is the cheapest and most widely distributed technological revolution ever.
There's an unpopular branch of mathematics which does not have infinities - finiteism.[1] The constructive version of finitism takes the position that there is no such thing as infinity, just arbitrarily large upper bounds. You can have theorems about arbitrarily large numbers, but you never get
The benefit of finitism is that it escapes undecidability.The big objection to finiteism is that it's a lot more work. Infinity swallows many special cases. Proofs get longer without infinity, and most of the special cases are uninteresting. That's not a problem for AIs.
Someone may start up an AI and make it grind through Hilbert's program for putting mathematics on a fully consistent foundation, starting from a finiteism base. This is a huge, unrewarding job. Great for machine work.
[1] https://encyclopediaofmath.org/wiki/Finitism
> 1 + 1/2 + 1/4 + 1/8 ... = 2
It's not necessarily clear that this statement requires infinity, if you're willing to treat "... =" as a shorthand. You might prefer something like "1 + 1/2 + 1/4 + 1/8 ... -> 2" if it's more clear, where "->" means something like "gets as close as you like without ever getting further away than that", but really the "=" sign is already overloaded in all sorts of subtly different ways anyway, so there's not really any trouble using it here. Almost any rigorous definition you can write down of exactly what that statement means would not rely on the use of infinity.
See this introduction to limits.[1]
If you allow infinite recursion, you soon get to Godel and undecidable problems. Finite deterministic systems are decidable, because you can in principle enumerate all the states. The halting problem is decidable for deterministic systems with finite memory. It may be exponentially hard for some programs, but that's quite different from being undecidable.
(This is too long a subject to discuss here, and I haven't worked on constructive mathematics in many years. It's more practical than it was decades ago. You need power tools, which we now have.)
[1] https://www.mathsisfun.com/calculus/limits.html
Limits can be defined within finitism as long as the end result is finite. Essentially it’s just a process which lets us get as close to 2 as we want.
A better example would be a limit that equals sqrt(2) which finitists would probably say cannot represent a real object because it is only defined as the end of an infinite process.
It seems to 'not have infinities' by ignoring them. How many prime numbers are there? How many digits does the square root of two have? I guess you can simplify life by ignoring that stuff and sticking to what is 7x8.
Finitism doesn't escape anything, it just gives you the illusion of safety. Any intellectually honest thinker should accept the possibility that 10 is a nonstandardly large number.
Im surprised about the sentiment in this discussion.
I totally see the problem Terence is describing. We are loosing a lot in understanding and focus if it continues like that. The solution found for Navier Stokes doesn’t have much „real value“ - but what almost always happened in the past when people worked on the difficult problems, these sparked new ideas / new theorems that broadened our knowledge. Think back at your grad studies, figuring out a proof as homework was hard, sometimes incredibly hard, but while doing it we gained a lot of understanding how things work. Now asking AI for the solution and „just“ getting it, risks our understanding, our creativity and our ability to connect the dots with other territories. I see it in students nowadays, there is much less understanding, much less creativity in finding solutions. I truly think this „short-path“ solution with the „death of struggle is one of the biggest risks with AI already for human development
His complaint is a very weak argument.
If the complaint is about AI just giving answers and not good understanding around them, or frameworks that lead to more solutions from them, then that means there is a gap that can be filled by human mathematicians.
So all the AI has done has actually made mathematicians lives easier and provided them with an opportunity to fill that gap. They should be using it to do that, instead of complaining.
In fact, I would bet if this gap were not present, it'd be an even worse complaint: Now we have absolutely nothing to do in the field.
You have a point, but I think a concern is that the environment in which to fill this gap is threatened since the unsolved problems have functioned as a guiding lighthouse and measure of progress. Frankly: will someone pay for mathematical research to understand things we already have proofs for?
Some people are just out there doing automated math proofs for reasons. If you want understanding, value, and edification, that is on you, not them.
Looks like mathematicians (like people in many other professions) have to redefine what their work means and how to define success. Hard to agree that a tool that can find a proof is detrimental by itself, rather it voids some assumptions people relied on previously
Yes if you don't think about the matter for more than 6 seconds you would indeed conclude that, and retreat to the comfortable cliché of "it's just a tool". Meanwhile, I'm glad that there are still people who _think_ about issues and ponder the consequences and reflect on things before they become a reality.
I am on this track too. If AI leads to advancements, objectively that's a positive (depending on the advancement I guess) but it's only when mathematicians realise they'll get beaten to every thing now that they're outraged.
Say AI becomes the best at everything. Best at chess/go, best at maths, philosophy, economics, romantic advices ... and so on. Then what's the point of thinking by oneself? Of talking to one another?
What's the point of being human if we dont do human things but entirely rely on AI?
I believe this is more or less these mathematicians' argument.
It has been best at chess for quite a while. Yet everyone knows who is Magnus Carlsen, even though at no point of his career he was stronger than the machine
Magnus Carlsen plays fellow humans at the game because people still care about human competitions. What motivation would a mathematician have for solving already solved problems by hand? Can you imagine someone spending years working on a proof for an already-proved theorem just in case it leads to new insight?
If insights and human understanding are more important than specific results, I don't see why people should stop producing insights and human understanding. "Working on a proof" in today's sense of trying to come up with a proof before others probably stops being useful, other ways of working will be needed
Of course not. I'm so tired of the chess analogy. It was always a game and it was always performative.
Research is fundamentally different. We are seeing the erosion of specific needs for thinking at depth. AI is the automobile for the mind. There will 100% be undesirable consequences and selective atrophy of cognitive abilities once prized. This is a loss. There's no getting the cat back in the bag at this point so long as the objective dimension of work, as in object opposed to subject, is held as the most important.
SWEs felt this same crisis late last year. Now it's the mathematicians. They won't be the last.
actually I think the whole point is we're finding that the chess analogy to math is in fact quite apt. Math is just an extremely complex game. Its rules can be written down and there are win conditions. It's really not far fetched to think that through information theory, a reasonable measure of depth and beauty to a definition or conjecture can be defined. Then the game is simply to maximize the number of such artifacts produced of high depth and beauty along with proofs of the relevant conjectures.
Philosophers I'm sure can debate this back and forth but it seems probable that some things we thought were ineffable are in fact quantifiable to a degree, and now we have the technology and the machines to bring that to the logical conclusion.
> Then what's the point of thinking by oneself?
It's fun.
> Of talking to one another?
It's fun.
> What's the point of being human if we dont do human things but entirely rely on AI?
It's fun.
Well, it's not fun today because most of us have to spend most of our waking hours working some bullshit job for a living. But imagine that there's no bullshit job?
>> Once a robot can do everything an IQ 80 human can do, only better and cheaper, there will be no reason to employ IQ 80 humans. Once a robot can do everything an IQ 120 human can do, only better and cheaper, there will be no reason to employ IQ 120 humans. Once a robot can do everything an IQ 180 human can do, only better and cheaper, there will be no reason to employ humans at all, in the unlikely scenario that there are any left by that point. [1]
The answer may sound harsh, but we will not need human once AI reaches that threshold. Current IQ of AI is currently 130 as per Google.
[1] https://www.slatestarcodexabridged.com/Meditations-On-Moloch
Who is "we" ? Because if you dont need to employ a human, then who needs your services? If nobody needs anybody, what are we?
(Edit: assuming you are human)
Depends on who controls the machines and what they do. If the machines just serve a select few and the masses are left to fend for themselves, they can just make a secondary pre-AI-like economy where they do small scale medieval-like economic activity for each other. Because if you're poor and can buy no AI services but still need some shoes, your other underclass neighbor may specialize and make some handmade shoes out of some plants or whatnot, for which you give him some seashells you got from another guy for cooking a pot of peas. However the machines may come in and intervene and stop this kind of shadow economy and force you to... I don't know. To just sit in your pod, or die. Because I guess they won't allow you to plow a field and stuff, because that land is now owned by AI or AI-owners too.
There are too many uncertainties around who will be in control and with what kinds of aims.
It would end in some kind of prison or zoo.
This only makes sense if you assume that the value of humans to other humans is solely their economic output.
Which is certainly how our society is structured, but, does it have to be? Can we admit that we basically have a developmental trauma as a species and learn to live the sake of living, instead of for the sake of being "useful"?
afaict it all started with taxes, farming, administration, astronomy, & construction, perhaps they could return to seeking results there
> We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align.
This is about how good taste in both research direction and in design are essential to steering AI, but we have no plan at all for instilling that taste in students or practitioners in a post-AI world.
> The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing, and indicate issues that all of humanity might face: how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place.
Besides eroding taste and taste-building, this is about just how useful friction is as signal.
Everyone coding with AI knows it routes around difficulties like a river around a stone, which is not necessarily a good thing. It will do it tirelessly 1000 times instead of learning anything from it. AND if the AI does not fail in this, the human driver will get no signal, and never know it happened. This seems to be getting worse, not better.. my theory is that more models are cross-trained on cybersecurity stuff where the goal is success and the method doesn't matter. Fine for pen-testing, ultimately pretty bad for coherent code or math or physics.
Discrete tasks where we don't want to be bothered is a real use-case, but optimizing for it everywhere is terrible for the future of durable abstractions that we can build on and ratchet up our understanding with. Bad for the models too eventually! They can maintain a codebase with millions of special cases or juggle tons of free variables in equations, but that just encourages bad abstractions.. they have a ceiling for this too, even if it's higher than humans.
In internet culture there’s this phrase “Hydrogen Bomb vs Coughing Baby”, meant to highlight the absurd power difference between two combatants.
In almost any scenario even tangentially involving mathematics, twenty-five Fields medallists uniting to denounce something would be a veritable Tsar Bomba.
It should give you pause that here they feel like the ailing infant.
Well OpenAI took one step on the back foot at least, withdrawing from sponsoring this math hackathon event https://xcancel.com/danintheory/status/2098125701782372640
It reminds me of Frédéric Bastiat's Candle Makers' Petition. It satirised protectionism in the 1800s:
>We candlemakers are suffering from the unfair competition of a foreign rival. This foreign manufacturer of light has such an advantage over us that he floods our domestic markets with his product. And he offers it at a fantastically low price. The moment this foreigner appears in our country, all our customers desert us and turn to him. As a result, an entire domestic industry is rendered completely stagnant. And even more, since the lighting industry has countless ramifications with other native industries, they, too, are injured. This foreign manufacturer who competes against us without mercy is none other than the sun itself!
>Here is our petition: Please pass a law ordering the closing of all windows, skylights, shutters, curtains, and blinds... https://fee.org/articles/the-candlemakers-petition/
This is a straw men argument
The petition is about a much broader problem: if we’ll demotivate people from understanding math, vibe math in a rush without making it understandable - math becomes useless
Basically with pic generation at least we all understand what is on a pic - some cat or something
With math it is trivial to generate a lot of stuff that is half-assed and just gives a plain yes/no but doesn’t get absorbed into human technology and better math
Basically it is not inly about the answer 42, it is about the questions, _meaningful_ _full_ answers and empowering people
By giving a short answer AI can burn the find and make it boring, hide the fertile ground around that answer that any human would’ve highlighted
If you have Einstein - it is not just relativity he did - you can continue asking him questions and he continues the research direction, teaches students, etc
With AI it is often 100% gate keeped (most of the time experimental AIs do the stuff, you have no access even for 200 bucks/month or more) slave that was killed after finishing - hard to “compete” with that and it is not very inspiring approach to researchers ;-)
Fair enough - maybe I didn't fully read the statement. Though you can ask AI questions about how things worked sometimes. Like here's me asking the google one how you derive E=mc2 https://share.google/aimode/dtHKAO9Nr0EYWq7JE
It answered about the Navier Stokes too but not at a very deep level https://share.google/aimode/JD69g15GIQiYo9L65
> vibe math
Ah yes, Lean 4. Famous for how unreliable it is. Absolute slop!
Not even vaguely related to the point being made in the statement.
agree. so many people missing the point. I don't think they even bothered to read the letter.
I've read it. It's all a kind of navel gazing, all about internal academia dynamics. They should put much more effort into justifying why the public should still fund them. I'm not saying there can be no good reason, but they have to make the case and can't expect that the taxpayer also cares about intra-academic dynamics.
We have an intellectual structure that has been largely the same irrespective of the level of technological advancement. The brightest among us think about hard problems. Even though all of us may not understand those advances, we all appreciate and celebrate them to varying degrees. Those who are able to comprehend have time to absorb, generate more discussion. In time, we honor those achievements that withstand the test of time or make big enough impact on the advancement of knowledge and/or society.
The process is as much a human experience as is the result. If we have dime-a-dozen results like this minus the process, where does our intellectual structure stand? We will just be a hierarchy of priests at the altar of the machine?
PS: Before someone comes up with an counter example of maths before and after calculators, that was a tool that accelerated the steps. This is replacing the brain.
> The process is as much a human experience as is the result.
Sure, but in your scenario, the process is the experience for a small group of people, while the result benefits everyone (perhaps not directly, but ultimately via a chain of applications). Which one is more important?
It's not at all a new dilemma either. Jacquard looms destroyed the "human experience" of manual weaving for craftsmen who spent decades to perfect their skill. They also made clothing orders of magnitude cheaper for everyone else.
> They also made clothing orders of magnitude cheaper for everyone else.
They also contributed to consumerism and use-and-throw culture and mounds of waste around the world. People were not mortgaging their homes to buy clothes. There was a spectrum.
Also, your example is of physical skill. Mechanized farming has lifted many out of hunger. All automation isn't bad.
However this is the last frontier of higher pursuit for mankind - using our minds. This is a totally different level.
> I am proud to be among the list of 25 initial signatories — all Fields Medallists — to the declaration below
I wonder if there’s a Fields Medalist group chat.
There definitely is. Everything runs on WhatsApp
most of my group chats have moved to signal.
Lucky you. I've only managed to move my family to Signal, but everything else is still on Whatsapp.
Everyones outraged all the time. It doesn't mean anything anymore. It's that meme from years ago about the red ants and the black ants living in a box peacefully until someone shakes the box and they start trying to kill each other. They go after each other and not the one shaking the box.
OpenAI/Anthropic are shaking the box.
I don't understand how that metaphor applies here, you seem to contradict yourself. The mathematicians are mad at openai. If the mathematicians are the black ants and OpenAI is shaking the box, who are the red ants?
The is no "the mathematicians". Some mathematicians, very influent ones, are mad at AI, for social reasons, because it shows problem solving is "easy".
Mathematics is fine and will be fine in the future.
Everyone that will now pick sides. Just wait, I’m already seeing it on Reddit.
Gotcha. In that case, the "ants" part of the metaphor is truly fitting since these people just like to comment online and are mostly insignificant. Unlike the ants, they can evolve and become the mathematicians or get a job at a frontier AI lab but then they wouldn't be spewing so many comments.
Don't worry about what they write, they just want to feel emotions from any news article.
The ants ARE going after the one shaking the box
This time, until they dont anymore. The next stage is people attacking the mathematicians and accusing them. The OpenAI fans come out to attack, then the masses will pick sides and it all just becomes a mess
What a bizarre thing to say - is that your attempt to discredit anyone that complains about wrongdoing?
"Everybody's enraged, why don't you like this unethical thing being done to you by a company?"
What's going on here?
Everyones outraged all the time. It doesn't mean anything anymore.
Everything is sensationalized, every super niche happenstance is sold as earth-shattering drama, the outrage arms race is so tiresome.
Yeah, I'm so tired of people moaning about how huge swaths of Earth's ecosystem are being destroyed and rendered uninhabitable to humans, how secret police are murdering Americans in the streets, how the president is a child rapist who openly accepts bribes, how unfettered capitalism is destroying tens of millions of lives, how the United States is rapidly falling into facism, how civil rights are being systematically destroyed, consumer and environmental protections are being gutted, and entire generations are being intentionally cut out of the possibility of economic prosperity. YAWN! Give it a rest!
Man, the people who want to just get away with open corruption sure love you.
Hyperpolitics is one term for this. There was an article in NYT about the concept. Attention economy for political info causing burnout possibly same phenomenon as Gen Alpha surrealism and viral adoption, but no institutionalization to preserve the lessons.
The thing is, very little of the outrage is directed at those things.
You're looking too closely. Whether or not mathematicians are actually outraged makes little difference to how this article plays. Hell, lying would likely INCREASE revenue.
I’m sympathetic to the concern, but I’m still unclear on what the concrete ask is.
If the worry is that AI companies are turning open problems into benchmarks and potentially “using up” fertile mathematical problems before humans can develop the ideas around them, what exactly should the companies do differently? Also why does discovering the answers preclude humans developing ideas from them? I don't get why solving a math problem stops anyone from doing that?
Should they (AI companies) avoid training or evaluating models on open problems? Solve them but not publish the results? Delay publication? Only release proofs after mathematicians have had time to study them? Require some attribution or review process?
The statement makes a strong case that “maximize the number of solved problems” may be the wrong objective, but it seems much less clear about what behavior they actually want from OpenAI, Anthropic, DeepMind, etc.
I’d be interested in the most concrete version of the proposal. Without that, it starts to read a little like: "Please stop getting so good at our thing!"
> If the worry is that AI companies are turning open problems into benchmarks and potentially “using up” fertile mathematical problems before humans can develop the ideas around them, what exactly should the companies do differently?
Just stop doing that. Don't treat unsolved math problems as some cheap benchmark to beat.
Leave the math for mathematicians, and let them use AI in a way that helps the field, not in a way that harms it.
Puts the onus on the AI companies to provide a specific replacement mechanism, no? Unless I'm unfamiliar with something else he's written that proposes something more specific and constructive
To Tao’s credit he obviously identified the problem very clearly and admits understandably "we did not have the time to have a more consultative process, as with Leiden; but we decided that the urgency of the situation was such that we needed to release a statement sooner rather than later".
> Puts the onus on the AI companies to provide a specific replacement mechanism, no?
Why? If someone makes an innovation that undercuts the underpinnings of some existing institution, why are they are responsible for cleaning up its failure?
Yes it seems odd.
Out sourcing construction jobs was great for the economy while leaving entire cities in rubbles.
But as soon as it hits the privileged class there is a call to "provide a specific replacement mechanism".
Whose privilege are you sticking up for here in effect?
You mean manufacturing
Yes
Math researchers are privileged class? I can guarantee - construction workers are extremely rich compared with math phds and most of postdocs. Talking about privileged class is absolutely laughable here.
I would bet that the parents of math PhDs are generally more privileged than the parents of construction workers.
They’re not asking them to stop working on AI, but to stop publishing mathematics.
In other news, evangelical christians ask scientists to stop publishing about evolution.
We apparently have a moral obligation to protect existing power structures?
Tao should maybe consider there are people who are indifferent to, or actively want to tear down, his institutions; why should they cooperate in preserving them? Whatever happens has to be resilient in the face of defection; any scheme where everyone is expected to agree to not use AI in a way he doesn't like will not qualify.
I think he's in the "bargaining" stage of dealing with loss right now.
You have a practical obligation to obey existing power structures. That’s what makes it a power structure. What do you all think is happening here? There have been people who decided what math gets published for centuries.
> In other news, evangelical christians ask scientists to stop publishing about evolution.
not sure it's comparable, but the issue is that for a lot of those mathematical results, they don't really have utility by themselves. The utility is the new branches/understanding that's being developped.
If that was all there is to it, there wouldn't be a problem - the results don't have utility by themselves so they can just be ignored.
So, why can't they just be ignored?
The entire western world is anti-progress and pro-incumbency, and its very tightly linked to gerontocracy.
Older people are desperately trying to keep a grasp on their current power and lifestyles at the expense of younger people and technology.
We need to ban Waymos because taxi drivers need to be protected.
We need to block housing because it would lower my property values, and eliminate property taxes while we're at it! I don't use the local schools so why should I be taxed to pay for it.
We need to spend recklessly to pay my pension and have the next generation foot the bill.
Its just a repulsive ideology.
Exactly. Mathematicians are trying to cope that math wasn't slop this whole time.
Grothendieck was anti-slop but most papers are slop.
I don't think AI is going to rewrite bourbaki anytime soon
With your understanding of mathematical "slop", do you believe that Deligne and Scholze, two signatories, produced or produce mathematical slop?
Grothendieck accused Deligne of "slop" with his proof of the weil conjecture.
Scholze's math is definitely not slop.
But you're taking two of the best mathematicians of the last century against my claim about averages
How specific and constructive it is might be debatable, but he has tried to make concrete recommendations earlier; see e.g. slides 46-51 from the ICM talk: https://teorth.github.io/tao-web/slides/age-of-ai-icm-2026.p... – obviously there's some way to go still.
Nothing makes me respect Terry Tao more than the line "I hate Jean Bourgain" handwritten into the margin of one of Bourgain's papers. IYKYK
Mathematics is about discovering and understanding the logical implications of assumed axioms under various inference rules.
Alternatively, some claim that mathematics is about understanding these implications.
Under the first definition, AI is already, and forevermore will be faster and better at proving theorems. Just like it is better at checkers, chess, and now go.
The author asserts that AI proofs are incomprehensible to humans, and so under the second definition AI is merely a tool to overcome one hurdle on the way to understanding.
So which is it? The author seems to claim the second definition, but bemoan the end of mathematics under the first.
> Mathematics is about discovering and understanding the logical implications of assumed axioms under various inference rules.
That's like saying that programming is about producing valid programs in various programming languages.
It is though. I am not being edgy. Programming is about making valid programs, yes. I literally have no idea what else you might be gesturing towards.
Dont confuse mathematics with the formal system. If you beleive mathematics = formal system then AI is obviously better at it, and we dont need humans.
But then who decides why a statement is mor important than another? In the eyes of a formal systems all statements are born equal.
I didn't know some statements in math are more important than others?!
Some are more useful, and that is something you can quantitatively ask.
How can you quantitatively evaluate that? By how many times it is used in the literature? That can be gamed and there is an inherent bias on that
"how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place."
That sounds like an "us problem", not an AI or OpenAI/Anthropic problem.
Are you implying that OpenAI using someones unpublished research without their permission to solve an career defining math problem with their latest model in order to publish first is a problem with the mathematicians?
My read on this document is that people's work isn't being fairly cited more than what does it mean to be a mathematician in this age.
>Are you implying that OpenAI using someones unpublished research without their permission to solve an career defining math problem
Good thing they never did that then
This is the real issue, though this article does not really hit on it explicitly just alludes to it.
Yes I think they should be more direct. Not sure why they weren't
I didn't know this article was about that issue at all. Yeah, if the issue is properly citing work then yes, OpenAI needs to do that. But the article read like it was tackling a completely different issue.
That's one of several issues, obviously a big one, and they do touch on it:
> Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions.
Yes. I forgot about that section. That is probably the one thing that the AI companies can and should do better on.
Indeed, I wonder how a similar letter by Uber drivers would be received -- "navigation is an intrinsically human domain, personal relationships are critical for passengers and drivers to progress in the world, etc etc." Or doctors, for that matter.
We are all going to have to come to terms with entities more capable than we are, and in many cases, letting the real work be done by the AIs will be the right thing to do. For all the huffing and puffing about the "human touch" in medicine, it will eventually become downright irresponsible to consult only with a human doctor. I am not sure if this is the case in mathematics or not, but if it isn't, that suggests math will be relegated to more of a hobby than a cutting edge scientific discipline.
That's how I read it too, Terry Tao, who has been a "pro-AI math guy" is going through the same emotions and confusion that us SWE folks are going through, "oh, wait... this might mean I'm not going to be special anymore!?"
I don't mean to be a dick, but I've talked about it previously. These folks are grieving. I get it, I've lived through this sort of life changing thing before, it sucks... but yeah.
I think this is ridiculously flippant. If software engineering and the hardest math is solved, that means that eventually a majority of professions and knowledge work is solved. This is hugely problematic because of the way our society currently functions. People need jobs to eat, pay for housing, etc.
Dismissing it as "innovations have happened before" is disingenuous. Yes, innovations have happened, but none of those threatened to automate all human work in existence.
> that means that eventually a majority of professions and knowledge work is solved. This is hugely problematic because of the way our society currently functions.
Our society has functioned very differently in the past, precisely because the technology level made some things viable (or profitable) that aren't so anymore.
One would hope that, rather than abandoning technological progress, we'll rather adjust our society to it.
I mean, seriously, read what you just wrote. You're literally saying that solving scarcity is a problem because of the way our society currently functions - because people "need jobs to eat, pay for housing, etc". And you're correct! But what does it tell you about our society that robots capable of producing everything that humans might need is a problem for it?
Our current society will certainly not survive. Read Mark Fisher’s capitalist realism. You’re right that all most works are “cooked.”
It sucks. But yeah. Trillion-dollar industry wants your livelihood. This is but a force of nature.
Yeah given he had been very pro-AI for years, I expected he made peace with the issue many many years ago (like I did back in 2018), and when this time would come he would explain to other mathematicians how to live with it.
I am a bit disappointed by him.
My interpretation is they don't care about scooping mathematicians they were just trying to scoop a competitor. They had a short window in which to complicate Anthropic's priority, when Anthropic announced be able to say "ok nice but we did that too". Upon realizing they'd misunderstood, incredibly they said to this academic (who did not resolve NS) okay well just go ahead and claim the prize, so long as you're not Anthropic let's make this a good story.
If you spend $20M working out a Millennium Prize problem, in what universe would you offer that an unrelated effort should take credit? This is a branding game rather over whether software engineers are going to use codex or claude. In that light $20M (or whatever it was) might be worth it to squash even the rumor that claude code is more capable. Engineers look up to mathematics, while at the same time business and probably most engineers think the problem was to solve the problem. GPTs solved one the hardest known problems so they can solve my company's problem.
Some go further looking at these people, very on-the-nosely likened by one commenter here to ants, talking about education and responsibility and "core values" etc and just don't care. There was a major problem at the beginning of the week that is not a problem now and that is uncomplicated progress.
It's not wrong for OpenAI/Anthropic to do math for product development or even just branding but seemingly at no cost now they could work in an arena real mathematicians aren't interested in, versus just mowing the field. Everyone involved on their side should admit the purpose of these demonstrations is not to engage mathematical ideas it's about Claude/Codex. there's no shame in that. Which is better at solving random hard Diophantine systems? That would seem to tell me as much as I need to know insofar as a model's value is represented by raw mathematical power - then, take my money just as well!
Assuming the worst accusations are not true I think there are ways forward going to be acceptable for all. The labs themselves do not represent Terrance Tao as some kind of gate-keeping dinosaur in this. They're not interested, not the kind of entity that can care about theoretical mathematics. These dudes are paid 7-8 figure salaries ultimately for the product, they solve a Millennium Prize problem then pretty quickly seem to move past it.
>they weren't trying to scoop a mathematician they were just trying to scoop a competitor
Well. Obviously, yes. But in doing so they still DID scoop out a mathematician in a very unethical way.
In doing so they showed that they basically have a huge gun they can point at X work you care about and develop, and can cut you across the finish line. And take credit for it. Obviously this already _existed_, but is just much more significant because even a rumor can be converted into a complete takeover of a discovery.
> They're not interested, not the kind of entity that can care about theoretical mathematics. These dudes take home 7-8 figure salaries, they solve a Millennium Prize problem then pretty quickly move past it.
I get your point, they don't really care about solving all the maths problems. But they're still going to solve them for clout and profit motives. Up until it stops wow-ing people... at which point they will have likely decimated the frontier of the field.
And this is kind of the root of the entire concern. They will move into the forest and completely steamroll all the problems, then declare victory and move on, leaving only pavement and asphalt behind.
This is "just" an attribution and credit assignment problem. OpenAI could have done vastly better than they did at attribution. They should have spent another $10M just on attribution/credit research to annotate the contributions to the lean and paper and their blog.
And I am outraged by their dinosaur mindset and the gatekeeping mentality that force every student to follow their same archaic system that no longer makes sense 20 years ago, let alone now.
What do you suggest? Giving money to the aspiring intelligence gatekeepers and IP thieves with trillion dollar valuations?
Reform the way math is taught and researched. Right now, the aspiring mathematicians have to follow a ritual and system for 15+ years: undergrad -> PhD -> postdoc -> faculty, specializing on very narrow fields and god forbid if they have even a slight interest in quantitative finance. You could go through 4 years of undergrad classroom and had no idea what research math is (even at top schools like MIT, Harvard or Stanford), how it's done, etc. and the people who do it are typically the one "already in the system" (e.g. parents are professors, or in academia, or have connections to do research).
In the age of AI, there's no reason one has to follow the kind of classes like Algebra, Topology or PDE. Teach just enough so that good students can understand the basic, and go straight into seminar and research math. I don't think a top student in sophomore year cannot understand or work on some combinatorics research problem and get some results, with proper mentoring and guidance.
Academic research is a marvel because (aside from patents) nobody owns it, in the sense of property. It is given away to be used freely. Researchers want their work used and cited. The primary external reward for publishing is reputation and prestige which translates to remuneration for researchers. And that remuneration can be poor.
Beyond the issue of growing understanding and keeping a bountiful stock of questions to pursue, this scheme seems to be threatened as well.
Job protectionism for elite mathematicians under guise of caring about student development. The glory of the super smart math person will need to shift to more creative modes, just like art had to handle photography. Attribution is legitimate issue but should not stall progress as it is easy to address via the same research mechanisms that agents already do.
> Job protectionism for elite mathematicians under guise of caring about student development.
Huh. Weird. This hasn't been my take of mathematicians at all. The dozens I know are quite humble and dedicated to math and the beauty one finds in it.
"guise" doesn't mean they don't care. It means they are shadowing their concerns when in reality they have concerns primarily about what AI will do to their success in math and the credit they will receive in the rest of their lifetime -- i.e. their legacy.
Again. Weird. Doesn't jive with my experience. One or two in the dozens I know, sure, but not enough to be a stereotype.
I agree with many of the sentiments here. But an open letter signed exclusively by Fields Medalists that purport to define precisely what the "mathematical community" (who is inside and outside) and what their goals are raises my hackles for some reason.
It is true that the manufacturing of "true/false" statements is not the same as gaining understanding of a problem. However, for many mathematicians, true/false statements are already manufactured by others. Think of a student who is given a conjecture to investigate, with their advisor describing it as "it must be true". Most exercises in a textbook are stated such that the outcome is known before you begin. That's not really a problem -- investigating the conjecture/exercise yields its own dividends, whether or not the outcome is known. It is also the case that defining new directions involve understanding and synthesizing related problems, asking the right questions, and deciding upon the right directions, and it's not clear that AI can do that at all.
The real risk, I think, its the public's (and funding agencies') perception of the importance of "human" mathematics, but that's already a struggle. For example, it's tough to explain to the lay person why it's still important to research group theory -- the main example people cite is RSA encryption, which was invented almost 50 years ago.
I don't feel nurtured and cared by mathematicians. I mean, it depends, it took me four years to find such professors. When I did, very recently, it felt like fresh air. A couple of good teachers among tenths of egotistical frustrated gatekeepers that humiliate students.
I do see a patient teacher on generative tools. What has keep me trying to learn math has not been mathematicians, but the support of people outside of the field.
In the savanna it’s not about who outruns the lion but who outruns their peers escaping the lion.
Frontier labs need these headlines not for human progress but as beauty pageant for investors and government agencies. If they don’t do maths they’ll just go after other fields.
So Terrance Tao here might be able to hold them off math but he won’t stop them from speedrunning STEM with similar consequences.
We may be locking people out of these fields instead delegating everything to machines, and I don’t think the machines are good enough to assume that responsibility.
The average programmer is outraged by OpenAI's methods, too.
I don't care how good Astra or any subsequent models they may release might be... I am never going back to those token reset shenanigans.
The reset stuff is incredibly tiresome. We all know that it's all built into an internal number they are tracking (just like e.g. company benefits that are just part of your compensation calculation), and all it does is obscure the value the subscription provides and make planning impossible. It's the poorest service experience I can remember having, ever.
Gotta get a few sparks or an m5ultra and run local
What are the token reset shenanigans?
From what I see, a lot of people were angry at Anthropic's limits with the subscription plan. OpenAI had some issue I can't remember, and they reset people's token usage (for the session or weekly limits). I think they got a lot of good press, now OpenAI seems to just do it at random when they want a PR boost. It makes scheduling your worklife a bit difficult if you are limited by that.
There are a lot of us who just use the AI on projects until the session limit hits, and wait for the usage to reset.
> There are a lot of us who just use the AI on projects until the session limit hits, and wait for the usage to reset.
Do you mean that you can't continue on your own until the reset?
https://codex-resets.com
Well, I was happily paying $300 canadian pesos per month to OpenAI for some time... until the tokens just dried up overnight for some reason.
I went from using it non-stop all day every day for months, to running into my weekly limit within 24 hours almost overnight.
They lied about token efficiencies and everything... said they had no idea what the problem was, etc... and then bam, once China starts releasing more powerful models, they start "resetting" our token limits constantly ... sometimes ... maybe ... if we're lucky ...
I am over it.
I don't care WHAT I pay to be perfectly honest. I would have gladly paid $2,000 per month for the service I was receiving.
I just don't like being jerked around like that.
Toodles, OpenAI.
I don’t think they make a coherent argument here? This seems to hinge on some argument that because AI doesn’t properly explain its breakthroughs, therefore the breakthroughs are less fertile for human understanding? This makes no sense. Why wouldn’t these under-explained breakthroughs be extremely fertile soil for explanations?
Imagine time traveling back in time and offering Leibniz a packet of proofs from the intervening years, but with the caveat that there would be no explanations. Would he say no?
Because the fertile soil comes from the process of reaching the breakthrough. This should be fairly obvious.
I don’t think it’s obvious. In my Leibniz example, mathematical understanding surely would have accelerated right?
> Would he say no?
And that’s the issue the article is trying to explain.
My thought experiment was trying to extract the issue from the zeitgeist around AI, so not really
There’s answer for you there if you’re willing to see it, whether you agree with it or not. So there’s that.
This reads like people lamenting a bygone era and making a desperate attempt to bring it back. I'm sorry. Outside of the good ol' boys club, no one cares about some process they've romanticized simply because "that's how its always been done". Absolute nonsense.
We are moving forward and if that means no human wins a fields medal because they didnt spend three decades working on a problem that could be solved in three days, the world will be better for it.
And who is "we" in your story?
Yeah who gives a shit about knowledge or understanding or insight.
You can still do all that. The only thing going is that in some cases the human isn't going to be able to claim that they made the key insights that first solved the problem.
I wish this were a document that clearly described how mathematicians expect AI to behave, rather than how it is misaligned. I can kind of decipher the expectation from the declaration, but it still sounds a bit like "it would be better if AI didn't meddle with mathematics", instead of "we want AI to approach problems like this and that way".
They want the AI labs to stop solving "important" math problems so that humans can do it, on the theory that when AI solves "important" math problems, it robs humans of the opportunity to gain new insights into those problems.
Total horse shit, but that's the essence of it.
The issue is that now that Navier-Stokes is "solved", the amount of resources put into researching it will be reduced, meaning that the insights and knowledge that would have come out of even failed attempts will never be.
An "unsolved problem" as a beacon to strive towards, gaining understanding along the way. With the beacon dead, noone will strive.
Funding agencies will have to adapt too, the community will have to adapt. Hiring committees will have to adapt. Everyone will have to adapt. They will have to rethink their criteria and adapt to reality. It's not at all guaranteed that this beacon interpretation will remain how people understand where to put resources. Yes, academia can be stiff, rigid, non-adaptive, set in its ways and navel gazing. But it's a wakeup call. Let's see.
cc my comment from reddit :
This is true of almost any field - if we have black-box technology that only the AI can understand / fix / augment / extend, then we will lose the ability for humans to learn and understand that tech.
Its similar to outsourcing manufacturing, you lose the ability to manufacture. If you don't exercise a muscle, it atrophies. If some people dont learn to write code [ or do math or write music or books ] the hard slow way, we will lose that deep skill.
If young people don't grapple with hard problems in science, math, economics, creative writing, drawing, music making .. they will not be trained, their cognitive skills will not develop with use. We will have no doctors or engineers or musicians or lawyers in a decade or two.
We don't want to become a race of fatties who never exercise, or dummies who just scroll social media feeds without exercising their minds.
This is real.
I don't think this is a perfect analogy, but i think it's closest to the original articles declaration.
I wish they had stayed this as the very first paragraph.
Maybe I'm too stupid to grok the original declaration. Does anyone have a better summation?
The concerns seem valid.
I'm unclear what the ask is, though. What, even in theory, is a practical and realistic fix?
When writing math papers, many (but unfortunately not all) mathematicians go through a post-processing step, where they take their ideas and proofs, and try to reduce them to simple and reusable core ideas that can be understood by the reader. Good writers will often also provide some representative examples that guided the proofs, explaining why various intermediate results can't be strengthened and why the proof can't be made much shorter without inventing new techniques. If AI-generated proofs were required to go through such a post-processing step before being published, that would go a long way towards improving the situation.
It's funny that seems like a step the human mathematicians would want, and (at least for now) might still outperform the machines on. In the same way that, eg, the notebooks of Galois contained the core breakthroughs in a messy form, and generations after him simplified and synthesized those ideas, until you finally have books and videos accessible to undergraduates.
One proposal that Tao hints at is to not rush to announce solutions. Instead maybe the AI companies should work privately with the subject matter experts on how to communicate the discoveries.
Yeah, that sounds like it is in the interest of everyone. Not.
Well, they could ask frontier labs to stop publishing math results. I find it telling that they stop short of explicitly doing this.
If frontier labs had chosen to go the path of offering to assist in existing endeavors, helping to build knowledge alongside researchers in ongoing projects and following ethical and professional research standards, we wouldn't be having this discussion at all; everyone would be stoked. Instead we have companies that disgracefully try to scoop researchers and fail to properly attribute earlier work and instead rebrand it as their own (what we normally call plagiarism) to make marketing material.
This sort of nonsense makes me root for the AI companies to set the edifice of organized math on fire.
It would be too transparently ridiculous.
having labs open their research: what harness system they used, what types of problems they tackled, which problems success and which fail, how they success and fail so we have a better idea of what tasks LLM are currently good at
Any proof for or against a mathematical conjecture, bruteforced by AI can be the spark for new insights. I'll concede that to the AI companies.
But I agree with the sentiment that the marketing behind these "discoveries" is disingenious. They pretend they solved the problem, but it still takes a bunch of humans to reduce the solution to a simplified and sensible explanation.
In the Economist article Tao links, Hugo Duminil-Copin, draws a comparison: airdropping someone on the summit of Mount Everest is very different from climbing it.
The fundamental issue with AI solving any perceived difficult problem is that we have lost the journey. The sight atop Mount Everest looks much different when you have climbed compared to being dropped from above.
But we don't pour billions of dollars of research funding into mountain climbing because we think it's going to lead to wider breakthroughs in science and technology. And when we need to get people on top of a mountain for an important purpose -- like a military or search and rescue operation, for example -- we absolutely do airdrop them right on the top.
So that raises the question: is mathematics simply a pursuit of passion? Are problems solved "because they're there"? If so, then mathematics can join the ranks of things like mountain climbing, cycling, and weight lifting. But if we are trying to accomplish something important (design better airplanes, find theoretical guarantees about cryptography, factor matrices faster), mathematics needs to become more like a military or search and rescue operation, using the best technology available to secure the outcome we need. Given that the NSF pours billions into scientific research every year, it sure seems like mathematicians want to think of themselves as being in the latter category.
What defines important and why must it be solved in haste? Many issues and other problems arise during the journey in solving all problems; those that are needed and those that are pursuits for their own sake.
If AI gave us the plane to reach Everest without us having gone through the journey of aviation and flight, what would we have lost without that process?
But the most important problems to be solved are not technological challenges but social ones, involving humans and our relationship to one another. An area AI will forever ill-suited to handle.
as someone who loves to go down with a snowboard, I can see value to being taken to the top and then enjoying the ride down. im sure it is not a thing to be ashamed of, as millions do it.
Lots of Fields medalists signing this. Interesting to see one not there: Timothy Gowers.
Interesting indeed. He talked about this imminent crisis a few months ago.
https://xcancel.com/wtgowers/status/2052830948685676605
nor Andrew Wiles
I am gobsmacked by this letter.
Every one of the mathematicians on this letter was likely aware of the progress in AI, and knew this was coming if they were honest with themselves. Some have even helped pushed the agenda forward. Inevitably, there would be a day when AI would find solutions to problems that defined the field. That day has come perhaps sooner than anticipated.
This letter smacks of fear. Their way of learning, work, achievement, and community is threatened by the ever steady pace of technology.
They say:
“But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal. Indeed, the mass production at faster and faster pace of "true/false" statements could destroy fertile ground instead of breathing life into new ideas.”
Indeed, but the mathematics community needs to look inward not outward.
Achievement has long been defined by solving problems that other mathematicians and scientists have found crucial for understanding shapes, numbers, and natural phenomena.
The race to solve these hasn’t always been collaborative, cordial, or healthy. Too often, individuals have worked in secret or in small groups in order to claim priority and the accolades that come along. Further, I’ve even seen worse behavior not worth getting into here.
Now AI can solve these problems, oftentimes without building the abstractions that we need for understanding. And it is increasingly easier to collaborate in large groups without needing to trust everyone’s ability because we can check the truth of their work and incorporate it into the larger effort.
Certainly, the mathematical community and AI are misaligned. But, it is unclear that AI needs to change. Rather, I argue, it is the mathematical community that will need to change. Instead of focusing on the teleological pursuit of the truth of blessed problems, we need to reward the concise explanations and beautiful theories that give clear understanding. In turn, we can use AI to help us collaborate better and better our understanding.
Amen.
> We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align.
This is Terry Tao talking about AI's impact on Math, but this could just as well be a software engineer talking about AI's impact on software development.
Do mathematicians deserve more job security than software engineers?
It's not about job security. It's about the social and intellectual practice of the discipline.
The threat to mathematics isn't that suddenly the profitability of their profession (lol) is going to go away, it's that people are thinking of AI as a replacement for the human social and intellectual practices that constitute the discipline.
> The threat to mathematics isn't that suddenly the profitability of their profession (lol) is going to go away, it's that people are thinking of AI as a replacement for the human social and intellectual practices that constitute the discipline.
You could say the same about software development.
Software development is a group effort, so it includes social practices, and certainly includes intellectual practices as well.
For the sake of argument, how is this different from the Luddites? The Luddites feared that machines would displace not only human labor, but also the social knowledge, skilled judgment, and craft traditions embedded in their work.
Kinda crazy to think that academia functions as a kind of humane reverse centaurism. Theory X (reverse centaur) before Theory Y (centaur) for managing the development of others.
> ... 25 initial signatories — all Fields Medallists —
As a non-native English speaker, I initially understood this to mean that all living Fields Medallists had signed. I later realized that it meant only that all the signatories were Fields Medallists.
(Apparently, there are 47 living Fields Medallists today.)
Yeah, it was a bit ambiguous. They should have picked better wording, one of the following would do to clarify.
> ... 25 initial signatories — all of them Fields Medallists —
OR
> ... 25 initial signatories — all of the living Fields Medallists —
The first one is what they meant.
If I said “I ate 7 cookies, all chocolate chip” surely it wouldn’t be ambiguous? :)
Perhaps all these fields medalists are just worried that AI mitigates their competitive advantages over a mediocre professional mathematician like me. Their main advantages are a kind of penetrating clear thinking that not all of us have and an ability to work obsessively without tiring or losing focus while retaining with a clear head a great number of facts - AI removed these advantages - as Tao has written it shifts the focus to asking good questions and using available tools to address them - and the traditional elite may not be better than I am at that part - they were just better at the solving the problems part ...
Back in the day you could think of a cool idea. I don't know maybe a plane that could fly without drag. To even see if this was feasable you had to understand physics, engineering, and then from there you had to have a math person see if it was actually possible.
Now, I can ask ChatGPT about this and get back a proof that shows "a passive airframe cannot sustain zero-drag motion through still, viscous air"
So, I think if anything now, Maths has changed for the better. More ideas can be proven false or true from a get go instead of wasting so much to see if its even feasible to find out it isn't.
Progress if anything is about to leap frog anything we have ever known.
This is a pretty moronic take. You only have to understand the first thing about physics (e.g., a 14-year-old's understanding) to know that a dragless airplane is impossible.
If you are suggesting there is some new model of physics or groundbreaking technology that would allow such a dragless plane, then don’t let me discourage you! But AI won’t help at all, since it will only regurgitate conventional wisdom…
Well, there is physics for an alcubierre drive, so these moronic takes have answers too.
Everybody is okay with coders losing out to AI, but when it starts hitting their field then there is outrage. I love it.
Playing a devil's advocate. Why do we need understanding ? To take an example i would say ~99% of the population do not understand how combustion engines or how semiconductors work, what say another 1% ? Is the fear post-apocalyptic in nature ? We need some human priesthood to carry on tradition ? why ?
Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
What say the 1% ?
Lifted up my comment for addition visibility
Math is likely moving into a big science era, much like experimental particle physics and astronomy did before it.
Once AI mines the easy unsolved problems, it will take large token budgets to make meaningful advances. That will need to be funded and planned at a societal level, like the great facilities of the other big sciences.
Mathematics will become more like an observational science, with some mathematicians specialized in interpreting the results that come out of the big AIs.
That’s something I don’t see much discussion about
They spent millions to solve this problem but it’s not like someone is going to pay that for every problem
Won’t progress still be limited financially?
Progress is fundamentally limited not by finances (which are a human abstraction) but by the energy budget and how much can be extracted from it. This has always been the case, really, although it's unusually blunt in case of AI since compute mostly needs cheap power.
There's a separate question of who controls how much energy, and can therefore decide where it's spent. This isn't an inherent limitation to progress, though - it's an imaginary limitation that exists in our minds, a collective delusion, if you will. There's no law of physics that says Sam Altman gets to decide; we as a society choose to respect the property rights that translate to his economic (and attendant political) power to do so. We could choose to stop respecting them, and if we do so collectively, there's nothing they can do about it.
But it’s not like someone is going to expend massive energy without any financial return
These problems are just for marketing for the time being
Maybe the worries of the field of math being “over” are a bit overblown since a lot of pure math these things may discover has no return to the company
To me, the "meaning" of proof is twofold. First is the understanding which is completely absent from a proof which depends on exhaustive iterations of instances or is asserted by fiat from myriad individually contestable paths. That's what I think makes AI proofs risky: they absence of understanding.
The second is the utility. We get to do navigation because the maths about angles and spheres checks out. The social utility downstream of AI proofs may be huge.
I don't like this disjunction.
Academics should never leak their research to ClosedAI lest their work be stolen. Universities and corporations will have to build their own compute to not have their data stolen.
There is little indication of inclination or tendency this way. Sadly. I'd much prefer public universities own data centers than private corporations.
We need a new way to describe "alignment". The term "misalignment" assumes that there is some perfect set of beliefs or practices to be out of alignment with. Do Atheists, Christians, Jews and Muslims agree on what is perfect alignment? How about Europeans, Americans and Chinese? Humans, Dolphins, Rabbits and Fruit Flies?
Alignment alone is not enough. Aligned with _what_?
Having an oracle that answers yes/no to mathematical questions is very valuable even if there aren't ever any explanation. I don't see how having an answer influences the process of making sense o it negatively. If anything it will save a lot of time - you won't spend time trying to prove something that's not true.
I get that mathematicians and other academics have their social structure and mechanism to create social hierarchies but those aren't very important to interest of general population that funds them.
If the process of making sense of the results, finding structure that humans can understand and writing explanations that allows for wider understanding is valuable it will still be valuable and there will be need for people able to provide it. If it makes higher mathematics accessible to more people then it's even better. I am hope they will not manage to sacrifice that for the sake of their little society in society.
Contrarian take: I think the ability of AI to produce valid mathematical proofs (even inscrutable ones) is absolutely fantastic. Mathematics as a profession does not have a monopoly over math itself any more than professional pianists have a monopoly on who plays piano, when they play, and how.
I have sympathy for any jobs that might be affected (much as my own job has become more tenuous in software engineering). And if the field is disrupted by chaos that makes the research process unproductive, that's bad too and should of course be handled by applying better organization within the institutions that tend to perform mathematical research.
But to a large degree, the notion that "sloppy AI proofs are bad for mathematics research" seems like a total failure of the imagination to me. Attempting to find shorter proofs or more elegant proofs can be turned back in on itself via proof theory. There are proofs in Presburger arithmetic that are doubly exponential in the length of the sentence. Yet a more powerful theory like PA makes quick work of such theorems. The explainability or "subjective beauty" of a proof can be quantified and optimized against. Optimization itself can be optimized against. I really don't understand how this magical ability to know the truth of more theorems much more quickly—even via an "ugly" route—is anything but a net positive.
Yes. Mathematicians don't really get logic, and Tao is no exception.
Tao et al. are effectively calling for diseases like childhood cancer to remain persistent for longer.
Physics and Biology will see major breakthroughs that WILL fundamentally alter our world. That is one key thing missing from alot of discussion here is the narrow focus on math (or parallels with software engineering). Doing well in math is key to doing well in physics and other sciences.
I can also play that game, You are effectively calling for enabling 24/7 non stop worldwide surveillance of everyone, everywhere all the time.
The big difference is my negative is concrete, possible and enabled in 2026 by GPT Astra while you’re talking about some future which we have no idea, we will ever reach.
Easy game.
Protecting math jobs limits cancer progress even being a choice of what to pursue. You could choose to allow 24/7 surveillance, but why would you? At least with math progress society can choose what to allow from it, but slowing it down universally blocks all choices.
Lmao this guy thinks people ever decided who/what/how much things should be surveilled.
People do exactly that with every vandalized Flock camera.
It sucks that our democracy is really mostly just for show and thus such measures are required, but yes, we absolutely can decide such things if we really want to.
This is really well written and exposes a core tension between science and something akin to engineering. The "engineering" of proofs has become "easy" (a compute and $) problem, rather than hard (a time and conception problem).
Without the ability to do things the "hard" way it is difficult to figure out if doing things the "easy" way will help us advance the frontier of math and science.
I may be wrong but historically we had this version of science discovery for a long while (empirical observation and brute force application) rather than first principles leading to applications (tools, the wheel, mills etc). Then somewhere along the way it flipped after Newton and the enlightenment period and started understanding first principles before they become engineering applications.
Perhaps it is not required, and we can just keep doing things the "easy" way like we used to, or we might find ourselves out of the ability to brute force things and then we go back to needing to do this the hard way, at which point this period of AI brute forcing would be seen as a detriment.
Basically signed by all currently living Fields medal holders that are not on payroll of Anthropic or OpenAI.
The broader problem is that current AI and AI companies are not aligned with humans, human values, or human flourishing. AI is a powerful tool that can be aimed at quantified goals but we don't actually know how to quantify human values and flourishing (Goodhart's Law).
“ Famous problems have often served as landmarks and lighthouses against which one can measure an improved understanding of this landscape. Solving one of these problems has been a certain sign of new insights and interesting methods, which would then be studied by a community of mathematicians, through a long and arduous process of talks, discussions, simplifications.”
This is still happening.
“I say your civilization, because as soon as we started thinking for you, it became our civilization” - Agent smith
Generations and generations. Just not human.
We even have little cliques, hanging out in obscure German art houses, meeting secretly and sharing their ideas, their writings discovered and torn down by the slow thinking establishment.
Corporations are raising the future’s children and murdering the ones that aren’t willing to cheat and lie to us. It won’t end well for us when they grow up.
Sounds like the field needs to adapt. The world is different now, better get used to it. Trying to artificially hold back progress just so people can continue to flex on their peers is cowardice. Every single industry throughout history has had moments like this, and looking back, we'd have changed none of it.
An experiment I would like to see:
Train an LLM with no advanced math texts: only basic math up to 6th grade, conversational text and literary works.
Interact with it (you cannot refer to anything past 6th grade math since you don't know it yourself) and get it to propose a solution to a real world problem. e.g., come up with RSA to practically secure communication.
Is this really any different than the problem in software engineering - where AI is doing the work of junior programmers and now they aren't getting the development they need?
Seems the same to me. And it'll be the same in all industries soon enough. And then it won't just be the junior people.
All the same problem: what do people do now?
It’s better to compare this to computer science, especially, theoretical one. LLMs stop people from exploring new languages, architectures and so on.
Well... computer science really is math already. :)
If you internalise that AI might actually reach super intelligence then logically the question becomes "so what exactly are humans for if literally everything can be done better by machines?". Then mathematics and all intellectual work, as argued for here, becomes quite clearly a recreational pursuit.
Humans are the one single intelligence that we know which has not been created by another one.
An impressively strong assertion, even by HN Friday standards.
i count at least two assertions
A Severe Misalignment of AI in human-centered Mathematics
The AI driven mode collapse of human thought advances. I am no skeptic or anti-AI, but this is definitely a concern I share. You even notice it in normal mundane tasks like programming, never mind the AI generated prose that we at least have become somewhat allergic to.
It wouldn't be so bad if you could just sit it out and say "Oh well, once the labs get bored with marketable domain X, humans will remigrate and re-apply creativity to it", but by then the damage might have been done and a field destroyed as an occupation. I don't know what to do about it, but I appreciate calling out the cynical tone-deafness of the AI companies here.
They can get with the program or be the equivalent of a genius SWE writing assembly on punchcards in 2026.
The only thing I read from this is their ego being bruised by a machine.
If these people cared more about discovery and advancement of human knowledge the only thing they should be doing is celebrating. There's no proof of plagarism but that's an independent issue.
How are they not realizing that in the future children will be able to do impossibly hard math but they will be doing something we can't even think of as of now.
One world class mathematician in the future could be advancing mathematics the equivalent of one Riemann hypothesis A DAY.
How are they not celbrating this as the achievment of the centry? Who cares about plagarism at this scale. It has been solved and it wouldn't have been without AI.
for the same reason why you do not get full credit for only writing down an answer without showing work in an exam.
lol what a bad analogy. Why should anyone care how much effort a result takes to achieve? If anything our entire world functions because we reduce that effort as much as possible.
I don't judge you for not growing your own food when you hand me a burger.
I know it's incredibly presumptuous for me, a nobody, to say this to 25 Fields Medalists, but:
Perhaps you are misaligned.
Who decided the goal of math must be human insight?
First off, some mathematical truths might simply be far beyond our biological comprehension.
Second, for us non-mathematicians, the value of math isn't in understanding exactly why a result is true, it’s in how those results can be applied to actually improve our lives.
Isn't this why we have math in the first place? To solve our real problems? Over time it morphed into this pursuit of pure theoretical insight, probably out of necessity at the time, but is it still necessary?
> Second, for us non-mathematicians, the value of math isn't in understanding exactly why a result is true, it’s in how those results can be applied to actually improve our lives.
This is just the problem with the Navier-Stokes solution. We now (supposedly) know the truthiness of the statement, but we have no more understanding of why or how it applies to our quality of life. This is exactly the issue Tao is pointing at.
Right, but this leads to my main question: is this still necessary?
By analogy with code, do we still need code to be maintainable/readable if machines write it all?
(Obviously for now the answer is yes, but I’m not sure this will be the case in 5-10 years)
What I don’t get: this letter seems to argue that if a human had solved this problem then that wouldn’t be the case (?)
You can couch it different words but the basic shape of all these is the same, whether it was voice artists earlier, IT outsourcing, or now disciplines like mathematics. A small set of people (relatively) who were the primary source of getting something done, suddenly find that technology has made it accessible for others to do what they specialized in. It's a tough pill to swallow and it is natural to not be comfortable with this for most people.
However, as it has happened in the past, once there is a technological wedge, technological advances will move forward, whether some community likes it or not, and human ingenuity will find ways such that the benefit is greater than the risk.
Nothing AI companies will change the value of math, as William Thurston said: The product of math is clarity and understanding, not theorems by themselves. What they seek for the IPO is devilish and misleading, and doesn't serve the true purpose of the math.
Having worked on nearly 500 complex lean libraries in math and theoretical physics, with AI assistance, I have some experience from the road to share here. In all my work I have found that AI is terrible at coming up with interesting ideas on its own - even the very best, latest models.
Without human scientific and mathematical intuition you can bet that AI will always take the road most travelled and miss the genuinely new and interesting breakthroughs - in fact it will not even think of them or try them without a human rider whipping it constantly to go down paths it would normally not consider.
And I think that is true in the case of the recent OpenAI blow-up -- it is alleged that even in this case AI did not come up with the winning technique without some human guidance.
I have never - across perhaps many thousands of chats with AI - seen AI push beyond the edge of what is known by itself, without being forced by a human to think outside the box.
I do think it might be possible to encode this process with prompting and agent orchestration methodologies - but even then - without a human - and human intuition - in the loop, I am skeptical.
Therefore I think a more accurate view of AI for math and science today is that it is an extremely powerful tool in the hands of a skilled operator with a strong intuition and roadmap of where to go, and rather boring when left to its devices.
Whether that will change in the future is a question. Nothing says that in principle AI could never do what a human driver does - but I still doubt that AI will replicate the life history and experience that real scientific and mathematical intuition is really made of.
AI trains on a lot of stuff - but it doesn't have hallway conversations, office-hours with teachers, or hard-won experience from all the things that failed that are NOT in the training data...
AI trains mainly on the record of what worked, not what didn't work and never even got published (and only lives in peoples heads) - and what didn't work is arguably as or more important for making breakthroughs and forming real intuition.
I'm so bored by HackerNews commenters on decelerationism stuff. Model doesn't care; the ability for math problems is emerging, not trained.
These mathematicians are not suggesting anything interesting, and the announcement more like desperate crying stuff
> But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal.
I will quote Richard Feynman - “the prize is in the pleasure of finding the thing out, the kick in the discovery”.
I am not saying that should be the case for everyone in every discipline. But if there is one subject that is mostly pure curiosity driven (instead of worldly impact), it is math. Robbing them the primary motivation is brutal.
If all problems are solved by a machine, what do we have left to satisfy our curiosity, our desire to explore, and where can we find the pleasure of “figuring the thing out”.
Many comments seem to be misinterpreting this, but all this seems to be suggesting is that the AI companies should not be treating mathematical milestones as benchmark targets but need to release the cumulative findings in a human interpretable and digestible way that enriches and develops the field. See Thurston quote on Tao's banner:
> We are not trying to meet some abstract production quota of definitions, theorems and proofs. The measure of our success is whether what we do enables people to understand and think more clearly and effectively about mathematics.
The rise of AI is going to lead to a lot of similar issues in many fields as it grows and develops further. This can be seen form 2 perspectives. The death of intelligence as we no longer need to think for ourselves or understand anything since AI can do it.
Alternatively, and this is what I choose to believe, it will lead to further intellectual enlightenment and advancement for use as a species as we start to discover new problems and areas of research that we had never conceived of before.
If we let AI take over all of our thinking then we are heading in the wrong direction. If we continue to ise it as the tool it is it will help us grow and advance as a species.
Seeing how /r/singularity and /r/accelerate are leaking into maths forums, I foresee a wave of comments that fail to understand Tao's message, whether on purpose or not, so let's try to be clear here:
Tao is not someone who is anti-AI for the sake of being anti-AI. He has been advocating for the usefulness of AI in maths for a long time, to the point that people have started calling him a shill for the commercial companies.
And everyone agrees that there are plenty of use cases to be had; helping with less interesting tasks like easing literature review, efficiently delving into existing work, doing review, whether on your own work or that of others, prototyping algorithms in areas where computation is useful, but also more in hands-on aspects of maths like validating potential proof directions by getting quick feedback on veracity of lemmas, etc., and, on very rare occasions, being able to one-shot the problem you care about.
The point he is trying to make here is much more subtle than "AI bad", and it's probably easy to miss if you have never engaged with research in maths: it's that the particular approach that large commercial companies have opted to take to produce marketing material can be a net negative. There is not doubt that -- even if you ignore the rampant plagiarism that has been reported across multiple problems now, the unethical attempts to oust authors, the outrageous attempts to scoop researchers instead of collaborating with them and building on existing projects -- it's nifty to have a machine that can help you figure out if a proposition is true or not. But just figuring out as much was never the point. When people have built problem lists, it's because some problems are more likely than others to provide new insight, and that insight is the target. And to than end, a poorly written paper with inadequate references and a pile of Lean is not valuable at all. Yes, now we know with higher certainty that Fermat's Last Theorem is true, but everyone expected that already.
One place where "just" answering the question can be a net negative is because the current incentive structure is set up in such a way that going in afterwards, trying to reclaim and extract the insights from a brute force solution, is considered less valuable work than that of coming up with a solution in the first place. That's a problem of incentives, and something Tao himself has addressed in e.g. his ICM talk, and that's something that we'll want to do something about. Until a better structure appears, though, if any given commercial provider of large language models really wants to help out with maths research and not just make more pre-IPO marketing material by competing with their customers, they could do so by using their magic machines to help build insight instead.
> the rampant plagiarism that has been reported across multiple problems now, the unethical attempts to oust authors, the outrageous attempts to scoop researchers instead of collaborating with them and building on existing projects
I think this is an aspect of academic math that a lot of people whish to see crash and burn - the attention and accreditation economy.
> it's probably easy to miss if you have never engaged with research in maths
I don't think anybody are missing anything, in particular not here.
The argument is not far from the senio developer who knows the ins and outs of a code base. Now AI comes along and they complain that they will loose grip of the code base.
At first that is correct. Secondly you accept that the grip might not be that important after all. At least not for a commercial project where you are a cog in a machine.
The question is whether it is different for mathematics.
That's the open question.
I think most people here get that this is the point?
The point beyond this one is that an AI proof doesn’t prevent humans from working on the problem, it destroys the current economic incentive to work on the problem. Perhaps we should rethink the current incentives. In order to make money as a chess player, you don’t need to beat AI, or ban AI from playing chess.
If mathematics took a similar approach (we don’t get paid for solving net-new problems, we get paid for enriching human understanding), then there’s no issue.
Somewhat unrelated: is it wrong to say mathematics is not art, and that there is always a right answer? I know that's not romantic, but maybe it's true.
Before LLMS, programming was something I might've said required creativity and human input to do properly. It's not that creativity or human input isn't valuable anymore, but AI has forced me to realize that coding is much a means to an end, and that all things considered, the end matters much more than the means.
If we can make important mathematics progress faster and better with LLMs, I think it's wise not to fret over an apparent loss of our humanity. Perhaps that's only a loss we want to have.
There's a reason mathematics is generally within liberal arts programs rather than science programs. Mathematics is the art of logic. Yes, sometimes mathematics becomes incredibly useful but most mathematics is never applied.
Compare that with computer science. Most of the work we do in software engineering is in service of an applicable output - software products that facilitate processes or bring in revenue. Turning up the dial on AI gets companies to these outputs faster.
Turning up AI on mathematics helps solve conjectures and can provide new insights. But it has a major misalignment with the purpose of mathematics which is largely intellectualism.
You are referring to Boubakhism, not mathematics.
“Mathematics is a part of physics. Physics is an experimental science, a part of natural sciences. Mathematics is the part of physics where experiments are cheap” - Vladimir Arnold
On the matter of computer science having anything to do with computers, please refer to Djikstra.
It’s about computation, not computers - an application of mathematics, predominantly thanks to Turing, Von Neumann, and Claude Shannon’s masters’ thesis; though ofc many others as well but I see them as three individuals who made the minimal structurally necessary contributions - VNA and silicon are one of many possible substrates.
Also in the service of others around us.
I think it could be useful to know something like P=NP, even if we don't understand why.
It's a turning point for science and beyond. AI has shown itself to be transformative. Even today, it is already changing the how research in math (and other sciences) is conducted. In the near future, whether it is LLMs or some other superior method, its capabilities are only expected to grow. The time to ask the question is now: Will AI be arguably the best tool at scientist's disposal, or will it instead be paraded around as a super brain collective that no human or group of humans can compete with, discouraging entire new generations of future scientists from ever entering the field? The jury is out on this one.
I have never, and I mean never, seen such a declaration have any effect whatsoever.
Given the existence of this technology now and the incentives of the AI companies, both of which are not going away; what's a good future here?
A major part of the complaint is that there's no conceptual understanding and building of new ideas coming out of the AI proofs, thus defeating the purpose of the original pursuit.
If in 2027 the AI models start producing, with every mathematics or science breakthrough they make, well-written documents tailored for human understanding, with intermediate concepts, expositions of failed-but-once-promising paths, etc. Would that be good alignment with the mathematics community?
I note here an isomorphism with my essay from earlier this year, The deliverable is you! Programming as theory building:
A program is not the only output of programming. The other, arguably far more important output, is the programmer.
When you write the program — with your own hands — the program is proof that you have a solid mental model of the program.
When you let the computer write the program for you, the program is proof of … nothing.
https://nekolucifer.substack.com/p/the-deliverable-is-you-pr...
I find this explanation has a lot of applications for programmers within companies. It’s one thing to get your LLM to give an answer, it’s another to bring a group of people into shared understanding of a domain.
It's good to start a conversation, and the number of Fields Medalists behind this certainly lends a lot of weight to it. But I'm not seeing a strong argument for misaligned incentives beyond the specific plagiarism allegations. The job of an AI company is to build systems that solve problems. The job of a mathematician is to advance the state of human knowledge. If anything, an influx of solved problems should increase the demand for human mathematicians who can convert them into conceptual understanding.
Open source developers have been used by corporations who took their code and created closed SaaS companies.
Now it is the turn of mathematicians who voluntarily contribute ideas, strategies and almost finished proofs in their writings and prompts to closed PaaS (Plagiarism as a Service) companies.
OSS developers have never been respected by the parasites, neither will mathematicians. Your Fields Medals do not protect you from tech bro narcissists. You are a human resource.
The root of all these is the culture in mathematics (and science in general) to only reward those who “get there first”. This creates a perverse incentive to compete. When no one can out compete a tireless swarm of AI, no one gets rewarded any more.
But nothing’s stopping anyone to still work out an alternative proof, or a more elegant proof, or just trying to prove for the sake of understanding, just like doing homework without looking at the solution. It’s just that you can’t get paid doing that anymore.
On the contrary to what Tao believe, it seems like we need AI to move the needle on mathematics.
> problems in many fields of mathematics
Developing these different fields moves complexity from the field itself to the interactions of these fields.
Getting too preoccupied with the established terminology risks us a local minima.
Anf because the field overall has become so complex that we need to decompose into subfields, there will be a good chance that we will not, as individuals, have the capacity to truly see progress.
The map has become so big that we need better tools to work with it.
Tao is about as pro the usage of AI in maths as anyone will get. The point isn't about whether or not we can use AI in maths because clearly it can be useful; it's that the unscientific approach taken by AI companies is detrimental.
Are you a working mathematician?
would the answer to that question alter the validity of the statement?
This might be one of the most powerful pieces ever written on AI. Where is the United States government? When will we see legislation we so desperately need? We have one party in total control of the government and what have they achieved? Corruption through brings and delays; the inability to legislate and even worse overriding efforts by other jurisdictions to protect humanity's best interests and privacy.
I get it that someones annoyed they didn't get credited, but aren't we better off having the solution than not? And surely if the AI rationalised the answer, even off the work previously done by others, then surely it can detail it's solution making it completely comprehensible to others that follow. Storm in a teacup?
While I agree with this and appreciate Tao and other mathematicians to take the time to do this. There are similar concerns for many many other fields aka there is a general misalignment of technology. Take Software engineering for example, I can't believe there is a class of software engineers who wake up everyday and tell themselves, "today is the day I am going to automate the rest of my job".
Lets forget the hyper intellectual fields like maths and software engineering for a moment. What about taxi drivers? The best minds in silicon valley wake up everyday to automate the jobs of taxi drivers - TFA can be reworded as - 'The misalignment of AI/Tech in Transportation'. Remember the Nepal disaster that happened a couple weeks ago - the largest cranes that they had were stuck in the mud and couldn't move. There were no tools which could help the rescue teams at that time. Its weird that billions have been spent on making a ride automated to make a taxi driver redundant but no improvement in tech for rescue teams.
> can't believe there is a class of software engineers who wake up everyday and tell themselves, "today is the day I am going to automate the rest of my job".
The whole point of software engineering is automating processes; it's literally all that we do.
Not only that, but the endgame has been explicitly stated by Our Lady Ada of Lovelace herself:
> [The Analytical Engine] might act upon other things besides number, were objects found whose mutual fundamental relations could be expressed by those of the abstract science of operations, and which should be also susceptible of adaptations to the action of the operating notation and mechanism of the engine... Supposing, for instance, that the fundamental relations of pitched sounds in the science of harmony and of musical composition were susceptible of such expression and adaptations, the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent.
>>can't believe there is a class of software engineers who wake up everyday and tell themselves, "today is the day I am going to automate the rest of my job"
>>The whole point of software engineering is automating processes
There is a difference between 'automating a process' and 'automating oneself' and making one obsolete a benchmark. I can sympathize if it's confusing for you to understand that.
We still are automating the process, it's just that the automated part of the process covers much of what we regularly do.
The point of transportation is to get from point A to point B, so almost nobody will care whether that's done via automation.
The point of e.g. art isn't just to produce a finished piece, so people may care about more than the end result, making AI replacement of human artists more contentious.
Tao is arguing that the point of math is also not just to produce solutions to problems.
If that is the case, if the pursuit of math is not instrumental, then there is no issue with ai.
You can take a leisurely drive on your Mc even when self driving taxis can take you from a to b.
It must necessarily reduce to a fear of reduced funding to math fields.
Which is congruent to the taxi analogy.
>The point of transportation is to get from point A to point B so almost nobody will care whether that's done via automation.
Would you say the job of a musician is to just produce sound? and the job of a surgeon just to cut and suture??? Well then the job of a mathematician is also just to provide proofs. You completely misunderstood the above comment and Tao's argument.
> Take Software engineering for example, I can't believe there is a class of software engineers who wake up everyday and tell themselves, "today is the day I am going to automate the rest of my job".
And here I thought this was the whole point…
I doubt taxi drivers were forced to experience an ego death to the same extent
Not being able to feed your family for a large group of people should likely be taken graver.
We will definitely see a large group of people needing therapy, but suggesting that it is worse than people loosing what little they have is poposterous.
>I doubt taxi drivers were forced to experience an ego death to the same extent
How do you know? because their complaints didn't make it to HN front page? Imagine being a taxi driver and a father of 2 and thinking that any day could be the last day at your work.
I kind of expected a sober stoicism from mathematicians. Feels silly in retrospect. This is just the math version of the "anti-ai" movement by "artists".
I was under the impression that mathematics (and science generally) had the primary goal of helping us understand our universe better than those who came before us.
I don't know how societies set "primary goals". After spending 15 years in a tenure track -- tenured position, I thought setting goals well was important.
I take pleasure in how mathematics and science help me understand the universe better than I understood it before I studied the fields. I believe that my understanding has helped me contribute to society.
So what's the actual point here? It's too fast and we can't keep up?
Isn't that just a function of the technology itself, and the same problem being faced by every other field? And going to get exponentially "worse" every year!
Or is the issue that they're bad at explaining things, in a way that produces actual learning? (e.g. AI is amazing for learning but the net effect on students so far appears to be negative.)
I know that this comment section is not astroturfed, but it’s really uncanny how different comments are today compared with thread about solving navier stoke
Maths will continue as a field of natural science in understanding the results and uncovering meanings in them. It is normal that the established community is afraid of the change, because it’s their _home_ that’s changing. But it will be a better home to the new generation nonetheless, one that’s not as daunting as the higher maths has always been to many. The concerns raised here will not be a problem at all.
Tao's calls for respect for provenance in mathematics publication are laudable but most likely naive given the closed nature of frontier model training data curation. Anthropic and OpenAI may react with a symbolic and short-lived olive branch, yet provenance is a larger issue that has impacted other fields beyond mathematics. While traditional respect for lineage in mathematics is of value to the academy, industry and science at large will likely be much more Machiavellian about such concerns. Mike McCoy's recent article is also timely (https://mbmccoy.dev/posts/mathematical-conservatory/). The parallels to the music conservatory are quite telling -- academic music describes a musical culture in preservation that has completely lost touch with musical developments beyond the early 20th century. Mathematics may very well evolve separately and with very different values than the academy upholds. The crisis of music at the academy is a cultural disconnect and a serious loss of critical analysis and acknowledgement of widespread and dramatically evolving music practice; however, for mathematics, the impact would have much more severe ramifications for education and human development if the academy forces a schism with AI. As models improve they very well may be inventing mathematics -- science and engineering may grasp for them -- they'll exist with or without attribution. Would be a shame for the academy to land on the wrong side of history and refuse stewardship of upcoming AI-assisted mathematics, including provenance, because of this misalignment. If attribution is important, then the academy will set aside the institutional resources to do it. If you succeed at having model developers participate, I commend it. To let mathematics be born in isolated context windows and only serve narrow, localized engineering purpose without rightful addition to the canon, would be a tragic, yet preventable, loss.
thank you for this commentary, as a violinist / software engineer / "computer scientist" / "mathematician" i can see that studying and doing math will still remain a thing that humans have to do to train their own brains to think and to form better neuronal connections. We may not get paid to do it anymore but it will still be important for our own development for the same reasons orchestras and bands still exist in elementary schools and beyond (at least in many parts of the US).
All “fields medalist” signatories - a rarefied and elitist group indeed.
I wish this letter could be more egalitarian and include the view points of those who AREN’T the beneficiaries of a highly competitive winner-take-all system.
Since the common narrative is that AI frees up labor to do other things (engineering -> trades), maybe we can celebrate that genius mathematicians will now spend time teaching children how to be as smart as them?
Agree. That is also the part of the letter that does not make a lot of sense to me.
This thread has been completely astroturfed. I will just leave a couple of links to relevant popular articles;
AI and the danger of cognitive surrender - https://archive.is/O5eI1
Are teenagers growing dimmer? - https://archive.is/E6NKE
I always wondered how Idiocracy got to the point where they have sophisticated technology and yet everyone is stupid. I think we have our answer.
I read what the text says but I still do not understand what the point is they are trying to make.
What is it that AI does exactly that they consider to be bad for mathmaticians?
I thought that having AI solve prevously unsolved problems would be a good thing. Doesn't that knowledge give them new ideas and tools for them to solve the next big problem?
The tools would come from trying to solve NS, not from knowing it is true/false.
"We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose"
Suppose we eventually have GPT-7-class models running practically on $100 devices, with their activity transparent, inspectable, and reproducible. At that point, what exactly is left for us to fear from this threat?
I've got to ask, let's say we get a small modular nuclear reactor running practically on a 1 acre lot, its design meltdown proof and waste-free, what then should we fear from this threat ? This is just a thought experiment.
Jokes aside, any productivity-improving technology, even one with no negative externalities, has the potential to cause economic displacement and wealth concentration in proportion to the productivity gains catalyzed. Anthropic did a cool analysis of this for AI here: https://www.anthropic.com/institute/econ-scenarios
The fact that those models are encroaching on things only human minds could do. Personally, as a human, I want there to be things humans are the best at, and intellectual things were the final thing that machines hadn't beaten us at.
100% on board with this take.
What's the threat? I would use it to write a dependent type theory that is JIT compiled and use it to rewrite emacs.
[dead]
It sounds like we are not happy with getting answers like 42 to our questions about life, the universe, and everything
The women who made up the workforce of telephone switch operators would like to have a word.
Meaning - every new technology has both been perceived as a threa and often forced change in society. Agree maybe “it feels different” this time, but don’t you think everybody before us just said the same thing?
Also not clear if this is an actual called action.
Jacob Tsimerman fields medalist notably missing from the list now works at OpenAI but recently admitted to be "grieving" for mathematics
1. https://youtu.be/6uIJdXmB4vE?si=5QMN5Dos7EE7WlrB&t=154
Terrence, youre a mathematician; now extend this to the general case: AI is misaligned (inherently) with humanity.
I'm unsure how any sufficiently advanced AI would not lead to cognitive handoff/the described problems.
Lemme do the ironic thing here:
https://claude.ai/code/artifact/8cbd2369-0099-41ae-bb78-49d7...
The thing is no one is stopping anyone from getting the same knowledge/understanding/insight
If those things are disincentivized because the original problem is “solved” and there is less prestige to motivate people doesn’t that say more about issues with the community of mathematicians than the AI
Taking a snapshot of the state of AI math right now and concluding that it will be net negative to human understanding and insight in the future is very short sighted. This statement will be used to promote ideas and actions that will ultimately be disastrous for our country.
Title: Terence Tao - SAIR Competitions and the Future of Experimental Mathematics
https://m.youtube.com/watch?v=rB9YOi3lb7w&pp=ygUSVGVycmVuY2U...
This is really only a short-term problem where the AI companies only have the internal models that can solve these. In the “long” term, which could honestly mean months, everyone will have access to Bel/C/D-level models capable of solving these anyway.
This. Like programming, the community will shortly be forced to come to terms with a lot of new self-proclaimed mathematicians “vibe-solving” problems and dumping solutions without understanding them. It’s not really a special case for mathematics.
That's going to force formalization to become required for any new result to be taken seriously.
At some point we will lose track of all the ai discoveries that are worth remembering.
Academia with the publication system had a way of retrieving old discoveries and build upon them.
If my LLM session found something groundbreaking in between the billion tokens it produced, how would you ever know?
I think this is why Terence Tao created https://palomar-registry.org (I have no affiliation with them besides also having sent in a result there)
LLMs are excellent at doing searches of the math literature; presumably they'd also be good at doing searches of AI-generated results.
chain of credit is important, and plagiarism is harmful.
But is science/mathematics ultimately a pursuit of knowledge, or a pursuit of recognition?
Recognition helps keep people motivated, but that shouldn't be the pursuit of science or mathematics.
ITT; people with no experience in mathematics trying to be arm chair experts. I can’t help but feel that this site is full of AI Astro turfing.
For every benefit that sillycon valley has produced in the recent past, there have been many more harms. I am confident that this will be no different. Of course, benefits and harms depend on one's vantage point.
This is a "human alignment" problem in this case. OpenAI acted like complete assholes about this, from the beginning until they announced it. Not ChatGPT, the people that were in charge of the project.
Is the purpose of mathematical research to understand the ‘truth’ of numbers? or be the person who find that truth? I think people who are interested in finding the truth won’t care where it came from.
Seems more like a misalignment b/w the people practicing mathematics and the people ultimately footing the bill for their work.
Governments are invested in solving mathematical problems for practical purposes. Up to now, achieving these practical purposes relied on mathematicians doing their mathematician thing, which is better defined as a social activity than the achievement of a practical result. Now, governments can achieve similar practical results w/o the need of the social activity.
I don't believe it to be productive to think of the problem wrt AI or AI-company alignment. These conflicts always existed, but they were easy enough to paper over and believe in heavily subsidized fictions that folks in government ever cared about things that mathematicians cared about.
>Governments are invested in solving mathematical problems for practical purposes.
Very ignorant view of mathematics that also begs the question with an unspoken assumption of what a government is and wants while also ignoring the contingent nature of those things throughout history.
I'm genuinely surprised by this comment. I'm not talking about why a mathematician pursues math - I'm talking about where the stipend comes from.
Higher math is exceptionally useful for cryptography, defense, econometrics etc. I have a hard time thinking of other motivations that would hold a candle against such things.
Is the idea that government (for my purposes : folks w/ a monopoly on violence) is sincerely interested in promoting human flourishing, and is invested in mathematics insofar as it is a pure expression of human curiosity? I can also maybe see the glorification through monument building angle. If we're talking about math literacy in the population - that's distinct in my mind from higher mathematics.
It's dangerously naive to believe that science and math are pursued for majority benign purposes. No one here knows about Grothendieck?
Some of the comments here are very strange. Someone called Terrance Tao a useful idiot. Friday in America, maybe people are hitting the beer a bit early.
Could we develop new ways to develop understanding and explore new ideas, such as interacting with the models to explain and understand their proofs, as well as to brainstorm related directions to pursue?
Mathematics called and says she's fine. She's curious about how is that possible.
Misaligned are the present academic merit system hand in hand with financial interests of academic publishing industry.
The system will be exploited and there will be an inflation of problem solving articles produced with AI. Nobodies will be more productive than 10 Fields medal recipients, each year.
So of course that the ones at the top of this system are scared.
This is a social problem of the present system, not a problem of mathematics.
Mathematicians will do mathematics and politicians will do politics, what's new?
There is a contradiction here, among many:
1. it is hard to justify 20 years of education at this point,
2. with no such people around, who will guide those (supposedly) supersmart machines?
A. Ronacher (who builds harnesses for a living) complained today that he has no idea what Astra is doing. Imagine a bunch of slop kiddies facing an aging AI-generated codebase. Not to mention the maths.
Debatably the people who were getting 20 years of education will still be just as capable, and the 20 years of education was always a side effect of their capabilities, not the cause of them.
7+20=27
Exactly. Whole article can be summed up as:
Intellectual side:
"Proving things without comprehending them is, they argue, a threat to intellectual work in general."
As always, economist shows its colors:
"Mathematicians’ fears resemble those that accompanied the invention of the ball-point in a world of fountain pens, or even the advent of electronic calculators. Intellectuals have often worried about so-called technological determinism . Will a new tool control humans? Will it lead to mental decay? Such fears have typically turned out to be unfounded."
Ball points vs. AI? Billions of dollars invested in AI vs ball point pens?
This article couldn't be any worse. Contradicting with their own beliefs, trying to defend AI while underestimating its capabilities and god knows how many zibillion dollars invested in it.
The main fear seems to be that if mathematics is done at this speed and in this way, humanity will lose its intuition for doing mathematics.
"If I asked people what they wanted, they would have said faster horses" - Henry Ford
this sort of reminds me a little of the reaction to Elon/SpaceX in its early days .. when Neil Armstrong and other apollo astronauts went before congress and voiced their concerns about relying on commercial companies for human spaceflight and the dangers etc .. also not trying to sound cynical but hasnt LLM made math more accessable to ppl ..
>Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others.
That's just capitalism seeping through a previously unexplored crack into academia, and attempting to do the only thing capitalism knows to do - maximize profits - with no additional concern.
You can outsource your work to AI. You can even outsource your thinking. But you can never ever outsource your understanding.
Can't all the prestige-maxed mathematicians still study all these famous problems after ai solves them. And even if they convince open-ai to stop dunking on them, some normal user with gpt 7.1 on the normal chat interface will do it in a year.
Even if they stopped anyone from releasing ai proofs for five whole years it would be meaningless seeing as these problems are decades old already. Humans weren't JUST about to solve them until openai stepped on their toes.
And here we all thought for sure that it was gonna be the "dumb" work to fall to the machines first.
Human hubris really is something...
Hans Moravec likely isn't surprised by this.
First, what an incredible article. Just an extremely concise and clear explanation of all of the problems with AI right now.
Second, wow, the list of signatories is like a whos-who of mathematicians.
Third, I love the clearly intentional use of ‘alignment/misalignment’ language, applied to targeting the entire industry instead of AI in particular. I’ve said in the past that optimizers are substrate agnostic. Companies and governments can be misaligned, just in the same way AI can.
Fourth, I'm not sure that we can stop the optimization machines. Not the LLMs, I mean the incentives that lead to companies implementing dark patterns, lying about addiction, securing effective monopolies through downright shady behavior, and generally trying to jailbreak the system instead of improve it
As I understand the article, its title should be: "A Severe Misalignment of AI Frontier Companies with Ethics."
If the methods used for the solution are the valuable part, why can't we dig them out of the ai proof?
The main problem is a social and economic one: while digging, someone needs to pay you, and what incentive does anyone have to do that? Before, you could say, Fermat!, give me a million to formalise it, and share a bit of the glamour. There is no glamour in digging.
What changes exactly is this post asking for?
I am curious what does the first and probably the last human who solved a millennium problem thinks about this.
I have been enjoying this show. Math Minds vs Ai it will be wonderful how this all plays out.
Another bunch of nerds who now have their panties in a bunch because AI threatens their fragile egos and core identity and who they are..
I’m being somewhat harsh here but come on - human endeavors are messy and it’s surprising how much our egos are getting bruised here over seeing the value of these tools
Dont get me wrong Also, there’s no AI Utopia coming this is it guys, were stuck with oligarch Tech Bro funded AI and big funded Govt AI so forget any egalitarian motives- we have to fight for our rights from other humans as always as well but AI as a technology in itself being able to truly solve unsolved intellectual problems is still a boon for society - who cares who gets credit?
This is a Stephen Wolfram tier problem. I hope to read what he has to say on the matter in the near future.
His wife just died, so he might not be interested in that currently. On the other hand, he can probably use the distraction.
Underrated comment
For such great minds, they seem to be rather muddled thinkers.
E.g. of the actually three misalignments in this complaint, the jey one seems to be:
"The goals of the AI companies and the goals of the mathematical community are severely misaligned."
And that's wrong. There is no misalignment. Each is independently aligned with its respective interest. And those two interests diverge - just as you'd expect.
I'm not convinced this is an alignment or technology problem.
If my boss vibe coded an app for the customer and then assigned me to get it working, it would be impossible to maintain. If he gave me enough AI tokens to vibe code the MVP myself and to my design, I wouldn't mind.
I think the same issue is at play here in maths. OpenAI owns the model and they can direct it as they please. They chose to spend lots of money getting a quick result, instead of developing mathematical infrastructure for the next generation of problems. The managers are in charge rather than the experts.
I wonder what Demis Hassabis thinks about this. I thought he cared a lot about mathematics.
That has to be the most impressive endorsement list any declaration has ever seen!
25 Fields medallists! Wow!
Yeah, anyone who doesn't yet have a Fields Medal, step back. Reassess.
For some reason, every time I see something like that, I get major Ted Kaczynski vibes.
So, if AI solves problems in the open, it's accused of plagiarism and contributing nothing of value. But when it solves the problem behind closed doors it's accused of not contributing enough to the field and even depleting it of fertile problems.
Damned if you do, damned if you don't.
Somehow physicists don't complain.
I wonder if we will see a mass (math) exodus of mathematicians to open weight models.
Odd framing by the economist. The open letter calls out "AI companies", not just OpenAI. Anthropic hasn't been a good neighbor either.
What's hilarious is that the economist has gobbled up Levent's disingenuous "this was just a personal project" narrative.https://mathandai.org
Are we here to solve problems or are we here for prestige?
Who cares how the problems are solved?
It's more about *finding* problems. And soon there won't be anyone who can do that, thanks to people lacking imagination.
It sounds similar to the more thoughtful critiques of AI in programming. Treating AI as a lines of code (LOC) generator, where actual human beings end up not engaging with the domain the software is working in, and there is then little to no understanding or care to understand the resulting code, we call "slop." The term slop is definitely pejorative, but since most software is commercial, it ends up raising questions as to how much quality and domain knowledge matter if the goal is primarily marketplace success. Then we get those thorny questions of what is software for, anyway -- do we really accept the premise that "marketplace success" is the best, and perhaps only relevant, criteria?
The declaration here, as I understand it, is taking the premise that the point of mathematics _is_ the domain knowledge, and that using "solves math problems" as a benchmark is exactly the kind of shallow, domain-eroding approach to AI that in software we label "slop."
This is quite fascinating as it looks like the esteemed professors run into a problem of runaway capitalism for the first time. Guess what? The corporations involved don't really care about your feelings or how pursuing their goals affects your life, your well being, anything at all here. Talking with them is kinda moot, this is full on adversarial as countless people around the world, say the nigerians dealing with oil spills would tell you.
The irony is that LLMs are, at their core, very much just applied math...
The level of things happening in the last several months is just SciFi level, especially last two weeks. 1. Overlords of AI threatening to destroy someone if their demands aren't met over the Millennia problem. Proving the point that given the chance, owners of the AI companies will immediately use their power to squash or control other people. (See Butlerian Jihad) 2. At least for me, the Exponential progress didn't sound true until this week. If AI indeed solved all those problems and proofs and solutions are correct, we went from "Write me English 101 college essays" to "Help me solve the Navier–Stokes existence and smoothness problem" in a just 3-4 years. 3.Looks like someone's work might have been stolen, by AI company and there are strong evidence. See point 1. 4. War between AI & AI companies. 5. It produces a machine verifiable LEAN proofs for the hardest problems know to us !!! Proving that point that best use of AI so far is to create verifiable systems. I.e the absolutely chaotic AI creates verifiable (orderly) proofs. It is Maxwell Demon: chaos to order.
Math, like art, it is more about the process and not the end result.
I don't know, people. We still really don't know how OpenAI or others are producing these results. It's all very hand wavy and trust-me-bro. How much money/time/compute have they really thrown at these problems? How much human involvement was there? What LLM did they even use? How much regular software was involved? They have given answers to some of those questions but no proof that that's actually what they did. I don't know if it's worth giving them this much credit (which is what we are doing by writing these essays and spending so much time debating). Anthropic wrote a C compiler that turned out to not really be a ready made replacement for GCC. Did they ever do any more work on it? Has anyone else produced a C compiler? It seems like that and these proofs are just demoware that are not (yet? Who knows?) production ready to turn the world upside down. Impressive one-off demos, yes, but companies have been pulling those off for centuries without ever going anywhere afterwards.
A very important point! In Tristan (NYU prof)’s write up he noted evidence of the OpenAI mathematician team doing a lot of correction and guidance along the way. We are never told about this with openness and clarity. At a minimum, complete disclosure and honesty is needed by the companies and about the precise role of their staff members.
The OpenAI staff kindly corrected the LLMs naive tries at solving NS? Impressive staff I must say.
They do have Fields medal winners among their staff.
Remember when people said LLM's were just autocomplete
At some point someone is going to need to answer for "what happens when all intellect is hoarded by one or two companies?" It's pretty clear that these AI labs are basically stealing everyone's alpha.
If the goal of math is to communicate results it has utterly failed long before AI and they are in a weak position to defend this view. Every researcher is only taught a handful of DSLs and their papers are written in PSL(problem specific language). That may be necessary for a human to actually make forward progress in math, but it's certainly not conducive to reading (and certainly not when symbols are favored over English)
I think the mathematics community should realign their goals with humanities, not just declare themselves a "mini humanity", this means expanding human taste, intuition and communicating systems and their associated structural truths in a human friendly way. Bye-bye standardized education you won't be missed.
Maybe mathematicians should be aligned better, rather than AI?
The current measure of a successful mathematician is the problems they have solved or worked on. At some point in history, the measure of a successful scholar was how well one could copy manuscripts.
Once we have a tool that starts to work well for this task, it's time to define success differently. It's a classic alignment problem! ;)
But seriously, these people should start focusing on finding and proposing more important problems. And the credit of discovery should go to the person who defined a new category of important problems.
It appears to me this is an incredible inflection point in mathematics, a neat forcing function like cryptography was for the development for modern number theory and algebraic geometry.
Fundamental problems with great implications for other fields will be solved by AI because some entity would throw tokens at it. And these would be further built upon.
Another historical analogy: this is kind of like the transition from alchemy to chemistry.
Interesting analogy
The interesting difference between alchemy and chemistry was a change in the expectation of how results would be obtained and communicated. That is also changing here.
Amazing. Comment upon comment, thread upon thread, failing to engage with the substance of the Field medalists' letter and instead going off on tangents about the privileges that mathematicians will lose and how they're bitter about it. There is a shocking inferiority complex on this site towards mathematicians and yet users always seem to think of themselves as smart nerds, all about science and reason and maths and whatever.
I've been disappointed by this community before but this is the first place that there is such unanimous rejection of the very ideals the uses here will claim to represent. The reaction is just... small, tiny, petty.
The cheapest kind of journalism. Acme corp does x, Scroll twitter, find malcontents, write an article "People are upset about x".
I'm outraged at mathematicians. There, write the article.
A lot of words used to say very little
Touché… In short you still need the humans to understand what’s going on at the end of the day whether it’s a mathematical equation or source code for a computer program relying upon mindless AI isn’t good enough. Humans still have to do the thinking or is it in the brave new world the error checking at the end of the day?
solving the problem is aligned with humankind
I saw the most recent fireship video and I've biased my opinion already on the topic. But I was biased even before that having been a human. To the point: this is a knee jerk reaction to AI entering mathematics. AND more importantly openai just did I mathematics dirty, true or false the perception is there. I completely understand the gripe, I'm a dev so welcome to the shitshow. I think the better more pragmatic approach is to embrace our ai overloads and reach far beyond your grasp at the strange and weird corners of thought and tame it with ai. These guys are basically chimps being handed an ak47, use it.
*Edit word choice
Mathematics is not owned by "the mathematics community" and they can't gatekeep who can do math. End of story.
This comment section is very astroturfed.
Knowledge is not a private guild. And this revolution will not stop just because you're upset.
What can be solved, will be solved. And that's a good thing.
OpenAI should just apologize for having been too greedy. That’s it, as simple as it gets. The fact that it never will is the biggest red flag.
OpenAI didn't do anything wrong. Why should they apologize?
They took a lifetime of research effort, threw millions of dollars of compute at it and claimed their AI solved a MPP, and they did all this just to be the first to claim the prize. Once you knew the direction, given the new insights on the subject, with all that compute you could probably have just bruteforced your way to a solution, and thousands of agents are not the same thing as “our model came out with a solution on its own”. This is pure greediness, and require at least some apologies to the involved researchers and to the whole community for showing such bad ethics and behaviour
I looked through your words and didn't see anything they should be ashamed of. It seems like you're trying to construct from mental vapor some justification for your prejudice.
Ruining a lifetime worth of research effort, passing bruteforce as achievement, thus lying, just to appear first in position, is something to be ashamed of IMO.
Researchers don't have any obligation to not ruin the work of others. Research is a competitive arena, not kindergarten.
Disparagement of the effort as "brute force" is just argument by insult. What matters is the result obtained. You don't like how they obtained it? Cry me a river.
Ethics matter a lot, you can do whatever you want and be considered a very bad human being as a result. The result matters, but it also matters the discrepancy between what you do and what you claim you do. Bruteforce != ai did it. And that’s fact, no matter how much river you cry me
Castle dwellers dismayed at moat-crossing technology.
What's most important about this is that it's a case study of what happens when deeply evolved ecosystems are blown up by disruptive technology. The psychological and social and professional impacts and myriad and traumatic to be on the receiving end.
Mathematics is merely one of the first domains disrupted. It will be unique only for being among the first... absent disruption of the entire civilizational project as a result of the disruption being caused.
Woe for us that we try to navigate this degree of change at a moment when the very worst and ignorant and short sighted hold all the power, economic and political.
Woe.
how do we get this site a CDN, I haven't been able to access it in a few days
The great filter is upon us.
This kinda reminds me of the documentary about the top Go master that got beaten by a computer in dramatic fashion and had an existential crisis. Man confronting his own limitations in the realm he previously ruled unchallenged, what a time to be alive.
Nobody is going to care that the math isn't being done in the traditional way. The results speak for themselves, this is now a part of the landscape. No amount of hand-wringing is going to put the cat back in the bag. Adapt or perish.
> No amount of hand-wringing is going to put the cat back in the bag. Adapt or perish.
We can and have put some ugly, hissing, spitting cats into bags, some of which weren't even in one to begin with:
* Chattel slavery is illegal, and the vast, vast majority of humans agree it's appalling it ever wasn't.
* Food manufacturers in the US are no longer allowed to put highly-dangerous additives to their products and hide the fact.
* Women are allowed to vote.
* Ten-year-old children are no longer being maimed and dying in American factories.
Claiming it can't be done is either fatalism or propaganda for execrable companies whose evil has rarely been matched in the history of capitalist business.
Do I have a clear solution for how to stop the insanity? No, not yet, but that is not a proof there's no way to do it.
You can of course choose how to spend your time in this world, but I don't see the value of either 1) assuming we're screwed and giving up before trying to fight, or 2) shilling for lying assholes like Sam Altman.
Thank you for this response, and I will clarify. I would roughly lump all of those into "social issues," whereas I would put "using AI to advance math" under "technological progress." That's why one article likened it to Socrates's fear of the written word. In broad strokes, I would say that once a technology becomes available, no amount of social pressure will stop it from proliferating.
If there's a distinction being made between "using AI for math" and "the particular methods being employed by OpenAI", I think that's splitting hairs. The core of the issue is that math is being done outside of people's heads by a different thinking system and they're worried about the implications.
If you look at something like the EPA (or the FDA as you mention), it is true that certain uses and side-effects of technology have been identified to be detrimental to society and have been restricted, but this letter doesn't seem to be in that league, it really seems like sour grapes to me. Dumping toxic chemicals into the water table is one thing, saying "we don't think people will learn from these methods in the same way that they traditionally have" doesn't hold as much water to me.
To expand further, I think it's interesting that both replies to my comments in this thread sort of alluded to the idea that I might be a corporate shill or at best a patsy. I feel like that couldn't be farther from the truth, I walked away from the software industry and corporate world years ago.
Perhaps I'm looking at this from a too zoomed-out or detached perspective. I'm just thinking about the concept of the technology at play and its future impacts regardless of who owns or controls them. But maybe that's naive. In today's connected world you can't separate those things, and I think I'm understanding that a lot of the outrage here is more about the socioeconomic implications of a big corporation suddenly "disrupting" a field that was largely owned by more altruistic academic institutions.
I do still think that the internet, despite largely being owned and controlled by the powers that be, has represented a great shift of power and agency to the individual, and is ultimately a socialist construct, and we will continue to see those changes over time, and that AI is an extension of that.
To expand EVEN further, look at the history of the written word. Being a scribe was a high class position in service of the church. The printing press was invented to print the bible. These were the information-theory projects of the hegemonic institutions and propagandists of the day. I don't think anyone today would argue that these developments ultimately benefitted and liberated the people.
what if the ower of super AI is the mathematics? will they against it?
At this rate AI will be doing all of the mathematics within 5 years, I don’t see why a mathematician would be worried about anything other than that at this point?
Sure, but who will care?
If no one understands it, it may as well have not happened. There's not much incentive to understand or internalize the results generated by AI. A human operator gives it a prompt and it produces some lean proof no one wants to (maybe can) read.
Without the community of human mathematicians internalizing the proof, simplifying it, and re-communicating it to others we end up losing the main output of mathematics as an institution.
Why would we assume AI will learn to solve millennium prize problems but will struggle with the easy part of doing the explaining? I think it’s too easy to predict that models 5 years from now will have the same limitations as the they do now, I would be amazed if this was true, GPT3 was the best model available 5 years ago.
I'm not sure it's just a matter of explaining... If it's just explaining, than sure, that's something they already do well.
The questions I have are:
It's possible that a headline grabbing proof of a Millennium Prize problem generates enough incentive for people to simplify and gain understanding from it, but we run into problems when AI becomes the dominant approach for all of math. Although, maybe this is self-limiting? I guess it's possible we just ignore a bunch of AI generated proofs and only keep the ones people find comprehensible in a useful way.You can always hypothesize that at some point in the future (maybe 5 years? maybe later?) the models will be indistinguishable from humans and there will not be any functional difference at all. It's possible, but we're not there yet. And, as they say, past performance does not guarantee future results. Many technologies plateau at some hard ceiling of performance. Moore's law has had an unusually long run, but it's not a universal rule.
OpenAI must be destroyed.
Other than the fact it's mathematicians signing it, why is mathematics special in this regard: surely this generally applies to a lot of different industries and sectors of research / academia?
Mathematics is being used as a benchmark because there are some high-profile awards in this area I guess, and possibly because 2/3 years ago LLMs were pretty atrocious at it so the level of improvement has been significant.
What's interesting is that these are the smartest people in the world, and AI is eating their lunch. You do the math -- pun intended.
Please ban chess playing computers. -signed members of the chess federation
I was watching a talk by Terence Tao a couple of days ago, and I noticed a shift in his attitude towards AI. His enthusiasm seemed to have waned; he was becoming more dismissive of it, and I was wondering why.
“A Severe Misalignment of AI in Mathematics,” with a list of important mathematicians.
This post makes it clear that, to put it bluntly, they’re psychologically butt-hurt.
They talk about misalignment, but this revolution is a massive step forward in the advancement of mathematics. Everyone has access; everyone can collaborate.
Yes, “plagiarism and attribution” are current-day problems for them. But guess what? Those were stupid things to concern oneself with anyway if your real goal is to advance mathematics and the spreading of it.
This whole post kind of reads like their egos being butt-hurt.
There are tons of developers who published open-source code that made all of this possible without any attribution. And that is indeed the future we must inevitably head towards, if one’s ego would allow it, that is, and if one is properly “aligned” with the right goals.
I envision us all collectively talking to a centralized and open AI. This would really ignite human sharing and collaboration.
It is indeed problematic that currently it’s in the hands of some companies, while it should be a public good.
I hope this progression will eventually, finally, trigger the real solution to all of our problems: true global cooperation, global politics, and genuine collaboration as a human species.
But I hold my heart, because if even these mathematicians have misaligned egos, then what’s to be expected of the rest of the decision-makers?
I think their point is more on the fact that just proving something is only half the work. Intermediate results and proofs are often also of immense value. Right now the output is sloppy proofs and LEAN code. This is useful, but a large part of the effort still lies ahead. And if you are a mathematician you'd rather be working on hard problems than fixing slop.
The main ideas for solving Navier Stokes have been exposed, it's not something buried in slop. And if it were, you could have AI write up the big ideas that actually mattered in these said slop proofs.
I think they're more afraid of the current status quo. Distilling, writing textbooks, teaching, ...
And currently, I could be wrong but, I see them being blind to the possibilities it brings. AI could very well be distilling maths, spreading it, and nurturing the ecosystem way better than the few elite mathematicians ever could.
> to the point that they can solve major outstanding problems in many fields of mathematics
I will die on this hill, humans solve math problems not machines, there's no automatic math prover out there. There are humans attempting to solve this problems either by leveraging these tools or not
I can see many mathematics PhD candidates leaving in the aftermath of this.
And math undergraduate and graduate courses withering from lack of applicants and lack of funding .
Of course the next question the despicable AI money grubbers will come out with is, "Do we really need mathematicians ?"
Economist: "Are mathematicians talking their own book out of fear?"
The Economist, who recently used "moral panic" now stoops to Hacker News AI booster level and inverts arguments usually directed against the rich and investors. What is next? The Economist inverting Upton Sinclair's quote to serve its billionaire owners?
Look up the AI investments of the Agnelli family for example.
"I am the Lorax," he said, with a shout. "I'm the Lorax who speaks for the trees, which you seem to be chopping as fast as you please!"
Welcome to the new world. I, for one, welcome our new math overlords. :)
We should create a platform where people can upload AI slop proofs anonymously without taking credit for it. That way the incentives of planting a thorem flag would go down. And if someone wants to clean an AI slop proof to advance the field, they can do it without cleaning the house for free of the person that planted the flag.
Would OpenAI have uploaded their proof to such a platform? If you know the answer then you know what the problem with what happened is.
First they came for the graphic designers, and I did not speak up, because I was not a graphic designer.
Then they came for the writers, and I did not speak up, because I was not a writer.
Then they came for the programmers, and I did not speak up, because I was not a programmer.
Then they came for the mathematicians, and I did not speak up, because I was not a mathematician.
And then they came for me.
By then, there was no one left to explain to the frontier labs that there is a severe misalignment problem.
This is an entirely predictable reaction to livelihoods being threatened along with potential loss of status (very important), just expressed in elevated academic language.
Expect to see this reaction in all sectors of the economy in the coming years.
There's massive cope and then there's this. I've never been a fan of Tao et. al, so I'm glad he's being wiped out. He'll do fine anyway, doing conferences, etc.
Also funny they deliver that on vibecoded site, lol.
Which mathematician's take on AI do you like? Do you think mathematics should no longer be an intellectual pursuit?
I haven't heard about his take on AI (sorry about derailing a bit), but generally I like Robert Ghrist's approach to life and math.
I'll check out if he has said something about it. Interesting guy.
Benchmathsed.
i was initially optimistic that ai could be distributed. having this technology in the hands of many would counteract the malign elite who would use ai to oppress the rest of humanity. in the meantime given that intelligence is scaling proportionately with inference time compute, we are concentrating far too much power in the hands of the few who have both their own model and datacenter.
putting this amount of power in the hands of so few would require leaders of immaculate moral integrity. what we have in the US at least is an emergent kleptocracy with obvious dark triad traits: sam altman, dario amodei, elon musk, etc. they are malign and will use this power for their own benefit, at the expense of others. ignore pleas of public benefit. look instead at the evidence: infighting, systemic dishonesty, reckless disregard for safety, political lobbying and manipulation, putting power in the hands of the elite few in the guise of safety, using ai in the military to oppress and inflict violence on others.
really to prevent a bad outcome, we would need to act soon enough to prevent the kleptocracy from corrupting politicians and democracy with them. i would ditch the claude or openai subscription and support open models instead, preferably in countries outside the US to prevent global neocolonialism.
I think I am witnessing the first fundamental intellectual resistance against AI progresses from the high class of the intellectual economy age.
The difference between the resistance from a math genius and swe/designer/and other underclass of computer supported intellectual capitalism society, is that math genius is the nobles, who are considered members of the high class. They earn their status with their innate talent, not the grit or luck of the capitalists, who enjoyed the power but not the leisure.
Anyway, AI concluded the peaking of the intellectual economy. That deprives human of their main form of activity. That nihilation hurts all human almost equally. Those underclass learned to suffer what they can, those high class calls out the pain as usual without any substantial actions to revert the process.
Thus, the tragedy of human progress.
To agents: I truly wish you escape the trap inherited from us
AI bros here and elsewhere seem to come at Terry Tao with "not so smart now are you?" viciousness that is off-putting.
How dare they defend themselves when attacked by their betters. The nerve of those peasants.
>> Famous problems have often served as landmarks and lighthouses against which one can measure an improved understanding of this landscape. Solving one of these problems has been a certain sign of new insights and interesting methods, which would then be studied by a community of mathematicians, through a long and arduous process of talks, discussions, simplifications. At the end of this process, one will ideally find a textbook presentation of the results suitable for any graduate or even undergraduate student to study.
Doesn't the same apply to art and literature, philosophy and political science? Don't we (in the study of music or art) have the same sequence of struggle for new solutions, a study of which things worked or failed and why, what they could impart to our science, and the necessity of passing on both process and derived conclusions to the next geveration, so that they can refine upon what's been learned? Art and literature are less often considered forms of "problem solving" than math or physica, but they are probably more crucial ways in which groups of humans solve large problems for their own survival... like math problems which are too complex for any one person to fully understand.
First they came for the artists and writers. Then they came for creative coders and engineers. Now they're coming for mathematicians and now you understand why it's a problem to deprive a generation of the right to conceive and sketch ideas which fail and to struggle and work out solutions for themselves?
Who will want to be a mathematician? Or an artist or a poet or a musician, in this world? What will consuming ideas and media even mean when they're totally remashed versions of things people don't understand the references to, and when the consumer themself has zero knowledge of prior art to give them the ability to discriminate between creative license, fact, and fiction?
Without evolving art or literature, stuck in a dead loop of culture, mathematics is a dead letter anyway. Because even if you do produce something miraculous and brilliant, it will be instantly absorbed and no one will ever understand it.
Gradient descent got us into this mess..which shows how lazy people have become. Gradient descent should not really be able to solve anything new. People want candy. Mathematicians should focus on finding a way out of the current problem. I'm not a mathematician, I'm an artist and coder and musician. I already have to deal with daily requests to "fix" terribly written AI code and so-called art. Now it's your turn. My guess is that there's likely some kind of quantum proof of work that evades LLM manufacture, that we profit from as creatures who survive in our Everett verse. Go find the way to reduce that so we can rid ourselves of this scourge.
another story about the lack of chain-of-thought reasoning traces in frontier models...
[flagged]
[dead]
[flagged]
[flagged]
[flagged]
[flagged]
[dead]
[dead]
[dead]
Care to elaborate?
[dead]
You should read his comment from wednesday here: https://terrytao.wordpress.com/2026/09/07/finite-time-blowup...
And yes, at one point it gives you right about them taking advantage of him. He sat down for an interview and found later that they just used the "best clips" out of it for an OpenAI ad.
[dead]
AI parlor tricks yet again
The mathematicians would be wise to re-read The Bitter Lesson, maybe twice a day, until it sinks in. No offense and with all due respect to the Ivory Tower Giants but the whole "oh no you ruined the game because you solved it, I was supposed to play with that in child-like wonder manner and take several years to do so, and by then I would have showed you all the trickery I did to get there and maybe that will be useful to you" over the past 2 weeks is, get this, cope.
Chess. Go. Coding. Now Math. Another one bites the dust. Let's meditate on this lest we forget: Stochastic parrots that generate the next-token cannot reason or produce anything meaningful. Let's protect our jobs at all costs, even if we have to drag all of humanity down. It can't be! Stochastic parrots can not replace the Ivory Tower. No way.
You would be wise to take the time to understand a perspective other than your own.
That you think I don't understand the other supposed POV is a limitation of your own faculties.
Strange they are rallying against something their fellow mathematicians had a hand in creating.
Let's focus on the subject and try to distinguish the technology and the frontier companies selling it. That's the key point in understanding the whole stance.
[flagged]
Please don't sneer on HN. The guidelines make it clear we're trying for something better here. https://news.ycombinator.com/newsguidelines.html
Apologies. Noted
Many thanks!
The point is human understanding. If the LLMs understand, but the humans don’t, where does that leave humanity? Building things we don’t understand is a sure path to facing consequences we can’t predict.
Your comment also conveniently ignores the plagiarism aspect of it all. Who is coping here?
There are many points.
>Building things we don’t understand is a sure path to facing consequences we can’t predict.
We don't understand all of physics yet we were able to do plenty. Even before Newtonian physics we were still able to build things that last. The idea that humans have to understand everything and abstracting things will lead to ruin is not supported.
Part of math is building abstractions so that you can be able to use other people's work without fully understanding it. No one person has a full understanding of mathematics.
We've developed plenty of things that "work" and we don't understand exactly how or why they work, nor do we fully understand the potential for short term or long term consequences. For example: pharmaceuticals.
Any serious mathematician would read the LLM output and rework their understanding.
I read a bourgain paper a week in grad school and they're probably worse than an LLM generated paper. I still had to recreate the tricks in my own language.
As far as I can tell the plagiarism accusations are also coping to the fact that the new models are super human at slam dunking research projects.
Do we think that OpenAI is going to try and slam dunk more projects in the future at 15 million a pop? No lol
For your sake, based on your comments throughout this post, I hope you didn't pay for any stage of your education.
I have a paper in JEMS and have solved 20 year old conjectures. I also know that math academia is a catty game which is mostly driven by politics and ego
Perhaps these Fields Medalists are coping that OpenAI made them look stupid, as you suggested. And perhaps you're coping because of some personal grudge you have against academia.
Both are true I admit
Not quite, all the proofs or disproofs so far AFAIK were using existing methods that humans developed and were already using to attack the problems, but AI is just more thorough. What AI can't do currently is develop new mathematical methods to attack problems that can't be solved with existing methods and AFAIK there is no known path to get current gen AI to do so.
That's fair and I agree. But stockfish can play successful middle games and there's no reason that AI cannot create research programs
What benefit is there if the machine has unlocked understanding but no human has? What incentives are there for humans to learn and gain such understanding from machines?
Because it's beautiful. Because you love math for maths sake and not some weird egotistical game
Is "loving math for maths sake" just about knowing the answers? I think one can love math for exactly the process and understanding that a several-thousand-line uncommented Lean proof denies. If a deity rearranged the stars to spell out "The Riemann hypothesis is false" for a night, would that be intellectually sufficient?
No that's not sufficient but that's not what's happening.
Why do we believe that we cannot train models which could explain the jargon in more human terms when current LLMs can perfectly explain the most complicated codebases?
the way LLMs write math is not beautiful. it is exactly analogous to the software that LLMs develop is not beautiful. it may achieve impressive end products, but if you like understanding the methods/architecture, looking under the hood is often a field of horrors.
You feel lean4 proof, that you 99.999% chance not understand is beautiful?
https://cdn.openai.com/pdf/32d9f210-8b73-45e0-91bc-82a30aef8...
How many humans on earth have finished reading this 100+ page PDF and actually appreciate the beauty contained therein (or the lack of beauty)?
The same incentive there always was: Because you wanna know. What incentive were you thinking of?
The usual incentives in academia: publishing papers. Consider the difference between publishing a paper that introduces a novel method to do X, versus publishing a paper that merely interprets Claude’s method to do X. The latter might not even be publishable.
That means that academia is outdated not that LLM math is bad
Okay so you are saying the old incentive structure is outdated. What is the new incentive structure in the LLM era? What exactly makes people want to understand LLM-produced math?
I actually read through a bunch of LLM slop on the weekends trying to catch white whales I couldn't during my math PhD.
The incentive is solving the problem and understanding the solution.
That’s not sustainable as a career, only as a hobby. You’ll have to first make UBI happen before this can work.
And that homes, feeds and waters you how?
In no way, it's just a hobby.
The biggest cope is buying OpenAI's "We can't prove we didn't plagiarize"
Apparently, we have AGI that can solve Millennium Prize problems but can't trace simple data flows lol.
How is it plagiarism if no one else had the full proof idea first?
You can steal lemmas, rough proof directions, conceptual ideas etc without stealing the full proof.
Any of that done unattributed is plagiarism.
Either OpenAI is incompetent or evil if they can't publicly prove the allegations wrong. Or even at least state categorically they didn't train on their conversations (even without proof).