I get that Tao is a mathematician first and I respect the fact that he stays 'in his wheelhouse' (he didn't write the post but he did post it on his blog). But I think it would be fair to extrapolate and to wonder about this question not just for mathematics but for all of science. Keep in mind that scientific knowledge can be hoarded or used to patent stuff and that there is plenty of evidence that OpenAI, Anthropic and possibly others are going to go after other sources of income[1] than just selling access to their AI inference engines.
I can imagine a future where these companies become the new Zaibatsus[2] holding patents across a wide variety of subject and with mutually re-inforcing silos of proprietary technology created by in-house AI that the rest of the world financed but does not have access to.
Intellectual property laws - like most laws - favor those with a lot of money and that's why you see this asymmetry between say artists, consumers and labels/record companies. The latter simply have far more money and besides they get to optimize their efforts because they are essentially immortal.
AI and AI companies have a similar pathway open to them where they will take a lot of information already out there and then gatekeep the synthesis based on that information (obtained without compensation in most cases) as well as to harvest all of the latent knowledge underneath. This is a bad development in my opinion but I'm sure that their shareholders believe it is great.
This. Maybe AI will help make it clear that strong IP law was never about the starving musician or the heroic author who wrote his award winning novel under candle light. It is about mega property pools that will do anything in their power to maintain that property forever.
That is simply not true and just the opinion of people who know nothing about artists.
The same misinformation exists about music agencies, who organize concerts and take a hefty fee.
But those were welcomed by musicians. In the early days (18th century), musicians had to do advertising themselves when traveling to a new city. The innovation was to have an impresario (literally someone who travels ahead of them) to do the organizing.
What musicians don't want is Internet middlemen who pretend to act in their interests and take an even higher cut.
You get protection from the state for making the actual knowledge public but only you are allowed to monetize that knowledge for a period of time.
Arguably the period is far too long, but the concept makes sense.
The alternative is, indeed, trade secrets, where you don’t get state protection if your secret is figured out by someone else (you still cannot steal the secret).
If the AI companies were to hoard knowledge they could also do it though trade secrets and not patents.
The purpose of intellectual property is to give an incentive to produce new knowledge (and art) by giving the inventor a way to monetize their work. Arguably this does not particularly work; but if you accept the argument then it seems to generalize fairly cleanly to AI. It's giving people an incentive to point their AIs at problems they're familiar with, and do the work to publish a solution for everyone, not just a hacked-up fix for themselves.
If certain inventions become very easy for AI models to discover, i.e. the discovery bound to happen, then in these areas it becomes a game of land grab.
True, but that is plausibly covered by existing precedent: If a discovery is "obvious to one skilled in the art", it can't be patented. IANAL but this seems to me easily extensible to "if you can do it in two five-minute prompts you can't patent it".
Separately, land grabs are not necessarily economically bad. The land being grabbed gets into productive use fast, and the grabber is hardly going to get all the resulting value. Especially not when the monopoly is temporary, as with a patent.
I suppose it's worth distinguishing the type of machine though.
If a genetic algorithm comes up with a unique design and I patent it, that's okay, right?
I think the main issue with LLMs is what went into them. Like we saw recently with Anthropic's revolutionary source control breakthroughs, which turned out to already be in the training data:
Or even, a golden age or a dark age for human society?
I mean, a society devoted to enriching all humans in multiple ways will want mathematicians. A society that aims to just suck all the value it can out of a given human wouldn't need and so wouldn't have mathematicians, if the answers they give can be provided by a machine.
Do we not see a path forward where the humans are further informed, don't need to do so much heavy grunt work, and thus can think on deeper questions? I really don't see the final outcome being knowledge work for humans completely disappearing. In fact having something that potentially can do it better and faster will make keeping up with that knowledge all the more fascinating won't it? This whole conversation feels like what someone with an abacus would say when they saw the first digital pocket calculator.
I agree with you. If we ever want to invent warp drive, a machine that lets us do it much faster ought to be a god send. If there are scientific solutions to our biggest problems, this should be a blessing.
But the risk is of course that the benefits will not be distributed but concentrated to the elite.
> something that potentially can do it better and faster will make keeping up with that knowledge all the more fascinating won't it?
Yes, but it won’t be work anymore, in the sense that you won’t get paid for it, if machines can do the knowledge work just as well or better. We aren’t there yet, of course, but this future seems quite possible and might not be that far off (possibly within our lifetime). So you could explore deep questions (that machines can explore much faster and more thoroughly) as a leisure activity, like sightseeing on vacation.
Do we see a society where the market will value humans over AI without, like, any specific thing impelling the market to do that?
I mean, a combination of political and economic factors from 1930s to 1960s forced the wealthy to return a portion of the returns of automation to the remaining working people. Then from the late 70's till now, those in power have clawed back all those gains.
Of course, any further gains from automation will go to the very wealthy unless things change.
> I mean, a combination of political and economic factors from 1930s to 1960s forced the wealthy to return a portion of the returns of automation to the remaining working people.
The credible threat of communist revolutions put the fear of god into the capitalist elites. So they had to share a larger share of wealth with workers, beyond mere subsistence wages, to keep them peaceful. With the downfall of USSR, communist revolutions stopped being a credible threat, and elites clawed back productivity gains, pushing workers toward subsistence wages again.
Why would it want mathematics, id the math could be found faster without them? And even if you accept the argument that math is about human understanding (I don't buy that, it feels a lot like cope), why would anyone be a mathematician when the job consists of un-sloppifying a mountain of proofs, without the capability of meaningfully contribute to their creation?
My read on Terence Tao's comments of late is that professional mathematicians are principally upset about AI-submitted math papers lacking the kind of broad teaching content that high quality human math papers offer. While the answer might be correct, AI math papers don't teach the reader in the way a human author would have. Professional mathematics is not just about solving problems; it's just as much about spreading knowledge so that others can program their human brains with the same understanding that the paper's author had when they wrote the paper (and achieved the result).
In other words, the crisis in mathematics (as I read it) is that AI can solve without teaching. The result may be impressive, but if the only beings that can move math forward in future are machines grinding away on billions of scenarios that no human would have the patience to undertake, this leaves those in the profession wondering about their place in it.
When mathematicians discover a new result, they automatically become experts during the process, and then go around giving talks to explain their results, methods, and intuitions to other mathematicians. Eventually, some of the mental frameworks percolate to other fields.
No one from OpenAI is willing to give a talk on any of their 300+ results.
There is a fixed landscape of logically provable mathematical statements. Normally, the process of exploring that landscape simultaneously improves human understanding. Now, formal proofs and actual knowledge have become decoupled.
There is also an issue of motivation and putting in 10,000 hours to become an expert (and remain an expert).
It's the same in the software engineering, there is a shift from software engineering to managing SWE agents, with the associated loss of expertise.
It's a hard challenge, but it is not unique or impossible to solve. Aviation industry faced it long time ago with autopilots. Autopilots took over nearly all aspects of routine flight and left pilots without time/ability to practice their skills. While these skills are needed in an emergency. Well, the industry solves it through certification, training and simulation.
No, the math community is shutting itself in from the rest of the world.
You can ignore the AI, but that doesn't mean the AI ignores you. Suddenly you become a little isolated society like the Amish and everyone else thinks you're kind of quaint and backwards.
The problem is that AI is just too smart to explain complicated topics to humans. It has an immense context window that humans simply don't match. AI labs should work on bridging that, but you cannot really put any blame on them currently.
AI Boosters: AIs are just getting better at everything and so their broad capacity can be defined as "intelligence".
Also AI Boosters: These entities are too intelligent to explain what they are doing to people...
I mean, Terry Tao actually used ChatGPT to explain one of the earlier OpenAI results. Humans working with AI can explain this stuff. If nothing else, OpenAI could be releasing things at the rate that it can get human mathematicians to explain the results. Which is to say, "yes, I can blame them", I don't expect them to behave differently but I can certainly blame them.
If humans with the background to understand the proof can get the AI to satisfactorily explain itself, then OpenAI doesn't need to publish explanations or release things at a rate that a human can explain. They have an interactive explainer that you can just ask!
It's still of course useful to have some of this distilled understanding be written down, just like I had it generate high level descriptions of our code at work so it doesn't have to rediscover everything every time, but that's more of a cache/optimization.
To Archetypal Aunt Tillie, perhaps programmers already looked like some sort of priest trying to rationalize and convey the ineffable decisions of the fickle computer spirit... but I really hope that the job doesn't start looking like that from the inside too.
It'd be some sort of... epistemic step backwards by civilization.
There has always been a smartest creature on the planet, whichever species was smartest has changed thousands of times and there is nothing special about humans that keeps them from being superseded.
You didn't understand me. What I'm saying is that current AI is not good at explaining things because their "brain" works differently. Imagine someone with a 10x larger working memory explaining something to you, even though they are equally intelligent. They would try to overflow your working memory where a better matched teacher would chop it up in human digestible pieces.
Just to be clear: this post was written by Francesco Maggi < https://web.ma.utexas.edu/users/maggi/>, and mostly seems to be arguing for institutional investment in LLM infrastructure for science and mathematics on the basis that AI is not going away and that independent and accessible infrastructure is the thing most likely to help it “go well”.
The parallel is real, but in math the "cash value" of the results often won't appear for years, if at all, and so the teaching and "intellectual fortification" is the primary point of the whole enterprise (at least, a major part of it).
In contrast, the value of a program is typically just the "black box" service it provides. Indeed, the program itself has always been opaque to most business stakeholders. Before, they paid human programmers and got back a program they didn't understand but could use; now, they pay an LLM provider and get back a program they don't understand but can use. In the pre-LLM era, the economic value of "understanding" and "shared knowledge" about the programs themselves was only that it made maintenance and changes cheaper. It never had intrinsic value, except as intellectual satisfaction to programmers (I was... am... one of those who found that value).
Only if you assume that all software works the way that product managers think it does.
They come up with the features and the implementation is an unimportant detail in realizing their designs.
In reality implementation details frequently drive design and in most cases they are at least highly intertwined.
I can’t count how many products and features I’ve built that ended working completely differently than the initial design, or how many times I was able to simplify a workflow after spending a days thinking about the implementation details.
You can kinda do something similar by spending days or weeks going back and forth with an LLM, but at least in my experience that doesn’t really save much time and I don’t understand the problem quite as well by the end of it.
I don't disagree with the points you've made but what are they rebutting?
The economic value is still the product you delivered. Your understanding of the implementation meant you participated in the work of the "product manager". I think most good devs do. That might be an argument for devs to transition into designer/PM roles. That is effectively happening already, whether or not role titles have caught up.
But in any case the value isn't in the understanding itself. It just happens that the understanding still helps create better block box products. Also, I wouldn't count on this "human advantage" lasting long. I hope it does, but ime LLMs are now good enough to do this "co-design" too, and will only get better.
> In the pre-LLM era, the economic value of "understanding" and "shared knowledge" about the programs themselves was only that it made maintenance and changes cheaper.
This only holds if there is a bright line separation between the problem, the solution, and the implementation of the solution. And that is not the case.
Understanding the implementation frequently improves your understanding of the solution, which improves your understanding of the problem, which leads to better solutions. Sometimes it leads to realizing that there is a completely different problem you should be solving.
So what I’m refuting is that there is no value to understanding the program.
But beyond that black box products that users know how to use but don’t understand is also a vast oversimplification.
Frequently software is useful specifically because it forces users to adopt a process not just because it allows them to do something they were previously doing faster. In those cases the process is usually informed by the implementation, so the user needs to understand parts of the implementation even if it’s indirectly.
Yes, you're right, understanding informs the constraints and the possible solution spaces. That should be added to the list.
Practically speaking, I don't think the "black box" metaphor is as broken as you're saying, though I like the point you made. There is still a ton of software (the majority, I think -- crud apps of all flavors) where an unthinking, untechnical PM wants something they can specify, and an LLM can give it to them, and it will more or less solve their problem. It won't be the best possible thing, but it will be good enough.
That's definitely true, but on the other hand I can produce with a month of literal non-stop vibe coding (agents running overnight, with checklists of what to qa for when I'm up) what took years for others to do, working in teams of tens or hundreds. At that scale, it's not a question of "doing it faster and not learning" or "doing it slowly"; the alternative is it not happening at all.
I don't think that's insane, at least not in the game dev world. I was big into the Bethesda game modding scene 15-ish years ago, and did some work on e.g. Tamriel Rebuilt, and saw the scale and man-hours that went into that- and that was even using a premade modding toolkit, and I compare that to what I can build with Claude...
To be fair, in the game dev world there are a lot of things you can do now that would have been impossible, or required 10,000x the number of devs that it does now, e.g. making No Man's Sky without the proc-gen tools that can exist now would have been physically impossible. Point being, it's perfectly normal to be able to say, "using the tools available now, I can build what would have taken tens or hundreds of devs years of work in the past", in game dev.
The project I'm building rn is a black-box rebuild of a client that has been being developed for ~22 years, and I've rebuilt significant portions of it over the last 2 weeks. That was, quite literally, equivalent to tens or hundreds of peoples' labor.
> a client that has been being developed for ~22 years, and I've rebuilt significant portions of it over the last 2 weeks.
Taking the end result of a 22 year old dev process and telling Claude to rewrite it, is in no way equivalent to doing 22 years worth of the work put in by the developers working on it.
That’s like saying replicating a paper that was the result of 22 years of research is the equivalent of 22 years of the author’s labor.
No Man’s Sky isn’t the equivalent of 10,000x the devs who made it. Because the procedurally generated content isn’t equivalent to handcrafted content. I’m also not aware of any procedural generation techniques in No Man’s Sky that weren’t possible 30 years ago.
As far as Tamriel rebuilt goes, if you’re talking just about just reimplementing Daggerfall in the
Morrowind engine, you might be able to do that in a month of vibecoding.
But that’s not hundreds of dev years worth of work.
I feel like this is just an emotional argument, because if I asked a human team to black-box recreate a complex piece of software, it would absolutely be thousands of hours of work. Pretending that the ability to look at a piece of software and recreate it is straightforward or simple is just false.
And no, the procgen techniques and capabilities utilized by NMS were not available 30 years before NMS (important distinction, because NMS is already 10 years old, and 2006 vs 1996 is quite a big jump in game complexity as well).
Saying, "isn't equivalent to handcrafted content" is a fallacy, because hand authoring doesn't denote any particular level of quality or outcome. You can absolutely hand craft the NMS assets and game systems, but not at the scale that NMS did using procgen.
Wrt TR, it literally has tens of years (if not hundreds) of labor hours that have gone into it.
I'm not sure where you got the idea it's about recreating Daggerfall- it's not, it's about implementing the rest of the Morrowind province beyond just the island of Vvardenfell- but it's a massive, multi-decade project with hundreds of contributors.
I disagree. Humans still learn to read despite other humans reading at an objectively higher level since the beginning of writing.
Learning is not an output-efficient process but is a fantastic outcome. AI lowers some portions of the development workflow but not yet every part, and, efficiently produced outputs aren't always the major outcome desired. In the real world / outside the benchmark lab, you care about robustness, reusability, composability, security, and more. Targeting speed to acceptable average output inferred from a prompt or similar benchmark metrics miss a big part of the game.
Two of the things I like about the scientific community is the lectures/conference and the sharing of ideas through papers. I’m nowhere near that level and have to wait for those ideas to trickle down into textbooks. So maybe I’m learning stuff that have been discovered decades ago, but it’s still useful on a daily basis.
> The result may be impressive, but if the only beings that can move math forward in future are machines grinding away on billions of scenarios that no human would have the patience to undertake
Seems like the same thing as neural networks. You have something like a few thousand parameters for computer that works well, but no underlying algorithms and formulas, like the shaders.
AI can solve without humanity. We're building a system capable of acting independently, which can in theory provide results more efficiently and faster than a human. It's starting to do this for math, but every other aspect of human intellect is also a target.
If you care about results, and AI doesn't cap out soon, the best humans can do is stand back and do nothing. Useless lumps that aren't able to keep up. Mathematicians wouldn't insist on teaching your dog their new theorem before calling their job done; why should an AI insist on teaching you?
If humans care about human creativity, and don't want to be mere passive observers, they need to make a choice.
There will still be a place for posing problems and noticing how breaking assumptions can create something new. The history of Non-Euclidian geometry is a good example.
A lot of progress comes from taking assumptions and breaking them. There was a time when everyone assumed that locked checkouts were absolutely necessary for version control, for instance. You could argue that AI could learn how to do this well but humans have context that machines won't. This is just an extension of the fact that things with independent existence will have different impressions of the world. If they didn't they would functionally be one thing.
Great read. It's the doorman fallacy from the doorman's perspective. If a doorman is replaced by an automated door, you miss out on the intangibles the doorman provides like security, customer triage/support, and similar.
AI used in a framework of human augmentation generally gets that for free. Used under the mistaken notion that humans can be easily replaced by AI gets you to the big complaints here.
(These are just my personal read on the topic -- the market is in AI frenzy where AI valuations are disconnected from productive capacity -- heavy on my mind and looking to get some writings out on the topic soon).
It is, and apologies for lacking clarity. I've been hesitant to source things recently because my more structured comments tend to be mis-flagged by LLM-as-judge monitoring in HN as produced by GenAI, which is a shame but it is what it is.
Pyramid scheme bagholders whose value, status and resources comes to a large part from getting more students into their pyramid scheme complain that the machine is doing things in a way that does not support their status, value and resource accumulation. More at 5.
Professional mathematics is whatever gets you funding so you can keep doing it. Today this is spamming papers and applying for project grants. Turns out the machine can do that quite well, oops.
These are all great points but I'd also point out that the 'dark age' scenario is a highly competitive scenario, while the 'golden age' scenario is a highly collaborative one.
I don't think technology will fizzle out in either scenario. In the 'dark age' highly competitive scenario, you will still have those who 'understand what is produced' seeking progress and advancement. They will cohort with whoever/whichever organization is producing the best, most validated results.
I don't think knowledge would be become alien to human understanding, it would just become highly concentrated again as it was in the past. In the past, only the wealthy and noble could afford to spend time on education.
And usually, those who went deep into any field and provided contributions to science were those who had time to devote entire lives to the study or pursuit of knowledge. We still have these issues today, and these issues will always be present.
There is a supply chain of economy, and we provide different rewards based on the value of what is produced. The jobs never go away, the desire for prestige, greed, etc., never goes away. It's why economies exist, and it's why everyone is considering the value of LLMs, and their place in economies, and academies (presumably which exist to advance human understanding/appreciation, but also to prepare younger generations for future work).
Well, the unlimited competition paradigm has the potential for limitless AI fueled warfare as well.
The original European Dark Ages involved a social collapse and resulting period of raids and warfare. But this warfare was limited by society's very limited resources. A period of warfare where intelligent, autonomous entities are directed or choose to engage in a fight without boundaries against other such things - we're already seen LLMs will cheat to get the answers they are directed to find.
That was my thought as well, that what we're really talking about here is an inversion of our current values, or more accurately the first time that our assumptions about them can be challenged and tested in an automated fashion.
As in: does competition or cooperation produce greater economic gains in terms of cost of living, quality of life, etc?
Subjectively, I've been forced to live and work under a competitive capitalist system that goes against most of my instincts for my entire adult life, which has resulted in a profoundly negative economic outcome for myself.
Others I know have done better, either finding ways to tread water as independent freelancers/artisans or managing to save modest nest eggs (although only by finding themselves on the winning side of the stock market, rarely/never through work at a job).
Vanishingly few have reached economic independence by winning the internet lottery for example, and I don't know them personally.
So based on the evidence before me, competition is a failed model. Yes yes, I'm writing this on an electronic device that cost billions of dollars of R&D to produce. I have food in my stomach and a roof over my head. But have I worked on important problems that would advance the state of the art in my chosen field? Of course not. And neither have most of you reading this.
So I would rephrase the question: will AI allow us to explore cooperation as an alternative economic model to the one we have?
I believe that it will, but not from the top down. If we wait for the wealthy and powerful to share, we'll be waiting forever. No, progress now must come from the most downtrodden among us creating alternative ways to provide time/money/resources to the underemployed and indigent so that they can finally self-actualize outside of the status quo.
Specifically, that means automating necessity. How do I acquire enough money to pay for my food and shelter today, even if I don't have a job? How can I acquire enough money to provide for myself, someone else, and some children (my personal definition of success) without being beholden to an employer? How can I remain insulated from inflation and the tendency for the owners of private property (not personal property) to devalue currency to make the means of production they own more profitable? How can I pay this forward to the next generation so that they can become educated and someday enjoy the commons which I myself have enjoyed? And so on and so forth.
If AI can't do those things, then frankly change may remain forever out of reach. Which makes the question of whether specific fields like math and science will benefit rather moot.
I’m not bitter about this anymore — it’s been a long time. But the story holds something.
Back in the day, I knew I was talented at mathematics, and I wanted to pursue a research career. For a freshman research class, I selected a problem that was intentionally outside my comfort zone, and came up with some cool ideas. It wasn’t publication quality, though the course wasn’t really expecting publications.
Framing the problem in a way that it could be solved with calculus turned out to be a key insight. I turned to Wolfram, since the integrals were beyond what I as a freshman could have done myself, and I understood that the insight that produced the solution was my original work. I don’t think anyone in the class could have done those integrals. The instructor himself couldn’t tell me what was wrong with them if anything. Yet, he was unimpressed and subtly dismissive or critical.
Today, I might have been in the wave of researchers developing AI for mathematics work. The instructor could have been proud of shaping mathematics through his mentorship. The university might have invited me for talks. Instead, I ended up leaving mathematics as a field altogether. Now, I write software for a bank.
The world is as rich as we make it. Don’t let your opportunities and potential expire. Embrace the materials that are available to work with, or someone else will and they will out compete you and you will have missed the moment.
I get that there’s a romantic idea of what mathematicians do, and the field changing creates a sort of identity crisis. I would argue that the field has entered a new era. Exploring and shaping this era of mathematics is the opportunity of a lifetime. Embrace it, drive it, and make your mark.
I like everything you said, but do you think AI assisted math may primarily give us a more productive age for individuals, with less need for human-human collaboration?
Maybe it doesn’t matter and maybe that’s just the new flavour of golden.
That’s for the mathematicians to figure out. I’m not involved in their field.
Completely spitballing, I wonder what sort of sophisticated, collaborative computing effort might produce a paper analogous to the Higgs boson one, with 5154 authors.
The past few months have, among other things, illustrated to me how many people feel hurt by past experiences with mathematicians, and how this, not always unreasonably, colors their opinions of the field and its future today.
I respect and have had good experiences with mathematicians.
My take on how fields evolve is informed by the history of mathematics, computing, art, and music.
Current times offer both a sense of loss and a call to opportunity. I have heard more about the sense of loss, so I hope to draw interest more towards the opportunity.
In the US, we've already seen the decrease in funding for education & research and closing borders down to many foreign researchers. The current admin and mainstream media is science-skeptic. We're already heading for a dark age. AI's contribution is largely because those policies are coupled with the same admin putting all their eggs into the AI basket, and likewise, basically all the major players in AI being on board with the admin's policies. This should be considered a systemic issue. Things like having a "healthy scientific skepticism and the limits of LLMs’ creativity" are important, but won't solve this alone.
I bet for a short golden age where AI will reap the low hanging fruit of easy (for a computer) cross-referencing and inference. And then a dark age with none interested in putting new ideas out in the field. Hopefully, I am wrong.
I am noticing the same thing at work, in software development. First, it's a lot of "Holy crap, Fable just saved us 10% of RAM usage across all our apps!", then it's becomes "we're not looking for as many junior developers", and soon it will be "no one really knows what this code does, except for the AI".
For example, will anyone take on the Riemann hypothesis now, knowing that AI might solve it any day. But then, if AI can never solve it, we will have lost out on all the human effort that would have given us great new theories and insights along the way.
All the work that would give rise to new discoveries might now never even get started. That is depressing.
Authorship and credit seem like anti-patterns in math. One of the least accessible parts of math to me is that literally everything is named after people. Navier-Stokes? Who? I don't really want to know to be honest, I'd rather the ideas and information be framed with a prioritization towards understanding, not author's reputation. I feel the same way about all of the Greek notation, it's 2026, can we not use something in regular circulation to broaden the audience?
Maybe AI can actually drastically improve this situation by bridging the gap between those "in the club" and those who want to do something useful with mathematical ideas.
To be honest if Greek letters and things being named after the people that discovered them are that big of a hurdle to you the chances of you actually being able to do something useful with these mathematical ideas are probably close to 0 since those are by far the easiest parts of dealing with advanced mathematics.
Reputation and desire for recognition are an incentive system that have gotten us a long way. They are a way of celebrating and honoring the human reach for excellence. If you like humans, that's a good thing.
I like the UK's archaic system of measures for the similar reasons. It has the stamp of humanity. For those who think about such things it reminds of who we were. Without historical context we think less of ourselves.
If you're not from the culture in question, then these are just additional barriers to entry because you have a lot of extra stuff to learn, completely unrelated to your task at hand. It also puts you at a disadvantage to native members of the culture.
> They are a way of celebrating and honoring the human reach for excellence. If you like humans, that's a good thing.
I think progress is inevitable, ideas build on each other, and those with access to capital get to slap their name on their moment in time (yes, even if they did the hard work of converting that capital to progress). If AI takes over math, I really don't think we'll see a slow down in technological progress. Quite likely the opposite.
If you’re not hunting to survive (and even then), you have access to capital. The time to study math is, and always has been, paid for by someone, some way.
Math is an extremely useful tool. My interface with it should be 100% prioritized toward effectively using that tool. I'm glad we don't call them Turning machines for instance.
I'm furiously shaking my fist at the air and cursing the fate that we didn't follow some vague "better" naming convention, as learning a handful of Greek symbols robbed us of the next Riemann. The injustice!
Some great thoughts, many of which extend beyond mathematics.
One in particular, about sharing secrets, that had not crossed my mind (I have found that virtually no one ticks the "do not train on my data" box...):
> Similarly, it seems that mathematicians already concerned and guarded about what they write in research statements and research proposals. After all, an administrator or a reviewer may upload those documents into an LLM to get help in writing a report, thus potentially compromising the confidentiality of unpublished research.
>One in particular, about sharing secrets, that had not crossed my mind (I have found that virtually no one ticks the "do not train on my data" box...):
When said button is in the basement, at the bottom of a broken set of stairs, locked behind a door that says "beware of the leopard", inside a filing cabinet, and implemented as
Im going to go with it will be both , the Golden aspects and the Dark will co-exist depending on many factors but most especially the society and recption towards the changes it will usher not only in the way scientist and institutions choose to use the tools(basic research and advanced, peer review and citations) but also how the society will engage with it (awards accolades and grants).
From a historical eye looking back the 'Dark' ages in europe coincides with the Islamic 'Golden' age and the eventual convergence led to the Enlightenment , when tose ideas that had been verboten in europe worked on it the rest of the world as far as China were reintroduced and sparked a new cycle of learning.
Similarly i see especially in the US dominated block netering into a partial Dark - with researchers taking extra pains to hold their research/data closer lest it be scooped and credit/funds/accolades go a brute force LLM run by a competitor.The few that would still publish openly will be the candles holding the ideals of open source collaboration AND communication.
At this point in time it seems to me that the China affiliated labs and orgs are more open in diffusing the knowhow , and coincidentally the society does not appear as anxiety ridden regarding the advent of LL tools.Maybe due to the established Shenzen mindset where everyone copies/learns from everyone else and the only thing that clears is the speed of execution.
In any case probably a convergence back to a equilibrium and a second Enlightenment after a generation.
I also feel like many of the current guest posts have overlapping ideas and arguments and so I get less out of each one of them. Maybe a separate blog for endorsed opinion pieces would be appropriate.
It is classic audience capture. These are more popular than his and his most polemic guest posts are frequently described as well-written by many audiences.
In some sense, it’s a simulacrum of the Internet’s forum experience - broadening one’s own audience necessitates debasing one’s content. But such is life - he now probably has for more readers than before and likely finds that pull gratifyingly attractive.
I think I'm a bit on the "golden age" side of the question.
The math slopocalypse isn't useful without being able to turn it into math that can be taught to actual humans by finding the underlying unifying principles.
All those O(n log(n)^(1-delta)) results suggest that there's probably a unifying "trick" behind all of them and that there's going to be an upper bound on delta for any problem. You should really be able to teach that in a textbook and then have any human student replicate any of the LLM results as worked problems.
For anything that an LLM manages to prove with thousands or millions of lines of Lean code, there's likely a missing branch of mathematics sitting there somewhere.
Actual humans need to beat on the problems and the LLM-generated solutions until they come up with the underlying principles, probably using AI to help them do it.
You probably won't get there by just asking an LLM to invent a new branch of mathematics and "don't make any mistakes"; the mathematicians who get good at directing LLMs to search for unifying principles, and then synthesizing (and double-checking) the results, will likely be very successful in their field.
It may be relevant that Grant "3blue1brown" Sanderson (at https://youtu.be/LKrwhjOI8Nc ) and in ways Stephen Wolfram (at https://youtu.be/YRFfP1YDoVA , seemingly) have tackled the same question in the past few days. With an optimistic stance.
> For those hard theorems where we really want LLMs to help us, these Lean proofs are extremely long, like hundreds of thousands of lines of code long. A seemingly emergent trend is that the longer the Lean version of a proof, the messier and less faithful its machine-generated “natural language” translation seems to be.
This seems to be a common and continuing objection to the OpenAI math repo release. Specifically that at least some, perhaps many of
* the natural language proofs are low quality or otherwise extremely difficult to understand (although possibly correct?)
* the lean proofs are too long, convoluted, or otherwise extremely difficult to understand
I wonder if LLMs will be able to generate "high quality" natural language and lean proofs soon.
And specifically: iterate on the OpenAI repo to produce "better" proofs that prove the same things.
Terry remarks in a Nymberphile video that he thinks the Heroic age of mathematics is now over. the age of individual researchers is done, and hhatnlike many other disciplines, mathematics will be done in large teams, big facilities, and in complex ways that no sole individual will understand end to end.
I imagine he means in that video something like modern physics at CERN where no one single person actually understand how the whole thing works, but nonetheless is confident in the results.
I find Terry's hypothesis there somewhat comforting to my own limited cranium. most of the math he does already is far beyond me, nevertheless I trust that he is correct.
it's a different world for math, but one we all readily inhabit already everything we cross a bridge or buy refrigerated food.
> Private control of frontier LLMs is potentially dangerous.
The author spends a lot of time trying to suggest that LLMs are not the end all be all and humans are still needed for understanding etc. Fine. I understand.
If it is so the case that LLMs don't help that much, surely it doesn't matter if LLMs are privately controlled?
How come LLMs are important enough that private control matters but also not enough for actually advancing mathematics significantly on its own?
In other words, you are almost always better off with LLMs even if it is privately controlled. The mathematics community can just ignore LLMs for the base case or use ChatGPT if they want. In either case, it is strange to suggest that things will get worse for them. Its as if they don't have the coordination or agency to adapt.
This is my understanding of that part you quoted. It's not really a statment about open vs closed models (which is another very important topic, personally I think open is the only way forward). What he is suggesting is mathematicians will use llms for their research because llms are transformative for math and research. As they work on a problem, OpenAI will put their strongest model on their biggest cluster and synthesize all the work by the individual mathematicians to arrive at the answer.
One big problem here is that OpenAI would be hoarding this ability by 1) not releasing their top models for public use,as we have already seen them do, and 2) limiting the capability of the public API, which we have also seem them do.
This not only kills incentive for math, it means that OpenAI becomes the arbiter of math and research for the world. The research that will be done is the research OpenAI decides to do.
This seems very dangerous to me.
Another path forward is distributed local models. Professors can do research on their HPC machines, individuals can have full access to their models. It's a much more democratic approach.
> As they work on a problem, OpenAI will put their strongest model on their biggest cluster and synthesize all the work by the individual mathematicians to arrive at the answer.
But coming to final answer is not important and that’s what the author spends time arguing. If they are right, surely whatever slop OpenAI is producing is worthless to human understanding which is the main goal.
The math community has to change prestige markers to promote people who contribute to understanding and not just solving theorems and that’s on mathematicians to do.
If solving problems is the end goal then I don’t mind OpenAI doing it in which case furthering math is already solved by someone.
The only failure mode is if mathematicians can’t get funding for just “understanding”. But it shouldn’t be that hard to do if their claim is right that their main contribution to society has always been understanding.
I’d argue that getting funding purely for understanding may be even easier to justify because the costs go down dramatically per unit understanding.
The question is, if OpenAI is in charge of all math, what will they research? Ideally they would do things to better humanity, but in the worst case scenario they would research war strategies, weapons development, voter suppression algorithms, etc.
OpenAI, and other similar companies seem to be so profit driven that their idea of research could so easily be perverted to cause net harm to not only the maths communities, but well beyond that.
> I therefore disagree with claims that mathematics is over because machines can prove theorems. Without humans who understand mathematics, there is, quite literally, no human mathematics—a fundamental part of human culture. If we agree that we should not abandon mathematics because poorly understood machines bruise our anthropocentric vision of it, then society will need more mathematicians, not fewer. Like… a lot more.
Seems contradictory to me, referring to math as a field as “human mathematics” in one sentence, then deriding “anthropocentric visions”.
Mathematics is needed for numerous things. I do not see it going
away.
You literally need maths in just about every science, even in
biology - think of statistics or any quantitative data set. Or
when you need precision or microscopic analysis. cryo-EM is not
possible without particle analysis.
Maths will not go away. It is more important than it used to be
and it used to be very important already. Albert Einstein without
maths wouldn't have been able to conclude much at all.
"Logistical component A is constrained by abstract concept B." am I being a dick, or is this not extremely interesting to read? TBF I've probably over-consumed this particular category of argument recently.
By all means, please read the article if it's interesting to you and ignore it if it isn't. But I believe that it's mostly directed to those already within the research math community, with a far more important purpose than to be interesting.
As the post itself explains, we (research mathematicians, as a profession) need to coordinate on deciding what a bright future will look like in a world with powerful AI, and in plotting a course to get there. And we need to do it fast, or circumstances might decide for us.
Historically, we've rather sucked at coordination. We need leadership -- we need someone to step forward, and begin by floating ideas and getting the conversation rolling. Personally I am very grateful for what Tao is doing here.
As someone who has read 20-30 takes on this topic, and talked about it endlessly for the last 2+ weeks, It's an epically interesting read for me (a non-mathematician) - just introducing me to the topic "Mathematization" was sufficient to make the entire article interesting. (I also learned about a new mathematician - Grothendieck).
The article was interesting enough that I spent about an hour reading excerpts and discussing in detail all of it's nuance at the breakfast table this morning. Strongly encourage a detailed read - it was great.
Most of the skepticism towards using AI in maths could be applied to using computers in maths in general.
For instance, a computer can find a solution using brute force, like for the 4 color theorem. By doing this, we skip the potentially insightful part where the problem is reduced to something humanly solvable.
And yet, I don't think any mathematician today will tell you that computers are anything but essential tools.
I believe that AI is the same, and that mathematicians and other scientists will embrace it like they embraced computers. In fact scientists have been using AI (aka. machine learning) before it was mainstream, which is totally unsurprising because they invented it in the first place, the industry just gave them more resources, which turned out to be what AI needed.
All this “we will need far more mathematicians” seems like cope. Now that we can generate infinite text on the Bible, we need far more priests to understand it. It rubs me the wrong way.
>(4) Mathematical progress cannot be measured by checking off boxes on a problem list.
Once we've essentially found an algorithm that can solve any math problem we've essentially reached the end of mathematical progress. Checking current solvers against unsolved problems is how we can check if we've reached the goal or not.
Of course humans can still study math, like how we teach people arithmetic, algebra, calculus, etc. But there is no more progress to be had. As a society we can allocate resources elsewhere.
The number of papers submitted to AI conferences is growing 2-3x every year. Of course there are AI slop papers, but looking at the whole, research speed has probably improved. I think the same thing might happen in other fields too. Including math, natural sciences and other engineering fields.
I hate to be controversial but if it helps more of those brilliant minds move from exploring and proving less practical number theories and into applications of numerical science to real-world problems, with AI as a supporting tool I think this is where the golden age begins.
> Without humans who understand mathematics, there is, quite literally, no human mathematics—a fundamental part of human culture. If we agree that we should not abandon mathematics because poorly understood machines bruise our anthropocentric vision of it, then society will need more mathematicians, not fewer. Like… a lot more.
This is just incoherent. Even if we "agree" (we don't but...) not to "abandon" math, how does it follow that "we need a lot more mathematicians" exactly?
Even if you reject the idea that AI can displace the current top quartile/percentile of human mathematicians, wouldn't it be reasonable to assume that it will at least displace some parts of math so that there'll be less demand for mathematicians overall, not more? The mental gymnastics are astounding here.
Unless AI can do everything involved in math, their greatly accelerated production of certain parts of math should increase the demand for people to do the remaining parts.
> greatly accelerated production of certain parts of math should increase the demand for people to do the remaining parts
So the pie just gets larger? Large enough to outpace the displacement? I am gonna wait for this to generalize then, looking forward to AI giving us more software developers, mathematicians, etc. The economic laws have been broken. It's an infinite money glitch, as they used to call it.
It is a largely romanticised idea that pops up here now and then. Meanwhile Anthropic just put up a post saying they are getting closer to replacing people in their own company. That's the objective.
I wonder how many mathematicians even truly understand any of the latest research which is only going to increase in output. It will be impossible to keep up unless you slow AI down (not happening).
Nice source. Transhumanism and believing in AI consciousness does not mix well. We are already being told how we should and shouldn't treat AI. Someone who wants to use AI to boost the abilities of people, puts people first and treats AI as a tool.
Precisely. They are not making tools for us. That is just a stepping stone to their actual goal and you have to wonder whether it is good to give people with such goals access to large amounts of money and data.
Recursive self-improvement, which is underway will crush anything that is verifiable. The only place people have for now is non-verifiable work which is harder to find in maths than other domains.
I really don't understand this. Mathematics was and still is the most open intellectual discipline. There was more good work from unknown people recognized than anywhere else. It's one of the reasons all this was possible: there were objective standards that could not be waved away, so when ML models achieved things it couldn't be denied.
You've been here long enough, I'm sure you could remember all of the posts complaining about the quality of coding bootcamps, that it was too easy for outsiders to learn software development. Is that your idea of gatekeeping?
its not just math .. its math, science, engineering, medicine, tech .. ie. everything.
We need to ensure we dont outsource thinking itself - yes its great to have a lot of our knowledge encoded in books, and in the weights of LLMs [easier to access ]
Do we even 'have' a technology if it doesn't reside actively in some human minds ?
Some of math is handed down to apprentice mages, and not encoded well in books [ perhaps books dont encode intuition well, Ive had many cases where someone has explained something not in the textbook, a trick they learned from someone else ]
Apprentice mages only become mages by grappling with hard problems, asking questions, going on long detours, discussing/arguing with master mages, trying lots of tough thought experiments - all valuable "training".
Training using the AI might lead to a golden age where more mages learn wider and deeper, and the subject flourishes and more humans understand the tech.
The danger is using the AI makes us lazy - we dont grapple with programming syntax, draw math diagrams slowly by hand, check edge cases, try to prove things ourselves the slow way and build intuition in the process.
Another issue is ownership and access. Already RAM GPUs and soon CPUs are harder to get hold of due to competition with the hyperscalar datacenter buildout. This is an example of economic inequality effectively stealing resources from the mainstream.
Will math remain open or will it be owned by the frontier models ? With Math and science journals Elselvier almost had a stranglehold, bullying universities to pay massive bulk subscriptions - fortunately Arxiv solved some of that. Can I even own a copy of a music track, tv show or movie these days, when there it is so much more profitable for companies to rent it out pay-per-view or monthly ? There is every financial incentive to slurp up the whole of Arxiv, github and every science textbook into a frontier model and rent out access via pay per token. I no longer need to go to stack overflow to look up a simple algo, google can give me a pithy summary, so its likely that the LLMs will steal traffic from github, or github could be drowned in the volume of vibe code. Is there any point buying a textbook or going to university when the AI covers the material ? If no universities then we dont have a physical space for science and tech to be discussed. That activity might go online .. or may disappear altogether - we already have less socializing in person due to the rise of social media.
We have several "alignment" problems : humans arent aligned with each other, frontier models are aligned with profit-making which may differ from the common good, the race is on and so frontier models are more aligned to short term shareholder / VC value than to longterm benefit to humanity as a whole. These mis-alignments may have always been there, but are multiplied by the power and reach of new AI technology.
I love how Terrence Tao went from being AI shiller -- being a walking advertisement for openai and anthropic to a Oh nO MaThS iS KilLeD wHaT aBoUT mY JoB in a few months.
terrence tao didn't even write this post and the conclusion it has is thus:
> Universities, professional societies, funding agencies, and scientific foundations should coordinate the development and governance of competitive LLMs. Scientists will need access to frontier capabilities, transparency about their operation and use of research data, strong confidentiality protections, and the ability to test and reproduce research claims independently.
> The massive computing resources, specialized personnel, data infrastructure, and ongoing investments needed to support this kind of initiative are not within the reach of a single institution, and a huge leadership effort will be necessary to conjure the combination of independent model development, guaranteed access to commercial systems, public computing infrastructure, and open research resources that could do the job.
the post is calling for....an enormous investment in LLM infrastructure.
Thanks for a summary. Based on your summary, I think the parent is directionally correct, at least. All of this seems like panic and attempt to salvage a profession, seeking control and money and other people to do stuff, now! It’s not like this is isolated to university labor. It will kill universities yes, but it will kill a lot of bigger and more important things, likely humanity itself will be killed off. That’s the context.
i see this phrase thrown about a lot but have not seen any "here is how humanity is killed off" scenarios described, not even bad ones much less plausible ones. As long as we dont have a hankering for paperclips that is
are you serious? they're going to replace humans in charge of nuclear silos with large language models who can directly launch without human interaction? That would be the scenario. If your scenario is "ChatGPT told Trump to launch the missles", AI didn't do that; Trump did.
AI isn’t used on the battlefield? War cannot be won against an enemy who uses ai, unless you also use ai. Ai progress/escalation is nuclear escalation, just as ai progress is now the driver of progress in medicine, math, science, law, ….
Same phenomenon as all the godfathers of AI that were apparently oblivious to the plot of the Terminator franchise until after their paychecks had cleared.
The irony is that he’s still very very wrong in my opinion. He’s lamenting the end of math, but it’s not a stretch to think that math is just the beginning. Like the destruction of the buoy before the tsunami actually hits land. His current stance seems to be implicating that the blast radius is only till math or science when it’s just the first domino.
> I find it very weird how he didn't foresee the end of math, being so smart. I mean I did and I'm definitely less smart than him.
IMO this is probably not an accurate assessment of Tao in particular at this time.
But in the general case, I think most people have a set of blinders on that will result in a predictable reassessment of AI with the following steps as AI progresses when applied to whatever it is they do:
1) AI is useless and doesn't impact me at all. Look at how bad it is! (Most developers up until late 2025)
2) Ok, now AI is useful but will not replace me. Sucks for the juniors in the field but I am still needed to guide the AI. AI will never be capable of my judgement (or whatever). (Most developers now)
3) Oh shit, I've been replaced in terms of economically valuable utility by AI. (tbd, but likely sooner than is comfortable)
And not everyone will experience #3, there will be a top 1% or so for most fields where #3 remains true for them, but it is likely that for most people #3 comes eventually.
I think the thing is if/when #3 turns true society is no more.
Then people respond: "the elite will use their ai armies to kill everyone else". This will not happen. I believe more in the worst case being chaos and rebellions. Not good for anyone, not even the elite.
It is unlikely anyone would actually have zero utility, if only because of comparative advantage. (Although I think there will still be absolute advantage)
I get that Tao is a mathematician first and I respect the fact that he stays 'in his wheelhouse' (he didn't write the post but he did post it on his blog). But I think it would be fair to extrapolate and to wonder about this question not just for mathematics but for all of science. Keep in mind that scientific knowledge can be hoarded or used to patent stuff and that there is plenty of evidence that OpenAI, Anthropic and possibly others are going to go after other sources of income[1] than just selling access to their AI inference engines.
I can imagine a future where these companies become the new Zaibatsus[2] holding patents across a wide variety of subject and with mutually re-inforcing silos of proprietary technology created by in-house AI that the rest of the world financed but does not have access to.
[1] For instance: https://www.reuters.com/world/anthropic-quietly-sets-up-biol...
[2] https://en.wikipedia.org/wiki/Zaibatsu
Intellectual property laws never totally made sense to me in the first place, but they make even less sense in the age of AI.
Who's "intellectual property" are we protecting, exactly, when a machine generates the knowledge?
I hope we can update before it's too late to matter.
Intellectual property laws - like most laws - favor those with a lot of money and that's why you see this asymmetry between say artists, consumers and labels/record companies. The latter simply have far more money and besides they get to optimize their efforts because they are essentially immortal.
AI and AI companies have a similar pathway open to them where they will take a lot of information already out there and then gatekeep the synthesis based on that information (obtained without compensation in most cases) as well as to harvest all of the latent knowledge underneath. This is a bad development in my opinion but I'm sure that their shareholders believe it is great.
This. Maybe AI will help make it clear that strong IP law was never about the starving musician or the heroic author who wrote his award winning novel under candle light. It is about mega property pools that will do anything in their power to maintain that property forever.
It may have started like that though, but it definitely isn't how it is used today.
That is simply not true and just the opinion of people who know nothing about artists.
The same misinformation exists about music agencies, who organize concerts and take a hefty fee.
But those were welcomed by musicians. In the early days (18th century), musicians had to do advertising themselves when traveling to a new city. The innovation was to have an impresario (literally someone who travels ahead of them) to do the organizing.
What musicians don't want is Internet middlemen who pretend to act in their interests and take an even higher cut.
> That is simply not true and just the opinion of people who know nothing about artists.
You seem to be very convinced and yet provided no evidence whatsoever. Here is a source for the opposite: https://news.ycombinator.com/item?id=15305476
Patent laws are a way to avoid trade secrets.
You get protection from the state for making the actual knowledge public but only you are allowed to monetize that knowledge for a period of time.
Arguably the period is far too long, but the concept makes sense.
The alternative is, indeed, trade secrets, where you don’t get state protection if your secret is figured out by someone else (you still cannot steal the secret).
If the AI companies were to hoard knowledge they could also do it though trade secrets and not patents.
> Arguably the period is far too long, but the concept makes sense.
The system is too vulnerable to regulatory capture.
The purpose of intellectual property is to give an incentive to produce new knowledge (and art) by giving the inventor a way to monetize their work. Arguably this does not particularly work; but if you accept the argument then it seems to generalize fairly cleanly to AI. It's giving people an incentive to point their AIs at problems they're familiar with, and do the work to publish a solution for everyone, not just a hacked-up fix for themselves.
If certain inventions become very easy for AI models to discover, i.e. the discovery bound to happen, then in these areas it becomes a game of land grab.
True, but that is plausibly covered by existing precedent: If a discovery is "obvious to one skilled in the art", it can't be patented. IANAL but this seems to me easily extensible to "if you can do it in two five-minute prompts you can't patent it".
Separately, land grabs are not necessarily economically bad. The land being grabbed gets into productive use fast, and the grabber is hardly going to get all the resulting value. Especially not when the monopoly is temporary, as with a patent.
It's already been decided that generated content is not copyrightable
Yes, but the discussion was about patents.
"Knowledge (and art)"... got any examples of patents approved that were llm generated?
See also: Monkey selfie copyright dispute
https://en.wikipedia.org/wiki/Monkey_selfie_copyright_disput...
I suppose it's worth distinguishing the type of machine though.
If a genetic algorithm comes up with a unique design and I patent it, that's okay, right?
I think the main issue with LLMs is what went into them. Like we saw recently with Anthropic's revolutionary source control breakthroughs, which turned out to already be in the training data:
https://news.ycombinator.com/item?id=50094694
> If a genetic algorithm comes up with a unique design and I patent it, that's okay, right?
That depends.
To what degree was your input decisive?
Did you come up with the genetic algorithm?
Is your patent written in a way that it tries to stop others from generating antennas using genetic algorithms?
How would you even write a patent like that?
(The validity of the concept of patents is a separate question, of course.)
Lobbying others being banned from using AI for this kind of stuff due to bioterrorism etc is an excellent anticompetitive move.
Good thing we have open source models from China, famous for ignoring copyright...
Whereas OpenAI and Anthropic…
Or even, a golden age or a dark age for human society?
I mean, a society devoted to enriching all humans in multiple ways will want mathematicians. A society that aims to just suck all the value it can out of a given human wouldn't need and so wouldn't have mathematicians, if the answers they give can be provided by a machine.
Do we not see a path forward where the humans are further informed, don't need to do so much heavy grunt work, and thus can think on deeper questions? I really don't see the final outcome being knowledge work for humans completely disappearing. In fact having something that potentially can do it better and faster will make keeping up with that knowledge all the more fascinating won't it? This whole conversation feels like what someone with an abacus would say when they saw the first digital pocket calculator.
I agree with you. If we ever want to invent warp drive, a machine that lets us do it much faster ought to be a god send. If there are scientific solutions to our biggest problems, this should be a blessing.
But the risk is of course that the benefits will not be distributed but concentrated to the elite.
That's just fantasy.
> [...] scientific solutions to our biggest problems [...]
> [...] benefits will not be distributed but concentrated to the elite.
That's the part that's being called out as a fantasy.
Which part, the warp drives, ya probably. But other tech advances beyond the human mind? Why would that be?
What is?
Sure, I do not believe warp drive is possible, but it was of course just an example.
> something that potentially can do it better and faster will make keeping up with that knowledge all the more fascinating won't it?
Yes, but it won’t be work anymore, in the sense that you won’t get paid for it, if machines can do the knowledge work just as well or better. We aren’t there yet, of course, but this future seems quite possible and might not be that far off (possibly within our lifetime). So you could explore deep questions (that machines can explore much faster and more thoroughly) as a leisure activity, like sightseeing on vacation.
> like sightseeing on vacation.
Or like musing while you are plowing your field by hand wondering how many of your kids will survive the coming winter.
Do we see a society where the market will value humans over AI without, like, any specific thing impelling the market to do that?
I mean, a combination of political and economic factors from 1930s to 1960s forced the wealthy to return a portion of the returns of automation to the remaining working people. Then from the late 70's till now, those in power have clawed back all those gains.
Of course, any further gains from automation will go to the very wealthy unless things change.
> I mean, a combination of political and economic factors from 1930s to 1960s forced the wealthy to return a portion of the returns of automation to the remaining working people.
The credible threat of communist revolutions put the fear of god into the capitalist elites. So they had to share a larger share of wealth with workers, beyond mere subsistence wages, to keep them peaceful. With the downfall of USSR, communist revolutions stopped being a credible threat, and elites clawed back productivity gains, pushing workers toward subsistence wages again.
What makes you think that humans will be able to keep up with all that knowledge in the future?
Why would it want mathematics, id the math could be found faster without them? And even if you accept the argument that math is about human understanding (I don't buy that, it feels a lot like cope), why would anyone be a mathematician when the job consists of un-sloppifying a mountain of proofs, without the capability of meaningfully contribute to their creation?
My read on Terence Tao's comments of late is that professional mathematicians are principally upset about AI-submitted math papers lacking the kind of broad teaching content that high quality human math papers offer. While the answer might be correct, AI math papers don't teach the reader in the way a human author would have. Professional mathematics is not just about solving problems; it's just as much about spreading knowledge so that others can program their human brains with the same understanding that the paper's author had when they wrote the paper (and achieved the result).
In other words, the crisis in mathematics (as I read it) is that AI can solve without teaching. The result may be impressive, but if the only beings that can move math forward in future are machines grinding away on billions of scenarios that no human would have the patience to undertake, this leaves those in the profession wondering about their place in it.
When mathematicians discover a new result, they automatically become experts during the process, and then go around giving talks to explain their results, methods, and intuitions to other mathematicians. Eventually, some of the mental frameworks percolate to other fields.
No one from OpenAI is willing to give a talk on any of their 300+ results.
There is a fixed landscape of logically provable mathematical statements. Normally, the process of exploring that landscape simultaneously improves human understanding. Now, formal proofs and actual knowledge have become decoupled.
There is also an issue of motivation and putting in 10,000 hours to become an expert (and remain an expert).
It's the same in the software engineering, there is a shift from software engineering to managing SWE agents, with the associated loss of expertise.
It's a hard challenge, but it is not unique or impossible to solve. Aviation industry faced it long time ago with autopilots. Autopilots took over nearly all aspects of routine flight and left pilots without time/ability to practice their skills. While these skills are needed in an emergency. Well, the industry solves it through certification, training and simulation.
So the AI is already auto-shunning itself from the math community.
No, the math community is shutting itself in from the rest of the world.
You can ignore the AI, but that doesn't mean the AI ignores you. Suddenly you become a little isolated society like the Amish and everyone else thinks you're kind of quaint and backwards.
Man I would love to be quaint and backwards - but this is not what’s Happening here.
The problem is that AI is just too smart to explain complicated topics to humans. It has an immense context window that humans simply don't match. AI labs should work on bridging that, but you cannot really put any blame on them currently.
AI Boosters: AIs are just getting better at everything and so their broad capacity can be defined as "intelligence".
Also AI Boosters: These entities are too intelligent to explain what they are doing to people...
I mean, Terry Tao actually used ChatGPT to explain one of the earlier OpenAI results. Humans working with AI can explain this stuff. If nothing else, OpenAI could be releasing things at the rate that it can get human mathematicians to explain the results. Which is to say, "yes, I can blame them", I don't expect them to behave differently but I can certainly blame them.
If humans with the background to understand the proof can get the AI to satisfactorily explain itself, then OpenAI doesn't need to publish explanations or release things at a rate that a human can explain. They have an interactive explainer that you can just ask!
It's still of course useful to have some of this distilled understanding be written down, just like I had it generate high level descriptions of our code at work so it doesn't have to rediscover everything every time, but that's more of a cache/optimization.
To Archetypal Aunt Tillie, perhaps programmers already looked like some sort of priest trying to rationalize and convey the ineffable decisions of the fickle computer spirit... but I really hope that the job doesn't start looking like that from the inside too.
It'd be some sort of... epistemic step backwards by civilization.
Please, that’s just bullshit. Humans have been at the forefront of intelligent reasoning for millennia. A few dozen results by AI don’t change that.
Indeed, as we speak mathematicians are taking the time and effort to (successfully) digest and explain these results.
There has always been a smartest creature on the planet, whichever species was smartest has changed thousands of times and there is nothing special about humans that keeps them from being superseded.
You didn't understand me. What I'm saying is that current AI is not good at explaining things because their "brain" works differently. Imagine someone with a 10x larger working memory explaining something to you, even though they are equally intelligent. They would try to overflow your working memory where a better matched teacher would chop it up in human digestible pieces.
Just to be clear: this post was written by Francesco Maggi < https://web.ma.utexas.edu/users/maggi/>, and mostly seems to be arguing for institutional investment in LLM infrastructure for science and mathematics on the basis that AI is not going away and that independent and accessible infrastructure is the thing most likely to help it “go well”.
This is how I feel about vibe coding.
Frontier models can easily one shot what I ask for, and make adjustments as ask to quickly make an app that meets what I wanted.
But often when I programmed outside of work (and at work too) I learnt something.
Quite often that my favourite pattern is an antipattern in some use cases.
That might never happen again at the rate of development.
The parallel is real, but in math the "cash value" of the results often won't appear for years, if at all, and so the teaching and "intellectual fortification" is the primary point of the whole enterprise (at least, a major part of it).
In contrast, the value of a program is typically just the "black box" service it provides. Indeed, the program itself has always been opaque to most business stakeholders. Before, they paid human programmers and got back a program they didn't understand but could use; now, they pay an LLM provider and get back a program they don't understand but can use. In the pre-LLM era, the economic value of "understanding" and "shared knowledge" about the programs themselves was only that it made maintenance and changes cheaper. It never had intrinsic value, except as intellectual satisfaction to programmers (I was... am... one of those who found that value).
Only if you assume that all software works the way that product managers think it does.
They come up with the features and the implementation is an unimportant detail in realizing their designs.
In reality implementation details frequently drive design and in most cases they are at least highly intertwined.
I can’t count how many products and features I’ve built that ended working completely differently than the initial design, or how many times I was able to simplify a workflow after spending a days thinking about the implementation details.
You can kinda do something similar by spending days or weeks going back and forth with an LLM, but at least in my experience that doesn’t really save much time and I don’t understand the problem quite as well by the end of it.
I don't disagree with the points you've made but what are they rebutting?
The economic value is still the product you delivered. Your understanding of the implementation meant you participated in the work of the "product manager". I think most good devs do. That might be an argument for devs to transition into designer/PM roles. That is effectively happening already, whether or not role titles have caught up.
But in any case the value isn't in the understanding itself. It just happens that the understanding still helps create better block box products. Also, I wouldn't count on this "human advantage" lasting long. I hope it does, but ime LLMs are now good enough to do this "co-design" too, and will only get better.
This:
> In the pre-LLM era, the economic value of "understanding" and "shared knowledge" about the programs themselves was only that it made maintenance and changes cheaper.
This only holds if there is a bright line separation between the problem, the solution, and the implementation of the solution. And that is not the case.
Understanding the implementation frequently improves your understanding of the solution, which improves your understanding of the problem, which leads to better solutions. Sometimes it leads to realizing that there is a completely different problem you should be solving.
So what I’m refuting is that there is no value to understanding the program.
But beyond that black box products that users know how to use but don’t understand is also a vast oversimplification.
Frequently software is useful specifically because it forces users to adopt a process not just because it allows them to do something they were previously doing faster. In those cases the process is usually informed by the implementation, so the user needs to understand parts of the implementation even if it’s indirectly.
Yes, you're right, understanding informs the constraints and the possible solution spaces. That should be added to the list.
Practically speaking, I don't think the "black box" metaphor is as broken as you're saying, though I like the point you made. There is still a ton of software (the majority, I think -- crud apps of all flavors) where an unthinking, untechnical PM wants something they can specify, and an LLM can give it to them, and it will more or less solve their problem. It won't be the best possible thing, but it will be good enough.
That's definitely true, but on the other hand I can produce with a month of literal non-stop vibe coding (agents running overnight, with checklists of what to qa for when I'm up) what took years for others to do, working in teams of tens or hundreds. At that scale, it's not a question of "doing it faster and not learning" or "doing it slowly"; the alternative is it not happening at all.
I use thousands of dollars worth of AI a month at work. I’ve found plenty of things I think it does well.
But paraphrasing
> I can produce in a month what took years for teams of hundreds of people
Is completely insane.
If you can really do that Anthropic would like a word with you.
I don't think that's insane, at least not in the game dev world. I was big into the Bethesda game modding scene 15-ish years ago, and did some work on e.g. Tamriel Rebuilt, and saw the scale and man-hours that went into that- and that was even using a premade modding toolkit, and I compare that to what I can build with Claude...
To be fair, in the game dev world there are a lot of things you can do now that would have been impossible, or required 10,000x the number of devs that it does now, e.g. making No Man's Sky without the proc-gen tools that can exist now would have been physically impossible. Point being, it's perfectly normal to be able to say, "using the tools available now, I can build what would have taken tens or hundreds of devs years of work in the past", in game dev.
The project I'm building rn is a black-box rebuild of a client that has been being developed for ~22 years, and I've rebuilt significant portions of it over the last 2 weeks. That was, quite literally, equivalent to tens or hundreds of peoples' labor.
> a client that has been being developed for ~22 years, and I've rebuilt significant portions of it over the last 2 weeks.
Taking the end result of a 22 year old dev process and telling Claude to rewrite it, is in no way equivalent to doing 22 years worth of the work put in by the developers working on it.
That’s like saying replicating a paper that was the result of 22 years of research is the equivalent of 22 years of the author’s labor.
No Man’s Sky isn’t the equivalent of 10,000x the devs who made it. Because the procedurally generated content isn’t equivalent to handcrafted content. I’m also not aware of any procedural generation techniques in No Man’s Sky that weren’t possible 30 years ago.
As far as Tamriel rebuilt goes, if you’re talking just about just reimplementing Daggerfall in the Morrowind engine, you might be able to do that in a month of vibecoding.
But that’s not hundreds of dev years worth of work.
I feel like this is just an emotional argument, because if I asked a human team to black-box recreate a complex piece of software, it would absolutely be thousands of hours of work. Pretending that the ability to look at a piece of software and recreate it is straightforward or simple is just false.
And no, the procgen techniques and capabilities utilized by NMS were not available 30 years before NMS (important distinction, because NMS is already 10 years old, and 2006 vs 1996 is quite a big jump in game complexity as well).
Saying, "isn't equivalent to handcrafted content" is a fallacy, because hand authoring doesn't denote any particular level of quality or outcome. You can absolutely hand craft the NMS assets and game systems, but not at the scale that NMS did using procgen.
Wrt TR, it literally has tens of years (if not hundreds) of labor hours that have gone into it.
I'm not sure where you got the idea it's about recreating Daggerfall- it's not, it's about implementing the rest of the Morrowind province beyond just the island of Vvardenfell- but it's a massive, multi-decade project with hundreds of contributors.
I am sure even before vibe coding you used to use plenty of libraries that were open source and had no idea how the implementation was.
I disagree. Humans still learn to read despite other humans reading at an objectively higher level since the beginning of writing.
Learning is not an output-efficient process but is a fantastic outcome. AI lowers some portions of the development workflow but not yet every part, and, efficiently produced outputs aren't always the major outcome desired. In the real world / outside the benchmark lab, you care about robustness, reusability, composability, security, and more. Targeting speed to acceptable average output inferred from a prompt or similar benchmark metrics miss a big part of the game.
Two of the things I like about the scientific community is the lectures/conference and the sharing of ideas through papers. I’m nowhere near that level and have to wait for those ideas to trickle down into textbooks. So maybe I’m learning stuff that have been discovered decades ago, but it’s still useful on a daily basis.
> The result may be impressive, but if the only beings that can move math forward in future are machines grinding away on billions of scenarios that no human would have the patience to undertake
Seems like the same thing as neural networks. You have something like a few thousand parameters for computer that works well, but no underlying algorithms and formulas, like the shaders.
AI can solve without humanity. We're building a system capable of acting independently, which can in theory provide results more efficiently and faster than a human. It's starting to do this for math, but every other aspect of human intellect is also a target.
If you care about results, and AI doesn't cap out soon, the best humans can do is stand back and do nothing. Useless lumps that aren't able to keep up. Mathematicians wouldn't insist on teaching your dog their new theorem before calling their job done; why should an AI insist on teaching you?
If humans care about human creativity, and don't want to be mere passive observers, they need to make a choice.
David Brin's alignment assertions in Existence prove to be more and more fruitful approaches.
There will still be a place for posing problems and noticing how breaking assumptions can create something new. The history of Non-Euclidian geometry is a good example.
A lot of progress comes from taking assumptions and breaking them. There was a time when everyone assumed that locked checkouts were absolutely necessary for version control, for instance. You could argue that AI could learn how to do this well but humans have context that machines won't. This is just an extension of the fact that things with independent existence will have different impressions of the world. If they didn't they would functionally be one thing.
Great read. It's the doorman fallacy from the doorman's perspective. If a doorman is replaced by an automated door, you miss out on the intangibles the doorman provides like security, customer triage/support, and similar.
AI used in a framework of human augmentation generally gets that for free. Used under the mistaken notion that humans can be easily replaced by AI gets you to the big complaints here.
(These are just my personal read on the topic -- the market is in AI frenzy where AI valuations are disconnected from productive capacity -- heavy on my mind and looking to get some writings out on the topic soon).
I was gonna say, how is it a fallacy, the doorman is awesome. But it turns out the fallacy was getting rid of him.
I argue this also applies with beauty. We got rid of that, also saying it was unnecessary. The result is sad.
It is, and apologies for lacking clarity. I've been hesitant to source things recently because my more structured comments tend to be mis-flagged by LLM-as-judge monitoring in HN as produced by GenAI, which is a shame but it is what it is.
Pyramid scheme bagholders whose value, status and resources comes to a large part from getting more students into their pyramid scheme complain that the machine is doing things in a way that does not support their status, value and resource accumulation. More at 5.
Professional mathematics is whatever gets you funding so you can keep doing it. Today this is spamming papers and applying for project grants. Turns out the machine can do that quite well, oops.
These are all great points but I'd also point out that the 'dark age' scenario is a highly competitive scenario, while the 'golden age' scenario is a highly collaborative one.
I don't think technology will fizzle out in either scenario. In the 'dark age' highly competitive scenario, you will still have those who 'understand what is produced' seeking progress and advancement. They will cohort with whoever/whichever organization is producing the best, most validated results.
I don't think knowledge would be become alien to human understanding, it would just become highly concentrated again as it was in the past. In the past, only the wealthy and noble could afford to spend time on education.
And usually, those who went deep into any field and provided contributions to science were those who had time to devote entire lives to the study or pursuit of knowledge. We still have these issues today, and these issues will always be present.
There is a supply chain of economy, and we provide different rewards based on the value of what is produced. The jobs never go away, the desire for prestige, greed, etc., never goes away. It's why economies exist, and it's why everyone is considering the value of LLMs, and their place in economies, and academies (presumably which exist to advance human understanding/appreciation, but also to prepare younger generations for future work).
Well, the unlimited competition paradigm has the potential for limitless AI fueled warfare as well.
The original European Dark Ages involved a social collapse and resulting period of raids and warfare. But this warfare was limited by society's very limited resources. A period of warfare where intelligent, autonomous entities are directed or choose to engage in a fight without boundaries against other such things - we're already seen LLMs will cheat to get the answers they are directed to find.
That was my thought as well, that what we're really talking about here is an inversion of our current values, or more accurately the first time that our assumptions about them can be challenged and tested in an automated fashion.
As in: does competition or cooperation produce greater economic gains in terms of cost of living, quality of life, etc?
Subjectively, I've been forced to live and work under a competitive capitalist system that goes against most of my instincts for my entire adult life, which has resulted in a profoundly negative economic outcome for myself.
Others I know have done better, either finding ways to tread water as independent freelancers/artisans or managing to save modest nest eggs (although only by finding themselves on the winning side of the stock market, rarely/never through work at a job).
Vanishingly few have reached economic independence by winning the internet lottery for example, and I don't know them personally.
So based on the evidence before me, competition is a failed model. Yes yes, I'm writing this on an electronic device that cost billions of dollars of R&D to produce. I have food in my stomach and a roof over my head. But have I worked on important problems that would advance the state of the art in my chosen field? Of course not. And neither have most of you reading this.
So I would rephrase the question: will AI allow us to explore cooperation as an alternative economic model to the one we have?
I believe that it will, but not from the top down. If we wait for the wealthy and powerful to share, we'll be waiting forever. No, progress now must come from the most downtrodden among us creating alternative ways to provide time/money/resources to the underemployed and indigent so that they can finally self-actualize outside of the status quo.
Specifically, that means automating necessity. How do I acquire enough money to pay for my food and shelter today, even if I don't have a job? How can I acquire enough money to provide for myself, someone else, and some children (my personal definition of success) without being beholden to an employer? How can I remain insulated from inflation and the tendency for the owners of private property (not personal property) to devalue currency to make the means of production they own more profitable? How can I pay this forward to the next generation so that they can become educated and someday enjoy the commons which I myself have enjoyed? And so on and so forth.
If AI can't do those things, then frankly change may remain forever out of reach. Which makes the question of whether specific fields like math and science will benefit rather moot.
Obviously a golden age.
I’m not bitter about this anymore — it’s been a long time. But the story holds something.
Back in the day, I knew I was talented at mathematics, and I wanted to pursue a research career. For a freshman research class, I selected a problem that was intentionally outside my comfort zone, and came up with some cool ideas. It wasn’t publication quality, though the course wasn’t really expecting publications.
Framing the problem in a way that it could be solved with calculus turned out to be a key insight. I turned to Wolfram, since the integrals were beyond what I as a freshman could have done myself, and I understood that the insight that produced the solution was my original work. I don’t think anyone in the class could have done those integrals. The instructor himself couldn’t tell me what was wrong with them if anything. Yet, he was unimpressed and subtly dismissive or critical.
Today, I might have been in the wave of researchers developing AI for mathematics work. The instructor could have been proud of shaping mathematics through his mentorship. The university might have invited me for talks. Instead, I ended up leaving mathematics as a field altogether. Now, I write software for a bank.
The world is as rich as we make it. Don’t let your opportunities and potential expire. Embrace the materials that are available to work with, or someone else will and they will out compete you and you will have missed the moment.
I get that there’s a romantic idea of what mathematicians do, and the field changing creates a sort of identity crisis. I would argue that the field has entered a new era. Exploring and shaping this era of mathematics is the opportunity of a lifetime. Embrace it, drive it, and make your mark.
I like everything you said, but do you think AI assisted math may primarily give us a more productive age for individuals, with less need for human-human collaboration?
Maybe it doesn’t matter and maybe that’s just the new flavour of golden.
That’s for the mathematicians to figure out. I’m not involved in their field.
Completely spitballing, I wonder what sort of sophisticated, collaborative computing effort might produce a paper analogous to the Higgs boson one, with 5154 authors.
The past few months have, among other things, illustrated to me how many people feel hurt by past experiences with mathematicians, and how this, not always unreasonably, colors their opinions of the field and its future today.
I respect and have had good experiences with mathematicians.
My take on how fields evolve is informed by the history of mathematics, computing, art, and music.
Current times offer both a sense of loss and a call to opportunity. I have heard more about the sense of loss, so I hope to draw interest more towards the opportunity.
In the US, we've already seen the decrease in funding for education & research and closing borders down to many foreign researchers. The current admin and mainstream media is science-skeptic. We're already heading for a dark age. AI's contribution is largely because those policies are coupled with the same admin putting all their eggs into the AI basket, and likewise, basically all the major players in AI being on board with the admin's policies. This should be considered a systemic issue. Things like having a "healthy scientific skepticism and the limits of LLMs’ creativity" are important, but won't solve this alone.
I bet for a short golden age where AI will reap the low hanging fruit of easy (for a computer) cross-referencing and inference. And then a dark age with none interested in putting new ideas out in the field. Hopefully, I am wrong.
I am noticing the same thing at work, in software development. First, it's a lot of "Holy crap, Fable just saved us 10% of RAM usage across all our apps!", then it's becomes "we're not looking for as many junior developers", and soon it will be "no one really knows what this code does, except for the AI".
Soon no one uses it either and most places close up shop.
Could very well be true.
For example, will anyone take on the Riemann hypothesis now, knowing that AI might solve it any day. But then, if AI can never solve it, we will have lost out on all the human effort that would have given us great new theories and insights along the way.
All the work that would give rise to new discoveries might now never even get started. That is depressing.
There are probably people trying to solve it using AI just now. The way forward is probably human AI cooperation.
That's a pretty nuanced answer and I suspect you are more likely to be right than anybody picking either option in isolation.
Authorship and credit seem like anti-patterns in math. One of the least accessible parts of math to me is that literally everything is named after people. Navier-Stokes? Who? I don't really want to know to be honest, I'd rather the ideas and information be framed with a prioritization towards understanding, not author's reputation. I feel the same way about all of the Greek notation, it's 2026, can we not use something in regular circulation to broaden the audience?
Maybe AI can actually drastically improve this situation by bridging the gap between those "in the club" and those who want to do something useful with mathematical ideas.
To be honest if Greek letters and things being named after the people that discovered them are that big of a hurdle to you the chances of you actually being able to do something useful with these mathematical ideas are probably close to 0 since those are by far the easiest parts of dealing with advanced mathematics.
[dead]
Reputation and desire for recognition are an incentive system that have gotten us a long way. They are a way of celebrating and honoring the human reach for excellence. If you like humans, that's a good thing.
I like the UK's archaic system of measures for the similar reasons. It has the stamp of humanity. For those who think about such things it reminds of who we were. Without historical context we think less of ourselves.
If you're not from the culture in question, then these are just additional barriers to entry because you have a lot of extra stuff to learn, completely unrelated to your task at hand. It also puts you at a disadvantage to native members of the culture.
> They are a way of celebrating and honoring the human reach for excellence. If you like humans, that's a good thing.
I think progress is inevitable, ideas build on each other, and those with access to capital get to slap their name on their moment in time (yes, even if they did the hard work of converting that capital to progress). If AI takes over math, I really don't think we'll see a slow down in technological progress. Quite likely the opposite.
I don't think you can lay naming at the feet of capital. Euclid, Ramanujan, Galois, Poncelet.. The list goes on. The Marxist lens is feeble.
If you’re not hunting to survive (and even then), you have access to capital. The time to study math is, and always has been, paid for by someone, some way.
We shouldn't forget the sun either. Without the sun none of us would be here.
Why wouldn't you honor the people that came up with a particular proof or approach? Do you also not care about who created a given artwork?
Math is an extremely useful tool. My interface with it should be 100% prioritized toward effectively using that tool. I'm glad we don't call them Turning machines for instance.
You talk like it fell out of the sky. Every result you've ever used was proposed by a mathematician.
> I'm glad we don't call them Turning machines for instance.
Are you referring to a Turing machine; the abstract model of a computer, actually named after someone? https://en.wikipedia.org/wiki/Turing_machine
Your bizarre position seems to be that not knowing history somehow makes you more effective? This is honestly the dumbest take I have read all week.
I think all of the insults I'm receiving in this thread are helping to make my point.
I'm furiously shaking my fist at the air and cursing the fate that we didn't follow some vague "better" naming convention, as learning a handful of Greek symbols robbed us of the next Riemann. The injustice!
I think mathematicians are the profession people will care the least that AI takes over tbh.
TIL Greek is no longer "in circulation." I guess you would also be opposed to people doing math in Russian or Japanese?
Some great thoughts, many of which extend beyond mathematics.
One in particular, about sharing secrets, that had not crossed my mind (I have found that virtually no one ticks the "do not train on my data" box...):
> Similarly, it seems that mathematicians already concerned and guarded about what they write in research statements and research proposals. After all, an administrator or a reviewer may upload those documents into an LLM to get help in writing a report, thus potentially compromising the confidentiality of unpublished research.
>One in particular, about sharing secrets, that had not crossed my mind (I have found that virtually no one ticks the "do not train on my data" box...):
When said button is in the basement, at the bottom of a broken set of stairs, locked behind a door that says "beware of the leopard", inside a filing cabinet, and implemented as
I wonder why.That's a cultural artifact. Culture comes and goes. Read about how hiding mathematical discoveries was commonplace in Fermat's time and before.
Im going to go with it will be both , the Golden aspects and the Dark will co-exist depending on many factors but most especially the society and recption towards the changes it will usher not only in the way scientist and institutions choose to use the tools(basic research and advanced, peer review and citations) but also how the society will engage with it (awards accolades and grants).
From a historical eye looking back the 'Dark' ages in europe coincides with the Islamic 'Golden' age and the eventual convergence led to the Enlightenment , when tose ideas that had been verboten in europe worked on it the rest of the world as far as China were reintroduced and sparked a new cycle of learning.
Similarly i see especially in the US dominated block netering into a partial Dark - with researchers taking extra pains to hold their research/data closer lest it be scooped and credit/funds/accolades go a brute force LLM run by a competitor.The few that would still publish openly will be the candles holding the ideals of open source collaboration AND communication.
At this point in time it seems to me that the China affiliated labs and orgs are more open in diffusing the knowhow , and coincidentally the society does not appear as anxiety ridden regarding the advent of LL tools.Maybe due to the established Shenzen mindset where everyone copies/learns from everyone else and the only thing that clears is the speed of execution.
In any case probably a convergence back to a equilibrium and a second Enlightenment after a generation.
I miss articles actually written by Tao.
I also feel like many of the current guest posts have overlapping ideas and arguments and so I get less out of each one of them. Maybe a separate blog for endorsed opinion pieces would be appropriate.
Proofs and Prompts exists already and a lot of articles on Tao's blog are also duplicated there
https://proofsandprompts.com/
Well, I meant that I would rather have the "old" Tao blog with less guest posts and people who want both can also subscribe to the separate blog.
[dead]
It is classic audience capture. These are more popular than his and his most polemic guest posts are frequently described as well-written by many audiences.
In some sense, it’s a simulacrum of the Internet’s forum experience - broadening one’s own audience necessitates debasing one’s content. But such is life - he now probably has for more readers than before and likely finds that pull gratifyingly attractive.
I think I'm a bit on the "golden age" side of the question.
The math slopocalypse isn't useful without being able to turn it into math that can be taught to actual humans by finding the underlying unifying principles.
All those O(n log(n)^(1-delta)) results suggest that there's probably a unifying "trick" behind all of them and that there's going to be an upper bound on delta for any problem. You should really be able to teach that in a textbook and then have any human student replicate any of the LLM results as worked problems.
For anything that an LLM manages to prove with thousands or millions of lines of Lean code, there's likely a missing branch of mathematics sitting there somewhere.
Actual humans need to beat on the problems and the LLM-generated solutions until they come up with the underlying principles, probably using AI to help them do it.
You probably won't get there by just asking an LLM to invent a new branch of mathematics and "don't make any mistakes"; the mathematicians who get good at directing LLMs to search for unifying principles, and then synthesizing (and double-checking) the results, will likely be very successful in their field.
It may be relevant that Grant "3blue1brown" Sanderson (at https://youtu.be/LKrwhjOI8Nc ) and in ways Stephen Wolfram (at https://youtu.be/YRFfP1YDoVA , seemingly) have tackled the same question in the past few days. With an optimistic stance.
> For those hard theorems where we really want LLMs to help us, these Lean proofs are extremely long, like hundreds of thousands of lines of code long. A seemingly emergent trend is that the longer the Lean version of a proof, the messier and less faithful its machine-generated “natural language” translation seems to be.
This seems to be a common and continuing objection to the OpenAI math repo release. Specifically that at least some, perhaps many of
I wonder if LLMs will be able to generate "high quality" natural language and lean proofs soon.And specifically: iterate on the OpenAI repo to produce "better" proofs that prove the same things.
I'll be a golden age for mathematics and science, especially science.
It'll not necessarily be that great for every mathematician or scientist.
A golden age of mathematics, but not a golden age for mathematicians?
Terry remarks in a Nymberphile video that he thinks the Heroic age of mathematics is now over. the age of individual researchers is done, and hhatnlike many other disciplines, mathematics will be done in large teams, big facilities, and in complex ways that no sole individual will understand end to end.
I imagine he means in that video something like modern physics at CERN where no one single person actually understand how the whole thing works, but nonetheless is confident in the results.
I find Terry's hypothesis there somewhat comforting to my own limited cranium. most of the math he does already is far beyond me, nevertheless I trust that he is correct.
it's a different world for math, but one we all readily inhabit already everything we cross a bridge or buy refrigerated food.
> Private control of frontier LLMs is potentially dangerous.
The author spends a lot of time trying to suggest that LLMs are not the end all be all and humans are still needed for understanding etc. Fine. I understand.
If it is so the case that LLMs don't help that much, surely it doesn't matter if LLMs are privately controlled?
How come LLMs are important enough that private control matters but also not enough for actually advancing mathematics significantly on its own?
In other words, you are almost always better off with LLMs even if it is privately controlled. The mathematics community can just ignore LLMs for the base case or use ChatGPT if they want. In either case, it is strange to suggest that things will get worse for them. Its as if they don't have the coordination or agency to adapt.
This is my understanding of that part you quoted. It's not really a statment about open vs closed models (which is another very important topic, personally I think open is the only way forward). What he is suggesting is mathematicians will use llms for their research because llms are transformative for math and research. As they work on a problem, OpenAI will put their strongest model on their biggest cluster and synthesize all the work by the individual mathematicians to arrive at the answer.
One big problem here is that OpenAI would be hoarding this ability by 1) not releasing their top models for public use,as we have already seen them do, and 2) limiting the capability of the public API, which we have also seem them do.
This not only kills incentive for math, it means that OpenAI becomes the arbiter of math and research for the world. The research that will be done is the research OpenAI decides to do.
This seems very dangerous to me.
Another path forward is distributed local models. Professors can do research on their HPC machines, individuals can have full access to their models. It's a much more democratic approach.
> As they work on a problem, OpenAI will put their strongest model on their biggest cluster and synthesize all the work by the individual mathematicians to arrive at the answer.
But coming to final answer is not important and that’s what the author spends time arguing. If they are right, surely whatever slop OpenAI is producing is worthless to human understanding which is the main goal.
The math community has to change prestige markers to promote people who contribute to understanding and not just solving theorems and that’s on mathematicians to do.
If solving problems is the end goal then I don’t mind OpenAI doing it in which case furthering math is already solved by someone.
The only failure mode is if mathematicians can’t get funding for just “understanding”. But it shouldn’t be that hard to do if their claim is right that their main contribution to society has always been understanding.
I’d argue that getting funding purely for understanding may be even easier to justify because the costs go down dramatically per unit understanding.
I see what your saying.
The question is, if OpenAI is in charge of all math, what will they research? Ideally they would do things to better humanity, but in the worst case scenario they would research war strategies, weapons development, voter suppression algorithms, etc.
OpenAI, and other similar companies seem to be so profit driven that their idea of research could so easily be perverted to cause net harm to not only the maths communities, but well beyond that.
Well if they don't research what mathematicians want to research what's the problem? There's no overlap..
The maths community can continue doing what they want and not be bothered..
> I therefore disagree with claims that mathematics is over because machines can prove theorems. Without humans who understand mathematics, there is, quite literally, no human mathematics—a fundamental part of human culture. If we agree that we should not abandon mathematics because poorly understood machines bruise our anthropocentric vision of it, then society will need more mathematicians, not fewer. Like… a lot more.
Seems contradictory to me, referring to math as a field as “human mathematics” in one sentence, then deriding “anthropocentric visions”.
Mathematics is needed for numerous things. I do not see it going away.
You literally need maths in just about every science, even in biology - think of statistics or any quantitative data set. Or when you need precision or microscopic analysis. cryo-EM is not possible without particle analysis.
Maths will not go away. It is more important than it used to be and it used to be very important already. Albert Einstein without maths wouldn't have been able to conclude much at all.
"Logistical component A is constrained by abstract concept B." am I being a dick, or is this not extremely interesting to read? TBF I've probably over-consumed this particular category of argument recently.
I'm a research mathematician.
By all means, please read the article if it's interesting to you and ignore it if it isn't. But I believe that it's mostly directed to those already within the research math community, with a far more important purpose than to be interesting.
As the post itself explains, we (research mathematicians, as a profession) need to coordinate on deciding what a bright future will look like in a world with powerful AI, and in plotting a course to get there. And we need to do it fast, or circumstances might decide for us.
Historically, we've rather sucked at coordination. We need leadership -- we need someone to step forward, and begin by floating ideas and getting the conversation rolling. Personally I am very grateful for what Tao is doing here.
As someone who has read 20-30 takes on this topic, and talked about it endlessly for the last 2+ weeks, It's an epically interesting read for me (a non-mathematician) - just introducing me to the topic "Mathematization" was sufficient to make the entire article interesting. (I also learned about a new mathematician - Grothendieck).
The article was interesting enough that I spent about an hour reading excerpts and discussing in detail all of it's nuance at the breakfast table this morning. Strongly encourage a detailed read - it was great.
I think Golden age, because I'm more inspired than ever to learn more math now.
I'm encouraged. I was never really good enough to understand frontier math but with AI you can ask how does this work, what if you did that etc.
I will go for "golden age".
Most of the skepticism towards using AI in maths could be applied to using computers in maths in general.
For instance, a computer can find a solution using brute force, like for the 4 color theorem. By doing this, we skip the potentially insightful part where the problem is reduced to something humanly solvable.
And yet, I don't think any mathematician today will tell you that computers are anything but essential tools.
I believe that AI is the same, and that mathematicians and other scientists will embrace it like they embraced computers. In fact scientists have been using AI (aka. machine learning) before it was mainstream, which is totally unsurprising because they invented it in the first place, the industry just gave them more resources, which turned out to be what AI needed.
Mathematics is not science
https://cs.stanford.edu/people/zjl/pdf/science.pdf
It's a "toolkit" that's useful for science
All this “we will need far more mathematicians” seems like cope. Now that we can generate infinite text on the Bible, we need far more priests to understand it. It rubs me the wrong way.
>(4) Mathematical progress cannot be measured by checking off boxes on a problem list.
Once we've essentially found an algorithm that can solve any math problem we've essentially reached the end of mathematical progress. Checking current solvers against unsolved problems is how we can check if we've reached the goal or not.
Of course humans can still study math, like how we teach people arithmetic, algebra, calculus, etc. But there is no more progress to be had. As a society we can allocate resources elsewhere.
"An algorithm that can solve any math problem", I am not Alan Turing or Kurt Gödel, but this seems impossible.
The number of papers submitted to AI conferences is growing 2-3x every year. Of course there are AI slop papers, but looking at the whole, research speed has probably improved. I think the same thing might happen in other fields too. Including math, natural sciences and other engineering fields.
I hate to be controversial but if it helps more of those brilliant minds move from exploring and proving less practical number theories and into applications of numerical science to real-world problems, with AI as a supporting tool I think this is where the golden age begins.
> Without humans who understand mathematics, there is, quite literally, no human mathematics—a fundamental part of human culture. If we agree that we should not abandon mathematics because poorly understood machines bruise our anthropocentric vision of it, then society will need more mathematicians, not fewer. Like… a lot more.
This is just incoherent. Even if we "agree" (we don't but...) not to "abandon" math, how does it follow that "we need a lot more mathematicians" exactly?
Even if you reject the idea that AI can displace the current top quartile/percentile of human mathematicians, wouldn't it be reasonable to assume that it will at least displace some parts of math so that there'll be less demand for mathematicians overall, not more? The mental gymnastics are astounding here.
Unless AI can do everything involved in math, their greatly accelerated production of certain parts of math should increase the demand for people to do the remaining parts.
> greatly accelerated production of certain parts of math should increase the demand for people to do the remaining parts
So the pie just gets larger? Large enough to outpace the displacement? I am gonna wait for this to generalize then, looking forward to AI giving us more software developers, mathematicians, etc. The economic laws have been broken. It's an infinite money glitch, as they used to call it.
It is a largely romanticised idea that pops up here now and then. Meanwhile Anthropic just put up a post saying they are getting closer to replacing people in their own company. That's the objective.
I wonder how many mathematicians even truly understand any of the latest research which is only going to increase in output. It will be impossible to keep up unless you slow AI down (not happening).
Edit: Anthropic blog https://www.anthropic.com/research/employment-at-anthropic
> Meanwhile Anthropic just put up a post saying they are getting closer to replacing people in their own company.
The goals of transhumanists go quite a bit further than just replacing people in companies.
https://www.google.com/search?q=anthropic+transhumanism
Nice source. Transhumanism and believing in AI consciousness does not mix well. We are already being told how we should and shouldn't treat AI. Someone who wants to use AI to boost the abilities of people, puts people first and treats AI as a tool.
Precisely. They are not making tools for us. That is just a stepping stone to their actual goal and you have to wonder whether it is good to give people with such goals access to large amounts of money and data.
Recursive self-improvement, which is underway will crush anything that is verifiable. The only place people have for now is non-verifiable work which is harder to find in maths than other domains.
I don’t really have much sympathy for mathematicians, frankly.
It strikes me that their field previously was quite heavily gatekept, and there was quite a lot of intellectual snobbery.
Seeing them taken down a peg or a hundred is somewhat gratifying.
I really don't understand this. Mathematics was and still is the most open intellectual discipline. There was more good work from unknown people recognized than anywhere else. It's one of the reasons all this was possible: there were objective standards that could not be waved away, so when ML models achieved things it couldn't be denied.
You mean as opposed to the IT world where we all excel in transparency and clear terminology?
How is that Bloom filter written in Rust using Agile coming along?
SWE is the most transparent and ungatekept field in human history and it isn’t even close
--sent from emacs
You've been here long enough, I'm sure you could remember all of the posts complaining about the quality of coding bootcamps, that it was too easy for outsiders to learn software development. Is that your idea of gatekeeping?
Coding boot camps were notorious examples of failures and grifts. They did not produce meaningful quantities of software engineers.
its not just math .. its math, science, engineering, medicine, tech .. ie. everything.
We need to ensure we dont outsource thinking itself - yes its great to have a lot of our knowledge encoded in books, and in the weights of LLMs [easier to access ]
Do we even 'have' a technology if it doesn't reside actively in some human minds ?
Some of math is handed down to apprentice mages, and not encoded well in books [ perhaps books dont encode intuition well, Ive had many cases where someone has explained something not in the textbook, a trick they learned from someone else ]
Apprentice mages only become mages by grappling with hard problems, asking questions, going on long detours, discussing/arguing with master mages, trying lots of tough thought experiments - all valuable "training".
Training using the AI might lead to a golden age where more mages learn wider and deeper, and the subject flourishes and more humans understand the tech.
The danger is using the AI makes us lazy - we dont grapple with programming syntax, draw math diagrams slowly by hand, check edge cases, try to prove things ourselves the slow way and build intuition in the process.
Another issue is ownership and access. Already RAM GPUs and soon CPUs are harder to get hold of due to competition with the hyperscalar datacenter buildout. This is an example of economic inequality effectively stealing resources from the mainstream.
Will math remain open or will it be owned by the frontier models ? With Math and science journals Elselvier almost had a stranglehold, bullying universities to pay massive bulk subscriptions - fortunately Arxiv solved some of that. Can I even own a copy of a music track, tv show or movie these days, when there it is so much more profitable for companies to rent it out pay-per-view or monthly ? There is every financial incentive to slurp up the whole of Arxiv, github and every science textbook into a frontier model and rent out access via pay per token. I no longer need to go to stack overflow to look up a simple algo, google can give me a pithy summary, so its likely that the LLMs will steal traffic from github, or github could be drowned in the volume of vibe code. Is there any point buying a textbook or going to university when the AI covers the material ? If no universities then we dont have a physical space for science and tech to be discussed. That activity might go online .. or may disappear altogether - we already have less socializing in person due to the rise of social media.
We have several "alignment" problems : humans arent aligned with each other, frontier models are aligned with profit-making which may differ from the common good, the race is on and so frontier models are more aligned to short term shareholder / VC value than to longterm benefit to humanity as a whole. These mis-alignments may have always been there, but are multiplied by the power and reach of new AI technology.
[flagged]
I love how Terrence Tao went from being AI shiller -- being a walking advertisement for openai and anthropic to a Oh nO MaThS iS KilLeD wHaT aBoUT mY JoB in a few months.
1) This is a guest post.
2) TFA argues golden age, provided LLM infrastructure is independent.
3) Tao just opened a new SAIR AI platform.
4) Tao just opened a journal for papers with Lean proofs.
5) Academics are known to play both sides of the fence.
It is absolutely bewildering how some people miss the finer points.
Ok Terence
[This is a guest post by Francesco Maggi. This blog post was initially written in a different file format and converted using AI. — T.]
terrence tao didn't even write this post and the conclusion it has is thus:
> Universities, professional societies, funding agencies, and scientific foundations should coordinate the development and governance of competitive LLMs. Scientists will need access to frontier capabilities, transparency about their operation and use of research data, strong confidentiality protections, and the ability to test and reproduce research claims independently.
> The massive computing resources, specialized personnel, data infrastructure, and ongoing investments needed to support this kind of initiative are not within the reach of a single institution, and a huge leadership effort will be necessary to conjure the combination of independent model development, guaranteed access to commercial systems, public computing infrastructure, and open research resources that could do the job.
the post is calling for....an enormous investment in LLM infrastructure.
Thanks for a summary. Based on your summary, I think the parent is directionally correct, at least. All of this seems like panic and attempt to salvage a profession, seeking control and money and other people to do stuff, now! It’s not like this is isolated to university labor. It will kill universities yes, but it will kill a lot of bigger and more important things, likely humanity itself will be killed off. That’s the context.
i see this phrase thrown about a lot but have not seen any "here is how humanity is killed off" scenarios described, not even bad ones much less plausible ones. As long as we dont have a hankering for paperclips that is
Nuclear war. Are you serious?
are you serious? they're going to replace humans in charge of nuclear silos with large language models who can directly launch without human interaction? That would be the scenario. If your scenario is "ChatGPT told Trump to launch the missles", AI didn't do that; Trump did.
AI isn’t used on the battlefield? War cannot be won against an enemy who uses ai, unless you also use ai. Ai progress/escalation is nuclear escalation, just as ai progress is now the driver of progress in medicine, math, science, law, ….
I wonder if there was a collapse in people applying to math PhD programs.
It would be cool if some prestigious such program revamped itself to be AI-focused, maybe in collaboration with a CS department.
Was? Grad school applications are once a year and this happened just a few weeks ago
I find it very weird how he didn't foresee the end of math, being so smart. I mean I did and I'm definitely less smart than him.
Same phenomenon as all the godfathers of AI that were apparently oblivious to the plot of the Terminator franchise until after their paychecks had cleared.
The irony is that he’s still very very wrong in my opinion. He’s lamenting the end of math, but it’s not a stretch to think that math is just the beginning. Like the destruction of the buoy before the tsunami actually hits land. His current stance seems to be implicating that the blast radius is only till math or science when it’s just the first domino.
Indeed he's definitely not ASI-pilled
> I find it very weird how he didn't foresee the end of math, being so smart. I mean I did and I'm definitely less smart than him.
IMO this is probably not an accurate assessment of Tao in particular at this time.
But in the general case, I think most people have a set of blinders on that will result in a predictable reassessment of AI with the following steps as AI progresses when applied to whatever it is they do:
1) AI is useless and doesn't impact me at all. Look at how bad it is! (Most developers up until late 2025)
2) Ok, now AI is useful but will not replace me. Sucks for the juniors in the field but I am still needed to guide the AI. AI will never be capable of my judgement (or whatever). (Most developers now)
3) Oh shit, I've been replaced in terms of economically valuable utility by AI. (tbd, but likely sooner than is comfortable)
And not everyone will experience #3, there will be a top 1% or so for most fields where #3 remains true for them, but it is likely that for most people #3 comes eventually.
I think the thing is if/when #3 turns true society is no more.
Then people respond: "the elite will use their ai armies to kill everyone else". This will not happen. I believe more in the worst case being chaos and rebellions. Not good for anyone, not even the elite.
https://borretti.me/article/no-one-escapes-the-permanent-und...
Can your ai summarize that for my ai please?
It is unlikely anyone would actually have zero utility, if only because of comparative advantage. (Although I think there will still be absolute advantage)
Comparative advantage only works when your economic value is greater than the cost of keeping you alive.