This is really a microcosm of one particular problem that AI presents to the world: what do people do when their labour is not required any longer?
Here the worry is the social structures of mathematics are eroded such that fewer humans become able to do the work and less well, similarly to how juniors are being recruited less in software engineering, breaking the ladder and leading to fewer seniors in the years to come.
The answer to this really depends strongly on what AI can actually accomplish, but I’ll assume the maximal case and say that AI can do everything economically necessary, and further even those things just desired, such that human labour isn’t required for anything that anyone wants in a practical sense.
Here, we don’t need a human understanding of mathematics to give people a perfect standard of living. We also don’t need humans involved with anything else.
Everything therefore becomes a hobby or a game. People do things because they enjoy them for their own sake, or because they are endeavours used as vehicles to socialise and enjoy others’ company, or because a shared social belief exists and is cultivated such that doing such and such a thing confers social status.
And I think that’s more or less it. I predict we may see some fairly strange sorts of things, such as games where the team structure looks like the descendant of a company org and they compete in an artificial economy. Likewise we may see gamified versions of universities and academia. All of these would be “tamed” such that the rougher parts of the experiences were sanded off.
Sort of like how we evolved in an ancestral environment, and we have certain drives and expectations driven by that environment even though they no longer matter for survival. Our social structures may be derived similarly from those of today, even after they have ceased to serve a real purpose, but changed and repurposed to give meaning and community.
> what do people do when their labour is not required any longer
I don't think this is the main issue (if at all) discussed here or in the field medalist letter. If anything, the majority of pure mathematics graduates are absorbed from the industry and a lot of exodus happens along the different scales of academia to there (though industry is also dealing with this issue but that's not what's concerned here). The issue discussed is what mathematics itself will be, which is much deeper. AI finding proofs to open problems does not solve at all the question of how to produce new problems, and there is no indication imo that there is way to go with that with AI.
Pure mathematics is not like applied sciences, as it is only tangentially influenced by external applications. Deciding which problems to tackle is a social process and a matter of taste/aesthetics, and is built largely through the exact friction that is more and more removed with AI. This is what makes it unclear how one can find problems without this friction, and none of these posts/letters have an answer really. Each one seems to describe just different standpoints than concrete, practical ideas.
[Sorry that none of the following is concrete, but perhaps elucidating the paradox contained within might open our minds to .. the shape of the paths to action ?]
Your productive friction (eutripsis? ~ negentropy? Viscosity!!!???) seems like a wonderful concept that the original letter should have flagged to rally the community ( Gowers might not have missed this point if they had a new name for it!)
Tao had a relevant talk about the paradox of efficiency..
Still abstract, but nearer to quantitative (mathematical anthrop(ic)ology even?): ordinary, bad friction is, eg, "size-consistent"
Coasean Ceiling: organizational size limit where the internal friction of managing a firm consumes all of its energy, leaving nothing left for actual production
That's a good point. And yeah I got the term from Tao, as I had not described it this way before but I think it elucidates well the issue.
The problem imo is that, from a purely psychological/phenomenological perspective, there is not always a perceivable difference between "eutripsis" and "dystripsis" (just made it up but "dys" is the opposite of "eu") as experienced. There is some reward coming from learning through friction (depending on personal interests, environment etc), but mostly it is effort and humans usually try to reduce or avoid effort.
Moreover, even if one tries to be fully mindful and choose where to employ friction and where not, there could be systemic factors to optimise away any kind of friction. Imo we already see that in software engineering, judging from a lot of different anecdotes, where increasing the pace of generating code sacrifising human understanding is already taking place. It is not like these forces are not already in place widely in academia too even before AI (eg optimising for paper output quantity), so AI reinforcing this direction sounds a reasonably probable scenario, unless some other action is taken.
Ah, contrary to what I mused elsewhere, concreteness can also lead to bad friction
Eg, KPIs, metrics, but of productivity, of "veracity", not understanding
Anecdotes--> better friction than data, sometimes, though :)
How about Inverse Metrics. of simplicity? Parsimony? Shortness of code? (Efficiency/compressibility is a sort of "intensive" metric, so it might not be especially relevant, thermodynamically speaking)
Just taxidermy, stamp collecting, and vibe-anthropologizing here TT
I agree with your argument of expected societal shift, but it's important to keep in mind that this is largely a developed country issue. It'll be a while before child-staffed cobalt mines or whatever is superseded. A significant amount of poverty could be alleviated through improved resource distribution and policy, without really needing improved productivity/increased resource production.
Did you even read the open letter? Mathematicians are annoyed because the proof is only meaningful if people understand it and came to it. I mean I’m basically just summarising it now but it’s annoying you didn’t read it or can’t comprehend its nuance.
"And maybe I could have tried to gain that respect in a different way, such as thinking very hard about an area of mathematics until I was able to demonstrate to others just how well I understood it. But I’m not sure how motivating that would have been for me. I very much hope that there is a pool of young people for whom it will be a powerful motivation, because I think the survival of a human mathematical tradition may well depend on it. [...]
Thus, the primary risk, as I see it, is that a lot of people who would have done a PhD in mathematics and gone on to become custodians of the mathematical tradition will no longer wish to do so. Those of us who have PhD students, including me, need to try as hard as we can to come up with imaginative ways for them to use their time productively (in consultation with the students themselves, obviously)."
When I read this I'm trmpted to read "imaginary" instead of imaginative - that is: in the sense lacanian psychoanalysis uses the term imaginary in contrast to symbolic - the latter of which would mean that it's having consequences within the symbolic social order. From their own, quite selfhonest evaluation the author assumes that without those they would most likely not have persued the mathematical profession. Which also relates to:
"A related risk is that the perception among policy-makers will be that mathematicians are no longer needed and that funding will become much harder to come by: we urgently need to come up with good ways of explaining the value of having a large pool of human mathematical experts, even if it is no longer part of their role to find new proofs of theorems."
Note that there isn't any attempt to establish a possible horizon as to how that coule happen.
It seems to me that the end of the article has a strong tendency to a somewhat stoic attitude of one might phrase as: it is going to happen anyways - the systemic context in which these corporations act makes it inevitable ("However much we might regret that, there is no chance that the impact of such models on mathematics will persuade AI companies to stop their release, though perhaps concerns about safety will lead to some delay and give us a bit more time to work out how to adapt. Assuming that they are released, there will be a flood of new results, whether we like it or not, and it will no longer be the AI companies producing them, though perhaps the pattern will continue that the AI companies will have access to more powerful models and so will obtain more than their fair share of headline results.")
- which makes me kind of wonder if it is not some sort of cognitive dissonance within the authors view of the situation to not also think 'it's not going to happen' in regard to their stated requirements (students that don't need to be motivated by a desire to be symbolically valuated as explorers of the mathematical frontier, and policy makers that acknowledge the value of funding the social "production" of human mathematical experts /enable such a community through funding).
So basically they see the same problems as the authors of the letter (which they also state in their letter - their should be no room for a misrepresentation of that fact: "I felt that I could not sign the letter, despite agreeing with much of what it said. Instead, it seemed better to do what I did with the Leiden Declaration and set out my own position in a blog post. But it should be understood that by doing that I am not setting myself up as a member of some opposing camp: indeed one of my worries at the moment is that the mathematical community might become bitterly divided, something I would very much like to avoid.")
So what stays is mostly their disagreement on that conceptual understanding is more important than the solving of problems for all mathematicians. - which kind of makes the problemfield of ai in mathematics somehow more urgent, as far as I have thought it through. And I'm not really convinced that the displacement of the problem into some sort of pedagogy (and it should be appreciated that the author implicate themselves in the responsibility to establish it) instead of finding solutions in the realm of policy and regulation. Seems realistic though, that they see that as an effort unlikely to succeed in a meaningful way.
Can someone argue against that reading? Am I missing something? My conclusion as such is that not only are they not opposing the authors of the fields medalist letter, but also that their evaluation of the situation is much more dire without them giving in to resignation.
I thought one of the implicit points of the open letter was that unsolved problems are not something that falls out of the sky, they are a curated resource that people have spent time on and shared for the benefit of like-minded peers and humanity as a whole. And the AI companies treat them like they treat absolutely everything else: natural resources, literature, art, code etc. as something to be chucked into the ravening maw and pooped out the back as profit. They don't care if mathematics advances, they don't care if they strip-mine the available problems and damage the field. In fact, as with programming I think they see that as in their long-term interest - soon there will be no intelligence or creativity but the one that Sam Altman bills you for.
Ok, so unresolved math problems are often something people discover while trying to solve a different math problem.
However, math problems are really there to solve a real world problem. We have unlimited real world problems no matter how smart AI gets. Therefore, we will always have unresolved math problems.
> math problems are really there to solve a real world problem
I think this is totally wrong. Math problems are almost by definition problems with a particular theory. That theory might be inspired by the real world, but the problem itself is purely theoretical. I can't think of any theoretical problems like this that actually support a practical problem, as opposed to being an internal knot in the theory that indicates something is wrong with it. Not to say that cannot happen - certain optimization problems were historically actually hard to solve and solving them helped us to genuinely optimize a real thing (rather than just explain why the answer we already had was correct, which is much more common). In particular, none of the millennium problems have anything to do with a "real" problem, including the Navier Stokes one.
Agreed, though for the hn audience I want to advocate a bit for the utility of mathematics. The development of applicable mathematics has often not been through the direct means of solving an open problem. It has however often depended on theory which was developed for the purpose of human understanding. It is difficult to pull concepts out of the aether on demand, but when there is a general milieu of human understanding economic applications can be developed in post.
I have in mind GPS, cryptography, numerical fluid simulation, lasers, etc…
Did the theoretical math lead to the invention of GPS, cryptography, lasers, etc? Or did we encounter a real physical problem, then we found that someone had done some theoretical math before that would be useful for this application? If we run into a real physical problem today, can we just have AI invent the math on the spot to solve the problem without a human having done the theoretical math in the past?
My point is that when the consumer application became apparent we already had the required concepts to build the technology on top of. In mathematics it still hasn’t happened that an LLM system has invented a conceptual framework. In most if not of the major AI announcements they’ve worked within known frameworks and assembled ideas across frameworks.
Moreover, it’s not clear that if (and when as I believe) they do, creating technologies with no human understanding of the framework is possible or desirable.
It might. It might not though. Would a 1300s superintelligence advocate for heliocentricism in the face of all institutional players advocating for geocentrism, or would it create the most refined epicycle model imaginable? I’d guess the latter.
If we are talking about pure/theoretical mathematics, then the vast majority of the problems people pose and solve have at best tangential relationship with applications, and a big part even is only related to other math problems. Of course quite a bit of mathematics historically emerged as this kind of intellectual endeavour to find applications later, but there is neither a way to predict which ones are that and how to get them, nor is there indication of this thing going on to the same proportion nowadays as it was, considering the mathematical production is much higher. In mathematics human mathematicians have to decide which problems matter, it does not come from somewhere.
Solving "real" problems in theoretical mathematics (as in problems directly related to applications) is a very small proportion compared to the vast majority of math work that does not. So if we are discussing about the future of mathematics as a field, we have to understand what the field of theoretical mathematics is actually about.
Technological progress is not bottlenecked by most of the millennium prize problems or erdos problems per se, or most of the rest open problems in theoretical math, ie that merely knowing the solution of them will help applications in some manner. I doubt the solution of such problems has any direct effect on technology progress at all, at least in any deterministic, foreseeable manner.
In fact, the relationship between theoretical mathematics and "real physical problems" is bidirectional, as in "real physical problems" informs to some degree some problems that may be interesting to research on in theoretical math, and at the same time pure mathematical research that is developed completely independent may find applications at some point. And even theoretical mathematicians working close to applications are mostly dealing with problems not directly addressing applications. Eg maybe they study properties of a certain function that arises often in application without any view to solving a specific "real physical problem" with it, and somebody after may find that useful for some application after some point, but that could be one out of 50 papers (random number) and it is hard to predict that. There is of course some work more related to specific real problems, but that's most often not what theoretical math is about, and not what these new developments with erdos problems, navier stokes etc are about.
So what could (in a chaotic sense) have effect in application is mathematical theories developed along the way of solving these pure math problems, which brings us back to the question of what happens if we remove this friction and if AI can do more than construct examples and proofs, ie actually build theories (autonomously or humans+AI). If anything, it is through building theories that mathematical progress germinates applied sciences, as this is the process that develops mathematical tools that can be taken up later, including whole mathematical fields. Building mathematical theories is a heavily social process, and it is the community that basically decides which directions are important to follow.
AI can also introduce its own "bad friction", or it (ppl?) can bypass good friction without necessarily directly be removed by ai, eg bikeshedding, slowly pivoting towards directions and problems that ai is better in tackling, because that can produce these accelerated results vs fields and problems that ai may not be able tackle as well and thus the output there is poor, demotivating people from following these fields and missing important insights from them. Of course this could have the opposite effect, depending on which direction the whole hype can go, or not happen at all if ai will be able to tackle everything equally well.
Yes, and here towards the bottom the author gets to the reason he didn’t sign with the other medalists:
> I felt that there was nothing to be gained from criticizing AI companies for generating too many solutions too quickly.
> Under the circumstances, I think the best we can do is recognise the changes that are coming and try to work out the least unsatisfactory way of dealing with them.
Basically let’s make it a short-term problem and deal with it. Groups of people can deal with short term emergencies. Don’t turn it into a structural issue.
And in my view what’s the alternative in the letter exactly? The tools exist. Is there going to be drama every time somebody decides to use them?
The tools exist, so let’s try to figure out as humans the best way to use them to promote humanity, and let’s advocate against ways of using them which are a detriment to humanity, which is exactly the charge the letter makes. There are cultural norms around the use of every technology.
Well I think in response to 'what are you going to do', what people are probably going to do is stop sharing on-going work, stop bothering to curate problems that are just going to inflate someone's IPO valuation, basically accelerating the tragedy of the commons that the AI companies are driving.
Seems legit. I speculate their are emergent mechanisms for the development of memory that are not dependent on what ever mechanism is behind the "improve $model for 'everyone'"*
Is someone knowlegeable on research that engages the question on the development of something that could be considered as some emergent mechanism of memory, that is independent of instances and their contextwindow an specifically also independent of the use of conversations for trainingsdata?
*(This is regarding the value of selfhosted models - maybe small selforganisations that share ressources to do though to do so? Generally the value of machine learning seems to be to big to reject)
> With that interpretation, the issue becomes slightly different: is it more important that the collective understanding of the mathematical community should be as advanced as possible or that there should be answers to as many problems as possible? Or are those two aims valuable in different ways, so that there is no point in declaring one of them more important? Or are they so inextricably linked that it makes no sense to argue that one is more important than the other? And when we say “important”, for whom are we saying it is important: for mathematicians, or for society as a whole?
This is easy to me. Truth should be the North Star. If there is a fundamental truth that can be found via mathematics, then the shortest route to that truth should be preferred. While LLMs are definitely capable of solving problems in search of truth, I agree with Tao that instant "true/false" results threaten to short-circuit the traditional avenues we have used to escape local minima in the search for truth. Their products may be the junk food that provides immediate satiation in exchange for long-term health. Perhaps it's wrong, though.
I think it is easy to just roll of a local maximum, however, you may get stuck in a local minimum.
I kid, of course, but I do wonder where the use of local "maximum" comes from, what is maximum there? Why do you not see this as a landscape of hills and valleys where marbles with certain energies may indeed get stuck in deep enough holes... Of course, I just assume and picture gravity pointing down in that landscape, but hey. I'm human, I feel it is expected of me.
The letter is not incompatible with “pro AI in math” unless one thinks arguing against extreme behavior such as corporations using millions of dollars to scoop results makes one “anti AI”, which is not a reasonable stance in my opinion.
> AI offers the potential of enhancing and accelerating genuine mathematical study and understanding. Mathematics as a profession will need to adapt to these changes in several ways. However, whether these changes ultimately benefit the field or have a destructive effect will in large part be determined by the decisions of the humans in control of this new technology.
It is getting a lot harder for those people to justify using OpenAI to assist such endeavours. Afterall, OpenAI might just front-run you if they hear a rumor you solved some marquee problem that they can brag about in PR campaigns.
This isn’t true, as there are plenty of things proven beyond reach.
And plenty of things, eventually solvable, can create major problems that could both be avoided and the problem solved by taking a much better path.
Having technology and the ability to safely and sanely use the technology needs to progress together at a similar rate. The failure to do this is even a reasonable and common solution to the Great Filter. Jared Diamonds book “Collapse” has ample examples of cultures that wiped themselves completely out via not having this balance, so it’s not simply a theory.
as a society of researchers we've tended to cultivate pretty effective strategies for escaping local maxima. I think of it like ants, where you can see if you place an obstacle in between their nest and a foot source, they develop a path that loops around it. if you remove the obstacle, for some time they continue to follow the old looped path. However, some ants deviate and go around at random, exploring. eventually by chance one happens to find a quicker route. he gets a couple of his friends to follow him, by pheremone, and over time more and more take the quicker route, and they end up abandoning the old route
It works this way with research, with most following the current trends, and some curious souls searching around for other ideas, be they contrarians, dreamers, or just convinced of some strange truth. But if we're right, signs tend to slowly begin to point their way, and we can shift the whole hulking edifice of science towards their point of view.
The problem of llms is that while they may be able to find a shorter route, we can't follow them unless we understand the route. So the forces that slowly begin to change everyone's behavior are lost
This pattern of argument keeps repeating in every place.
Pro tech people: technology removes bottlenecks. Sometimes we use those bottlenecks as a side effect to build muscle and so on. But removing bottlenecks gives us much higher degrees of freedom. It is up to us to coordinate and make use of the technology.
Anti tech people: bottlenecks are fundamentally useful. They should remain and technology shouldn't remove those so easily. Humans cannot coordinate as well when the bottlenecks are removed, so lets not remove them so quickly.
I don’t think the letter is even anti-tech progress. I also don’t think the tech companies are primarily pro-math progress (not that I think you think that). It’s primarily “anti-unnecessarily-destroy-the-human-systems-of-mathematics-just-for-benchmarks-and-marketing”
> … the push by AI companies to solve mathematical problems as a benchmark is detrimental to the science of mathematics, and to the mathematical community. The goals of the AI companies and the goals of the mathematical community are severely misaligned.
Its not a conspiracy. Its 25 fields medalists who think exactly like how I put it. Ideally, they can just take whatever the technology gives as soon as possible and use it. They don't want it because they don't think the community can rearrange and coordinate such that they can make use of the new found degrees of freedom.
That's literally all there is to it - they don't believe in the rearrangement.
Counterpoint: I'm not anti tech at all. In fact, I sell an AI harness for legal.
I think you've set up a false dichotomy. I'd propose to you the middle ground that a lot of us are concerned that VC-backed AI slop is "solving" problems in indigestible ways that hollow out the core. This applies in OSS as well as mathematics.
Please help me understand this and I'm asking this in good faith.
Why can't OpenAI publish whatever it wants. And the math community can use it or not use it. Fundamentally OpenAI's solutions are high signal - they are incentivised to not deliberately mislead people. Let the individuals in math community choose to read it or understand it? If OpenAI wants to publish something, let them do it in the current channels using peer review using whatever time is required.
What's wrong with this? The math community thinks this will destroy previously unwritten ways of prestige allocation and remove incentives that used to exist. I say that the community can rearrange and allocate prestige and time in different ways to maximally use the technology.
They can use the new proofs as foundations for future proofs.
But many folks just won't be motivated to attack or help digest solved problems because there's less extrinsic value in doing so. Tao and others argue that it's this human effort that finds human-relatable abstractions which spurn further investigation. Take the humans out of the loop, and they'll stay there, is the argument as I understand it.
I read this multiple times and also recollected the original letter. You can’t really blame me for thinking this is a straaaange request.
“Don’t help me solve my problems because it removes the incentive to be in this field” is how I hear it. Honestly.
I would say the incentives HAVE to change because of new found capabilities.
Imagine accountants asking companies to reconsider producing calculators because we used to value accountants for how well they did arithmetic!!
Edit: my main point is that the math community has full agency to do what it wants with new found tech. Instead, asking to change external entities like OpenAI to be careful with releases is a strange thing to do. Why not change yourself and adapt?
You're describing an improvised surgery on a living organism. Developing a complex system involving humans that is productive and doesn't collapse is extremely hard, so if it ain't broke don't fix it.
The Math community can "not use" a proof? How would that work? It's a little like coloured functions (async etc) - you have 'human proved' vs 'machine proved'?
I'm pointing out that the way Mathematics works as a discipline is that a new proof builds on existing proofs. If we have a bunch of machine-generated proofs, then a human Mathematician has to decide whether to reference any relevant machine proof that has been put out there.
If, as you suggest, human Mathematicians 'choose to ignore' a machine proof, then another human decides not to, what then? Mathematics bifurcates into 'pure human' proofs and 'mixed machine-human' or maybe 'pure machine'?
Consider that 'calculator' used to be a title for a role a human used to do. That went away, which is probably fine as there were other jobs and likely it was tedious.
Then consider that "just adapt" could be "adapt or die", and that maybe, just maybe there is no way to adapt to well-funded corporations churning out millions of proofs a minute.
I don't know, maybe AI companies have "agency" to not break everyone else's stuff, just to get more funding and a higher share price?
One thing I've found in my own use of AI, which Gowers touches on in his point at the end but which I think deserves more attention: AI makes some previously non-trivial tasks trivial, but that doesn't make the work itself trivial. You naturally expand the scope of what you attempt and take on harder problems than you could before. The floor rises, and so does the ceiling. The open question is whether that still holds once models can also do the harder problems, or whether choosing and framing those problems remains the human part.
I think the author did not mention that the way humans construct the knowledge ladder requires low complexity because each step gives power to the next, but the AI with the exponential exploration can give biggers steps but then it stagnates because the next step can be beyond the exponential exploration complexity. In chess a good strategy can be the best tool, in the game of go our experience suggest the same, but it could happen that mathematical thinking requires a type of policy that could be beyond the current ideas. I can not fathom a LLM could conceive from scratch concepts like the real numbers by Dedekind.
> I can not fathom a LLM could conceive from scratch concepts like the real numbers by Dedekind.
What stops you? I can fathom it alright.
I am always suspicious of these arguments like "a machine won't do thing x which humans can". These always basically pre-suppose that humans aren't machines. If that's what you're arguing for, say it outright.
A cynical could say that AI could not interpolate the Dedekind cut but it could extrapolate to create a lot of money but just using an imaginary extra point.
The ironic part is that generating a joke like this requires jumping across three distant fields: real analysis, machine learning limits, and VC market cynicism. For an AI to discover that specific overlap through statistical search, the combinatorial space is absurdly huge. Yet a human brain connects them in a fraction of a second. This tiny joke is a micro-proof of the macro-argument: human conceptual leaps routinely bypass exponential search spaces.
Just to add that the Dedekind cut example seems to stand beyond any RL policy used in chess or go, AlphaZero or AlphaProof. In the classical RL there is an state-action space. If the solution requires jumping to a totally difference action space (that must be created) the local policy stalls. Dedekind cut is an example of a out-of-distribution state-space generation.
Solving a theorem is like climbing a new mountain. The mountain is already there, and there is a list of the hardest known mountains to climb. The point of climbing them and not just dropping with a plane on top of the peak is to help develop human climbing skills and expand our knowledge and abilities. Also, already conquered mountains are climbed all the time to test new strategies.
Now, if an LLM proves a theorem, it's like discovering a new mountain and knowing what its peak looks like. Does that mean the problem is finished? No, we still need climbers to actually do the work and advance the field with human understanding.
The job of a mathematician just shifts from proving to inventing new theories and finding new conjectures with the help of AI, which should be even more fun.
> we urgently need to come up with good ways of explaining the value of having a large pool of human mathematical experts, even if it is no longer part of their role to find new proofs of theorems.
This is the main issue, and while I fully agree with that value sentiment, the referenced letter failed to provide convincing arguments for why mathematicians should widely receive funding for merely understanding things, and how competition for postdoc and tenure positions would work.
> One way that might happen is that AI disrupts society so much, or even kills vast numbers of us, that the preservation of something like the current mathematical tradition ceases to be of any concern: all that will matter is the survival of the human race. But that again is a topic for a different blog post (which in fact I am in the middle of writing).
This is pretty disappointing I have to say. The letter actually has a “pro AI” stance (see quote below), and Gowers’ apparent reason for not signing it is so subtle (not wanting to offend people whose goal in mathematics is not understanding) that I can’t help but view this as being more about optics than its actual content; i.e. he seems to be cleverer than I am in recognizing most people will not see the letter as a push to use AI in ways which are healthier for human flourishing but as advocating for one of two sides in a highly artificial binary (“pro AI” or “anti AI”). Gowers draws a distinction without a difference here and should have just signed the letter.
> AI offers the potential of enhancing and accelerating genuine mathematical study and understanding. Mathematics as a profession will need to adapt to these changes in several ways. However, whether these changes ultimately benefit the field or have a destructive effect will in large part be determined by the decisions of the humans in control of this new technology.
Watching the discussion unfold, here is what I feel:
Before arguing whether mathematics must strictly be done by humans, there are different motivations at play. Some people love the sense of solidarity within the community that forms during the process. Those excluded from that community might resent it, while others just purely want to solve problems.
Many things are being discussed, but looking at the overarching narrative, it seems that AI's true function isn't necessarily opening new horizons of specific knowledge, but rather excelling at 'serializing' topics that have been heavily fragmented until now.
In that sense, the concern is that because AI is solving the very problems needed to cultivate mathematicians internally, the stepping stones required for human growth are disappearing.
However, on the other hand, as the world and industries become increasingly complex and hyper-specialized, you could also argue that AI is the exact tool needed to unify this fragmentation across academia and industry. It is a highly complex dilemma.
From the perspective of researchers and the mathematical community, those 'problems for growth' must remain. But conversely, AI has the distinct ability to serialize siloed disciplines. Usually, when you go to graduate school, you often hear professors say that even within the exact same major, they cannot understand each other if their sub-specialties differ.
[I have not read Gower's article. It's only that your response is prompting me to share what i've been thinking recently]
>the stepping stones required for human growth are disappearing.
Actually only in an institutional sense, imho. Maybe I'm exaggerating, but reddit.com/r/math* or even mathstackexchange will be so back with users analyzing and distilling proofs with AI. Maybe in 5 years (when lean attains 10% popularity of rust, and/or gpt7 level models cost ~USD5 on average, per month, inflation adjusted or not) this types of submissions will make those sites as fun/educational as mathoverflow [has always been for me]. One dreams that by that time openAI and Anthropic will have taken down your ethno-cultural "compatriot" Masayoshi with them.. (I prefer his brother whom he did not regret giving a physical beating to. Only purely on principle) so that they can't acquire these sites
Such forums can then replace math grad school, if the profs/alpoges who drop by get into the habit of constructive criticism (as they do on mathSE already). It's already starting, I see personally interesting 1 AI-aided submissions every 1.5-3 weeks starting 2 months ago
It would be like rust discussions on HN for you I bet.
This seems unlikely when s/math/physics/, or s/Reddit/HN/ because HN hates AI-aided posts so ideologically (sorry mods, I don't mean you guys). MO is also not as welcoming to outsiders/lay, congruent to academia (only online) PhysicsSE/overflow had become dumb and arrogant last decade
HN, even pg seem anti-intellectual in effect tbh whenever they sneer at AI _writing_ (again sorry mods, you will figure this out soon I believe, do you or do you not want HN to end up as a reservation for meatbrains), though I mostly am on their coder side with regards to this fields medallist letter
The AI debate is, in fact, somewhat ideological. The problem is that in the process, whether advocating for it or pushing for progress, there is a failure to face its limitations directly. In other words, it is emotional. Because AI's mere existence is a threat to knowledge workers, much like machines were a threat to blue-collar workers. That is, if the value of intellectual labor drops, it damages the value of the labor force through which workers earn capital in a capitalist system.
I have tried solving a few math problems with AI (they were Erdős problems), but because I know absolutely nothing about that math, I couldn't just take the AI's word for it, so I am actually a bit skeptical. A problem arises where you arrive at the answer without actually understanding it.
It's not that AI is bad. The problem is that AI destroys the equilibrium between the knowledge I have and the knowledge I lack. That boundary collapses, making it feel as if I can know everything.
Actually, academia is fundamentally about mental models. It's a kind of internal worldview, and that worldview is shared. When you actually listen to the thoughts of scholars and professors, there are subtly different aspects. That forms the person's worldview... and I get the feeling that sharing it is what constitutes intellectual activity.
However, as you mentioned, unlike academics like yourself, academia and knowledge communities seem disconnected to someone like me (meaning they lack accessibility). Even if I were to make a discovery, it would probably be hard for me to become recognized, and I do think AI could actually play a role in opening up those closed communities.
But apart from that, I find it hard to say that this only has positive aspects.
I followed Karpathy's research from start to finish to build a small LLM like nanoGPT on my own, and people say that because of the positive transfer that comes from feeding diverse data through modern LLM multimodal encoders, there will be new discoveries. But in reality, the types of problems AI excels at are generally those that humans have found but overlooked.
In my opinion, rather than being a knowledge machine, LLMs (or AI) make me feel that what we call "intellectual activity" is closer to a kind of serialization work. A method of stacking things up one by one in sequence, so to speak? After all, the actual operating principle of an LLM proceeds according to the probability of the token coming in the next sequence.
In other words, I think the serialization method of our knowledge activities is similar to how LLMs operate, but I also think a different kind of thinking might be necessary. I am not that smart, and I have never interacted with scholars... (As you know, I am a subcontract worker. Of course, I have been hired by startups run by professors in my country, but it's not like I modeled that intellectual design myself.)
On the contrary, I feel that the evolutionary approach will slow down after GPT 6 Astra. They can continue to increase the size, but the issue lies in the cost-effectiveness of token costs.
Anyway, I agree with most of what you said in your discussion, but rather than anti-intellectualism, I consider this a direct threat to survival.
In value terms, the question becomes: who has the right to translate their money into what sorts of meaning? Who controls the medium through which, and the institutions through which, our actions become meaningful to ourselves, by the very act of being publicly recognized in some kind of public arena? It seems to me that while if one is trying to understand the strategies by which people can move back and forth between “fields”, and especially, by which some are excluded from them, Bourdieu’s models are pretty much indispensable, THEY DO LITTLE to tell us why anyone wishes to enter certain fields to begin with.
The evolution of fields, of domains of enquiries. AI-independent
First of all, I take it for granted there is really no such thing as “intelligence”
Writing like this from 2005, feels like a personal bedtime Bible for AI age
This is really a microcosm of one particular problem that AI presents to the world: what do people do when their labour is not required any longer?
Here the worry is the social structures of mathematics are eroded such that fewer humans become able to do the work and less well, similarly to how juniors are being recruited less in software engineering, breaking the ladder and leading to fewer seniors in the years to come.
The answer to this really depends strongly on what AI can actually accomplish, but I’ll assume the maximal case and say that AI can do everything economically necessary, and further even those things just desired, such that human labour isn’t required for anything that anyone wants in a practical sense.
Here, we don’t need a human understanding of mathematics to give people a perfect standard of living. We also don’t need humans involved with anything else.
Everything therefore becomes a hobby or a game. People do things because they enjoy them for their own sake, or because they are endeavours used as vehicles to socialise and enjoy others’ company, or because a shared social belief exists and is cultivated such that doing such and such a thing confers social status.
And I think that’s more or less it. I predict we may see some fairly strange sorts of things, such as games where the team structure looks like the descendant of a company org and they compete in an artificial economy. Likewise we may see gamified versions of universities and academia. All of these would be “tamed” such that the rougher parts of the experiences were sanded off.
Sort of like how we evolved in an ancestral environment, and we have certain drives and expectations driven by that environment even though they no longer matter for survival. Our social structures may be derived similarly from those of today, even after they have ceased to serve a real purpose, but changed and repurposed to give meaning and community.
> what do people do when their labour is not required any longer
I don't think this is the main issue (if at all) discussed here or in the field medalist letter. If anything, the majority of pure mathematics graduates are absorbed from the industry and a lot of exodus happens along the different scales of academia to there (though industry is also dealing with this issue but that's not what's concerned here). The issue discussed is what mathematics itself will be, which is much deeper. AI finding proofs to open problems does not solve at all the question of how to produce new problems, and there is no indication imo that there is way to go with that with AI.
Pure mathematics is not like applied sciences, as it is only tangentially influenced by external applications. Deciding which problems to tackle is a social process and a matter of taste/aesthetics, and is built largely through the exact friction that is more and more removed with AI. This is what makes it unclear how one can find problems without this friction, and none of these posts/letters have an answer really. Each one seems to describe just different standpoints than concrete, practical ideas.
[Sorry that none of the following is concrete, but perhaps elucidating the paradox contained within might open our minds to .. the shape of the paths to action ?]
Your productive friction (eutripsis? ~ negentropy? Viscosity!!!???) seems like a wonderful concept that the original letter should have flagged to rally the community ( Gowers might not have missed this point if they had a new name for it!)
Tao had a relevant talk about the paradox of efficiency..
https://youtu.be/svl_1upFpQo
It's not clear to me that AI necessarily removes this eutripsis. The threat though, might become real if users don't see the threat :)
Also reminiscent of Keat's
https://en.wikipedia.org/wiki/Negative_capability#Reception
https://www.poetryfoundation.org/education/glossary/negative...
Still abstract, but nearer to quantitative (mathematical anthrop(ic)ology even?): ordinary, bad friction is, eg, "size-consistent"
So.. for eutripsis, Coasean Floor? LolThat's a good point. And yeah I got the term from Tao, as I had not described it this way before but I think it elucidates well the issue.
The problem imo is that, from a purely psychological/phenomenological perspective, there is not always a perceivable difference between "eutripsis" and "dystripsis" (just made it up but "dys" is the opposite of "eu") as experienced. There is some reward coming from learning through friction (depending on personal interests, environment etc), but mostly it is effort and humans usually try to reduce or avoid effort.
Moreover, even if one tries to be fully mindful and choose where to employ friction and where not, there could be systemic factors to optimise away any kind of friction. Imo we already see that in software engineering, judging from a lot of different anecdotes, where increasing the pace of generating code sacrifising human understanding is already taking place. It is not like these forces are not already in place widely in academia too even before AI (eg optimising for paper output quantity), so AI reinforcing this direction sounds a reasonably probable scenario, unless some other action is taken.
Ah, contrary to what I mused elsewhere, concreteness can also lead to bad friction
Eg, KPIs, metrics, but of productivity, of "veracity", not understanding
Anecdotes--> better friction than data, sometimes, though :)
How about Inverse Metrics. of simplicity? Parsimony? Shortness of code? (Efficiency/compressibility is a sort of "intensive" metric, so it might not be especially relevant, thermodynamically speaking)
Just taxidermy, stamp collecting, and vibe-anthropologizing here TT
I agree with your argument of expected societal shift, but it's important to keep in mind that this is largely a developed country issue. It'll be a while before child-staffed cobalt mines or whatever is superseded. A significant amount of poverty could be alleviated through improved resource distribution and policy, without really needing improved productivity/increased resource production.
Maybe our AI overlords won't appreciate our tendencies to abuse and exploit each other and crack down on things like "child-staffed cobalt mines"...
Next stop the Culture!
Did you even read the open letter? Mathematicians are annoyed because the proof is only meaningful if people understand it and came to it. I mean I’m basically just summarising it now but it’s annoying you didn’t read it or can’t comprehend its nuance.
Did you read the blog post that we’re actually talking about?
"And maybe I could have tried to gain that respect in a different way, such as thinking very hard about an area of mathematics until I was able to demonstrate to others just how well I understood it. But I’m not sure how motivating that would have been for me. I very much hope that there is a pool of young people for whom it will be a powerful motivation, because I think the survival of a human mathematical tradition may well depend on it. [...] Thus, the primary risk, as I see it, is that a lot of people who would have done a PhD in mathematics and gone on to become custodians of the mathematical tradition will no longer wish to do so. Those of us who have PhD students, including me, need to try as hard as we can to come up with imaginative ways for them to use their time productively (in consultation with the students themselves, obviously)."
When I read this I'm trmpted to read "imaginary" instead of imaginative - that is: in the sense lacanian psychoanalysis uses the term imaginary in contrast to symbolic - the latter of which would mean that it's having consequences within the symbolic social order. From their own, quite selfhonest evaluation the author assumes that without those they would most likely not have persued the mathematical profession. Which also relates to:
"A related risk is that the perception among policy-makers will be that mathematicians are no longer needed and that funding will become much harder to come by: we urgently need to come up with good ways of explaining the value of having a large pool of human mathematical experts, even if it is no longer part of their role to find new proofs of theorems."
Note that there isn't any attempt to establish a possible horizon as to how that coule happen.
It seems to me that the end of the article has a strong tendency to a somewhat stoic attitude of one might phrase as: it is going to happen anyways - the systemic context in which these corporations act makes it inevitable ("However much we might regret that, there is no chance that the impact of such models on mathematics will persuade AI companies to stop their release, though perhaps concerns about safety will lead to some delay and give us a bit more time to work out how to adapt. Assuming that they are released, there will be a flood of new results, whether we like it or not, and it will no longer be the AI companies producing them, though perhaps the pattern will continue that the AI companies will have access to more powerful models and so will obtain more than their fair share of headline results.") - which makes me kind of wonder if it is not some sort of cognitive dissonance within the authors view of the situation to not also think 'it's not going to happen' in regard to their stated requirements (students that don't need to be motivated by a desire to be symbolically valuated as explorers of the mathematical frontier, and policy makers that acknowledge the value of funding the social "production" of human mathematical experts /enable such a community through funding).
So basically they see the same problems as the authors of the letter (which they also state in their letter - their should be no room for a misrepresentation of that fact: "I felt that I could not sign the letter, despite agreeing with much of what it said. Instead, it seemed better to do what I did with the Leiden Declaration and set out my own position in a blog post. But it should be understood that by doing that I am not setting myself up as a member of some opposing camp: indeed one of my worries at the moment is that the mathematical community might become bitterly divided, something I would very much like to avoid.")
So what stays is mostly their disagreement on that conceptual understanding is more important than the solving of problems for all mathematicians. - which kind of makes the problemfield of ai in mathematics somehow more urgent, as far as I have thought it through. And I'm not really convinced that the displacement of the problem into some sort of pedagogy (and it should be appreciated that the author implicate themselves in the responsibility to establish it) instead of finding solutions in the realm of policy and regulation. Seems realistic though, that they see that as an effort unlikely to succeed in a meaningful way.
Can someone argue against that reading? Am I missing something? My conclusion as such is that not only are they not opposing the authors of the fields medalist letter, but also that their evaluation of the situation is much more dire without them giving in to resignation.
I thought one of the implicit points of the open letter was that unsolved problems are not something that falls out of the sky, they are a curated resource that people have spent time on and shared for the benefit of like-minded peers and humanity as a whole. And the AI companies treat them like they treat absolutely everything else: natural resources, literature, art, code etc. as something to be chucked into the ravening maw and pooped out the back as profit. They don't care if mathematics advances, they don't care if they strip-mine the available problems and damage the field. In fact, as with programming I think they see that as in their long-term interest - soon there will be no intelligence or creativity but the one that Sam Altman bills you for.
I don't understand this logic.
Ok, so unresolved math problems are often something people discover while trying to solve a different math problem.
However, math problems are really there to solve a real world problem. We have unlimited real world problems no matter how smart AI gets. Therefore, we will always have unresolved math problems.
> math problems are really there to solve a real world problem
I think this is totally wrong. Math problems are almost by definition problems with a particular theory. That theory might be inspired by the real world, but the problem itself is purely theoretical. I can't think of any theoretical problems like this that actually support a practical problem, as opposed to being an internal knot in the theory that indicates something is wrong with it. Not to say that cannot happen - certain optimization problems were historically actually hard to solve and solving them helped us to genuinely optimize a real thing (rather than just explain why the answer we already had was correct, which is much more common). In particular, none of the millennium problems have anything to do with a "real" problem, including the Navier Stokes one.
Agreed, though for the hn audience I want to advocate a bit for the utility of mathematics. The development of applicable mathematics has often not been through the direct means of solving an open problem. It has however often depended on theory which was developed for the purpose of human understanding. It is difficult to pull concepts out of the aether on demand, but when there is a general milieu of human understanding economic applications can be developed in post.
I have in mind GPS, cryptography, numerical fluid simulation, lasers, etc…
Question:
Did the theoretical math lead to the invention of GPS, cryptography, lasers, etc? Or did we encounter a real physical problem, then we found that someone had done some theoretical math before that would be useful for this application? If we run into a real physical problem today, can we just have AI invent the math on the spot to solve the problem without a human having done the theoretical math in the past?
My point is that when the consumer application became apparent we already had the required concepts to build the technology on top of. In mathematics it still hasn’t happened that an LLM system has invented a conceptual framework. In most if not of the major AI announcements they’ve worked within known frameworks and assembled ideas across frameworks.
Moreover, it’s not clear that if (and when as I believe) they do, creating technologies with no human understanding of the framework is possible or desirable.
Do we know that AI can't invent a conceptual framework?
If we give it a real problem to solve, it may just have to invent a new form of math to solve.
It might. It might not though. Would a 1300s superintelligence advocate for heliocentricism in the face of all institutional players advocating for geocentrism, or would it create the most refined epicycle model imaginable? I’d guess the latter.
Humans only invest in solving problems that matter one way or another.
I also disagree that none of them solve "real" problems. They clearly do. Solving them have implications on real world problems.
If we are talking about pure/theoretical mathematics, then the vast majority of the problems people pose and solve have at best tangential relationship with applications, and a big part even is only related to other math problems. Of course quite a bit of mathematics historically emerged as this kind of intellectual endeavour to find applications later, but there is neither a way to predict which ones are that and how to get them, nor is there indication of this thing going on to the same proportion nowadays as it was, considering the mathematical production is much higher. In mathematics human mathematicians have to decide which problems matter, it does not come from somewhere.
Solving "real" problems in theoretical mathematics (as in problems directly related to applications) is a very small proportion compared to the vast majority of math work that does not. So if we are discussing about the future of mathematics as a field, we have to understand what the field of theoretical mathematics is actually about.
So my question is this:
Would this slow technological progress? Or make it go faster?
That math problems are found and solved as we run into real physical problems.
Technological progress is not bottlenecked by most of the millennium prize problems or erdos problems per se, or most of the rest open problems in theoretical math, ie that merely knowing the solution of them will help applications in some manner. I doubt the solution of such problems has any direct effect on technology progress at all, at least in any deterministic, foreseeable manner.
In fact, the relationship between theoretical mathematics and "real physical problems" is bidirectional, as in "real physical problems" informs to some degree some problems that may be interesting to research on in theoretical math, and at the same time pure mathematical research that is developed completely independent may find applications at some point. And even theoretical mathematicians working close to applications are mostly dealing with problems not directly addressing applications. Eg maybe they study properties of a certain function that arises often in application without any view to solving a specific "real physical problem" with it, and somebody after may find that useful for some application after some point, but that could be one out of 50 papers (random number) and it is hard to predict that. There is of course some work more related to specific real problems, but that's most often not what theoretical math is about, and not what these new developments with erdos problems, navier stokes etc are about.
So what could (in a chaotic sense) have effect in application is mathematical theories developed along the way of solving these pure math problems, which brings us back to the question of what happens if we remove this friction and if AI can do more than construct examples and proofs, ie actually build theories (autonomously or humans+AI). If anything, it is through building theories that mathematical progress germinates applied sciences, as this is the process that develops mathematical tools that can be taken up later, including whole mathematical fields. Building mathematical theories is a heavily social process, and it is the community that basically decides which directions are important to follow.
Agree, just adding the observation that
problem specificity, concreteness, engineering relevance, or even "empiricity" seems (vaguely)
proportional to how much "good friction" can be generated.
There's also bad friction related to "meta-ness", "bad names", "aesthetics", etc, I presume. Like bikeshedding and its relatives. Is yakshaving?
Eutripsis? Vs just tripsis
AI can also introduce its own "bad friction", or it (ppl?) can bypass good friction without necessarily directly be removed by ai, eg bikeshedding, slowly pivoting towards directions and problems that ai is better in tackling, because that can produce these accelerated results vs fields and problems that ai may not be able tackle as well and thus the output there is poor, demotivating people from following these fields and missing important insights from them. Of course this could have the opposite effect, depending on which direction the whole hype can go, or not happen at all if ai will be able to tackle everything equally well.
What real-world consequences are implied by a solution to Navier-Stokes?
That depends on the solution, right?
Yes, and here towards the bottom the author gets to the reason he didn’t sign with the other medalists:
> I felt that there was nothing to be gained from criticizing AI companies for generating too many solutions too quickly.
> Under the circumstances, I think the best we can do is recognise the changes that are coming and try to work out the least unsatisfactory way of dealing with them.
Basically let’s make it a short-term problem and deal with it. Groups of people can deal with short term emergencies. Don’t turn it into a structural issue.
And in my view what’s the alternative in the letter exactly? The tools exist. Is there going to be drama every time somebody decides to use them?
The tools exist, so let’s try to figure out as humans the best way to use them to promote humanity, and let’s advocate against ways of using them which are a detriment to humanity, which is exactly the charge the letter makes. There are cultural norms around the use of every technology.
Well I think in response to 'what are you going to do', what people are probably going to do is stop sharing on-going work, stop bothering to curate problems that are just going to inflate someone's IPO valuation, basically accelerating the tragedy of the commons that the AI companies are driving.
Seems legit. I speculate their are emergent mechanisms for the development of memory that are not dependent on what ever mechanism is behind the "improve $model for 'everyone'"*
Is someone knowlegeable on research that engages the question on the development of something that could be considered as some emergent mechanism of memory, that is independent of instances and their contextwindow an specifically also independent of the use of conversations for trainingsdata?
*(This is regarding the value of selfhosted models - maybe small selforganisations that share ressources to do though to do so? Generally the value of machine learning seems to be to big to reject)
> With that interpretation, the issue becomes slightly different: is it more important that the collective understanding of the mathematical community should be as advanced as possible or that there should be answers to as many problems as possible? Or are those two aims valuable in different ways, so that there is no point in declaring one of them more important? Or are they so inextricably linked that it makes no sense to argue that one is more important than the other? And when we say “important”, for whom are we saying it is important: for mathematicians, or for society as a whole?
This is easy to me. Truth should be the North Star. If there is a fundamental truth that can be found via mathematics, then the shortest route to that truth should be preferred. While LLMs are definitely capable of solving problems in search of truth, I agree with Tao that instant "true/false" results threaten to short-circuit the traditional avenues we have used to escape local minima in the search for truth. Their products may be the junk food that provides immediate satiation in exchange for long-term health. Perhaps it's wrong, though.
The whole point of the letter was that if we adopt this gradient search strategy the humanity might get stuck in a local maximum forever.
I think it is easy to just roll of a local maximum, however, you may get stuck in a local minimum.
I kid, of course, but I do wonder where the use of local "maximum" comes from, what is maximum there? Why do you not see this as a landscape of hills and valleys where marbles with certain energies may indeed get stuck in deep enough holes... Of course, I just assume and picture gravity pointing down in that landscape, but hey. I'm human, I feel it is expected of me.
Thing is, you can say that about any technology (not being here necessarily pro AI in math, but I think we need better arguments).
The letter is not incompatible with “pro AI in math” unless one thinks arguing against extreme behavior such as corporations using millions of dollars to scoop results makes one “anti AI”, which is not a reasonable stance in my opinion.
> AI offers the potential of enhancing and accelerating genuine mathematical study and understanding. Mathematics as a profession will need to adapt to these changes in several ways. However, whether these changes ultimately benefit the field or have a destructive effect will in large part be determined by the decisions of the humans in control of this new technology.
I mean sure? It's clear PLT and other applied CS fields are now as good as dead. You're never seeing new React or Python emerging ever again.
And unfortunately mathematics is much more fundamental to human endeavor than this.
I don't think that's true. Domain experts with the desire to create and experiment aren't going away any time soon.
It is getting a lot harder for those people to justify using OpenAI to assist such endeavours. Afterall, OpenAI might just front-run you if they hear a rumor you solved some marquee problem that they can brag about in PR campaigns.
Sure we are free to tinker, like the guys doing model railways. With about the same impact.
We'll have to check back in 5 or 10 years to see if anything new has been created.
Then we develop ways of breaking out of local maxima.
Wherever we can recognise a problem we can solve it.
Nobody is going to recognize anything if the institutions of mathematics are torn down.
This isn’t true, as there are plenty of things proven beyond reach.
And plenty of things, eventually solvable, can create major problems that could both be avoided and the problem solved by taking a much better path.
Having technology and the ability to safely and sanely use the technology needs to progress together at a similar rate. The failure to do this is even a reasonable and common solution to the Great Filter. Jared Diamonds book “Collapse” has ample examples of cultures that wiped themselves completely out via not having this balance, so it’s not simply a theory.
Tao's point is that we have a method already.
Can't we get stuck in local maxima because of the lack of LLMs?
The letter is not advocating not to use LLMs. It’s advocating to use them in ways which don’t erode human understanding.
as a society of researchers we've tended to cultivate pretty effective strategies for escaping local maxima. I think of it like ants, where you can see if you place an obstacle in between their nest and a foot source, they develop a path that loops around it. if you remove the obstacle, for some time they continue to follow the old looped path. However, some ants deviate and go around at random, exploring. eventually by chance one happens to find a quicker route. he gets a couple of his friends to follow him, by pheremone, and over time more and more take the quicker route, and they end up abandoning the old route
It works this way with research, with most following the current trends, and some curious souls searching around for other ideas, be they contrarians, dreamers, or just convinced of some strange truth. But if we're right, signs tend to slowly begin to point their way, and we can shift the whole hulking edifice of science towards their point of view.
The problem of llms is that while they may be able to find a shorter route, we can't follow them unless we understand the route. So the forces that slowly begin to change everyone's behavior are lost
“Forever” is a long time. That’s quite the claim.
This pattern of argument keeps repeating in every place.
Pro tech people: technology removes bottlenecks. Sometimes we use those bottlenecks as a side effect to build muscle and so on. But removing bottlenecks gives us much higher degrees of freedom. It is up to us to coordinate and make use of the technology.
Anti tech people: bottlenecks are fundamentally useful. They should remain and technology shouldn't remove those so easily. Humans cannot coordinate as well when the bottlenecks are removed, so lets not remove them so quickly.
I can only suggest reading the letter with open mind and literally, without uncharitable bias towards the authors or conspiracy mindset.
FWIW it didn't sound accusatory to me, it made me think the anti tech people might have a point.
It’s not even anti-tech though. Terence Tao signed it. The commenter above reduces the conversation to a false dichotomy.
Anti immediate-tech-progess-as-primary-objective?
I don’t think the letter is even anti-tech progress. I also don’t think the tech companies are primarily pro-math progress (not that I think you think that). It’s primarily “anti-unnecessarily-destroy-the-human-systems-of-mathematics-just-for-benchmarks-and-marketing”
> … the push by AI companies to solve mathematical problems as a benchmark is detrimental to the science of mathematics, and to the mathematical community. The goals of the AI companies and the goals of the mathematical community are severely misaligned.
Its not a conspiracy. Its 25 fields medalists who think exactly like how I put it. Ideally, they can just take whatever the technology gives as soon as possible and use it. They don't want it because they don't think the community can rearrange and coordinate such that they can make use of the new found degrees of freedom.
That's literally all there is to it - they don't believe in the rearrangement.
I can only envy the insight you have into thought processes of 25 fields medalists. Wish I could have that.
you must have been so smug writing this. But I wanted to engage in a discussion.
From your other replies it's clear you still haven't read the letter. There is little to discuss so far.
Discussion is indeed impossible if you're trying to insist that this garbage anti-AI letter is anything other than just that.
Counterpoint: I'm not anti tech at all. In fact, I sell an AI harness for legal.
I think you've set up a false dichotomy. I'd propose to you the middle ground that a lot of us are concerned that VC-backed AI slop is "solving" problems in indigestible ways that hollow out the core. This applies in OSS as well as mathematics.
Please help me understand this and I'm asking this in good faith.
Why can't OpenAI publish whatever it wants. And the math community can use it or not use it. Fundamentally OpenAI's solutions are high signal - they are incentivised to not deliberately mislead people. Let the individuals in math community choose to read it or understand it? If OpenAI wants to publish something, let them do it in the current channels using peer review using whatever time is required.
What's wrong with this? The math community thinks this will destroy previously unwritten ways of prestige allocation and remove incentives that used to exist. I say that the community can rearrange and allocate prestige and time in different ways to maximally use the technology.
They can use the new proofs as foundations for future proofs.
But many folks just won't be motivated to attack or help digest solved problems because there's less extrinsic value in doing so. Tao and others argue that it's this human effort that finds human-relatable abstractions which spurn further investigation. Take the humans out of the loop, and they'll stay there, is the argument as I understand it.
I read this multiple times and also recollected the original letter. You can’t really blame me for thinking this is a straaaange request.
“Don’t help me solve my problems because it removes the incentive to be in this field” is how I hear it. Honestly.
I would say the incentives HAVE to change because of new found capabilities.
Imagine accountants asking companies to reconsider producing calculators because we used to value accountants for how well they did arithmetic!!
Edit: my main point is that the math community has full agency to do what it wants with new found tech. Instead, asking to change external entities like OpenAI to be careful with releases is a strange thing to do. Why not change yourself and adapt?
>community can rearrange
You're describing an improvised surgery on a living organism. Developing a complex system involving humans that is productive and doesn't collapse is extremely hard, so if it ain't broke don't fix it.
But man this is ridiculous. 25 field medalists don’t accept agency in their own community but try to push external people to change their process??
Why can’t the letter say “guys new tech dropped. We need to reorganise. Understanding is crucial” instead of asking OpenAI to change their process.
The Math community can "not use" a proof? How would that work? It's a little like coloured functions (async etc) - you have 'human proved' vs 'machine proved'?
The dichotomy is more 'human understood' vs 'not human understood', I think
Are you suggesting that a human can't have the agency to decide whether to use it or not?
No, obviously not ...
I'm pointing out that the way Mathematics works as a discipline is that a new proof builds on existing proofs. If we have a bunch of machine-generated proofs, then a human Mathematician has to decide whether to reference any relevant machine proof that has been put out there.
If, as you suggest, human Mathematicians 'choose to ignore' a machine proof, then another human decides not to, what then? Mathematics bifurcates into 'pure human' proofs and 'mixed machine-human' or maybe 'pure machine'?
Well ok I understand your point. I’m trying to say the system has to change to adapt.
Imagine accountants saying the same thing against calculators. They had to adapt. So do mathematicians. So do I in software development.
Consider that 'calculator' used to be a title for a role a human used to do. That went away, which is probably fine as there were other jobs and likely it was tedious.
Then consider that "just adapt" could be "adapt or die", and that maybe, just maybe there is no way to adapt to well-funded corporations churning out millions of proofs a minute.
I don't know, maybe AI companies have "agency" to not break everyone else's stuff, just to get more funding and a higher share price?
One thing I've found in my own use of AI, which Gowers touches on in his point at the end but which I think deserves more attention: AI makes some previously non-trivial tasks trivial, but that doesn't make the work itself trivial. You naturally expand the scope of what you attempt and take on harder problems than you could before. The floor rises, and so does the ceiling. The open question is whether that still holds once models can also do the harder problems, or whether choosing and framing those problems remains the human part.
Your position is compatible with the letter.
I think the author did not mention that the way humans construct the knowledge ladder requires low complexity because each step gives power to the next, but the AI with the exponential exploration can give biggers steps but then it stagnates because the next step can be beyond the exponential exploration complexity. In chess a good strategy can be the best tool, in the game of go our experience suggest the same, but it could happen that mathematical thinking requires a type of policy that could be beyond the current ideas. I can not fathom a LLM could conceive from scratch concepts like the real numbers by Dedekind.
> I can not fathom a LLM could conceive from scratch concepts like the real numbers by Dedekind.
What stops you? I can fathom it alright.
I am always suspicious of these arguments like "a machine won't do thing x which humans can". These always basically pre-suppose that humans aren't machines. If that's what you're arguing for, say it outright.
A cynical could say that AI could not interpolate the Dedekind cut but it could extrapolate to create a lot of money but just using an imaginary extra point.
The ironic part is that generating a joke like this requires jumping across three distant fields: real analysis, machine learning limits, and VC market cynicism. For an AI to discover that specific overlap through statistical search, the combinatorial space is absurdly huge. Yet a human brain connects them in a fraction of a second. This tiny joke is a micro-proof of the macro-argument: human conceptual leaps routinely bypass exponential search spaces.
Just to add that the Dedekind cut example seems to stand beyond any RL policy used in chess or go, AlphaZero or AlphaProof. In the classical RL there is an state-action space. If the solution requires jumping to a totally difference action space (that must be created) the local policy stalls. Dedekind cut is an example of a out-of-distribution state-space generation.
Solving a theorem is like climbing a new mountain. The mountain is already there, and there is a list of the hardest known mountains to climb. The point of climbing them and not just dropping with a plane on top of the peak is to help develop human climbing skills and expand our knowledge and abilities. Also, already conquered mountains are climbed all the time to test new strategies.
Now, if an LLM proves a theorem, it's like discovering a new mountain and knowing what its peak looks like. Does that mean the problem is finished? No, we still need climbers to actually do the work and advance the field with human understanding.
The job of a mathematician just shifts from proving to inventing new theories and finding new conjectures with the help of AI, which should be even more fun.
> proving to inventing new theories and finding new conjectures with the help of AI
This is exactly the activity Tao and others worry will be hollowed out by instant "true/false" responses to novel questions.
inventing new theories is the interesting part.
finding new conjectures is the second interesting part
"instant "true/false" responses to novel questions."
Fair enough. Then we can add ordering and analyzing theorems with the help of AI as a third activity, admittedly the least interesting one.
> we urgently need to come up with good ways of explaining the value of having a large pool of human mathematical experts, even if it is no longer part of their role to find new proofs of theorems.
This is the main issue, and while I fully agree with that value sentiment, the referenced letter failed to provide convincing arguments for why mathematicians should widely receive funding for merely understanding things, and how competition for postdoc and tenure positions would work.
Knowledge wants to be free.
Tokens want to be paid for
> One way that might happen is that AI disrupts society so much, or even kills vast numbers of us, that the preservation of something like the current mathematical tradition ceases to be of any concern: all that will matter is the survival of the human race. But that again is a topic for a different blog post (which in fact I am in the middle of writing).
This is pretty disappointing I have to say. The letter actually has a “pro AI” stance (see quote below), and Gowers’ apparent reason for not signing it is so subtle (not wanting to offend people whose goal in mathematics is not understanding) that I can’t help but view this as being more about optics than its actual content; i.e. he seems to be cleverer than I am in recognizing most people will not see the letter as a push to use AI in ways which are healthier for human flourishing but as advocating for one of two sides in a highly artificial binary (“pro AI” or “anti AI”). Gowers draws a distinction without a difference here and should have just signed the letter.
> AI offers the potential of enhancing and accelerating genuine mathematical study and understanding. Mathematics as a profession will need to adapt to these changes in several ways. However, whether these changes ultimately benefit the field or have a destructive effect will in large part be determined by the decisions of the humans in control of this new technology.
Watching the discussion unfold, here is what I feel:
Before arguing whether mathematics must strictly be done by humans, there are different motivations at play. Some people love the sense of solidarity within the community that forms during the process. Those excluded from that community might resent it, while others just purely want to solve problems.
Many things are being discussed, but looking at the overarching narrative, it seems that AI's true function isn't necessarily opening new horizons of specific knowledge, but rather excelling at 'serializing' topics that have been heavily fragmented until now.
In that sense, the concern is that because AI is solving the very problems needed to cultivate mathematicians internally, the stepping stones required for human growth are disappearing.
However, on the other hand, as the world and industries become increasingly complex and hyper-specialized, you could also argue that AI is the exact tool needed to unify this fragmentation across academia and industry. It is a highly complex dilemma.
From the perspective of researchers and the mathematical community, those 'problems for growth' must remain. But conversely, AI has the distinct ability to serialize siloed disciplines. Usually, when you go to graduate school, you often hear professors say that even within the exact same major, they cannot understand each other if their sub-specialties differ.
[I have not read Gower's article. It's only that your response is prompting me to share what i've been thinking recently]
>the stepping stones required for human growth are disappearing.
Actually only in an institutional sense, imho. Maybe I'm exaggerating, but reddit.com/r/math* or even mathstackexchange will be so back with users analyzing and distilling proofs with AI. Maybe in 5 years (when lean attains 10% popularity of rust, and/or gpt7 level models cost ~USD5 on average, per month, inflation adjusted or not) this types of submissions will make those sites as fun/educational as mathoverflow [has always been for me]. One dreams that by that time openAI and Anthropic will have taken down your ethno-cultural "compatriot" Masayoshi with them.. (I prefer his brother whom he did not regret giving a physical beating to. Only purely on principle) so that they can't acquire these sites
Such forums can then replace math grad school, if the profs/alpoges who drop by get into the habit of constructive criticism (as they do on mathSE already). It's already starting, I see personally interesting 1 AI-aided submissions every 1.5-3 weeks starting 2 months ago
It would be like rust discussions on HN for you I bet.
This seems unlikely when s/math/physics/, or s/Reddit/HN/ because HN hates AI-aided posts so ideologically (sorry mods, I don't mean you guys). MO is also not as welcoming to outsiders/lay, congruent to academia (only online) PhysicsSE/overflow had become dumb and arrogant last decade
HN, even pg seem anti-intellectual in effect tbh whenever they sneer at AI _writing_ (again sorry mods, you will figure this out soon I believe, do you or do you not want HN to end up as a reservation for meatbrains), though I mostly am on their coder side with regards to this fields medallist letter
The AI debate is, in fact, somewhat ideological. The problem is that in the process, whether advocating for it or pushing for progress, there is a failure to face its limitations directly. In other words, it is emotional. Because AI's mere existence is a threat to knowledge workers, much like machines were a threat to blue-collar workers. That is, if the value of intellectual labor drops, it damages the value of the labor force through which workers earn capital in a capitalist system.
I have tried solving a few math problems with AI (they were Erdős problems), but because I know absolutely nothing about that math, I couldn't just take the AI's word for it, so I am actually a bit skeptical. A problem arises where you arrive at the answer without actually understanding it. It's not that AI is bad. The problem is that AI destroys the equilibrium between the knowledge I have and the knowledge I lack. That boundary collapses, making it feel as if I can know everything.
Actually, academia is fundamentally about mental models. It's a kind of internal worldview, and that worldview is shared. When you actually listen to the thoughts of scholars and professors, there are subtly different aspects. That forms the person's worldview... and I get the feeling that sharing it is what constitutes intellectual activity.
However, as you mentioned, unlike academics like yourself, academia and knowledge communities seem disconnected to someone like me (meaning they lack accessibility). Even if I were to make a discovery, it would probably be hard for me to become recognized, and I do think AI could actually play a role in opening up those closed communities. But apart from that, I find it hard to say that this only has positive aspects.
I followed Karpathy's research from start to finish to build a small LLM like nanoGPT on my own, and people say that because of the positive transfer that comes from feeding diverse data through modern LLM multimodal encoders, there will be new discoveries. But in reality, the types of problems AI excels at are generally those that humans have found but overlooked. In my opinion, rather than being a knowledge machine, LLMs (or AI) make me feel that what we call "intellectual activity" is closer to a kind of serialization work. A method of stacking things up one by one in sequence, so to speak? After all, the actual operating principle of an LLM proceeds according to the probability of the token coming in the next sequence.
In other words, I think the serialization method of our knowledge activities is similar to how LLMs operate, but I also think a different kind of thinking might be necessary. I am not that smart, and I have never interacted with scholars... (As you know, I am a subcontract worker. Of course, I have been hired by startups run by professors in my country, but it's not like I modeled that intellectual design myself.)
On the contrary, I feel that the evolutionary approach will slow down after GPT 6 Astra. They can continue to increase the size, but the issue lies in the cost-effectiveness of token costs.
Anyway, I agree with most of what you said in your discussion, but rather than anti-intellectualism, I consider this a direct threat to survival.
>types of problems AI excels at are generally those that humans have found but overlooked.
Seems to be a deep and interesting angle lurking here, especially when coupled to
>evolutionary approach will slow down
Meaning something I will have to think about (while keeping Graeber in the background[0]) or even plan around :)
Will respond after I sleep on it
[0] https://davidgraeber.org/articles/value-as-the-importance-of...
In value terms, the question becomes: who has the right to translate their money into what sorts of meaning? Who controls the medium through which, and the institutions through which, our actions become meaningful to ourselves, by the very act of being publicly recognized in some kind of public arena? It seems to me that while if one is trying to understand the strategies by which people can move back and forth between “fields”, and especially, by which some are excluded from them, Bourdieu’s models are pretty much indispensable, THEY DO LITTLE to tell us why anyone wishes to enter certain fields to begin with.
The evolution of fields, of domains of enquiries. AI-independent
First of all, I take it for granted there is really no such thing as “intelligence”
Writing like this from 2005, feels like a personal bedtime Bible for AI age
Siri, can you call the bicameral order, the captain has published again..