The author argues for evaluating Ph.D. candidates based more on the oral thesis defense than on the actual thesis.
By essentially the same reasoning, I’ve been arguing for prioritizing in-person design/code reviews over code-only async PR comments.
The important thing is to verify that the human has a coherent design in mind and can demonstrate that it got implemented, regardless of who or what was at the keyboard. “I dunno, I guess Claude thought this was a good idea” is not a coherent design.
An oral defense is ultimately a meeting, with the same weaknesses as other types of meetings. In particular, if new information comes up in a meeting, you cannot reasonably expect to get useful responses from the other participants. First impressions maybe, and plausible-sounding bullshit from those prone to generating that. If you want anything more, you need to provide the information in advance or schedule another meeting later.
Oral defenses in academia are largely rituals. If a student fails their defense, it's almost always the supervisor's fault. The supervisor is supposed to be the primary quality control. With their regular meetings with the student over the years, they should be able to tell whether the student has achieved sufficient understanding and contributed enough to graduate. If the supervisor thinks the student is ready to defend, the reasonable expectation is that the student will pass and graduate. The defense is mostly there to let the other examiners validate the supervisor's judgment.
Here's another example of a similar but slightly different failure mode of the live defense: The department where I got my degree had a practice around PhD defense where there would be a prof each representing the four research pillars + the dean. It was well known that certain professors disliked each other enough that they would snipe each other's candidates. You'd just hope not to work for one whose enemies got picked for the panel. If you were a superstar, you'd pass, but if you were an okay candidate or someone strong but with stage fright, you be could toast even if you did good research.
The defense was only briefly about the actual thesis, then switched over to whatever research interest the committee members had, and they'd drill into their pet subjects. This was in physics, hardly politics forward normally, and at a very well respected university.
You are missing the entire point of the oral defense while almost stating it. The existence of the oral defense is what extrinsically motivates the supervisor to do their job well. If the student fails, the supervisor loses faces in front of their peers, which obviously they try to avoid. Removing the oral defense would massively reduce the quality of PhDs because supervisors will not do their job.
Also in my observational experience in physics, a fair number of students (probably like 10%) get some sort of major corrections to their thesis. After writing this, I did a search, and in the UK, the number across all fields is 16% [1].
And even then, the other examiner will only fail the candidate under egregiously extreme conditions. Rather, if the supervisor wants to pass someone who is unfit but not egregiously, this will just erode the supervisor's reputation among peers. Because there is a grapevine of course. So it can't be regular. But academics also really care about face and embarrassment, so they won't make a scene, u less there is preexisting drama around the supervisor. The only fails during the oral I've heard of happened when the candidate insisted to defend against their advisor's wish.
Person B: “key pieces are delayed due to the developer not understanding all of the LLM doesn’t and implementation.”
Manager: “does it work? What are the risks?”
Person B: “well yes it works for now but we’re accumulating tech debt due to a lack of understanding and potential flaws that haven’t been thought out yet”
Manager: “they want feature X, ship it, we can deal with it later, I don’t care if it’s not coherent as long as it works.”
How many decades at this point has these been a push for functionality over everything at all costs? And you have a mechanical snow plow now. Most businesses don’t care about later risk or any future planning beyond the quarter horizon, they’re not concerned about how it will effect their performance in 3 quarters or lead to instability or issues, those are future problems for a future person and we’re here for money now.
The simulated dialog you have provided is very much representative of what many of us have heard first-hand. However, I disagree with:
Most businesses don’t care about later risk or any future
planning beyond the quarter horizon, they’re not concerned
about how it will effect their performance in 3 quarters or
lead to instability or issues, those are future problems
for a future person and we’re here for money now.
Businesses care about "later risk" and what it implies.
Individuals within an organization do not unless it will specifically affect their bonuses/promotions.
Another way to say this is that this has worked fine for decades— ie, software businesses have done just fine prioritizing features over code cathedrals. What I find more surprising is that many engineers still can’t seem to accept this basic fact, after decades.
I suspect AI will actually change this. The opportunity presented to business in dramatically relaxing both our understanding of our codebases and our tastes and preferences is big enough to justify letting go of folks who don’t get it.
I think upweighting the live components of academia is inevitable in the age of automatically produced writing, but I also find it depressing that people think so little of writing that they imagine it obsolete because of AI.
AI writing is aggressively, aggressively mediocre. It is only good for how cheap it is. If you're indifferent to how much better good human writing is than AI writing, you should not be the one to evaluate human writing.
The tricky part is that live review is much more expensive than async review. But maybe that just means we were getting away with a cheap proxy because it happened to work reasonably well.
Unfortunately, from my experience, forcing people to talk about their research to an audience mostly just condemns introverts and people with social anxiety. Being able to formulate your thoughts clearly in writing and in speech are two radically different skills. I've seen world class scientists give the most horrible conference presentations to the point where everyone in the audience agreed they'd rather have just read the paper when it came out, because they had no clue what he said while the author is amazing at writing. If we want to switch academia and to purely oral skills, you'll probably need to replace a ton of people. And in the industry these fields are already dominated by fast talking bullshitters.
I do this already if there’s any kind of issue. Just talk the person if they’re next to you or zoom them if not (don’t set up a meeting). A five minute conversation can save hours of back and forth.
>“I dunno, I guess Claude thought this was a good idea” is not a coherent design.
I am sorry, but the writing on the wall is that this is where software industry wants to go. As a software developer, I cannot but help notice how unrealistic this is.
We are entering the next level of bloated, inefficient and unreliable software....
Excellent optimistic post in a sea of negativity, and with actual suggestions, too.
After reading, my mental image is this: think of Olympiads in Ancient Greece.
* A weightlifter was only awarded a laureate if he were able to lift a heavy stone (have no idea what they were lifting, for illustrative purposes only :-)
* Along comes Archimedes who invents what we would call an exoskeleton. Now any regular guy can lift twice as much as last year’s athlete.
* What to do? You can cancel the Olympiads, but they are actually useful as training, motivation, etc So now you have to give the prize on other factors, eg how well he can lift, has he opened a gym in the city, etc
BTW, physics and bio are not exempt, so those researchers better read and try to stay ahead.
I don´t think so. For programming agents can run code, check compiler output, etc. For mathematics, it is almost the same once you factor in the usage of lean.
For the reality, you can´t close the loop that fast, or with that precision. You will have to slow down by several orders of magnitude.
Doesn’t have to. There are petabytes of experimental physics data that can be fed to AI to extract additional insights. The only holdup is this is slightly harder to than math.
With bio, you’re right, generally designing and conducting an experiment goes hand in hand and theoretical biologist is not a common label.
> We already interview faculty hires; we must now do the same for graduate admissions
They hire PhD students without hearing them give a talk and then doing interviews? In Germany, the applicant gives a talk (30-40 min) to the research group they want to join, usually presenting their master's thesis, engage in discussion, often share lunch with the group, then do 1 on 1s with individual members of the group and a longer one with the PI. Obviously this can vary within Germany too, but I couldn't imagine hiring someone without something like this.
1. US a lot of international PhD applicants, so traveling before even being accepted is difficult, and 2. lots of people don't do a masters.
when I applied to PhD programs (not in math) it was basically CV + personal statement + recommendation letters + short chats with interested faculty :shrug: Maybe it was because my CV was "strong" but the chats were more see if interests were aligned, rather than actually interviewing me.
Remote talks are also a common alternative in such cases. Zoom etc.
I also know that another major difference is that American universities tend to hire without a professors involvement, into a generic "program", then the PhD student seeks an advisor after being accepted, so applicants have to woo some unconnected committee pursuing various goals misaligned from the PIs instead of convincing the PI. In much of Europe a professor basically "owns" a chair and really is boss and decides hiring pretty much alone.
Anyways, if they don't have a masters' yet, they could present their bachelor thesis. But I already think it's a bad model to combine the masters (courses) and the phd (research) into this American hybrid that's the direct-bachelor-to-PhD jump, but that is somewhat unrelated.
I agree the combined masters + phd is weird tbh. I think almost everyone treats it like "just focus on research and spend as little time on course work as possible." I would prefer it if courses were more flexible.
I'm not a big fan of the US application setup. IDK how it is in Europe, but in the US, it feels like there's a lot of not-very-meritocratic "secret" stuff you need to know to up your chances.
> I'm not a big fan of the US application setup. IDK how it is in Europe, but in the US, it feels like there's a lot of not-very-meritocratic "secret" stuff you need to know to up your chances.
In Germany (countries differ quite a bit on these details) a significant chunk (though not necessarily majority) is directly hiring a master's student who worked at the same chair on a part-time student research assistant job and/or a master's thesis and did a good job and typically turned the thesis into a paper submission. In other cases it's still somewhat inside baseball like getting recommendations from other scientists that the PI knows, or coming from a university that the professor knows well. It can also be from an impressive email, from a chat at a conference poster, etc. There is much less of this global casting that committees do, like "we need this and this type of character now" to have a vibrant student body composition, since each PI makes a decision just about their lab, without having a global view, and their goals are basically to get research out of that person, papers, results useful for funding applications etc. So they want to see aptitude in that. Through talk quality, communication efficiency when talking with existing team member one on one, recommendations, sometimes even phone calls with the PI with whom the student worked (these are small worlds). I think it's more organic and natural, though still prone to exclusionary gatekeeping, than the HR-style narrative essay-and-extracurriculars-based admissions though maybe that's more prominent for undergrad admissions even in the US.
Unlike Europe, in the US, most PhD applicants do not have a masters. 10 years ago, most had not even conducted substantial senior projects (e.g a semester or year). Though increasingly senior projects / undergraduate "theses" have become more common.
In my opinion, that's not a good time to really know you want to pursue academic research. A masters gives you a little taste at least and allows to show aptitude. Doing a student research assistant job, working on a thesis, maybe even a paper submission, diving deep into specialized courses. Then you can know if it's for you as well as the PI can know if it's for you. Straight from a bachelor and them interleaving the phd with coursework is a bad model I think, but of course I'm from Europe so it's what I'm used to.
In the US, usually you do a summer research program if you want to be a PhD student, so you can get a taste of research.
At most universities as well, if you decide the PhD isn't for you, there is always the option of leaving with a Master's. Because you get free tuition and a stipend, you also come out ahead of if you had gone straight for the MS (which you generally would pay for).
In the UK I had a panel interview with 4 people, in person. But for international students who often brought their own funding sponsored by their government that didn’t happen.
That's an interesting wrinkle. If anything, externally funded international students seem like exactly the case where you'd still want the interview, since the funding answers "who pays for this?" but not "does this person have the background and understanding to do the work?"
As someone who has a degree in math, I still can't help but think mathematicians are getting a little bit of a comeuppance. In a lot of areas of mathematics there had been little effort to make the work understandable and leaves numerous folks who could benefit from the knowledge on the outside looking in. Now AI comes along and do the same to mathematicians. Makes me chuckle a little bit.
This feels backwards. I frequently joke "physics is the subset of mathematics that reflects the observable world". Math can be as abstract as it wants, but physics has a constraint. It must model an observable world (related, this us part of why people say String Theory is math and not physics)
Never been my experience of maths, which is one of the most accessible disciplines to have ever existed.
I also have a degree, but mostly taught myself the important undergrad-level concepts as a kid. I just went to the library and borrowed any of the hundreds of books written to clearly communicate maths to beginners or downloaded any of the free ebooks / lecture notes.
Name a single field of endeavor that is more open in 2026. Software certainly isn't one of them - the best stuff has always been gatekept.
I too have a degree in math and it's easy to see some of what's happening as too much of an emphasis on intellectual chest thumping and to little emphasis on showing why understanding the concepts is useful for everyone. I will say the problem may well increased here by the American education system which denigrates such understanding and isn't influenced by mathematicians.
But also, with things rapidly changing, perhaps in ten years chat programs will not only do superhuman but make their proofs marvelously accessible and provide incredible tutoring sufficient to bring any curious up to a super high level quickly. Then what can you say and what can you complain of.
But I think "do everything machines" are necessarily inevitable but the situation does make it uncertain where the limits are.
I think you're a little naive if you think this is because of gate-keeping by mathematicians rather than the essential complexity of mathematics. Mathematicians individually and as a group would love nothing more than a world which is capable of understanding their work more deeply.
In the exact same way that LLMs allow anyone to vibe code an app but do not replace real understanding of system design due to its essential complexity, non-mathematicians will quickly learn that asking an LLM to pump out advanced mathematical statements to you, even if they are correct (and even if you could verify them) does not constitute understanding, and that the human brain is the bottleneck either way.
It is only if the LLM is super-human at simplification and explaining that a difference will be noted. This would be excellent for mathematics but its not a foregone conclusion (and the argument of most mathematicians, such as Terence Tao, is that this distillation process is one of the key parts of doing mathematics, and that LLMs so far seem to be going in the opposite direction. I suspect its probably user error and leveraging the tools better will produce different outcomes, but mathematicians are only just starting the journey that software developers have been going through, so patience is needed).
Folks keep using the word "gate keeping" -- I never used that term nor implied intentional gate keeping. It's simply not caring if those outside the club understand -- there is no one guarding the door its just folks don't care if anyone finds it.
Gatekeeping is a human universal. Even those who preach maximum inclusion do exclude plenty, of course there is always explanation why that doesn't count. If you try to run any community you quickly learn the importance of gatekeeping. Eternal September etc. See any site or fandom that gets mainstream etc. I don't think anyone has any duty to actively pull in as many as possible new people. Zen masters used to chase away students or make them sit on their doorstep for days and send them away anyway. Now, the west is essentially Christian and so the missionary impulse is real, but it doesn't work without the other stuff in the package.
Again with the gatekeeping. An AI generating a convoluted proof for some conjecture is not gatekeeping. It just does not care if you understand. It only cares enough that it can be verified -- usually using some other opaque proof tool like Lean.
AI is much less gatekeeping and much more welcoming for beginners than human experts are in general. It has infinite patience to break things down for your level of understanding. So AI has less gatekeeping which I like when I'm gatekept out of something otherwise. But I recognize the social function of gatekeeping and I don't see it as an inherent evil.
Why should we expect everyone to be both a great researcher and communicator? The obvious result is that it is as effective as engineering managers. Sure, there's some amazing ones, but most aren't. Though that also doesn't mean no expertise in the field (i.e. non-engineering manager) is any better. It is just that managing/communicating is a different and orthogonal skill.
What needs to happen is we need to make it okay for people to specialize in more things. More nuance to this rather than trying to throw everyone into nice easy to manage buckets. Those buckets are just unrealistic abstractions filled with hope, denial, and laziness. Reality is surprisingly complex. Math can do a really good job helping you understand that, but it's a sufficient condition, not a necessary one
It is not lowering the bar to find better ways to demystify and explain things. In fact, I would say those who can explain it well understand it the best. Richard Feynman would be my best example.
Do you have any examples of places you felt like there was a lot of gatekeeping? Perhaps having studied mathematics I am a bit blind to the issue here and would like to learn more.
I agree making simple things sound complicated to appear more impressive is bad but there are limits. Even with Feynman he could only go so far, e.g. his interview about why questions and magnetism.
I am not thinking of intentional gatekeeping, but more of the kind where mathematicians build themselves their own island of concepts and notation with no thought of building a passageway for others to engage and make use of the theory.
I would find modern algebraic geometry highly useful as someone who works in computer graphics and computer vision, but much of the theory is akin to learning a new language and I don't get the sense that those who publish their work in this field care if I enter their world.
I do think in a lot of fields there is a lot of impact/influence to be had by people who are willing to do work to bridge different fields. I wonder to what degree this is because practitioners don't always see how their work could be used elsewhere.
I would say that it is lowering the bar to "find better ways to demystify and explain things." But there is no harm to this bar falling, because this is a bar for entry. There is a separate bar for making research discoveries (being credited for them, specifically), and that one should not be lowered, because it is a bar of standards.
Half of modern academia, for one. Universities have turned into degree mills with the expectation that >50% of the population requires a college degree, regardless of whether they actually have any interest or need for one beyond doing it because it's a prerequisite for a good career, independent of whether said career actually uses the knowledge in any way.
Not that I actually agree that math was at the right level of gatekeeping. It definitely feels intentionally opaque beyond reason, and I think it's why LLMs are able to cut through the obfuscation and solve problems that maybe wouldn't actually have been considered quite so hard if mathematicians did a better job of making their work accessible.
We have seen the thing in programming. I'm not a gatekeeper, I love seeing more people to code. On the other hand, along the way, we lost the joy of the journey and only fixated at the destination.
Result is more software at lower quality. The reason is statistics. When you increase the population, you increase the population of every kind of programmer, and people who want the result are favored in most competitive sectors because corporations want something somewhat working yesterday.
...and here we are.
Now programmers talking about code quality is stoned en-masse. If it's somewhat working then it's good. Efficiency, resiliency, maintainability and sustainability is an afterthought. Some of my friends who loved debating programming language theory now don't even care about the code. They don't write it, just vibe, and they don't plan to come back to "older, caveman style of development".
Some universities are also adding fuel to the fire: They "prepare students for the job", not teaching the science, but the parts that corporations need for the job only.
Though, hardware was cheap and people were expensive, and now code is cheap and hardware is expensive now. We'll see.
The point I was trying to make was not "enjoying the process for the sake of it", but paying attention and spending effort on the journey created better software at the end.
When you look at older software, most of it was higher quality than the things we have today. A web site contained more information in a more readable way, more features in a smaller footprint. Same for native applications.
Now we slap what we found online together and calling it done. Everything is sparse, takes ages to load, centuries to submit and everything is so disconnected and async that some simple features are straight out impossible.
This is what fixating on the destination brought us. I dare you to download something you purchased 3 months ago via a 4096 character S3 link they have sent you, and double dare you to ask customer support for a new link. I'll bet that with a 80% chance they have no way to verify your serial number, even.
Have people completely forgotten how to read? I'm saying your analogy between software and math is completely, irreconcilably flawed. I don't need to read the rest of your screed because this is a thread about math not software.
Somebody said that lowering the bar is not good. Somebody else asked for examples, and I provided an example. So, if you want to be pedantic, that doesn't track well, because what I answered was not about mathematics.
If you want something about mathematics, computation is mathematics, as software is. So, my example tracks the same way in mathematics.
Finding solutions without understanding its parts or the path is equally detrimental to mathematics as it is detrimental to software.
Maybe you need to read a bit slower and think along the way. Using AI too much blunts critical thinking skills in some, as I read.
What was the end of the proof thing you mathematicians use, was it "Q.E.D."?
> computation is mathematics, as software is. So, my example tracks the same way in mathematics.
brother i already addressed this literally in my first response to you: the reason software went to shit is because it became commodified (ie a thing produced in a factory) not because it's computational nor because the bar got lowered. seriously read my whole original reply again and see whether you really disagree before just re-asserting your rant on SWE.
> Finding solutions without understanding its parts or the path
you didn't read the post at all did you? it's emphatically about deep understanding over blind results.
> Using AI too much blunts critical thinking skills in some, as I read.
Are you arguing that all the metal we have in our cars, buildings, airplanes is still being forged by hand? That that the furniture people are buying off Amazon is handmade? That stonework isn't primarily done these days with concrete pours?
>(which is that people who enjoy math will continue to be able to).
You posited that code is not like art, because it's primarily written for money and not for pleasure. But that's a stupid fucking argument, because there are plenty of crafts out there that are both done for money and pleasure (often by the same people!).
I would say the research university is getting comeuppance.
Pedagogy is the primary purpose of educational institutions. That includes universities. However, when research is placed first, you are often left with mediocre teachers, because "who cares?" They were hired to do research and lend the university that kind of prestige; teaching is an afterthought squeezed into the spaces that remain.
And frankly, we haven't the faintest clue today what education is even for. That doesn't bode well.
Add to this the other troubles facing the university, like declining attendance, skyrocketing costs, and AI cheating and we're in for a very interesting transition indeed.
I love how this is (without slighting the problems entailed) coming at it from a perspective of infinitude and abundance (we will always have more problems to solve) - which is the correct framing, particularly when dealing with ideas, or fields in the which ideas are the driver/product/output/material, and ideas themselves, the field itself, are infinite.-
People had the same complaints that the code produced by early coding models was messy, lazy, poorly commented, had terrible architecture and so forth. The central complaint was that it was just too difficult for humans to review. The answer is just to improve the models and move on.
Similarly now we're getting AI doing math. The proofs compile but are a mess. So just make the models better at writing clean proofs and explaining what they're doing to humans. That's the end of it.
Rather than just go on and on about how it's the end of the world if we don't do this, why don't we just do it?
> Similarly now we're getting AI doing math. The proofs compile but are a mess. So just make the models better at writing clean proofs and explaining what they're doing to humans. That's the end of it.
The assumption here is that the true/false of the theorem is the important outcome. While it is certainly part of it, a big part of maths is the understanding you gain from a proof. Many of the best proofs elegantly explain some aspect of the maths which was previously unclear and expand our understanding of the world.
To use a programming related example, imagine that an LLM spits out a solution to the travelling salesman problem which works in O(n) time. On the one hand that's very convenient for whatever problem you happen to be trying to solve at the time...but there's also an answer to P=NP in there! The former means your delivery drivers app works a bit faster on their busy days, the latter fundamentally shifts how humanity thinks about certain problems.
Going back to the maths, there have been theorems that were proved (by people) where the proof is broadly seen as 'unsatisfactory' in that it doesn't really expand our understanding. I assume some of these LLM proofs are a bit like that: we now know that the thing is true, but we really want to know why it's true, and how that changes our understanding.
No, I see this argument repeated all the time in these threads. "It will just give true/false, while humans need intuitive explanations". Why will the AI only give true/false? Where did this assumption come from? The nested citation was about AI that can give humans an explanation. The reply was "but true/false is not enough". It is as if it was a reply to another comment. How is that a great comment?
I think it's pretty analogous to coding, honestly. When it comes to gaining true understanding of a piece of software, I still find today's models almost as useless as they were a year ago.
They can write great code, they can gain understanding of something for themselves, but they still kinda suck at explaining it to people. As the article says, if you want understanding, it seems like there is no replacement for getting in the weeds yourself. A model can help, but it's not going to magically download knowledge into your brain.
One funny thing is that in order to tune the models to make what they're doing explainable to humans, you need to have humans involved in the RL pipeline to indicate which explanations are good.
You can understand this as learning a mapping between the model's internal "world" (i.e., 'meaning,' which is hopefully coherent and consistent -- but definitely not always! see, e.g., https://arxiv.org/html/2505.11581v1) and language (i.e. 'form') that reflects that world.
For this to work, you need both coherent / consistent internal model worlds, and also good mappings onto human language. Supervision by mathematicians has provided the signal for both internal coherence (though this can also come from interacting with a proof oracle) and for good explanations. If models exceed human capacities, you could imagine that aligning their explanations potentially becomes harder (though not necessarily). Also, humans naturally have to do the same thing: as researchers we must find analogies to make our work legible to collaborators or laypeople. Often in doing this, we further clarify our own understanding!
More deeply I think the "end of the world" vibe arises not only from the practical need to have models that explain, but also Litt's (and many other fields' researchers) grappling with being relegating to not mattering.
There's a fundamental difference between the goal of code and math. The goal of code is to produce software that does something useful. As long as the code does what it's supposed to do, arguably, it's good to ship. (As you imply, we want the code to be good enough to also be reasonably certain there are not too many bugs, that it is maintainable and can be extended etc. This is what early models failed at but now seems broadly fine.)
But for math: what is the point of a proof if no one will read it and no one uses its result? To quote the article, "AI systems will [...] result in the production of an abundance of PDFs. The contents of some of those PDFs may even have important applications." But if there's no one reading the PDFs, what's the point - no matter how good your AI model.
The point of math is understanding. So mathematicians should feel free to use AI as much as you want, but in the end, they should've gained some understanding on what happened.
If the point of math is understanding then maybe our incentive structure is wrong. Maybe it should be teaching the concepts to as many people as possible rather than just continuing to write papers that 10 people in the world understand, which is the current state of a lot of math.
I won't understand it, but can benefit from it in better data structures with proven invariants, faster algorithms, convergence guarantees for some iterative computations, better practical linear algebra and matrix factorizations, deriving things in statistical hypothesis testing, tightening upper and lower bounds etc. I don't care if some mathematician somewhere who isn't me "understands" it or not. It has practical use. I know that practical is dirty peasant word for many ivory tower mathematicians but that's their problem and I don't interact with them much, except when they throw a hissy fit like this and try to ban restrict matchmaking to their guild and ban the plebs from getting math from anywhere but them on their terms.
The primary goal of all basic sciences is human understanding. "Truth" is no more a goal for mathematicians than the physical laws are a goal to physicists; they simply exist in nature. The goal is rather to develop useful language and conceptual frameworks for reasoning and communicating. That understanding underpins all practical applications.
Sciences don't have goals, people have goals and they differ. Some are fans of pure math as a kind of religious or almost erotic activity in elegance and beauty, others are application minded. Some are in it for the community and outreach and conferences, some are in it to just sit in an office alone and be left alone to do it in a zen like flow state all day and night. Some treat it as a 9-5 to pay the bills with a skill they happen to be fit for but aren't especially passionate about.
I think article is more about the goal than the how. Perhaps it will be better to have ai produce human accessible explanations. No one is saying not to do what you propose. The goal remains the same.
The more I see these posts about mathematics institutions reforms and challenges from AI advancements the more it looks like they may need to go through a death. Or to put it another way they may need to start again from first principles.
If math is truly about spreading intuition and understanding then our institutions have dropped the ball decades ago and have not been able to grab hold of it since (if they ever had it to begin with)
Not sure why you're down the bottom when the current top post says pretty much the same thing. I agree that the reevaluation and refocusing that is being forced by AI is one the maths establishment could fruitfully have had a long time ago.
"A computer or monkey could easily start at the axioms of ZFC and iteratively apply deduction rules....simply conjecture all mathematical propositions in alphabetical order...The prospect of automating mathematics by enumerating all conjectures, and all proofs of ZFC, is probably not so disturbing to you."
I thought we were going to get at least some brief comment on Godel here?
Gödel effectively says ZFC must be incomplete, otherwise it would not be sound, but does that stop you from listing all mathematical propositions it can generate in some well-defined order?
That's right. You could list all those propositions and search for proofs of them. In fact this is very similar to Hilbert's very program to which Godel's First Incompleteness Thm was a response (https://en.wikipedia.org/wiki/Hilbert%27s_program).
Godel showed that there are true statements that cannot be proven, and also that among the unprovable statements from within the system is the consistency of the system itself.
As I understand, mathematicians are still trying to figure out how much this matters. One of the best examples of its mattering is may be the Continuum Hypothesis: CH is consistent with ZFC, and ~CH (not CH) is also consistent with ZFC. In other words, you have enough flexibility in constructing your ZFC world such that in some ZFC-consistent worlds CH is true, and in others CH is false.
Litt's statement is not wrong; it's just that what he wrote sounds so much like Hilbert's program, that I'm surprised we didn't get some even minor comment on what kinds of truths we could reach if we embarked on such an effort.
People always bring up the Continuum Hypothesis as though it has something to do with Gödel's incompleteness theorems, but it doesn't really. The Continuum Hypothesis being neither proven nor disproven by some particular axioms is a similar phenomenon as that the group axioms neither prove nor disprove commutativity, the ordered field axioms neither prove nor disprove the existence of a square root of 2, etc. There's no particular reason to expect any particular formal system to be complete, sans some demonstration that is.
Gödelian incompleteness is the specific kind established by Gödel's proof, where theory T can't prove Con(T) without being inconsistent. But the inability of ZFC to consistently decide the Continuum Hypothesis is established in a completely different manner, with the Continuum Hypothesis not consistently decided by ZFC + Con(ZFC) either, or any such thing.
> There's no particular reason to expect any particular formal system to be complete, sans some demonstration that is.
Well, Godel's First Incompleteness Thm says that any consistent, effectively axiomatized theory that is strong enough to represent basic arithmetic must be incomplete. So there cannot exist a demonstration of completeness for any such system. ZFC is such a system/theory.
(And yes, of course some formal theories are complete and can be demonstrated to be complete, like Presburger arithmetic).
> Gödelian incompleteness is the specific kind established by Gödel's proof, where theory T can't prove Con(T) without being inconsistent....
That's the Second Incompleteness Thm.
But CH is an example of a (an important; or believed to be important) mathematical statement that cannot be proven / refuted from within ZFC. So it is an example of a statement to which the First Incompleteness Thm applies.
You're right that Incompletness #1 does not prove CH is undecidable; (assuming ZFC is consistent) it proves that undecidable sentences must exist. CH is one of those sentences.
>...is a similar phenomenon as that the group axioms neither prove nor disprove commutativity
I haven't done a course in abstract algebra (though I have studied Incompleteness), but the brief research I just did suggests that there is a meaningful difference in ZFC and the ordinary group axioms. The latter are not sufficiently axiomatized to represent arithmetic, so Godel's thms don't apply. ZFC is sufficiently axiomatized to represent arithmetic (and much more).
The ordinary group axioms are so weak that they can describe many different structures; they also cannot represent arithmetic. So the ordinary group axioms are incomplete, but they are not Godel Incomplete.
I like the quote from Hilbert that was brought up in the article: "we must know, we will know".
With AI, it might be the case that we don't know, we won't know, but the machine does.
The central question, namely whether humans should be in the loop, will be repeated again and again in the years to come for all industries, starting with mathematics.
I am not a mathematician, but I can’t see how we are going to address the problem which we already see in coding:
Impossibility to independently validate all AI results
And in math it goes even worse. In coding code reviews are typically still the form of action you do within days. In math, historically, the lifecycle of proof is months if not years. Take as an example Millennium problems. They require at least two years of validity after publishing. Two years! In modern times with amount of output AI can produce, it feels like infinity.
We are inches close if not at the moment already when humans can’t reliable validate proofs and mathematics produced by AI. Then next research will be based on this AI-written-no-human-in-the-loop results. And we will end up in just few years in a world where novel and frontier problems will be articulated by AI and proven by AI based on AI results and humans will be incapable of understating the mere nature of the solution.
I don't think that's actually the real problem. Along with the progress in answering mathematical questions, recent progress on AI-powered autoformalisation has been astonishing. All the recent AI discoveries have been accompanied by Lean proofs.
And, yes: that doesn't absolutely guarantee correctness. The Lean kernel has had soundness bugs, and may have some still. But it's pretty strong evidence of correctness nevertheless.
The concern among mathematicians is not mainly that they doubt the correctness of any of these discoveries, but that human understanding may be devalued.
I am not that worried, but rather just observing. Humanity is about to enter the phase when we will be using things based on ideas no human ever properly understands. This thought … disturbing, somehow?
It is perfectly valid counterpoint to say that we already do it. We everyday use myriad of things, tools, and software we have 0 clue how it operates. But for us as humans it was reassuring that we know that at least there are a few other alive humans who know it, who create it and who can explain it.
>Humanity is about to enter the phase when we will be using things based on ideas no human ever properly understands. This thought … disturbing, somehow?
This is just normal though. We were building sophisticated bronze and steel tools long before any complex understanding of metallurgy or chemistry. Medicine is still the wild west.
I first wanted to say fire, even though it's a cliche, but then I thought that in antiquity we used like everything without anything that would qualify today as understanding. Also now we have a lot of stuff that we "know" it works based on complicated numerical simulation.
I think the most "understanding" we ever had was in the 40s-50s designing nuclear bombs with slide rules. It was the culture that produced the idea of psychohistory.
> Medicine is still the wild west.
Reminder that we have no idea how anesthesia works.
But one of the things AI also excels in is summarizing and can do so hierarchically. One of my favorite things to do with a concept I'm new at is "ELI5" then "explain like I'm a high school student" then "explain like a bright undergrad in XYZ" then "explain to a working professional in this domain". It's a lot of steps, but I've found it very effective (for me) to learn with -- and I've done something similar with code and math (although not math proofs -- I'm not a mathematician). But my point is that I think we can use AI to also teach us these proofs they're building in a way that I don't understand today about human proofs.
If we use AI well here we could actually understand math much better than we do now.
I actually think it's the opposite: Lean proofs and autoformalization make it very easy to announce proofs alongside proofs of the correctness of those proofs (Lean certificates). It's not an absolutely fool-proof combination (the Lean kernel could still contain bugs), but it does immediately attach a very substantial degree of credibility to the result.
And that I think is essential to why some of the world's leading mathematicians are taking this so hard. In a world where we "merely" have AI systems capable of superhuman informal reasoning, verification, correctness, and acceptance could still only be conferred or anointed by human mathematicians. But a world that combines superhuman informal reasoning with superhuman autoformalization is a fundamental shakeup in the institutional order.
The recent proof of Fermats Last Theorem is interesting: it is (iirc) 13 million lines of lean code. And type-checking takes 5 hours or so on a pretty beefy machine. I cannot independently verify the proof, and I have to take Anthropics word for it that it actually type-checks.
That seems like a red herring. Have you independently verified the human generated proof of FLT? Surely someone else will try to verify Anthropic's formalization on different hardware. Plus, it seems likely that FLT formalizations will improve / get shorter over time, requiring less compute. And computers (and type-checkers) will continue to get faster over time as well. So maybe in 5 years you could own a computer fast enough to verify a/the proof in say a week, instead of 5 hours.
It'd cost you $30-$50 on AWS for a clean Lean build or maybe $100-$150 for the full verification suite. You are, in fact, capable of independently verifying the proof yourself if you don't trust all the people who've already done so.
With formalized math, you only need to validate the problem statement (in theory, in practice agents have already managed to exploit Lean compiler bugs, but the incidence of those should decrease enough to be practically lusable for 'blind' validation of AI proofs in the foreseeable future).
> resulting in the production of an abundance of PDFs. The contents of some of those PDFs may even have important applications.
I hope, from the depths of my soul, that the static typeset report format for transmitting knowledge and understanding will finally die and be laid to rest.
As a researcher in theoretical computer science, I love PDFs more than any other format when it comes to mathematics.
There simply is no contender to LaTeX and PDFs.
Lucky for you, almost all research in math, cs, and physics, are put on arxiv, where you can download the source code (.tex) as well as get an HTML render.
My worry is that the frontier of math is too far away for most humans to reach. AI is only going to make that worse. If today it takes twenty years of math study to reach the frontier (in a narrow field), what's it going to be like when it takes forty years? Or four-hundred years? Will the fields just get narrower and narrower to accommodate finite human intelligence?
> If today it takes twenty years of math study to reach the frontier ...
Are you counting from when a kid first learn 1 + 1 = 2? That seems a misleading metric. The rate at which kids are learning math in the first half to 2/3 of their schooling is very low. Or are you asserting that Ph.D.'s don't reach the frontier until age 40+?
Actually, the frontier of math is too far away for most humans to reach because >99% of humans don't have the drive and talent to study and learn enough advanced math. Similar barriers exclude >99% of people from Olympic-level athletics, or being famed violinists, or being important architects, or being US Senators, or being successful ancient historians, or many other things.
> I think so. This machine might produce answers we value, but it would not, in itself, produce human understanding of those answers.
It's nice that the author is optimistic, but won't the AI be best placed to dumb down its increasingly complex proofs into a language us lowly humans can understand? To keep thinking until it can refactor complex proofs into ones from 'the book'?
As they go on to explain, a human understandable proof is different than a human actually understanding the proof. That actual human understanding (like, in a brain of a human) is one of their stated goals.
Producing human understandable proofs is possibly a job best for humans today, but the author appears to agree with you that this is probably fleeting (and argues that even if you disagree, it should probably be treated as if it is fleeting when planning for the future):
> Right now AI systems arguably underperform us at theory-building, asking questions, exposition, … so we could prioritize and reward those skills. I think this is unwise: compare the speed at which the academy adapts to the speed at which model capabilities improve. We need to consider the endgame. If the models remain incapable in some domain, we can adjust later.
AI will never be able to dumb down a proof to a level simple enough for someone to understand who has never studied math and put a lot of effort into it. Some concepts just need time and effort to absorb no matter how relatively simply they are phrased.
Sure, but some proofs are so complex that even experts can't follow them. As AI progresses, this may constitute an increasing fraction of proofs. I meant that AI could work to find the simplest possible proof.
I'm heartened to see this kind of thoughtful, reasoned discussion around long-needed reforms in the educational community, with AI as the catalyst. We certainly have interesting times ahead.
> I propose the following reconceptualization of the goal of a mathematics PhD: to become a world expert on some interesting, deep topic, and to be able to convey that interest and understanding to others. Part of operationalizing this might be a thesis, but the degree would be awarded primarily on the basis of a rigorous defense, in which the student explains the topic to their examiners until they are satisfied.
I think this is a refreshingly forward looking idea and I agree with it 100%, especially the the "rigorous defense" part. That is a good measure of how well the topic has been researched and understood by the researcher. This is where the humans can be "in the loop".
> How different would this look from current PhDs? I think students would still meet with an advisor, who might suggest a topic. That topic could be explored with AI assistance, or not, but the student would be responsible for understanding it; it might be much more open-ended and larger than the typical PhD is currently.
Interesting point about "more open-ended" and "...larger than the typical PhD". I think the author has a point. Earlier, the bottleneck was the candidate's/researcher's understanding and knowledge. Now with AI tools, it is so much easier to zero in to relevant knowledge, get your questions answered quickly which might lead to understanding more quickly.
For e.g., before the advent of public libraries and printing press, the knowledge was inaccessible and guarded. So that was the bottleneck.
Then books became ubiquitous and the bottleneck to knowledge and understanding was people's motivation AND knowledge of WHAT books and topics to research.
Then came the internet and free PDFs of books and research articles. Now, the bottleneck was still people's motivation and a mild version of what books and topics to research. I say "mild" because one can lookup articles and newsletters, and book reviews and come up with a list of reading.
Now comes AI and it looks like the only bottleneck is people's motivation.
I believe there was also a silent, yet potent, bottleneck all along which is also removed by AI: personal tutor/coach/teacher/professor etc. Let's say if I am reading a textbook on manifolds or some research paper and I have a question about a specific theorem or even a mathematical operator being used. Before AI my only way to get my questions answered was to read more books (PDFs or print), or ask on math exchange or math overflow and wait for someone to answer, or to ask a professor. This could take up to a week.
Now all of that has been cut down to 1 hour or less with an interactive chatting session.
!!!!!
So....the only bottleneck is people's motivation! QED
I'm not sure why an advanced degree is necessary for that at all, besides the pride of a vanity title. It's pretty much what Bill Nye does for science.
The idea that most any modern "interesting" aspect of mathematics is going to be understood (or often even explained in enough detail to reveal what is interesting) in an hour is pretty rare. Either the student's aptitude, the tutorial, or the mathematics are unique. There is a reason that these are PhDs and not undergraduate HW sets.
I think we often delude ourselves as to how well we understand problems and their solutions. Some instructors even make you feel that you understand better than you do by pointing to a few approximations or simple solution spaces that obscure the larger complexity. Just looking in wonder at the many categories of three-body solutions (currently on hnews) is enough to remind me of this.
I am not implying that this is a new idea and I don't believe the author did either. If you read the full article, the context is clear.
The author is re-emphasizing the importance of thesis defense old school style and highlighting that being familiar with one's own material rigorously should still be a requirement.
Everything that leads up to the thesis defense perhaps changes signficantly.
Another bottleneck is money required to pay corporations who own the technology to "do mathematics". In that future, there will never be another Ramajunan.
Yeah I think it was in Terence Tao's recent post where he pointed out that the "low-hanging fruits of mathematics" are being harvested by the AI companies, when traditionally those problems were available for aspiring mathematicians to prove themselves in the community. Maybe there's a parallel in the software world too.
There absolutely is a parallel in the software world. I have 16+ years of experience in software and I have enough coding under my belt to guide and inform coding agents to solve the problems.
This is the reason why hiring is down for fresh graduates. We don't have low hanging fruits types of problems to give them anymore.
So until the industry figures out the workflow, the hiring for fresh graduates is going to be tricky.
What is the incentive for a person to sit though seminars and evaluations? People already hate redundant meetings. What is the incentive to change the system from the existing one to one that rewards this verification somehow? Who benefits from this change?
One of the things that struck me was that most people have an area of their workflow that they would be happy to hand off to an AI, so that they can focus on the thing they love. However, that area is different for each person. One person's grind-work is another person's love-work.
This is why it's so hard to build consensus around where the "red lines" are in this space.
I’m not a mathematician, but if I were I would imagine the main attraction would be the act of thinking hard and solving problems. Maths felt like the ultimate profession for someone who loved puzzle solving. If you delegate the thinking, any percent of it to an LLM, I don’t think it’s maths anymore. Might as well join any other job and earn better money now. A completely ludicrous proposition, that maths will no longer involve mathematical problem solving.
Modern incentives, unfortunately, reward producing solutions rather than going after the joy of solving problems. And it's the latter that inspires others, not the former.
1. It will take much longer to understand the output of the machine that it takes to prompt and create it.
2. The only? best? one? way to /verify/ that you /in fact/ understand the output of the machine is to explain it to someone else.
So there will be a machine generating koans which need to be meditated upon and discussed with human social back-pressure validating understanding. I think this could be much more cooperative and at a minimum this will be a way different math social construct.
As with the rest of the domains, AI/LLMs will do syntax and search better than any human.
In code, any developer whose differentiaton was clean code and knowledge of different technologies is now average.
In math, any mathematitian whose differentiation was to manipulate formal systems and know tricks of different domains will be average.
Fortunately, humans do more than syntax and search.
The bad news for developers is that if you know what the output of your program should be (which happens most of the time), almost all of the job is syntax and search to build the code that reproduces the output.
The good news for mathematitians is that for the majority of problems you never know the output, or you just know the output is either "True" or "False". There are some cases where you need something else, for example "a solution that blows up in finite time". For those cases AI will outperform you easily (see new Navier-Stokes solution)
So if as a mathematitian you were doing more than syntax and search, then keep doing that and use AI just for what its best.
While I broadly agree with the premise of re-directing the “purpose” of math, I quite detest the idea that judgement might be primarily based upon some in person discussion, or
oral presentation, and the claim that written mathematics that is not orally communicated might be less worthwhile in some sense (i know this isn’t the exact statement of the authors proposition).
There are a good deal of people, whom, falter much more in oral discussions, whether this be for a psychological thing, stage fright, or difficulty explaining things on the spot. There are also certainly brilliant people, who can’t give an informative, discussion inviting talk to save their lives, but given enough time, can formalize their thoughts in writing at the highest levels of their field, and that writing is likewise very enlightening (sometimes).
It’s not clear to me, that, AI as is, could not pose successfully in an oral discussion of a topic. I mention this because it seems that one implication of the article is that AI might write things that are logically correct, but devoid of understanding. I suggest rather that 1) it is not extremely improbably that AI is incapable of generating mathematics that furthers human understanding and if 2) it is indeed highly likely that they cannot generate mathematics that furthers human understanding in a textual format, then surely one could also differentiate between human and AI on a textual level, and judge the contribution of a human, without the need of oral discussion?
I suppose another aside is, one might claim that the existence of AI means people have much much more text to filter for, and so, it becomes difficult to find one person’s good writing amidst a sea of, logically correct, yet understanding devoid textual content. But by and large much or mathematical academia certainly operates off of some reputation/vouching system presently anyways, that already serves as a “filter” in some sense. Perhaps the existence of such a system/culture is not a good thing, but oral discussions/seminars certainly aren’t immune from such predilections.
Perhaps I’m babbling like an idiot, but the entire and sole purpose of this comment is just to say: for the love of god please don’t let the standard be judged by oral presentation
Imagine, eventually, a large ever growing library full of unread books. One could roam around forever and discover new things.
Might be possible to generate a 3d world using some taxonomy estimate the amount of titles and generate the books as the area is explored.
Like the manuscripts in Sakya Monastery in Tibet and like the millions(?) of cuneiform clay tablets. Vast amounts of data that may contain information.
The key shift seems to be from "can you produce mathematics?" to "do you understand mathematics?"... Those used to be correlated strongly enough that a thesis or paper could serve as evidence of both. If AI breaks that correlation, evaluating people through defenses, discussion and live problem solving suddenly makes a lot more sense.
I think this goes in the right direction. You have to rethink the role of human work in math, can't put your head in the sand and cling to your comfy institutions and system just because you got to know it's ins and outs and just want it to be like that forever.
But the bigger picture is: while I understand the author know his field best and wants to keep the post focused, the same issue will hit many more fields. We need to also have a broader discussion that involves more fields of knowledge work, largely academic scholarship but also regular office work, then it will come to engineering design, medicine, it's already coming for 3d modeling and vfx, software dev, it will come for a lot of middleman services. Not at the same rate, but we have to understand that it's not just that math will change.
Change will be the default. Everything will change. It will be much easier to make math change because all things will change. You shouldn't worry and imagine that funding criteria will be like today or that journals or academia or politicians expectations will be like today. No, everything will adjust with some timing differences of course but it won't be a static world and then math changing and having to justify and fight to explain the change to other actors who are baffled. They won't be baffled they will themselves have to change.
The world will transform as much as it did when society moved from feudal agrarian to urban capitalist industrial, or from the vast majority doing physical labor to a service economy with a huge amount of desk jobs. I can't tell how it will change exactly but it will be bigger than what we have seen in the last couple of generations or maybe more.
Another mathematician waxing poetically about a future that will not come to happen. Call me a pessimist if you'd like, but capital has no incentive to ensure mathematicians maintain their current status in society.
If you're a mathematician you are in the same boat as the software engineer, and the Dodo.
Better plan is to shut down PhD and make students take a oral thesis at bachelor and master level and help them become a productive economic participant as soon as possible.
The author argues for evaluating Ph.D. candidates based more on the oral thesis defense than on the actual thesis.
By essentially the same reasoning, I’ve been arguing for prioritizing in-person design/code reviews over code-only async PR comments.
The important thing is to verify that the human has a coherent design in mind and can demonstrate that it got implemented, regardless of who or what was at the keyboard. “I dunno, I guess Claude thought this was a good idea” is not a coherent design.
An oral defense is ultimately a meeting, with the same weaknesses as other types of meetings. In particular, if new information comes up in a meeting, you cannot reasonably expect to get useful responses from the other participants. First impressions maybe, and plausible-sounding bullshit from those prone to generating that. If you want anything more, you need to provide the information in advance or schedule another meeting later.
Oral defenses in academia are largely rituals. If a student fails their defense, it's almost always the supervisor's fault. The supervisor is supposed to be the primary quality control. With their regular meetings with the student over the years, they should be able to tell whether the student has achieved sufficient understanding and contributed enough to graduate. If the supervisor thinks the student is ready to defend, the reasonable expectation is that the student will pass and graduate. The defense is mostly there to let the other examiners validate the supervisor's judgment.
Here's another example of a similar but slightly different failure mode of the live defense: The department where I got my degree had a practice around PhD defense where there would be a prof each representing the four research pillars + the dean. It was well known that certain professors disliked each other enough that they would snipe each other's candidates. You'd just hope not to work for one whose enemies got picked for the panel. If you were a superstar, you'd pass, but if you were an okay candidate or someone strong but with stage fright, you be could toast even if you did good research.
The defense was only briefly about the actual thesis, then switched over to whatever research interest the committee members had, and they'd drill into their pet subjects. This was in physics, hardly politics forward normally, and at a very well respected university.
You are missing the entire point of the oral defense while almost stating it. The existence of the oral defense is what extrinsically motivates the supervisor to do their job well. If the student fails, the supervisor loses faces in front of their peers, which obviously they try to avoid. Removing the oral defense would massively reduce the quality of PhDs because supervisors will not do their job.
Also in my observational experience in physics, a fair number of students (probably like 10%) get some sort of major corrections to their thesis. After writing this, I did a search, and in the UK, the number across all fields is 16% [1].
[1] https://www.lexacademic.com/blog/how-common-is-passing-with-...
And even then, the other examiner will only fail the candidate under egregiously extreme conditions. Rather, if the supervisor wants to pass someone who is unfit but not egregiously, this will just erode the supervisor's reputation among peers. Because there is a grapevine of course. So it can't be regular. But academics also really care about face and embarrassment, so they won't make a scene, u less there is preexisting drama around the supervisor. The only fails during the oral I've heard of happened when the candidate insisted to defend against their advisor's wish.
I think this is true of defenses as they exist today, but that may be partly because the thesis currently does most of the signaling.
I can see how this trend is going to go.
Manager: “Why isn’t feature X available?”
Person B: “key pieces are delayed due to the developer not understanding all of the LLM doesn’t and implementation.”
Manager: “does it work? What are the risks?”
Person B: “well yes it works for now but we’re accumulating tech debt due to a lack of understanding and potential flaws that haven’t been thought out yet”
Manager: “they want feature X, ship it, we can deal with it later, I don’t care if it’s not coherent as long as it works.”
How many decades at this point has these been a push for functionality over everything at all costs? And you have a mechanical snow plow now. Most businesses don’t care about later risk or any future planning beyond the quarter horizon, they’re not concerned about how it will effect their performance in 3 quarters or lead to instability or issues, those are future problems for a future person and we’re here for money now.
The simulated dialog you have provided is very much representative of what many of us have heard first-hand. However, I disagree with:
Businesses care about "later risk" and what it implies.Individuals within an organization do not unless it will specifically affect their bonuses/promotions.
It is also true developers can have a myopic view of what should be valued and that can be incongruent with the business.
You can make a perfectly rational business choice to take on tech debt to get a feature sooner.
IMO this is exactly how software business operates in practice.
Immaculate codebases we can effortlessly extended are a fairy tale of consultants. The world runs on tech debt that doesn’t matter enough to fix.
Another way to say this is that this has worked fine for decades— ie, software businesses have done just fine prioritizing features over code cathedrals. What I find more surprising is that many engineers still can’t seem to accept this basic fact, after decades.
I suspect AI will actually change this. The opportunity presented to business in dramatically relaxing both our understanding of our codebases and our tastes and preferences is big enough to justify letting go of folks who don’t get it.
I think upweighting the live components of academia is inevitable in the age of automatically produced writing, but I also find it depressing that people think so little of writing that they imagine it obsolete because of AI.
AI writing is aggressively, aggressively mediocre. It is only good for how cheap it is. If you're indifferent to how much better good human writing is than AI writing, you should not be the one to evaluate human writing.
The tricky part is that live review is much more expensive than async review. But maybe that just means we were getting away with a cheap proxy because it happened to work reasonably well.
Unfortunately, from my experience, forcing people to talk about their research to an audience mostly just condemns introverts and people with social anxiety. Being able to formulate your thoughts clearly in writing and in speech are two radically different skills. I've seen world class scientists give the most horrible conference presentations to the point where everyone in the audience agreed they'd rather have just read the paper when it came out, because they had no clue what he said while the author is amazing at writing. If we want to switch academia and to purely oral skills, you'll probably need to replace a ton of people. And in the industry these fields are already dominated by fast talking bullshitters.
I do this already if there’s any kind of issue. Just talk the person if they’re next to you or zoom them if not (don’t set up a meeting). A five minute conversation can save hours of back and forth.
>“I dunno, I guess Claude thought this was a good idea” is not a coherent design.
I am sorry, but the writing on the wall is that this is where software industry wants to go. As a software developer, I cannot but help notice how unrealistic this is.
We are entering the next level of bloated, inefficient and unreliable software....
Excellent optimistic post in a sea of negativity, and with actual suggestions, too. After reading, my mental image is this: think of Olympiads in Ancient Greece.
* A weightlifter was only awarded a laureate if he were able to lift a heavy stone (have no idea what they were lifting, for illustrative purposes only :-)
* Along comes Archimedes who invents what we would call an exoskeleton. Now any regular guy can lift twice as much as last year’s athlete.
* What to do? You can cancel the Olympiads, but they are actually useful as training, motivation, etc So now you have to give the prize on other factors, eg how well he can lift, has he opened a gym in the city, etc
BTW, physics and bio are not exempt, so those researchers better read and try to stay ahead.
I don´t think so. For programming agents can run code, check compiler output, etc. For mathematics, it is almost the same once you factor in the usage of lean.
For the reality, you can´t close the loop that fast, or with that precision. You will have to slow down by several orders of magnitude.
Doesn’t have to. There are petabytes of experimental physics data that can be fed to AI to extract additional insights. The only holdup is this is slightly harder to than math. With bio, you’re right, generally designing and conducting an experiment goes hand in hand and theoretical biologist is not a common label.
>There are petabytes of experimental physics data..
There is one thing you are missing. Math is precise. Physical measurements are arbitrary imprecise...
> We already interview faculty hires; we must now do the same for graduate admissions
They hire PhD students without hearing them give a talk and then doing interviews? In Germany, the applicant gives a talk (30-40 min) to the research group they want to join, usually presenting their master's thesis, engage in discussion, often share lunch with the group, then do 1 on 1s with individual members of the group and a longer one with the PI. Obviously this can vary within Germany too, but I couldn't imagine hiring someone without something like this.
1. US a lot of international PhD applicants, so traveling before even being accepted is difficult, and 2. lots of people don't do a masters.
when I applied to PhD programs (not in math) it was basically CV + personal statement + recommendation letters + short chats with interested faculty :shrug: Maybe it was because my CV was "strong" but the chats were more see if interests were aligned, rather than actually interviewing me.
Remote talks are also a common alternative in such cases. Zoom etc.
I also know that another major difference is that American universities tend to hire without a professors involvement, into a generic "program", then the PhD student seeks an advisor after being accepted, so applicants have to woo some unconnected committee pursuing various goals misaligned from the PIs instead of convincing the PI. In much of Europe a professor basically "owns" a chair and really is boss and decides hiring pretty much alone.
Anyways, if they don't have a masters' yet, they could present their bachelor thesis. But I already think it's a bad model to combine the masters (courses) and the phd (research) into this American hybrid that's the direct-bachelor-to-PhD jump, but that is somewhat unrelated.
I agree the combined masters + phd is weird tbh. I think almost everyone treats it like "just focus on research and spend as little time on course work as possible." I would prefer it if courses were more flexible.
I'm not a big fan of the US application setup. IDK how it is in Europe, but in the US, it feels like there's a lot of not-very-meritocratic "secret" stuff you need to know to up your chances.
> I'm not a big fan of the US application setup. IDK how it is in Europe, but in the US, it feels like there's a lot of not-very-meritocratic "secret" stuff you need to know to up your chances.
In Germany (countries differ quite a bit on these details) a significant chunk (though not necessarily majority) is directly hiring a master's student who worked at the same chair on a part-time student research assistant job and/or a master's thesis and did a good job and typically turned the thesis into a paper submission. In other cases it's still somewhat inside baseball like getting recommendations from other scientists that the PI knows, or coming from a university that the professor knows well. It can also be from an impressive email, from a chat at a conference poster, etc. There is much less of this global casting that committees do, like "we need this and this type of character now" to have a vibrant student body composition, since each PI makes a decision just about their lab, without having a global view, and their goals are basically to get research out of that person, papers, results useful for funding applications etc. So they want to see aptitude in that. Through talk quality, communication efficiency when talking with existing team member one on one, recommendations, sometimes even phone calls with the PI with whom the student worked (these are small worlds). I think it's more organic and natural, though still prone to exclusionary gatekeeping, than the HR-style narrative essay-and-extracurriculars-based admissions though maybe that's more prominent for undergrad admissions even in the US.
Unlike Europe, in the US, most PhD applicants do not have a masters. 10 years ago, most had not even conducted substantial senior projects (e.g a semester or year). Though increasingly senior projects / undergraduate "theses" have become more common.
In my opinion, that's not a good time to really know you want to pursue academic research. A masters gives you a little taste at least and allows to show aptitude. Doing a student research assistant job, working on a thesis, maybe even a paper submission, diving deep into specialized courses. Then you can know if it's for you as well as the PI can know if it's for you. Straight from a bachelor and them interleaving the phd with coursework is a bad model I think, but of course I'm from Europe so it's what I'm used to.
In the US, usually you do a summer research program if you want to be a PhD student, so you can get a taste of research.
At most universities as well, if you decide the PhD isn't for you, there is always the option of leaving with a Master's. Because you get free tuition and a stipend, you also come out ahead of if you had gone straight for the MS (which you generally would pay for).
In the UK I had a panel interview with 4 people, in person. But for international students who often brought their own funding sponsored by their government that didn’t happen.
That's an interesting wrinkle. If anything, externally funded international students seem like exactly the case where you'd still want the interview, since the funding answers "who pays for this?" but not "does this person have the background and understanding to do the work?"
[dead]
As someone who has a degree in math, I still can't help but think mathematicians are getting a little bit of a comeuppance. In a lot of areas of mathematics there had been little effort to make the work understandable and leaves numerous folks who could benefit from the knowledge on the outside looking in. Now AI comes along and do the same to mathematicians. Makes me chuckle a little bit.
Kind of makes me think of Arnold's, "On Teaching Mathematics":
https://www.maths.tcd.ie/pub/Maths/Courseware/ProblemSolving...
What about the next 3,900 words after the first sentence? (yes, it is that V.I. Arnold https://en.wikipedia.org/wiki/Vladimir_Arnold )
Never been my experience of maths, which is one of the most accessible disciplines to have ever existed.
I also have a degree, but mostly taught myself the important undergrad-level concepts as a kid. I just went to the library and borrowed any of the hundreds of books written to clearly communicate maths to beginners or downloaded any of the free ebooks / lecture notes.
Name a single field of endeavor that is more open in 2026. Software certainly isn't one of them - the best stuff has always been gatekept.
I too have a degree in math and it's easy to see some of what's happening as too much of an emphasis on intellectual chest thumping and to little emphasis on showing why understanding the concepts is useful for everyone. I will say the problem may well increased here by the American education system which denigrates such understanding and isn't influenced by mathematicians.
But also, with things rapidly changing, perhaps in ten years chat programs will not only do superhuman but make their proofs marvelously accessible and provide incredible tutoring sufficient to bring any curious up to a super high level quickly. Then what can you say and what can you complain of.
But I think "do everything machines" are necessarily inevitable but the situation does make it uncertain where the limits are.
I think you're a little naive if you think this is because of gate-keeping by mathematicians rather than the essential complexity of mathematics. Mathematicians individually and as a group would love nothing more than a world which is capable of understanding their work more deeply.
In the exact same way that LLMs allow anyone to vibe code an app but do not replace real understanding of system design due to its essential complexity, non-mathematicians will quickly learn that asking an LLM to pump out advanced mathematical statements to you, even if they are correct (and even if you could verify them) does not constitute understanding, and that the human brain is the bottleneck either way.
It is only if the LLM is super-human at simplification and explaining that a difference will be noted. This would be excellent for mathematics but its not a foregone conclusion (and the argument of most mathematicians, such as Terence Tao, is that this distillation process is one of the key parts of doing mathematics, and that LLMs so far seem to be going in the opposite direction. I suspect its probably user error and leveraging the tools better will produce different outcomes, but mathematicians are only just starting the journey that software developers have been going through, so patience is needed).
Folks keep using the word "gate keeping" -- I never used that term nor implied intentional gate keeping. It's simply not caring if those outside the club understand -- there is no one guarding the door its just folks don't care if anyone finds it.
Gatekeeping is a human universal. Even those who preach maximum inclusion do exclude plenty, of course there is always explanation why that doesn't count. If you try to run any community you quickly learn the importance of gatekeeping. Eternal September etc. See any site or fandom that gets mainstream etc. I don't think anyone has any duty to actively pull in as many as possible new people. Zen masters used to chase away students or make them sit on their doorstep for days and send them away anyway. Now, the west is essentially Christian and so the missionary impulse is real, but it doesn't work without the other stuff in the package.
Again with the gatekeeping. An AI generating a convoluted proof for some conjecture is not gatekeeping. It just does not care if you understand. It only cares enough that it can be verified -- usually using some other opaque proof tool like Lean.
AI is much less gatekeeping and much more welcoming for beginners than human experts are in general. It has infinite patience to break things down for your level of understanding. So AI has less gatekeeping which I like when I'm gatekept out of something otherwise. But I recognize the social function of gatekeeping and I don't see it as an inherent evil.
Not everyone needs to be a communicator.
Why should we expect everyone to be both a great researcher and communicator? The obvious result is that it is as effective as engineering managers. Sure, there's some amazing ones, but most aren't. Though that also doesn't mean no expertise in the field (i.e. non-engineering manager) is any better. It is just that managing/communicating is a different and orthogonal skill.
What needs to happen is we need to make it okay for people to specialize in more things. More nuance to this rather than trying to throw everyone into nice easy to manage buckets. Those buckets are just unrealistic abstractions filled with hope, denial, and laziness. Reality is surprisingly complex. Math can do a really good job helping you understand that, but it's a sufficient condition, not a necessary one
Lowering the bar for the masses isn't always a good thing
It is not lowering the bar to find better ways to demystify and explain things. In fact, I would say those who can explain it well understand it the best. Richard Feynman would be my best example.
Do you have any examples of places you felt like there was a lot of gatekeeping? Perhaps having studied mathematics I am a bit blind to the issue here and would like to learn more.
I agree making simple things sound complicated to appear more impressive is bad but there are limits. Even with Feynman he could only go so far, e.g. his interview about why questions and magnetism.
I am not thinking of intentional gatekeeping, but more of the kind where mathematicians build themselves their own island of concepts and notation with no thought of building a passageway for others to engage and make use of the theory. I would find modern algebraic geometry highly useful as someone who works in computer graphics and computer vision, but much of the theory is akin to learning a new language and I don't get the sense that those who publish their work in this field care if I enter their world.
Ah that's fair.
I do think in a lot of fields there is a lot of impact/influence to be had by people who are willing to do work to bridge different fields. I wonder to what degree this is because practitioners don't always see how their work could be used elsewhere.
I would say that it is lowering the bar to "find better ways to demystify and explain things." But there is no harm to this bar falling, because this is a bar for entry. There is a separate bar for making research discoveries (being credited for them, specifically), and that one should not be lowered, because it is a bar of standards.
Examples, preferably academic?
Half of modern academia, for one. Universities have turned into degree mills with the expectation that >50% of the population requires a college degree, regardless of whether they actually have any interest or need for one beyond doing it because it's a prerequisite for a good career, independent of whether said career actually uses the knowledge in any way.
Not that I actually agree that math was at the right level of gatekeeping. It definitely feels intentionally opaque beyond reason, and I think it's why LLMs are able to cut through the obfuscation and solve problems that maybe wouldn't actually have been considered quite so hard if mathematicians did a better job of making their work accessible.
<pedant>28% of adult Americans</pedant>
66% in Japan. Who said anything about America?
You know, where most of the universities are.
And, who said anything about Japan? Confused.
We have seen the thing in programming. I'm not a gatekeeper, I love seeing more people to code. On the other hand, along the way, we lost the joy of the journey and only fixated at the destination.
Result is more software at lower quality. The reason is statistics. When you increase the population, you increase the population of every kind of programmer, and people who want the result are favored in most competitive sectors because corporations want something somewhat working yesterday.
...and here we are.
Now programmers talking about code quality is stoned en-masse. If it's somewhat working then it's good. Efficiency, resiliency, maintainability and sustainability is an afterthought. Some of my friends who loved debating programming language theory now don't even care about the code. They don't write it, just vibe, and they don't plan to come back to "older, caveman style of development".
Some universities are also adding fuel to the fire: They "prepare students for the job", not teaching the science, but the parts that corporations need for the job only.
Though, hardware was cheap and people were expensive, and now code is cheap and hardware is expensive now. We'll see.
this comparison makes little sense.
> we lost the joy of the journey and only fixated at the destination.
that's because you can make money with code and so it went the same way as everything else that's capitalized/commodified.
here a better comparison would be art which is generally still pursued on its own merits/pleasures.
The point I was trying to make was not "enjoying the process for the sake of it", but paying attention and spending effort on the journey created better software at the end.
When you look at older software, most of it was higher quality than the things we have today. A web site contained more information in a more readable way, more features in a smaller footprint. Same for native applications.
Now we slap what we found online together and calling it done. Everything is sparse, takes ages to load, centuries to submit and everything is so disconnected and async that some simple features are straight out impossible.
This is what fixating on the destination brought us. I dare you to download something you purchased 3 months ago via a 4096 character S3 link they have sent you, and double dare you to ask customer support for a new link. I'll bet that with a 80% chance they have no way to verify your serial number, even.
Have people completely forgotten how to read? I'm saying your analogy between software and math is completely, irreconcilably flawed. I don't need to read the rest of your screed because this is a thread about math not software.
I see.
Somebody said that lowering the bar is not good. Somebody else asked for examples, and I provided an example. So, if you want to be pedantic, that doesn't track well, because what I answered was not about mathematics.
If you want something about mathematics, computation is mathematics, as software is. So, my example tracks the same way in mathematics.
Finding solutions without understanding its parts or the path is equally detrimental to mathematics as it is detrimental to software.
Maybe you need to read a bit slower and think along the way. Using AI too much blunts critical thinking skills in some, as I read.
What was the end of the proof thing you mathematicians use, was it "Q.E.D."?
Q.E.D.
> computation is mathematics, as software is. So, my example tracks the same way in mathematics.
brother i already addressed this literally in my first response to you: the reason software went to shit is because it became commodified (ie a thing produced in a factory) not because it's computational nor because the bar got lowered. seriously read my whole original reply again and see whether you really disagree before just re-asserting your rant on SWE.
> Finding solutions without understanding its parts or the path
you didn't read the post at all did you? it's emphatically about deep understanding over blind results.
> Using AI too much blunts critical thinking skills in some, as I read.
irony
>here a better comparison would be art which is generally still pursued on its own merits/pleasures.
What about the crafts? Carpentry, blacksmithing, stonesmithing, etc. are all commodified yet people still pursue these for pleasure.
>this comparison makes little sense.
Stop gatekeeping what people find personally fulfilling.
> Carpentry, blacksmithing, stonesmithing
they're literaly not - do you know what the word artisanal means?
> Stop gatekeeping what people find personally fulfilling
wut? i can't even parse this in relation to what i wrote (which is that people who enjoy math will continue to be able to).
>do you know what the word artisanal means
https://www.merriam-webster.com/dictionary/artisan
Do you?
>they're literaly not
Are you arguing that all the metal we have in our cars, buildings, airplanes is still being forged by hand? That that the furniture people are buying off Amazon is handmade? That stonework isn't primarily done these days with concrete pours?
>(which is that people who enjoy math will continue to be able to).
You posited that code is not like art, because it's primarily written for money and not for pleasure. But that's a stupid fucking argument, because there are plenty of crafts out there that are both done for money and pleasure (often by the same people!).
I would say the research university is getting comeuppance.
Pedagogy is the primary purpose of educational institutions. That includes universities. However, when research is placed first, you are often left with mediocre teachers, because "who cares?" They were hired to do research and lend the university that kind of prestige; teaching is an afterthought squeezed into the spaces that remain.
And frankly, we haven't the faintest clue today what education is even for. That doesn't bode well.
Add to this the other troubles facing the university, like declining attendance, skyrocketing costs, and AI cheating and we're in for a very interesting transition indeed.
I love how this is (without slighting the problems entailed) coming at it from a perspective of infinitude and abundance (we will always have more problems to solve) - which is the correct framing, particularly when dealing with ideas, or fields in the which ideas are the driver/product/output/material, and ideas themselves, the field itself, are infinite.-
PS. The validation problem, being one.-
I admit I just don't understand this attitude.
People had the same complaints that the code produced by early coding models was messy, lazy, poorly commented, had terrible architecture and so forth. The central complaint was that it was just too difficult for humans to review. The answer is just to improve the models and move on.
Similarly now we're getting AI doing math. The proofs compile but are a mess. So just make the models better at writing clean proofs and explaining what they're doing to humans. That's the end of it.
Rather than just go on and on about how it's the end of the world if we don't do this, why don't we just do it?
> Similarly now we're getting AI doing math. The proofs compile but are a mess. So just make the models better at writing clean proofs and explaining what they're doing to humans. That's the end of it.
The assumption here is that the true/false of the theorem is the important outcome. While it is certainly part of it, a big part of maths is the understanding you gain from a proof. Many of the best proofs elegantly explain some aspect of the maths which was previously unclear and expand our understanding of the world.
To use a programming related example, imagine that an LLM spits out a solution to the travelling salesman problem which works in O(n) time. On the one hand that's very convenient for whatever problem you happen to be trying to solve at the time...but there's also an answer to P=NP in there! The former means your delivery drivers app works a bit faster on their busy days, the latter fundamentally shifts how humanity thinks about certain problems.
Going back to the maths, there have been theorems that were proved (by people) where the proof is broadly seen as 'unsatisfactory' in that it doesn't really expand our understanding. I assume some of these LLM proofs are a bit like that: we now know that the thing is true, but we really want to know why it's true, and how that changes our understanding.
> > models better at writing clean proofs and explaining what they're doing to humans. That's the end of it.
> The assumption here is that the true/false of the theorem is the important outcome
You're replying to a comment about "clean proofs" and explaining to humans. You talk about true/false anyway. Re-read please.
Are you always an ass? He had a great comment, and you respond with this low effort hostile bullshit.
No, I see this argument repeated all the time in these threads. "It will just give true/false, while humans need intuitive explanations". Why will the AI only give true/false? Where did this assumption come from? The nested citation was about AI that can give humans an explanation. The reply was "but true/false is not enough". It is as if it was a reply to another comment. How is that a great comment?
I think it's pretty analogous to coding, honestly. When it comes to gaining true understanding of a piece of software, I still find today's models almost as useless as they were a year ago.
They can write great code, they can gain understanding of something for themselves, but they still kinda suck at explaining it to people. As the article says, if you want understanding, it seems like there is no replacement for getting in the weeds yourself. A model can help, but it's not going to magically download knowledge into your brain.
One funny thing is that in order to tune the models to make what they're doing explainable to humans, you need to have humans involved in the RL pipeline to indicate which explanations are good.
You can understand this as learning a mapping between the model's internal "world" (i.e., 'meaning,' which is hopefully coherent and consistent -- but definitely not always! see, e.g., https://arxiv.org/html/2505.11581v1) and language (i.e. 'form') that reflects that world.
For this to work, you need both coherent / consistent internal model worlds, and also good mappings onto human language. Supervision by mathematicians has provided the signal for both internal coherence (though this can also come from interacting with a proof oracle) and for good explanations. If models exceed human capacities, you could imagine that aligning their explanations potentially becomes harder (though not necessarily). Also, humans naturally have to do the same thing: as researchers we must find analogies to make our work legible to collaborators or laypeople. Often in doing this, we further clarify our own understanding!
More deeply I think the "end of the world" vibe arises not only from the practical need to have models that explain, but also Litt's (and many other fields' researchers) grappling with being relegating to not mattering.
There's a fundamental difference between the goal of code and math. The goal of code is to produce software that does something useful. As long as the code does what it's supposed to do, arguably, it's good to ship. (As you imply, we want the code to be good enough to also be reasonably certain there are not too many bugs, that it is maintainable and can be extended etc. This is what early models failed at but now seems broadly fine.)
But for math: what is the point of a proof if no one will read it and no one uses its result? To quote the article, "AI systems will [...] result in the production of an abundance of PDFs. The contents of some of those PDFs may even have important applications." But if there's no one reading the PDFs, what's the point - no matter how good your AI model.
The point of math is understanding. So mathematicians should feel free to use AI as much as you want, but in the end, they should've gained some understanding on what happened.
If the point of math is understanding then maybe our incentive structure is wrong. Maybe it should be teaching the concepts to as many people as possible rather than just continuing to write papers that 10 people in the world understand, which is the current state of a lot of math.
I won't understand it, but can benefit from it in better data structures with proven invariants, faster algorithms, convergence guarantees for some iterative computations, better practical linear algebra and matrix factorizations, deriving things in statistical hypothesis testing, tightening upper and lower bounds etc. I don't care if some mathematician somewhere who isn't me "understands" it or not. It has practical use. I know that practical is dirty peasant word for many ivory tower mathematicians but that's their problem and I don't interact with them much, except when they throw a hissy fit like this and try to ban restrict matchmaking to their guild and ban the plebs from getting math from anywhere but them on their terms.
I keep seeing people repeat that the goal of math is "understanding".
I do think that is one goal of math but I don't think it's the only one.
I think an additional goal is simply "truth", which can be found without understanding as we've seen with these human-incomprehensible proofs.
Yet another is practical applications. While there's less of these in pure mathematics than in most domains, they do still exist.
The primary goal of all basic sciences is human understanding. "Truth" is no more a goal for mathematicians than the physical laws are a goal to physicists; they simply exist in nature. The goal is rather to develop useful language and conceptual frameworks for reasoning and communicating. That understanding underpins all practical applications.
Sciences don't have goals, people have goals and they differ. Some are fans of pure math as a kind of religious or almost erotic activity in elegance and beauty, others are application minded. Some are in it for the community and outreach and conferences, some are in it to just sit in an office alone and be left alone to do it in a zen like flow state all day and night. Some treat it as a 9-5 to pay the bills with a skill they happen to be fit for but aren't especially passionate about.
> But for math: what is the point of a proof if no one will read it and no one uses its result?
What is the point of writing software if nobody will run it?
> So mathematicians should feel free to use AI as much as you want, but in the end, they should've gained some understanding on what happened.
So they ask the AI to explain the proof.
I think article is more about the goal than the how. Perhaps it will be better to have ai produce human accessible explanations. No one is saying not to do what you propose. The goal remains the same.
Code has a very different purpose than math tough. The development in basic science follows a different motivation system.
The more I see these posts about mathematics institutions reforms and challenges from AI advancements the more it looks like they may need to go through a death. Or to put it another way they may need to start again from first principles.
If math is truly about spreading intuition and understanding then our institutions have dropped the ball decades ago and have not been able to grab hold of it since (if they ever had it to begin with)
Not sure why you're down the bottom when the current top post says pretty much the same thing. I agree that the reevaluation and refocusing that is being forced by AI is one the maths establishment could fruitfully have had a long time ago.
"A computer or monkey could easily start at the axioms of ZFC and iteratively apply deduction rules....simply conjecture all mathematical propositions in alphabetical order...The prospect of automating mathematics by enumerating all conjectures, and all proofs of ZFC, is probably not so disturbing to you."
I thought we were going to get at least some brief comment on Godel here?
Gödel effectively says ZFC must be incomplete, otherwise it would not be sound, but does that stop you from listing all mathematical propositions it can generate in some well-defined order?
That's right. You could list all those propositions and search for proofs of them. In fact this is very similar to Hilbert's very program to which Godel's First Incompleteness Thm was a response (https://en.wikipedia.org/wiki/Hilbert%27s_program).
Godel showed that there are true statements that cannot be proven, and also that among the unprovable statements from within the system is the consistency of the system itself.
As I understand, mathematicians are still trying to figure out how much this matters. One of the best examples of its mattering is may be the Continuum Hypothesis: CH is consistent with ZFC, and ~CH (not CH) is also consistent with ZFC. In other words, you have enough flexibility in constructing your ZFC world such that in some ZFC-consistent worlds CH is true, and in others CH is false.
Litt's statement is not wrong; it's just that what he wrote sounds so much like Hilbert's program, that I'm surprised we didn't get some even minor comment on what kinds of truths we could reach if we embarked on such an effort.
People always bring up the Continuum Hypothesis as though it has something to do with Gödel's incompleteness theorems, but it doesn't really. The Continuum Hypothesis being neither proven nor disproven by some particular axioms is a similar phenomenon as that the group axioms neither prove nor disprove commutativity, the ordered field axioms neither prove nor disprove the existence of a square root of 2, etc. There's no particular reason to expect any particular formal system to be complete, sans some demonstration that is.
Gödelian incompleteness is the specific kind established by Gödel's proof, where theory T can't prove Con(T) without being inconsistent. But the inability of ZFC to consistently decide the Continuum Hypothesis is established in a completely different manner, with the Continuum Hypothesis not consistently decided by ZFC + Con(ZFC) either, or any such thing.
> There's no particular reason to expect any particular formal system to be complete, sans some demonstration that is.
Well, Godel's First Incompleteness Thm says that any consistent, effectively axiomatized theory that is strong enough to represent basic arithmetic must be incomplete. So there cannot exist a demonstration of completeness for any such system. ZFC is such a system/theory.
(And yes, of course some formal theories are complete and can be demonstrated to be complete, like Presburger arithmetic).
> Gödelian incompleteness is the specific kind established by Gödel's proof, where theory T can't prove Con(T) without being inconsistent....
That's the Second Incompleteness Thm.
But CH is an example of a (an important; or believed to be important) mathematical statement that cannot be proven / refuted from within ZFC. So it is an example of a statement to which the First Incompleteness Thm applies.
You're right that Incompletness #1 does not prove CH is undecidable; (assuming ZFC is consistent) it proves that undecidable sentences must exist. CH is one of those sentences.
>...is a similar phenomenon as that the group axioms neither prove nor disprove commutativity
I haven't done a course in abstract algebra (though I have studied Incompleteness), but the brief research I just did suggests that there is a meaningful difference in ZFC and the ordinary group axioms. The latter are not sufficiently axiomatized to represent arithmetic, so Godel's thms don't apply. ZFC is sufficiently axiomatized to represent arithmetic (and much more).
The ordinary group axioms are so weak that they can describe many different structures; they also cannot represent arithmetic. So the ordinary group axioms are incomplete, but they are not Godel Incomplete.
I like the quote from Hilbert that was brought up in the article: "we must know, we will know".
With AI, it might be the case that we don't know, we won't know, but the machine does.
The central question, namely whether humans should be in the loop, will be repeated again and again in the years to come for all industries, starting with mathematics.
I am not a mathematician, but I can’t see how we are going to address the problem which we already see in coding:
Impossibility to independently validate all AI results
And in math it goes even worse. In coding code reviews are typically still the form of action you do within days. In math, historically, the lifecycle of proof is months if not years. Take as an example Millennium problems. They require at least two years of validity after publishing. Two years! In modern times with amount of output AI can produce, it feels like infinity.
We are inches close if not at the moment already when humans can’t reliable validate proofs and mathematics produced by AI. Then next research will be based on this AI-written-no-human-in-the-loop results. And we will end up in just few years in a world where novel and frontier problems will be articulated by AI and proven by AI based on AI results and humans will be incapable of understating the mere nature of the solution.
I don't think that's actually the real problem. Along with the progress in answering mathematical questions, recent progress on AI-powered autoformalisation has been astonishing. All the recent AI discoveries have been accompanied by Lean proofs.
And, yes: that doesn't absolutely guarantee correctness. The Lean kernel has had soundness bugs, and may have some still. But it's pretty strong evidence of correctness nevertheless.
The concern among mathematicians is not mainly that they doubt the correctness of any of these discoveries, but that human understanding may be devalued.
I am not that worried, but rather just observing. Humanity is about to enter the phase when we will be using things based on ideas no human ever properly understands. This thought … disturbing, somehow?
It is perfectly valid counterpoint to say that we already do it. We everyday use myriad of things, tools, and software we have 0 clue how it operates. But for us as humans it was reassuring that we know that at least there are a few other alive humans who know it, who create it and who can explain it.
With AI soon that comforting zone will be gone.
>Humanity is about to enter the phase when we will be using things based on ideas no human ever properly understands. This thought … disturbing, somehow?
This is just normal though. We were building sophisticated bronze and steel tools long before any complex understanding of metallurgy or chemistry. Medicine is still the wild west.
I first wanted to say fire, even though it's a cliche, but then I thought that in antiquity we used like everything without anything that would qualify today as understanding. Also now we have a lot of stuff that we "know" it works based on complicated numerical simulation.
I think the most "understanding" we ever had was in the 40s-50s designing nuclear bombs with slide rules. It was the culture that produced the idea of psychohistory.
> Medicine is still the wild west.
Reminder that we have no idea how anesthesia works.
>Humanity is about to enter the phase when we will be using things based on ideas no human ever properly understands.
Do any one person even understand the humble pencil?
https://dn790006.ca.archive.org/0/items/i-pencil-pdf-2019/I%...
But one of the things AI also excels in is summarizing and can do so hierarchically. One of my favorite things to do with a concept I'm new at is "ELI5" then "explain like I'm a high school student" then "explain like a bright undergrad in XYZ" then "explain to a working professional in this domain". It's a lot of steps, but I've found it very effective (for me) to learn with -- and I've done something similar with code and math (although not math proofs -- I'm not a mathematician). But my point is that I think we can use AI to also teach us these proofs they're building in a way that I don't understand today about human proofs.
If we use AI well here we could actually understand math much better than we do now.
I actually think it's the opposite: Lean proofs and autoformalization make it very easy to announce proofs alongside proofs of the correctness of those proofs (Lean certificates). It's not an absolutely fool-proof combination (the Lean kernel could still contain bugs), but it does immediately attach a very substantial degree of credibility to the result.
And that I think is essential to why some of the world's leading mathematicians are taking this so hard. In a world where we "merely" have AI systems capable of superhuman informal reasoning, verification, correctness, and acceptance could still only be conferred or anointed by human mathematicians. But a world that combines superhuman informal reasoning with superhuman autoformalization is a fundamental shakeup in the institutional order.
The recent proof of Fermats Last Theorem is interesting: it is (iirc) 13 million lines of lean code. And type-checking takes 5 hours or so on a pretty beefy machine. I cannot independently verify the proof, and I have to take Anthropics word for it that it actually type-checks.
That seems like a red herring. Have you independently verified the human generated proof of FLT? Surely someone else will try to verify Anthropic's formalization on different hardware. Plus, it seems likely that FLT formalizations will improve / get shorter over time, requiring less compute. And computers (and type-checkers) will continue to get faster over time as well. So maybe in 5 years you could own a computer fast enough to verify a/the proof in say a week, instead of 5 hours.
With human proofs, I have some trust in process behind it.
It'd cost you $30-$50 on AWS for a clean Lean build or maybe $100-$150 for the full verification suite. You are, in fact, capable of independently verifying the proof yourself if you don't trust all the people who've already done so.
With formalized math, you only need to validate the problem statement (in theory, in practice agents have already managed to exploit Lean compiler bugs, but the incidence of those should decrease enough to be practically lusable for 'blind' validation of AI proofs in the foreseeable future).
> resulting in the production of an abundance of PDFs. The contents of some of those PDFs may even have important applications.
I hope, from the depths of my soul, that the static typeset report format for transmitting knowledge and understanding will finally die and be laid to rest.
As a researcher in theoretical computer science, I love PDFs more than any other format when it comes to mathematics.
There simply is no contender to LaTeX and PDFs.
Lucky for you, almost all research in math, cs, and physics, are put on arxiv, where you can download the source code (.tex) as well as get an HTML render.
What would you prefer instead of "static typeset report"?
Not the OP, but how about things like:
https://ciechanow.ski/archives/
...for starters?
My worry is that the frontier of math is too far away for most humans to reach. AI is only going to make that worse. If today it takes twenty years of math study to reach the frontier (in a narrow field), what's it going to be like when it takes forty years? Or four-hundred years? Will the fields just get narrower and narrower to accommodate finite human intelligence?
> If today it takes twenty years of math study to reach the frontier ...
Are you counting from when a kid first learn 1 + 1 = 2? That seems a misleading metric. The rate at which kids are learning math in the first half to 2/3 of their schooling is very low. Or are you asserting that Ph.D.'s don't reach the frontier until age 40+?
Actually, the frontier of math is too far away for most humans to reach because >99% of humans don't have the drive and talent to study and learn enough advanced math. Similar barriers exclude >99% of people from Olympic-level athletics, or being famed violinists, or being important architects, or being US Senators, or being successful ancient historians, or many other things.
> I think so. This machine might produce answers we value, but it would not, in itself, produce human understanding of those answers.
It's nice that the author is optimistic, but won't the AI be best placed to dumb down its increasingly complex proofs into a language us lowly humans can understand? To keep thinking until it can refactor complex proofs into ones from 'the book'?
As they go on to explain, a human understandable proof is different than a human actually understanding the proof. That actual human understanding (like, in a brain of a human) is one of their stated goals.
Producing human understandable proofs is possibly a job best for humans today, but the author appears to agree with you that this is probably fleeting (and argues that even if you disagree, it should probably be treated as if it is fleeting when planning for the future):
> Right now AI systems arguably underperform us at theory-building, asking questions, exposition, … so we could prioritize and reward those skills. I think this is unwise: compare the speed at which the academy adapts to the speed at which model capabilities improve. We need to consider the endgame. If the models remain incapable in some domain, we can adjust later.
AI will never be able to dumb down a proof to a level simple enough for someone to understand who has never studied math and put a lot of effort into it. Some concepts just need time and effort to absorb no matter how relatively simply they are phrased.
Sure, but some proofs are so complex that even experts can't follow them. As AI progresses, this may constitute an increasing fraction of proofs. I meant that AI could work to find the simplest possible proof.
> Sure, but some proofs are so complex that even experts can't follow them.
If that's the case then I don't see much reason for them to exist in the first place except mental one-upmanship.
This is an excellent essay, and especially relevant when read together with the MIT report on the future of education in an AI era: https://aiandeducation.mit.edu/report/ (and my own https://oberbrunner.com/blog/ai-future-of-mathematics).
I'm heartened to see this kind of thoughtful, reasoned discussion around long-needed reforms in the educational community, with AI as the catalyst. We certainly have interesting times ahead.
> I propose the following reconceptualization of the goal of a mathematics PhD: to become a world expert on some interesting, deep topic, and to be able to convey that interest and understanding to others. Part of operationalizing this might be a thesis, but the degree would be awarded primarily on the basis of a rigorous defense, in which the student explains the topic to their examiners until they are satisfied.
I think this is a refreshingly forward looking idea and I agree with it 100%, especially the the "rigorous defense" part. That is a good measure of how well the topic has been researched and understood by the researcher. This is where the humans can be "in the loop".
> How different would this look from current PhDs? I think students would still meet with an advisor, who might suggest a topic. That topic could be explored with AI assistance, or not, but the student would be responsible for understanding it; it might be much more open-ended and larger than the typical PhD is currently.
Interesting point about "more open-ended" and "...larger than the typical PhD". I think the author has a point. Earlier, the bottleneck was the candidate's/researcher's understanding and knowledge. Now with AI tools, it is so much easier to zero in to relevant knowledge, get your questions answered quickly which might lead to understanding more quickly.
For e.g., before the advent of public libraries and printing press, the knowledge was inaccessible and guarded. So that was the bottleneck.
Then books became ubiquitous and the bottleneck to knowledge and understanding was people's motivation AND knowledge of WHAT books and topics to research.
Then came the internet and free PDFs of books and research articles. Now, the bottleneck was still people's motivation and a mild version of what books and topics to research. I say "mild" because one can lookup articles and newsletters, and book reviews and come up with a list of reading.
Now comes AI and it looks like the only bottleneck is people's motivation.
I believe there was also a silent, yet potent, bottleneck all along which is also removed by AI: personal tutor/coach/teacher/professor etc. Let's say if I am reading a textbook on manifolds or some research paper and I have a question about a specific theorem or even a mathematical operator being used. Before AI my only way to get my questions answered was to read more books (PDFs or print), or ask on math exchange or math overflow and wait for someone to answer, or to ask a professor. This could take up to a week.
Now all of that has been cut down to 1 hour or less with an interactive chatting session.
!!!!!
So....the only bottleneck is people's motivation! QED
Exciting time!
Why is "Doctor of Philosophy" the correct certificate of "becoming expert in a topic"?
That's a radical departure from "PhD" being a certificate that someone is qualified to produce new research.
What you describe is more like a Masters Degree.
I'm not sure why an advanced degree is necessary for that at all, besides the pride of a vanity title. It's pretty much what Bill Nye does for science.
I think you have misunderstood.
PhD has nothing to do with expertness.
If you have a PhD, you have completed some kind of research training.
That's all there is. Says nothing about knowledge or whether or not you're a genius.
You cannot conclude anything else, and nobody claims that you can.
If someone has a PhD, they have some training in doing research.
Looks like @gowld is agreeing with you.
The idea that most any modern "interesting" aspect of mathematics is going to be understood (or often even explained in enough detail to reveal what is interesting) in an hour is pretty rare. Either the student's aptitude, the tutorial, or the mathematics are unique. There is a reason that these are PhDs and not undergraduate HW sets.
I think we often delude ourselves as to how well we understand problems and their solutions. Some instructors even make you feel that you understand better than you do by pointing to a few approximations or simple solution spaces that obscure the larger complexity. Just looking in wonder at the many categories of three-body solutions (currently on hnews) is enough to remind me of this.
> the "rigorous defense" part
In my country, that's exactly how it is.
Yes, you need to have a thesis to defend, but ultimately it all comes down to the (oral and live) defense/disputation.
I'd say this is the most optimistic scenario. there are really difficult problems to be solved in terms of access to AI.
> Now comes AI and it looks like the only bottleneck is people's motivation.
Who pays the salary of the mathematicians and the cost of accessing the AI?
> a refreshingly forward looking idea
What?
"a rigorous defense, in which the student explains the topic to their examiners until they are satisfied."
is exactly how PhDs were awarded for hundreds of years.
Even my BSc in Applied Physics (1977) had a viva voce that was a substantial fraction of the final exam.
I am not implying that this is a new idea and I don't believe the author did either. If you read the full article, the context is clear.
The author is re-emphasizing the importance of thesis defense old school style and highlighting that being familiar with one's own material rigorously should still be a requirement.
Everything that leads up to the thesis defense perhaps changes signficantly.
Another bottleneck is money required to pay corporations who own the technology to "do mathematics". In that future, there will never be another Ramajunan.
That's a good point. Low level (for a lack of better term) mathematics will become extinct/saturated.
Perhaps a parallel to this is what's happening with youth sports in the US. It is becoming increasingly inaccessible.
Yeah I think it was in Terence Tao's recent post where he pointed out that the "low-hanging fruits of mathematics" are being harvested by the AI companies, when traditionally those problems were available for aspiring mathematicians to prove themselves in the community. Maybe there's a parallel in the software world too.
There absolutely is a parallel in the software world. I have 16+ years of experience in software and I have enough coding under my belt to guide and inform coding agents to solve the problems.
This is the reason why hiring is down for fresh graduates. We don't have low hanging fruits types of problems to give them anymore.
So until the industry figures out the workflow, the hiring for fresh graduates is going to be tricky.
What is the incentive for a person to sit though seminars and evaluations? People already hate redundant meetings. What is the incentive to change the system from the existing one to one that rewards this verification somehow? Who benefits from this change?
I was involved in this research a few years ago: https://metamiditoolkit.com/research-paper/
One of the things that struck me was that most people have an area of their workflow that they would be happy to hand off to an AI, so that they can focus on the thing they love. However, that area is different for each person. One person's grind-work is another person's love-work.
This is why it's so hard to build consensus around where the "red lines" are in this space.
I’m not a mathematician, but if I were I would imagine the main attraction would be the act of thinking hard and solving problems. Maths felt like the ultimate profession for someone who loved puzzle solving. If you delegate the thinking, any percent of it to an LLM, I don’t think it’s maths anymore. Might as well join any other job and earn better money now. A completely ludicrous proposition, that maths will no longer involve mathematical problem solving.
Modern incentives, unfortunately, reward producing solutions rather than going after the joy of solving problems. And it's the latter that inspires others, not the former.
Yes the fascinating thing is:
1. It will take much longer to understand the output of the machine that it takes to prompt and create it. 2. The only? best? one? way to /verify/ that you /in fact/ understand the output of the machine is to explain it to someone else.
So there will be a machine generating koans which need to be meditated upon and discussed with human social back-pressure validating understanding. I think this could be much more cooperative and at a minimum this will be a way different math social construct.
Misanthropic and Open Artificial Ignorance are misleading the public.
They are leeching off the work of mathematicians and pretending that new math was invented.
As with the rest of the domains, AI/LLMs will do syntax and search better than any human.
In code, any developer whose differentiaton was clean code and knowledge of different technologies is now average.
In math, any mathematitian whose differentiation was to manipulate formal systems and know tricks of different domains will be average.
Fortunately, humans do more than syntax and search.
The bad news for developers is that if you know what the output of your program should be (which happens most of the time), almost all of the job is syntax and search to build the code that reproduces the output.
The good news for mathematitians is that for the majority of problems you never know the output, or you just know the output is either "True" or "False". There are some cases where you need something else, for example "a solution that blows up in finite time". For those cases AI will outperform you easily (see new Navier-Stokes solution)
So if as a mathematitian you were doing more than syntax and search, then keep doing that and use AI just for what its best.
As a recent CompSci graduate and software engineer, I can sympathise with the author’s position. At the very least, these times are highly unsettling.
Pretty neat how these abstract origins still shape our world, especially in CS. Kinda makes you appreciate the shoulders we stand on.
Always fascinated by how foundational math concepts built up. Euclid's Elements still blows my mind for its logical rigor.
Does anyone have a rough estimate of how many research mathematicians there are in the U.S. or world?
> AI does not care if you are anti-AI.
This >>>
While I broadly agree with the premise of re-directing the “purpose” of math, I quite detest the idea that judgement might be primarily based upon some in person discussion, or oral presentation, and the claim that written mathematics that is not orally communicated might be less worthwhile in some sense (i know this isn’t the exact statement of the authors proposition).
There are a good deal of people, whom, falter much more in oral discussions, whether this be for a psychological thing, stage fright, or difficulty explaining things on the spot. There are also certainly brilliant people, who can’t give an informative, discussion inviting talk to save their lives, but given enough time, can formalize their thoughts in writing at the highest levels of their field, and that writing is likewise very enlightening (sometimes).
It’s not clear to me, that, AI as is, could not pose successfully in an oral discussion of a topic. I mention this because it seems that one implication of the article is that AI might write things that are logically correct, but devoid of understanding. I suggest rather that 1) it is not extremely improbably that AI is incapable of generating mathematics that furthers human understanding and if 2) it is indeed highly likely that they cannot generate mathematics that furthers human understanding in a textual format, then surely one could also differentiate between human and AI on a textual level, and judge the contribution of a human, without the need of oral discussion?
I suppose another aside is, one might claim that the existence of AI means people have much much more text to filter for, and so, it becomes difficult to find one person’s good writing amidst a sea of, logically correct, yet understanding devoid textual content. But by and large much or mathematical academia certainly operates off of some reputation/vouching system presently anyways, that already serves as a “filter” in some sense. Perhaps the existence of such a system/culture is not a good thing, but oral discussions/seminars certainly aren’t immune from such predilections.
Perhaps I’m babbling like an idiot, but the entire and sole purpose of this comment is just to say: for the love of god please don’t let the standard be judged by oral presentation
Imagine, eventually, a large ever growing library full of unread books. One could roam around forever and discover new things.
Might be possible to generate a 3d world using some taxonomy estimate the amount of titles and generate the books as the area is explored.
Like the manuscripts in Sakya Monastery in Tibet and like the millions(?) of cuneiform clay tablets. Vast amounts of data that may contain information.
The key shift seems to be from "can you produce mathematics?" to "do you understand mathematics?"... Those used to be correlated strongly enough that a thesis or paper could serve as evidence of both. If AI breaks that correlation, evaluating people through defenses, discussion and live problem solving suddenly makes a lot more sense.
I think this goes in the right direction. You have to rethink the role of human work in math, can't put your head in the sand and cling to your comfy institutions and system just because you got to know it's ins and outs and just want it to be like that forever.
But the bigger picture is: while I understand the author know his field best and wants to keep the post focused, the same issue will hit many more fields. We need to also have a broader discussion that involves more fields of knowledge work, largely academic scholarship but also regular office work, then it will come to engineering design, medicine, it's already coming for 3d modeling and vfx, software dev, it will come for a lot of middleman services. Not at the same rate, but we have to understand that it's not just that math will change.
Change will be the default. Everything will change. It will be much easier to make math change because all things will change. You shouldn't worry and imagine that funding criteria will be like today or that journals or academia or politicians expectations will be like today. No, everything will adjust with some timing differences of course but it won't be a static world and then math changing and having to justify and fight to explain the change to other actors who are baffled. They won't be baffled they will themselves have to change.
The world will transform as much as it did when society moved from feudal agrarian to urban capitalist industrial, or from the vast majority doing physical labor to a service economy with a huge amount of desk jobs. I can't tell how it will change exactly but it will be bigger than what we have seen in the last couple of generations or maybe more.
[flagged]
[flagged]
[dead]
[dead]
Another mathematician waxing poetically about a future that will not come to happen. Call me a pessimist if you'd like, but capital has no incentive to ensure mathematicians maintain their current status in society.
If you're a mathematician you are in the same boat as the software engineer, and the Dodo.
Better learn a trade buddy /s
Better plan is to shut down PhD and make students take a oral thesis at bachelor and master level and help them become a productive economic participant as soon as possible.