I always get annoyed when people misinterpret Occam’s razor. It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple.
It’s just like the Hopper quote. She said it’s better to ask for forgiveness during the fog of war, doing something you thought was right, not to do something you knew they were going to say no to and now you are trying to get away with something.
And I like the bayesian interpretation too. Murphy's "Probabilistic machine learning" has an occam's razor section.
The idea is that a complex model explains many more configurations (datasets) than a simple one. So its (prior) probability distribution is lower on the data seen (to compensate for the other possibilities it might explain). So the (marginal) likelihood that the simple model is correct is higher if it fits the data well enough.
This is a longstanding principle in model-fitting. More parameters, almost always, improves the ability of the model to fit to any particular data, in-sample. The model with the least parameters is both the simplest in principle and has the best chance of not overfitting.
This is provably not true, and you can use the marginal likelihood / PAC-Bayes to prove it (or any other framework for measuring model quality). Increase the number of parameters in a linear model way beyond the point of interpolation, and concentrate the likelihood around the zero loss set. Then reduce the variance on a Gaussian prior. You can balance the two temperatures at exactly the right rate so that any measure of model quality will monotonically increase with model size and achieve a maximum at infinite model size.
Even easier, just take a limit of polynomial regression to a Gaussian process while optimizing the marginal likelihood over the prior temperature.
In all of these cases, the model with the least parameters is not the simplest in principle and does not have the best chance of not overfitting. The reality is significantly more nuanced.
Are you sure that doing this after seeing the data is valid and does not suffer from the equivalent of peeking-into-the-test-set problem ? There are ways to address the peeking problem but that requires additional machinery.
I don't dispute your broad claim but the first counterexample you quote seems problematic.
You can choose the prior according to any selection rule that does not see the data (actually, you can do more, but justifying this is the realm of empirical Bayes and requires some more precise arguments). In this case, you can choose it according to the model size and provided that your Jacobian is full rank, you will get increasing marginal likelihood.
What threw me off was the (possibly misunderstood) suggestion for minimizing the generalization bound over the prior after the data has been incorporated.
Ah, sorry for the misunderstanding, I can see how my comment reads that way. That is done in the Gaussian process context, not in my first example, and yes, it's a dirty idea, but you can justify it using differential privacy arguments (basically you are optimizing few parameters and these do not have full interaction with the data).
Yeah, I had read one of your parallel comments and understood what you had meant. Differential privacy is a good formulation (well, the only one I know) to deal with the peeking problem in general.
You are saying something interesting, but talking like Grok and skipping a lot of the details, without any references to common check-in points like terminology or specific studies.
> and concentrate the likelihood around the zero loss set. Then reduce the variance on a Gaussian prior.
Those phrases could mean a lot of different things. What are you proposing?
> so that any measure of model quality will monotonically increase with model size and achieve a maximum at infinite model size.
any measure of model quality? You must have some bounds of any measure, since trivially that's false because "fewer parameters is better" is a measure of model quality, even if dumb.
It's hard to even engage when you're being so imprecise, and not even giving one specific example.
Apologies, I'm skipping details, because that's how I speak with my colleagues, but I realize this is an external environment without context. No references since this is folklore (you can look at Hastie et al's Surprises in High-Dimensional Ridgeless Regression paper for the non-Bayesian version, Bruno Loureiro or Andrew Gordon Wilson probably have a paper with something similar).
Concentrating a density around a zero set means that I raise it to the power of 1/gamma (appropriately normalizing) and then take gamma to zero. If the likelihood was Gaussian, this would be equivalent to taking the variance to zero (yielding a point mass). But in overparameterized settings, this concentrates on a submanifold describing the set of interpolating solutions. In least-squares linear regression, that is the solution space. Reducing the variance on a Gaussian prior is treated as an asymptotic expansion by Laplace's method. If you choose the variance to decrease (inversely proportional to the parameter size, for example), then the marginal likelihood will increase monotonically with model size.
By any measure of model size, I mean that you can pick your favourite among the common ones, such as information metrics (e.g. mutual information / KL), statistical metrics (e.g. marginal likelihood), test error. You should be able to show the same phenomenon happening for all of them, so it isn't a quirk of marginal likelihood. It is concentration of measure working in your favor to reduce the variance in the estimator.
No, I am talking about out of sample error and estimates thereof. It is "overfitting" to data, but it also has lower out of sample error than the case where you do not "overfit".
This is why the notion of overfitting is not nearly as cut and dry as a basic ML course would have you believe. Just because you fit data exactly does not mean that your estimator has high error on out of sample data. A trivial counterexample is a spiking model that spikes to fit to the data but otherwise follows the correct trend outside of the dataset. The bias variance tradeoff gets thrown out at enormous scale and overfitting is not a meaningful concept. What matters is regularization and robustness, not how well you fit the data.
The reason why bias variance tradeoff and considerations of model size are a good approximation for smaller models is due to concentration of measure in the data which effectively kills any regularization in your modelling procedure. Once you enter settings where concentration of measure begins to bite in parameter space, everything changes. This isn't really that mysterious; any textbook on Gaussian processes (e.g. Rasmussen and Williams) will tell you this.
I don't understand what you mean. Test error is literally out of sample error. Marginal likelihood is designed to estimate out of sample error. The whole discussion is about out of sample; nothing has been about in-sample error. The in-sample error for my examples are all trivially zero, so only out of sample error is worth discussing.
Another useful link [0], page 91. Effectively, the more complex the solution, the heavier is the upper bound on true risk. It doesn't mean a simpler model is necessarily better. But the complexity brings its own larger support for mistakes to live in. The _likelihood_ of being _more correct_ is probably related to larger sample required to learn a more complex model.
Absolutely not. This link is a reference on PAC learning, which is thoroughly misleading in the land of deep learning and inevitably leads to vacuous bounds. This is common knowledge in deep learning. I would not recommend that any student learn any part of this theory at this point, since we have far better alternatives in terms of simplicity, accuracy, and generality.
PAC-Bayes is genuinely superior, for example. Instead of a uniform weighting over all hypotheses (effectively encoded in the supremum), you get to weight the hypothesis class in virtually any way that you want. This is critical to ensure that you exclude absurd hypotheses that you have almost nil chance of reaching. If you do consider a uniform weighting, then you can just easily reduce to PAC anyway, but you do so in a cleaner package.
"Can be misleading" is an accurate characterization.
It is an uniform bound and will have problems with very large hypothesis classes but its statement isnt wrong (just that the tool is a little heavy handed). For simpler models they are adequately useful. So I would suggest new users to learn if they want to use simpler models rather than deep learning methods.
Real world isn't the adversarial/ worst-case that these models of generalization assume. So the generalization performance you experience is usually a better than what PAC indicates, but estimating that experience was not PAC's charter. It's charter was to quantify a adversarial/legal guarantee, the minimum (infimum) guaranteed program.
I too like PAC-Bayes a lot (see my other comments) but it does have a cop out in that by definition you do not know how good your prior is, yet the quantitative estimate of generalization error depends on how good it was.
Nevertheless, PAC-Bayesian and statistical physics based bounds are closer to what we experience, but you can't guarantee that the worst case will not bite you.
I agree that this is a good nuanced take. However, I find that students who have learned PAC (which usually takes quite some time) often have to unlearn certain principles to do PAC-Bayes, so my comments come from a fair amount of frustration with the topic. On the other hand, I find that teaching students PAC-Bayes from the get-go is easier, it still works for simpler models (you can derive the same PAC bounds, so you can't do any worse), and they get the full story. Obviously for those of us who learned both, it is good to know both. But I am skeptical for the next generation coming through whether it is worth teaching PAC at all.
I had not considered the pedagogical value of starting with PAC Bayesian bounds. It will be interesting to see ML courses that take that route.
Similar issues show up in traditional statics when considering multi-hypothesis testing. I wonder if betting/gambling over hypotheses might be a more accessible way to convey these ideas to new students.
Just to add on top of the quality reference provided by srean, I like to first drill in Bayesian principles and then use this article to derive PAC-Bayes from that: https://arxiv.org/abs/1605.08636
Regular PAC falls out by taking a uniform prior over a finite hypothesis class (and then building up VC dimension if desired, but usually by this point you realise why the bounds are unlikely to be good).
There are also various metaphysical theories that posit that the universe is algorithmically generated in some sense or the other, and from many of those theories it follows that simplicity is a fundamental feature of reality, which yields an even stronger version of Occam’s Razor.
You don't need the onerous assumption that the universe is algorithmically generated, just that all ways to reason about and communicate intelligently for the purpose of making decisions is digital.
The notion of "simplicity" can be completely arbitrary, though. It's enough that there are only finitely many hypotheses simpler than the best hypothesis (assuming there's such a thing as a best hypothesis). So as you eliminate hypotheses incompatible with the data, at some point you'll have eliminated all simpler hypotheses, and the simplest hypothesis left will also be the best hypothesis. If simpler hypotheses are also more likely to be correct, you get there faster, but it's not required.
It doesn't have to be arbitrary. The Kolmogorov/Solomonoff definition, that the simpler models are the shortest programs that can generate what we know about a domain, measured in bits, have a solid mathematical foundation, based on information entropy and even thermodynamic entropy if you cross over into computer hardware.
Entropy is a metric targeted in LLM training which is likely why LLMs are overfitting less and less as they improve and why they subjectively seem to obey Occam's razor.
There is some art to it, especially in model architecture choices.
But your smarter data scientists will try to get the best information density on test data to approximate generalization. MDL, as championed here, or AIC/BIC if working with more established and acceptable methods.
Except for the fact that eventually we are all dead.
So it is kind of important to get there faster.
For complicated hypotheses, where complicated is defined appropriately, it takes many many examples to realize that it was a wrong hypothesis all along. There lies the rub.
For a particular instance of a learning problem we can't tell much, however using a Occams razor over many instances, one would be correct more often than not. Provided, of course, the PAC assumptions are true or they are not very far from being true.
How far is not very far ? That gets very hairy to quantify.
To be clear, I was doing a reductio ad absurdum. PAC is precisely the kind of theoretical framework that concerns itself with asymptotic long-run behavior. An appropriate definition of simplicity certainly gets you to the goal much faster than the worst-case bound. For example, you could order hypotheses from most to less likely. But that would render the claim that simpler hypotheses are more likely rather tautological.
More typical definitions of simplicity, e.g. using the bit length of some kind of natural encoding, aren't guaranteed to offer any special advantages in terms of likelihood. So if you have prior knowledge that a particular hypothesis is more likely than another, but the less likely one is simpler in an intuitive sense, you shouldn't let that override you, but still prefer the more likely option. (And you don't need to take the circuitous route of coming up with a new encoding where the more likely hypothesis has a shorter bit length, either.)
PAC isn't asymptotic in general. It gives finite estimates for finite sizes of training data.
If you say, and I think you are indeed saying so, that PAC is ridiculously pessimistic, I would be in violently agreement with you. That's one reason why for practical training data sizes and practical (infinite version space) hypothesis classes PAC gives bound such as -- probability of error is less than 41K. This isn't exactly incorrect but not very useful or informative.
A far more useful formulation is PAC-Bayesian where you get error bounds less than 1 guaranteed and usually less than 0.5 on reasonable sized training data sets.
That's not true. It's pretty clear that she meant "do something you knew they were going to say no to and now you are trying to get away with something."
> So I want to tell something to all the young people here on many many occasions you'll find it is much easier to apologize than it is to get permission.
You do it then when somebody comes after you and say are you supposed to do that, "oh gee I didn't know I wasn't supposed
to do that" ...
so just remember it's frequently much easier to apologize than it is to get permission do it
She goes on further, explaining how to deceive your superiors to manipulate them to get what you want.
But I still don’t think that means eat all the cookies in the cookie jar and then apologize after because nobody would have given permission. That’s still about doing what you believe to be right. She even frames the fallout as “where you supposed to do that?” and not “you shouldn’t have done that”.
Analogies only go so far, but I think eating the cookies in the cookie jar is the wrong one, because you're only feeding yourself with that one. If you're stealing flour from the baker to make a surprise cake for the whole troupe to enjoy, then you can apologize later, instead of asking first and ruining the surprise.
Not sure about Hopper, as I recall biographers of Lawrence of Arabia certainly made it seem like he was using the fog of war to do things he knew his superiors may object to.
Regardless, even if its misinterpreted it still has a kernal of truth and separate utility than your version, that is: the people in the field closest to the action have an operational awareness that may result in better decisions in times of urgency.
“You should prefer it, because it’s simple” is just restating Occam’s razor, not giving any explanation. “The simplest is more likely to be accurate” is a much better interpretation than yours.
I think the most accessible example of Occam’s razor is fitting a line to some points; you can always use a high enough order polynomial to fit the seen points exactly, but a straight line is likely closer to representing the underlying distribution.
simpler is not the right word either. it's the one that makes the least assumptions, not the simplest. The simplest would be "god did it" pretty much everytime.
> The simplest would be "god did it" pretty much everytime.
An out-of universe entity, that is by definition too large to be understandable to anything in universe, is a lot, but not simple. Are you sure, you are not confusing easy and simple?
It is simple. There aren't any explanations needed because "god is omnipotent" already covers everything.
But its simplicity involves making MASSIVE assumptions.
And the fact that Occam's razor explicitly says "fewest assumptions" and not "simple" is unchanged.
Quantum mechanics is complicated. Assuming electrons orbit nuclei is simple. But quantum mechanics is backed by countless experiments, so needs less assumptions. If you have to choose between quantum mechanics' description of the atom, or the old, outdated, knowledge of an atom, Occam's Razor would say to choose quantum mechanics.
By that measure Quantum mechanics as explanation is simple, because "it's according to Quantum mechanics" already covers everything.
However that is not what I think of when I hear "complexity of an explanation", I do not think of the complexity of the statement referring to the explanation, but of the complexity of the explanation itself, which is only referred to by the statement.
Also I disagree with using the number of experiments needed to arrive at the conclusion as a measure for complexity. By that measure "things fall down" is complicated, when you have done a lot of experiments. I would rather take the number of "steps" needed to derive it from first principles once you know it. But also that sounds like what you said in your last sentence, so you are contradicting yourself?
It's partly because of disagreement about what "complexity" and "simplicity" actually are. Many simple statements are in fact backed by massively complex, unstated assumptions. In attempting to deal with assumptions, scientists necessarily end up having to deal with the complexity involved in that. But the problem is if you don't engage with that, how do you know what is really more conformant with Occam's razor, as opposed to just satisfying what is readily expressed in common language?
Likewise in software development a C developer will say abstractions are not simple, a java programmer will argue that dealing with low levels details is not simple. They're both kinda right but will resort to framings which back their world view.
Ah, over-the-top larger-than-life LLM-isms, they are really funny when you see them in a company blog, but they are vomitive when it's your coworker copy-pasting it and insisting you on reading it.
These models are trained on human language, which belongs to us, we shouldn’t surrender it to them. Keep the em-dashes. IMO don’t overuse negative parallelisms though, they were always bad and lazy.
For the first time ever in my life, I consider the fact that English is not my first language and that I never had any formal education in it as an advantage.
Nobody is going to consider my weird and barbarian prose with the well polished product of a SOTA model.
English as a 2nd language speakers often have better grammar than natives in my experience! I wouldn't be surprised if the average native english speaker doesn't know the different between there and their.
> A validation set is one you consult repeatedly while building the model — to compare candidates, tune hyperparameters, and decide what to try next.
The validation can be done like that but it is better to do a cross-fold validation or selecting a random subset. These are equally valid. Keeping the validation like a hold-out set can end up creating biases if the statistics don't match. Honestly, this is true about the test set too. Though I've found that most people really don't think about this much. It's mundane, boring, hard to do, but deceptively important.
"The more your listener already knows, the shorter the message you need to send. An expert ML engineer needs only a few sentences; a newcomer needs the whole manual."
Literally from the first words on the page. 1st sentence passed barely. 2nd sentence ending "a newcomer needs the whole manual" and it's Claude.
Why is this being published as a blog post and not as a peer-reviewed submission? If it's going to be a blog post, why isn't there a corresponding scientific version for me to look at?
If anything, the latest generation of AI models, Astra and Fable, are prime example of overfitting—whereas benchmarks suggest they’re AGI-tier, users (including myself) report the same old gaslighting, hallucination, context rot, cheating, incomprehensibility patterns as with prior models, sometimes even more pronounced.
Fable and Opus 5, I suspect, will become textbook examples of RL collapse.
Compression in this modern day and age is so slop.
Yes, I'm familiar with keystone results such as Solomonoff induction. It's a direct counterexample to compression - your intensional algorithm can completely outrun reality. I can literally specify a huge mega-algorithm that just searches over all possible Turing machines and evaluates them, and it's an optimal compressor. It's completely vacuous though. You can always hide the "heavy work" in your mappings and descriptions. It's ironic that a kolomogorov complexity minimizer is so loaded that it's vacuous.
This is pretty much why I roll my eyes at this point at all the compression is intelligence memes.
I wonder when intervention and causality will hit the mainstream. These tools were designed specifically to counteract purely predictive theories. But your average compression dude will hold tight to their paradigms and slogans, not realize their internal contradictions (that their own field has brought up), and then whenever a new paradigm suddenly becomes visible and mainstream, they'll latch onto that. It's not principled at all.
And to be clear - I do think intelligence is some amount of compression, and I am well aware of formal results such as the arithmetic decoding theoretical and empricial result. Just annoyed. It's literally no different than the whole Bayesianism meme. If you're not actually practicing that type of intelligence as a basis, then you don't get to go around beating the drum about how it's the ultimate reality. You're just spouting dogma to feel like part of an in-group.
I think that the ratio of work done by priors and search is the interesting question at this point. We aren't quite at the point where reading off the most likely hypothesis decompressed from a transformer is sufficient, but it's a lot closer than I originally suspected when we were ~solving chess. I think AlphaGo was kind of the watershed moment that prior-guided search was so much better than either alone.
I think the adversarial policies against Go AIs directly show the gap between intelligence and compression/priors.
I’m guessing your downvotes were for tone? You’re correct though regarding Solomonoff induction, as the choice of reference universal partial recursive function gives drastically different results for predictions based on finite data (even with access to a halting oracle). Asymptotically, any choice eventually converges to the same predictions, but that’s no help when there are infinitely many choices for U and no obvious natural prior over universal functions. And I don’t find the argument that our natural environment “implements some choice of U” particularly convincing. There’s definitely an open mystery there.
The pattern that RSS feeds best supports is subscribing to individual creators who produce things you value. Presumably, if you don't like AI word-vomit, you wouldn't subscribe to a blog that uses it heavily. This is in contrast to the social-media state of affairs (like HN, reddit, twitter, etc), wherein much of the posts you read are recommended to you by either a person or an algorithm, regardless of whether you believe that author will write something you like.
A shorter way of writing this is that RSS promotes writing stuff good enough to get into somebody's "favorite authors" list, and social media instead incentivizes "lots of people will share this."
I don't take issue with that. Attempting memorized pattern generalization through holdout / validation strategies is a big part of ML that you would not typically see with econometrics / psychometrics / possibly sabermetrics / most other -metrics. Philosophically the explain versus predict divide. https://www.stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf
That's a bit pedantic no. Memorization in ML refers to the model having the wrong level of capacity such that it's too hard to optimise it such that it doesn't memorize the _training examples_ themselves.
I always get annoyed when people misinterpret Occam’s razor. It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple.
It’s just like the Hopper quote. She said it’s better to ask for forgiveness during the fog of war, doing something you thought was right, not to do something you knew they were going to say no to and now you are trying to get away with something.
I think you should get less annoyed.
> It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple.
I don't know what Occam meant, but if you accept the formalism of PAC learning, it is more likely to be correct
https://web.archive.org/web/20170428225156/http://www.cse.bu...
https://web.archive.org/web/20130412062821/http://cs.ecs.bay...
And I like the bayesian interpretation too. Murphy's "Probabilistic machine learning" has an occam's razor section.
The idea is that a complex model explains many more configurations (datasets) than a simple one. So its (prior) probability distribution is lower on the data seen (to compensate for the other possibilities it might explain). So the (marginal) likelihood that the simple model is correct is higher if it fits the data well enough.
[1] https://probml.github.io/pml-book/book1.html
This is a longstanding principle in model-fitting. More parameters, almost always, improves the ability of the model to fit to any particular data, in-sample. The model with the least parameters is both the simplest in principle and has the best chance of not overfitting.
This is provably not true, and you can use the marginal likelihood / PAC-Bayes to prove it (or any other framework for measuring model quality). Increase the number of parameters in a linear model way beyond the point of interpolation, and concentrate the likelihood around the zero loss set. Then reduce the variance on a Gaussian prior. You can balance the two temperatures at exactly the right rate so that any measure of model quality will monotonically increase with model size and achieve a maximum at infinite model size.
Even easier, just take a limit of polynomial regression to a Gaussian process while optimizing the marginal likelihood over the prior temperature.
In all of these cases, the model with the least parameters is not the simplest in principle and does not have the best chance of not overfitting. The reality is significantly more nuanced.
> Then reduce the variance on a Gaussian prior.
Are you sure that doing this after seeing the data is valid and does not suffer from the equivalent of peeking-into-the-test-set problem ? There are ways to address the peeking problem but that requires additional machinery.
I don't dispute your broad claim but the first counterexample you quote seems problematic.
You can choose the prior according to any selection rule that does not see the data (actually, you can do more, but justifying this is the realm of empirical Bayes and requires some more precise arguments). In this case, you can choose it according to the model size and provided that your Jacobian is full rank, you will get increasing marginal likelihood.
What threw me off was the (possibly misunderstood) suggestion for minimizing the generalization bound over the prior after the data has been incorporated.
Ah, sorry for the misunderstanding, I can see how my comment reads that way. That is done in the Gaussian process context, not in my first example, and yes, it's a dirty idea, but you can justify it using differential privacy arguments (basically you are optimizing few parameters and these do not have full interaction with the data).
Yeah, I had read one of your parallel comments and understood what you had meant. Differential privacy is a good formulation (well, the only one I know) to deal with the peeking problem in general.
You are saying something interesting, but talking like Grok and skipping a lot of the details, without any references to common check-in points like terminology or specific studies.
> and concentrate the likelihood around the zero loss set. Then reduce the variance on a Gaussian prior.
Those phrases could mean a lot of different things. What are you proposing?
> so that any measure of model quality will monotonically increase with model size and achieve a maximum at infinite model size.
any measure of model quality? You must have some bounds of any measure, since trivially that's false because "fewer parameters is better" is a measure of model quality, even if dumb.
It's hard to even engage when you're being so imprecise, and not even giving one specific example.
Apologies, I'm skipping details, because that's how I speak with my colleagues, but I realize this is an external environment without context. No references since this is folklore (you can look at Hastie et al's Surprises in High-Dimensional Ridgeless Regression paper for the non-Bayesian version, Bruno Loureiro or Andrew Gordon Wilson probably have a paper with something similar).
Concentrating a density around a zero set means that I raise it to the power of 1/gamma (appropriately normalizing) and then take gamma to zero. If the likelihood was Gaussian, this would be equivalent to taking the variance to zero (yielding a point mass). But in overparameterized settings, this concentrates on a submanifold describing the set of interpolating solutions. In least-squares linear regression, that is the solution space. Reducing the variance on a Gaussian prior is treated as an asymptotic expansion by Laplace's method. If you choose the variance to decrease (inversely proportional to the parameter size, for example), then the marginal likelihood will increase monotonically with model size.
By any measure of model size, I mean that you can pick your favourite among the common ones, such as information metrics (e.g. mutual information / KL), statistical metrics (e.g. marginal likelihood), test error. You should be able to show the same phenomenon happening for all of them, so it isn't a quirk of marginal likelihood. It is concentration of measure working in your favor to reduce the variance in the estimator.
Okay, and that's all in-sample, which is the entire point, it won't necessarily hold out of sample.
E.g. over-fitting.
No, I am talking about out of sample error and estimates thereof. It is "overfitting" to data, but it also has lower out of sample error than the case where you do not "overfit".
This is why the notion of overfitting is not nearly as cut and dry as a basic ML course would have you believe. Just because you fit data exactly does not mean that your estimator has high error on out of sample data. A trivial counterexample is a spiking model that spikes to fit to the data but otherwise follows the correct trend outside of the dataset. The bias variance tradeoff gets thrown out at enormous scale and overfitting is not a meaningful concept. What matters is regularization and robustness, not how well you fit the data.
The reason why bias variance tradeoff and considerations of model size are a good approximation for smaller models is due to concentration of measure in the data which effectively kills any regularization in your modelling procedure. Once you enter settings where concentration of measure begins to bite in parameter space, everything changes. This isn't really that mysterious; any textbook on Gaussian processes (e.g. Rasmussen and Williams) will tell you this.
Nothing in your reply gets at the connection to out of sample data?
I don't understand what you mean. Test error is literally out of sample error. Marginal likelihood is designed to estimate out of sample error. The whole discussion is about out of sample; nothing has been about in-sample error. The in-sample error for my examples are all trivially zero, so only out of sample error is worth discussing.
True.
It so happens that one gets the best generalization error bounds when one combines PAC with Bayesian ideas -- the PAC-Bayesian bounds.
Another useful link [0], page 91. Effectively, the more complex the solution, the heavier is the upper bound on true risk. It doesn't mean a simpler model is necessarily better. But the complexity brings its own larger support for mistakes to live in. The _likelihood_ of being _more correct_ is probably related to larger sample required to learn a more complex model.
[0]: https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearnin...
Absolutely not. This link is a reference on PAC learning, which is thoroughly misleading in the land of deep learning and inevitably leads to vacuous bounds. This is common knowledge in deep learning. I would not recommend that any student learn any part of this theory at this point, since we have far better alternatives in terms of simplicity, accuracy, and generality.
PAC-Bayes is genuinely superior, for example. Instead of a uniform weighting over all hypotheses (effectively encoded in the supremum), you get to weight the hypothesis class in virtually any way that you want. This is critical to ensure that you exclude absurd hypotheses that you have almost nil chance of reaching. If you do consider a uniform weighting, then you can just easily reduce to PAC anyway, but you do so in a cleaner package.
"Can be misleading" is an accurate characterization.
It is an uniform bound and will have problems with very large hypothesis classes but its statement isnt wrong (just that the tool is a little heavy handed). For simpler models they are adequately useful. So I would suggest new users to learn if they want to use simpler models rather than deep learning methods.
Real world isn't the adversarial/ worst-case that these models of generalization assume. So the generalization performance you experience is usually a better than what PAC indicates, but estimating that experience was not PAC's charter. It's charter was to quantify a adversarial/legal guarantee, the minimum (infimum) guaranteed program.
I too like PAC-Bayes a lot (see my other comments) but it does have a cop out in that by definition you do not know how good your prior is, yet the quantitative estimate of generalization error depends on how good it was.
Nevertheless, PAC-Bayesian and statistical physics based bounds are closer to what we experience, but you can't guarantee that the worst case will not bite you.
I agree that this is a good nuanced take. However, I find that students who have learned PAC (which usually takes quite some time) often have to unlearn certain principles to do PAC-Bayes, so my comments come from a fair amount of frustration with the topic. On the other hand, I find that teaching students PAC-Bayes from the get-go is easier, it still works for simpler models (you can derive the same PAC bounds, so you can't do any worse), and they get the full story. Obviously for those of us who learned both, it is good to know both. But I am skeptical for the next generation coming through whether it is worth teaching PAC at all.
You raise a good point.
I had not considered the pedagogical value of starting with PAC Bayesian bounds. It will be interesting to see ML courses that take that route.
Similar issues show up in traditional statics when considering multi-hypothesis testing. I wonder if betting/gambling over hypotheses might be a more accessible way to convey these ideas to new students.
Do you have a good textbook to reference to where the theory starts from PAC-Bayes?
You can try this one
https://books.google.co.in/books/about/User_friendly_Introdu...
Free download here
https://arxiv.org/abs/2110.11216
One of the ICMLs had a nice tutorial by Langford and Banerjee on the relationship between the different style of bounds. 2003, I think.
Just to add on top of the quality reference provided by srean, I like to first drill in Bayesian principles and then use this article to derive PAC-Bayes from that: https://arxiv.org/abs/1605.08636
Regular PAC falls out by taking a uniform prior over a finite hypothesis class (and then building up VC dimension if desired, but usually by this point you realise why the bounds are unlikely to be good).
Seems I was misremembering the dates. The Langford and Banerjee papers/turorials I had in mind were
On Bayesian Bounds https://dl.acm.org/doi/10.1145/1143844.1143855
Tutorial on Practical Prediction Theory for Classification https://jmlr.csail.mit.edu/papers/v6/langford05a.html
The first one is quite in the same spirit that you like.
There are also various metaphysical theories that posit that the universe is algorithmically generated in some sense or the other, and from many of those theories it follows that simplicity is a fundamental feature of reality, which yields an even stronger version of Occam’s Razor.
You don't need the onerous assumption that the universe is algorithmically generated, just that all ways to reason about and communicate intelligently for the purpose of making decisions is digital.
None of your links work for me.
Ah! from my very dated and messy bibtex file comments. Wait, let me search for them on archive.org.
Fixed.
The notion of "simplicity" can be completely arbitrary, though. It's enough that there are only finitely many hypotheses simpler than the best hypothesis (assuming there's such a thing as a best hypothesis). So as you eliminate hypotheses incompatible with the data, at some point you'll have eliminated all simpler hypotheses, and the simplest hypothesis left will also be the best hypothesis. If simpler hypotheses are also more likely to be correct, you get there faster, but it's not required.
It doesn't have to be arbitrary. The Kolmogorov/Solomonoff definition, that the simpler models are the shortest programs that can generate what we know about a domain, measured in bits, have a solid mathematical foundation, based on information entropy and even thermodynamic entropy if you cross over into computer hardware.
Entropy is a metric targeted in LLM training which is likely why LLMs are overfitting less and less as they improve and why they subjectively seem to obey Occam's razor.
There's lesson for software engineering in general behind that kind of simplicity. https://benoitessiambre.com/entropy.html
How is the Kolmogorov definition not arbitrary? It depends on an arbitrary selection of the language you use to write the program.
[dead]
There is some art to it, especially in model architecture choices.
But your smarter data scientists will try to get the best information density on test data to approximate generalization. MDL, as championed here, or AIC/BIC if working with more established and acceptable methods.
Except for the fact that eventually we are all dead. So it is kind of important to get there faster.
For complicated hypotheses, where complicated is defined appropriately, it takes many many examples to realize that it was a wrong hypothesis all along. There lies the rub.
For a particular instance of a learning problem we can't tell much, however using a Occams razor over many instances, one would be correct more often than not. Provided, of course, the PAC assumptions are true or they are not very far from being true.
How far is not very far ? That gets very hairy to quantify.
To be clear, I was doing a reductio ad absurdum. PAC is precisely the kind of theoretical framework that concerns itself with asymptotic long-run behavior. An appropriate definition of simplicity certainly gets you to the goal much faster than the worst-case bound. For example, you could order hypotheses from most to less likely. But that would render the claim that simpler hypotheses are more likely rather tautological.
More typical definitions of simplicity, e.g. using the bit length of some kind of natural encoding, aren't guaranteed to offer any special advantages in terms of likelihood. So if you have prior knowledge that a particular hypothesis is more likely than another, but the less likely one is simpler in an intuitive sense, you shouldn't let that override you, but still prefer the more likely option. (And you don't need to take the circuitous route of coming up with a new encoding where the more likely hypothesis has a shorter bit length, either.)
PAC isn't asymptotic in general. It gives finite estimates for finite sizes of training data.
If you say, and I think you are indeed saying so, that PAC is ridiculously pessimistic, I would be in violently agreement with you. That's one reason why for practical training data sizes and practical (infinite version space) hypothesis classes PAC gives bound such as -- probability of error is less than 41K. This isn't exactly incorrect but not very useful or informative.
A far more useful formulation is PAC-Bayesian where you get error bounds less than 1 guaranteed and usually less than 0.5 on reasonable sized training data sets.
You choice of basis matters, eg, wavelet versus sinusoid.
That's not true. It's pretty clear that she meant "do something you knew they were going to say no to and now you are trying to get away with something."
https://youtu.be/wHdHCoeUbU4?t=861s
> So I want to tell something to all the young people here on many many occasions you'll find it is much easier to apologize than it is to get permission. You do it then when somebody comes after you and say are you supposed to do that, "oh gee I didn't know I wasn't supposed to do that" ... so just remember it's frequently much easier to apologize than it is to get permission do it
She goes on further, explaining how to deceive your superiors to manipulate them to get what you want.
But I still don’t think that means eat all the cookies in the cookie jar and then apologize after because nobody would have given permission. That’s still about doing what you believe to be right. She even frames the fallout as “where you supposed to do that?” and not “you shouldn’t have done that”.
Analogies only go so far, but I think eating the cookies in the cookie jar is the wrong one, because you're only feeding yourself with that one. If you're stealing flour from the baker to make a surprise cake for the whole troupe to enjoy, then you can apologize later, instead of asking first and ruining the surprise.
Damn no wonder she got a supercomputer named after her
>It’s just like the Hopper quote.
Not sure about Hopper, as I recall biographers of Lawrence of Arabia certainly made it seem like he was using the fog of war to do things he knew his superiors may object to.
Regardless, even if its misinterpreted it still has a kernal of truth and separate utility than your version, that is: the people in the field closest to the action have an operational awareness that may result in better decisions in times of urgency.
“You should prefer it, because it’s simple” is just restating Occam’s razor, not giving any explanation. “The simplest is more likely to be accurate” is a much better interpretation than yours.
I think the most accessible example of Occam’s razor is fitting a line to some points; you can always use a high enough order polynomial to fit the seen points exactly, but a straight line is likely closer to representing the underlying distribution.
simpler is not the right word either. it's the one that makes the least assumptions, not the simplest. The simplest would be "god did it" pretty much everytime.
> The simplest would be "god did it" pretty much everytime.
An out-of universe entity, that is by definition too large to be understandable to anything in universe, is a lot, but not simple. Are you sure, you are not confusing easy and simple?
It is simple. There aren't any explanations needed because "god is omnipotent" already covers everything.
But its simplicity involves making MASSIVE assumptions.
And the fact that Occam's razor explicitly says "fewest assumptions" and not "simple" is unchanged.
Quantum mechanics is complicated. Assuming electrons orbit nuclei is simple. But quantum mechanics is backed by countless experiments, so needs less assumptions. If you have to choose between quantum mechanics' description of the atom, or the old, outdated, knowledge of an atom, Occam's Razor would say to choose quantum mechanics.
By that measure Quantum mechanics as explanation is simple, because "it's according to Quantum mechanics" already covers everything.
However that is not what I think of when I hear "complexity of an explanation", I do not think of the complexity of the statement referring to the explanation, but of the complexity of the explanation itself, which is only referred to by the statement.
Also I disagree with using the number of experiments needed to arrive at the conclusion as a measure for complexity. By that measure "things fall down" is complicated, when you have done a lot of experiments. I would rather take the number of "steps" needed to derive it from first principles once you know it. But also that sounds like what you said in your last sentence, so you are contradicting yourself?
Marcus Hutter formalized this in his AIXI work.
And sadly, in academia, complexity (opposite of Occam's razor) is what gets you published.
It's partly because of disagreement about what "complexity" and "simplicity" actually are. Many simple statements are in fact backed by massively complex, unstated assumptions. In attempting to deal with assumptions, scientists necessarily end up having to deal with the complexity involved in that. But the problem is if you don't engage with that, how do you know what is really more conformant with Occam's razor, as opposed to just satisfying what is readily expressed in common language?
Likewise in software development a C developer will say abstractions are not simple, a java programmer will argue that dealing with low levels details is not simple. They're both kinda right but will resort to framings which back their world view.
Even tech giants are putting out articles seemingly fully written by Claude.
The animated graphic labeled "Occam's razor, formalized" is bizarre. Is that really visualizing "Occam's razor, formalized"?
Ah, over-the-top larger-than-life LLM-isms, they are really funny when you see them in a company blog, but they are vomitive when it's your coworker copy-pasting it and insisting you on reading it.
I was expecting Occam wearing a suit.
Animated SVG of a pelican shaving with Occam's razor?
Time to first detected slop in this article is <1s. Claudisms per paragraph is also very high.
Is it too much to ask from people to read their own article anymore?
If anyone read this at all, they would have had the ick, and would have fired off a prompt to get rid of the most popular AI slop tells...
What I really dislike is having to edit my own non-LLM assisted writing to make sure I'm not accidentally confused with AI.
I caught myself writing "And that matters because..." in a HN comment but had to edit myself. Also miss uising emdashes.
These models are trained on human language, which belongs to us, we shouldn’t surrender it to them. Keep the em-dashes. IMO don’t overuse negative parallelisms though, they were always bad and lazy.
An arms race on style would be interesting. Essentially a real life GAN.
For the first time ever in my life, I consider the fact that English is not my first language and that I never had any formal education in it as an advantage.
Nobody is going to consider my weird and barbarian prose with the well polished product of a SOTA model.
English as a 2nd language speakers often have better grammar than natives in my experience! I wouldn't be surprised if the average native english speaker doesn't know the different between there and their.
If you saw what goes on in middle school classrooms of native English speakers, I’d think you’d be in for a surprise…
Do tell!
Their, their, mate. You mispelled they're as there in your comment they're
It is even wrong about things
The validation can be done like that but it is better to do a cross-fold validation or selecting a random subset. These are equally valid. Keeping the validation like a hold-out set can end up creating biases if the statistics don't match. Honestly, this is true about the test set too. Though I've found that most people really don't think about this much. It's mundane, boring, hard to do, but deceptively important."The more your listener already knows, the shorter the message you need to send. An expert ML engineer needs only a few sentences; a newcomer needs the whole manual."
Literally from the first words on the page. 1st sentence passed barely. 2nd sentence ending "a newcomer needs the whole manual" and it's Claude.
Nobody wants to work anymore!
[dead]
> Why don't machine learning research agents overfit?
they do.
Why is this being published as a blog post and not as a peer-reviewed submission? If it's going to be a blog post, why isn't there a corresponding scientific version for me to look at?
Someone else already found it. I don't understand why the link isn't in the blog post. https://arxiv.org/abs/2606.11045
Use of claude for writing it should be disclosed.
arXiv link: https://arxiv.org/abs/2606.11045
The methodology is partially based on https://www.offconvex.org/2021/04/07/ripvanwinkle/ , for those thinking this sounded familiar.
If anything, the latest generation of AI models, Astra and Fable, are prime example of overfitting—whereas benchmarks suggest they’re AGI-tier, users (including myself) report the same old gaslighting, hallucination, context rot, cheating, incomprehensibility patterns as with prior models, sometimes even more pronounced.
Fable and Opus 5, I suspect, will become textbook examples of RL collapse.
[dead]
Wherein Claude gives an honest assessment that it genuinely does not overfit. I also had Grok telling me that it isn't quantized.
Do the submitters really not notice that this is AI slop? Do they like this? It is a complete pain to read.
Do you know what the scaling law actually is? Overfit everything as much as you can.
They tend not to overfit ... when there are way more data points than parameters.
Or when there are way more parameters than data points
https://en.wikipedia.org/wiki/Double_descent
What fits (into few tokens) doesn't overfit.
Compression in this modern day and age is so slop.
Yes, I'm familiar with keystone results such as Solomonoff induction. It's a direct counterexample to compression - your intensional algorithm can completely outrun reality. I can literally specify a huge mega-algorithm that just searches over all possible Turing machines and evaluates them, and it's an optimal compressor. It's completely vacuous though. You can always hide the "heavy work" in your mappings and descriptions. It's ironic that a kolomogorov complexity minimizer is so loaded that it's vacuous.
This is pretty much why I roll my eyes at this point at all the compression is intelligence memes.
I wonder when intervention and causality will hit the mainstream. These tools were designed specifically to counteract purely predictive theories. But your average compression dude will hold tight to their paradigms and slogans, not realize their internal contradictions (that their own field has brought up), and then whenever a new paradigm suddenly becomes visible and mainstream, they'll latch onto that. It's not principled at all.
And to be clear - I do think intelligence is some amount of compression, and I am well aware of formal results such as the arithmetic decoding theoretical and empricial result. Just annoyed. It's literally no different than the whole Bayesianism meme. If you're not actually practicing that type of intelligence as a basis, then you don't get to go around beating the drum about how it's the ultimate reality. You're just spouting dogma to feel like part of an in-group.
I think that the ratio of work done by priors and search is the interesting question at this point. We aren't quite at the point where reading off the most likely hypothesis decompressed from a transformer is sufficient, but it's a lot closer than I originally suspected when we were ~solving chess. I think AlphaGo was kind of the watershed moment that prior-guided search was so much better than either alone.
I think the adversarial policies against Go AIs directly show the gap between intelligence and compression/priors.
> I can literally specify a huge mega-algorithm that just searches over all possible Turing machines and evaluates them
Then why don't you ? and did you mean all or those that halt ? I presume you have a way of separating those.
I’m guessing your downvotes were for tone? You’re correct though regarding Solomonoff induction, as the choice of reference universal partial recursive function gives drastically different results for predictions based on finite data (even with access to a halting oracle). Asymptotically, any choice eventually converges to the same predictions, but that’s no help when there are infinitely many choices for U and no obvious natural prior over universal functions. And I don’t find the argument that our natural environment “implements some choice of U” particularly convincing. There’s definitely an open mystery there.
No point in reading anything AI related anymore. It's all slop.
We need to retvrn to rss feeds
Is there a study demonstrating that AI-assisted articles are less substantive or is it purely a kneejerk bias towards humanslop?
what would returning to rss feeds achieve?
The pattern that RSS feeds best supports is subscribing to individual creators who produce things you value. Presumably, if you don't like AI word-vomit, you wouldn't subscribe to a blog that uses it heavily. This is in contrast to the social-media state of affairs (like HN, reddit, twitter, etc), wherein much of the posts you read are recommended to you by either a person or an algorithm, regardless of whether you believe that author will write something you like.
A shorter way of writing this is that RSS promotes writing stuff good enough to get into somebody's "favorite authors" list, and social media instead incentivizes "lots of people will share this."
Indeed. The other problem with the social-media way is that it gives outsized weight to headlines and soundbites.
I actually couldn't have said it better. I'm brain rotted from LLM usage
Force the clankers to support rss!
[flagged]
[flagged]
> Machine learning, at its core, is about generalization, not memorization.
Well they memorize the patterns.
memorization doesnt mean rote learning.
I don't take issue with that. Attempting memorized pattern generalization through holdout / validation strategies is a big part of ML that you would not typically see with econometrics / psychometrics / possibly sabermetrics / most other -metrics. Philosophically the explain versus predict divide. https://www.stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf
That's a bit pedantic no. Memorization in ML refers to the model having the wrong level of capacity such that it's too hard to optimise it such that it doesn't memorize the _training examples_ themselves.
Does '110001111000000011111111111' contain n>1∧∀d(d|n→(d=1∨d=n)) as well as infinite number of other generalizations?