Co-author here - first paper from my postdoc accepted to NeurIPS main conference.
We built a way to sample diverse ideas in machine learning - ideas that make scientific sense, but that no existing research community is positioned to propose.
In short, the way it works is like this:
1. Take 16,068 LLM papers (NeurIPS, ICLR, ICML, ACL, EMNLP, 2017 to 2025) are broken into 273 "idea atoms". Every paper, and every candidate idea, becomes a set of atoms.
2. Train a Coherence Model: a 42.8M-parameter transformer trained on papers as sequences of atoms. Basically a language model whose tokens are ideas.
3. Train a Cognitive-Availability Model: a dual encoder trained on the publication histories of 39,791 researchers, which predicts where each person could plausibly go next.
4. Use those models to score sampled tuples using reciprocal rank fusion (you want ideas that go together well, but that are not cognitively available to anyone).
We tested it in a bunch of ways. One interesting one was that we took all the papers at the end of 2024, then checked which high-scoring tuples showed up in 2025 papers. It worked well - one examples is an ICLR 2025 oral, goes from rank 1,489 with coherence alone to the top 25 of 1.14M samples.
A funny one: if you ask Claude to come up with an ML research idea, it is often about interpretability. If you then run an autoresearch pipeline with some A100s on the idea, the result tends to be interpretability 8/10 times. Our Alien Sampling method stayed at 0 interpretability ideas even after letting claude cook on them.
We also paid 50 people with graduate degrees in CS/AI to judge the research reports and found that Alien’s reports had the best average quality rank and produced 3 of the top 5 reports.
Oh and then we scaled it way the hell up to a couple million papers and 10,000,000's of concept elements and generated 25,000 new research proposals that you can search over here: https://lacuna.tiptreesystems.com/explore?view=hypotheses
I was really impressed by the scale of the lacuna tool you created, but then I dug into the niche topics (steganography, covert communication) that constitute my PhD research area and my expectations were tempered.
There’s a lot of noise in the research literature. Lots of papers get published that really don’t contribute much of anything to the field, but the existence of a bunch of papers often sends a signal that some concept is important and needs to be studied.
I find that often a big cluster of papers around a particular topic, none of which seems to move the needle much, is actually an indication that we need a new approach and writing more and more incremental papers just digs deeper into a local minimum.
We haven't done concrete measurements yet, but my intuition is that the quality of Lacuna proposals is a function of density of topics / research directions. Lacuna is most dense around AI topics; for neighboring ones (e.g. bio, physics), I'd join you in tempering expectations.
> I find that often a big cluster of papers around a particular topic, none of which seems to move the needle much, is actually an indication that we need a new approach and writing more and more incremental papers just digs deeper into a local minimum.
Yes, I think this can work. We use a similar idea to categorize what papers have been influential or foundational -- look at the ones that have changed the trajectory (in time) of a research direction significantly.
> big cluster of papers around a particular topic, none of which seems to move the needle much, is actually an indication that we need a new approach
I feel like when it comes to "Alien Spaces" or new approaches, there's no other way than to babble a lot of stuff that doesn't move the needle much but then leads through the implicit to language and communication mechanisms to a spike out of the local minimum. Didn't the ancient guard have hundreds more notes than ended up in their useful collections?
Nowadays you have so much research that nobody really cares to just ramble with attempts and approaches ... and instead skims the work that others certainly tried already ... distancing themselves from the possibility of new approaches ...
Thanks for checking it out! We're working on a collaborative discussion / editing feature mediated by Althea (our AI agent). That should provide a way for domain experts like yourself to contribute to improving it. Hopefully we'll have that soon!
The problem of course, is how to measure the "movement of the needle." It might help to weight papers and research in the corpus based on some notion of "significance". One metric I know people have used for this is based on citations, on the reasonable premise that influential work gets cited much more. IIRC the original PageRank paper was inspired by work on citation analysis as well.
We actually have a high-risk research idea for how to do this. It involves looking at the clustering as a vocabulary that compresses the concept space, and (one form) of significance as how much your paper / addition of concepts changes the clustering/vocabulary. I.e., if you change the optimal compression of the space, that's probably important.
Oooh, very interesting! I'm way out of my depth here but is the intuition that new concepts that by a) connecting other concepts across that space, or b) introducing entirely new dimensions, or c) some mix of both, change the distribution / clustering of concepts within that space, which impacts how well it compresses, which could be a measure of significance based on the change in compression?
It's like information theory for abstract concepts.
Yea pretty much. The picture I have in my head is of those arcade machines with the sort of snow shovel moving back and forth that sometimes pushes quarters off a shelf (called a coin pusher https://en.wikipedia.org/wiki/Coin_pusher).
You try to figure out where there’s lots of coins on the edge and drop your coin in to get it to push more coins out.
So some fields are ready to get pushed, and maybe density or patterns in concept space can show you that
> It will never, ever, come up with Velcro. If a concept does not exist on the Internet, or the corpus of training material, LLM’s will not invent it.
So this guy made up an imaginary test, provided a completely unjustified result to it- which casually confirms his ideas- and he's so proud he was right! What a good test he invented!
I think if an LLM had read any discussions on how best to remove burrs from pet coats then it might eventually get to Velcro - I believe the inventor of Velcro got the idea from burrs?
> "I believe the inventor of Velcro got the idea from burrs?"
I think there's a provocative human-intelligence analog to be found there: "humans will never discover anything that doesn't already exist somewhere in nature. We can only recombine concepts we've seen—never invent truly new ones".
It's said that mathematics is "unreasonably effective" at describing the natural world. But what about the other way around? What if mathematics is unreasonably ineffective at escaping the convex-hull of concepts and patterns found in nature?
I'm working on a piece inspired in large part by William Thurston's "On proof and progress in mathematics"[1]. Should publish it next week.
Part of why math is effective is that its extraordinarily compressible. And you can take those ideas and apply them elsewhere (composition works well). The natural world has a lot of detail and is therefore hard to compress. That's why it's a good source of new ideas. Consider the relation between the McCulloch–Pitts neuron (MCP lol)[2], developed to model biological spiking neural networks, and an inspiration for modern artificial neural networks.
So I though about "we can only recombine concepts that exist in nature - never invent truly new ones".
The best counterexample I could come up with was lasers. Yeah, all the pieces are there in quantum mechanics, but so far as I know, nothing in nature lases.
Semiconductor chips. Again, yes, all the physics is there, but you're going to be reaching pretty far to find something in nature that's like a CPU chip.
Dyson spheres.
Interrupt vector tables.
Riemannian geometry.
Databases.
I'm sure that people can poke holes in some of these (with varying degrees of far-fetchedness as to how directly the same concept is in nature). But I think there are enough counterexamples, where nature's "prior art" is indirect enough, to make the absolute version of the statement false.
"Humans rarely discover something that doesn't already exist somewhere in nature"? Yeah, that's probably true.
> What if mathematics is unreasonably ineffective at escaping the convex-hull of concepts and patterns found in nature?
Interesting. My intuition here is quite the opposite, that math is vastly overpowered for physics. I don't think we invest a lot of work in figuring out math that isn't useful to describe the natural world.
Or if it ran a couple million soft-body simulations with the goal of creating a reusable temporary link between certain kinds of materials, I think it would probably get there. Just another argument from lack of imagination on the part of the author.
ya - I think my intuitions around velcro are firmly intertwined with my meat-space actuators and perceptual equipment.
In other words, I experience high-resolution multi-modal qualia all day and it's really noisy and I think a lot of new ideas and concepts come from obsessively exploring a small subset of those details. Maybe that's just me posthoc rationalizing my phd, though.
That's a bold claim. It was almost certainly true for models in 2023, when it was written. I think that the recent spate of mathematical discoveries at least significantly narrow the implications for even current models (one can argue that they are only re-mixing existing mathematical concepts to solve problems that humans initially generated, but in my opinion, at some point you are really narrowing what it means for an idea to be "novel"). And I would seriously consider taking odds against it for models within the next 3 years (since that's how long it's been since it was published).
We are basically already at the point where if what current frontier models can do isn't truly "novel", then the vast majority of humans have never had a novel idea in their lives. My three years from now guess is that if whatever it is that frontier models can do at that point isn't "novel", then no human ever will have had a novel idea.
LLMs assign non-zero probability to every output (unless you you sampling modifiers like top-k), so if you sampled enough times the model would definitely come up with velcro. It's the same as the proverbial monkey with a typewriter.
The real question is whether it would happen in a practical timeframe.
> The real question is whether it would happen in a practical timeframe.
How are we defining a practical timeframe? How many billions of man-years did it take for a human to come up with velcro? In a world without velcro I could easily imagine an llm proposing the idea long before a pure human team.
One part of the explanation is that we focus primarily on AI research, and a substantial chunk of physicists gravitate (heh) towards AI research, so we end up capturing more of their work.
Another part of the explanation could be that (AI +) physics papers are just easier to access, so we end up processing more of those.
Great question - the "universe" of venues we address is centered on peer-reviewed machine learning like ICML, ICLR, NeurIPS, COLM because (a) we have expertise in this area and so can therefore judge the quality and (b) it's all open access.
Therefore, the further you get away from the topic of ML, the less dense our coverage. Frankly, it's a tough to editorial choice to determine which research directions and therefore which proposals to include. ML is quite multi-disciplinary. And people who have a few papers in ML but many in (e.g.) medicine get upset when their profile is sparse. But we can't possibly produce good proposals in medicine due to its tendency to be closed-access.
Hmm, interesting. Yea I think ML is a much smaller field, too. ArXiv + all the conference / applied ml stuff is ~150-250k papers per year. PubMed is 1.5-1.7 million new records per year, and like 40,000,000 total records, whereas in ML we only had to process a couple million. Plus, pretty much all of ML is open-access, whereas medical is ~50% closed.
If somebody gave us a $10-$20m grant for compute, and a few (dozen) helpful doctors showed up to help us QA / figure out how to approach it, it could probably be done.
Thanks! I wish we could take full credit for the paper name, but it's based on a line of work from James Evans (Chicago-based computational social scientist) investigating "Alien artificial intelligences" https://x.com/profjamesevans/highlights
- Are there any issues with vocabulary drift? I know the LLM field in 2017 is not super different, but is there no atoms from 2017 that ended up being irrelevant to 2026 and/or which are just not possible to combine with more recent atoms?
- What does the cost of this project look like? (Ignoring the research iterations, just the costs relevant to the final product).
Yes, vocab drift matters a lot. Most of the pipeline is basically compressing out any of the weird vocab/language stuff, so it doesn't have numbers and proper nouns and is in a sufficiently abstract concept space to cluster nicely. We use a density-based clustering method (HDBSCAN) which when you set min_samples=1 works as an online clustering (i.e., incrementally adding new things is the same as if you re-ran the clustering in batch mode).
Oof - it's not like goodfire - we have quantitative results and a clearly defined method!
But yes, we might be bad at marketing. Internally the term we use more often is "concept elements".
The intuition is that if you take a complex treatment of a topic (like an academic paper), you should be able to break it down into some more abstract self-standing ideas. Then, critically, these units are composable and easily embeddable (i.e. they don't have proper names and numbers).
Here's a couple examples of these idea atoms pulled from our paper https://lacuna.tiptreesystems.com/work/the-alien-space-of-sc...: "Scientific ideation often conflates the logical coherence of a research direction with its likelihood of being discovered by the current community. This oversight makes it difficult to identify research paths that are scientifically sound yet unlikely to be discovered due to the existing distribution of expertise and collaboration networks."
Another example: "A generative model estimates the plausibility of concept combinations by treating conceptual units as tokens and calculating their length-normalized log-likelihood. By training on random orderings of these units, the model learns to identify which components are likely to co-occur in a valid research contribution regardless of their sequence."
> Oof - it's not like goodfire - we have quantitative results and a clearly defined method!
Goodfire is a discovery from research, an ahaha moment.
"concept elements" would have been a better descriptor in both cases. The V for the cats and the cars, and for your idea.
> 16,068 LLM ... The Coherence Model is a 42.8M-parameter transformer trained
My question is this: was this the smallest model you could train on the 16k papers? Isn't there an optimal small size to identify everything that is common?
Wouldn't it make more sense to use such a model to strip OUT the commonality? Get rid of everything where the "existing distribution of expertise and collaboration networks" are already optimal - and what's left is the novelty?
Ultimately I would be curious to see if your existing "atoms" happen to follow Zipfs law...
> I made it to the first figure in the paper before I raised an eyebrow at this term.
I came up with that name, and fair eyebrow. Think of an "atom" as a unit of a working vocabulary (a "token"), not a Platonic form. It's a way of compressing the corpus (paraphrases of the same idea across papers get merged into one symbol, a bit like a tokenizer's vocabulary).
We also talk about some limitations of this framing in page 11 (towards the top).
Congratulations, this is a really cool approach and result. I lead a research team and I'm very interested in tooling for improved collaboration, especially between human and agent researchers.
Do you have any evidence that this approach scales to cross-domain research?
I'd say science is somewhat fractal in this sense. A research area contains topics, those topics have sub-topics, those sub-topics have competing lines that "talk" to each other, etc. So if it works in core ML that means it's working for RL and deep learning and computer vision and NLP -- which is kind of cross-domain.
That said - we believe the method does need to have sufficient data to model the conceptual density. So it'll not work in closed-science areas.
One other thing I will share: we have been working on a way to enable verified academics with sufficient topical domain expertise to engage in small group discussions that update these documents. You can think of it like Althea (our agent) being an area chair and moderating a discussion to determine how to update pages. The reason we think this will work is that you can have an anonymous discussion with the AI system (and be blunt), but we can weigh your expertise when updating the documents.
This sounds really useful. That concept of weighing contributions from different sources (especially in relation to their expertise in the domain of discussion) is very interesting. Congrats again on the paper and the great response!
The idea is to allocate academic reviewer time based on the estimated quality of a submitted paper. But critically, you should maintain uncertainty bounds in your estimate. It's kind of like an UCB (upper confidence bound) bandit approach but inspired by cascaded speech translation (where you really should not throw away information at any point in your pipeline).
I did that with Claude research. It collected tons of studies from two or more dozen platforms and crosschecked whether my hypothesis was already considered anywhere and it created this gigantic set of data where it showed me all kinds of fields where the many parts of my hypothesis--that Claude rephrased over and over based on contiguous expansions and elaborations in layman's terms--were already or actively researched and how much of it was proved in theory and/or practice. It turns out the specific connections I made were unique and the underlying theory existed, much of it peer-reviewed and in application. I did not check whether some of those applications, peer-reviews and several thousands of studies were challenged but people were applying the science and there was demand for more and lots of work to be done, requiring specialists from diverse fields.
I just quickly checked the 25,000 new research proposals and found a few that indeed touched my hypothesis and the underlying theories but no direct connection.
Honestly, as so often found in criticism of academia, your approach seems very technical and alienated--pun intended--from the people, their lives and interactions with each other and the world, including tech, and it thus seems that your objective is "estranged" from the very thing you are trying to find.
If Claude's and other LLMs surprise when they find rare occurrences of ideation and inferential trains of thought is worth anything, then that it's an appreciation of training sets lack of humanity and their individual thinking.
Here's some idea I likely will never get to: "linguistic conceptualization of psycho-social reverse pareidolia" or "when, why and how humans were taught to not identify meaningful connections". The "doesn't look like anything to me" equivalent so imperative to the narrative of the TV Show Westworld, created by Jonathan Nolan
and Lisa Joy.
As much of a merry-go-round, cough, it is, science philosophy and personal narratives and perspectives is where scientific novelty resides, so getting LLMs to extract more out of the cattle, sorry, people, might not be such a bad idea, after all.
> " ... a ligand that binds tightly and stays bound a long time should produce a strong intracellular signal, a weak binder a faint one."
When a ligand binds to the ligand binding pocket, the contact points it makes with the proteins inside (called "cofactors") and the strength of each contact point (measured in van der Waals force) modulate the distinct transcriptional effect.
This is called "conformation": the shape it takes in the ligand binding pocket.
So it is not necessarily residence time and overall binding affinity that modulate transcriptional effect.
A very good example of this is the androgen Trestolone/MENT, which is an order of magnitude more potent than Testosterone but displays similar relative binding affinity for the androgen receptor:
Thanks for digging into one!
I had a project back on this back in my Probabilistic Graphical Model days with Simon Lacost-Julien (I believe this was my worst-ever grade in a course)... so I’ll defer to you on the mechanism.
I do know you're right that affinity/residence time and efficacy are separate axes. Bio is one of our sparsest regions (due to closed access papers, and our focus on core ML), so proposals there are more likely to have issues like this.
We've got some ideas on how to improve these issues, but I really appreciate you helping us by pointing this one out!
Monotropism and neurodivergence make strange bedfellows.
I believe that I may be AuDHD, not just because it's trendy right now, but because I immediately see the most likely solution(s) to a problem (as if the branches of the search space fall away instantly), while simultaneously seeing more branches than most people seem to (so that I go down rabbit holes and get distracted).
I might compare it to having nearsightedness and farsightedness simultaneously, so that I can drive the ball hundreds of yards but my short game stinks because I'm thinking the trajectory of the putt is a random walk due to the grass. It used to take a lot of effort to step back and not overthink everything, until I had to silence my inner monologue completely to salvage my mental health from total annihilation in a neurotypical world that seems to be oblivious to nothing yet take everything at face value and somehow be productive.
Another analogy might be having depression and anxiety simultaneously, which results in something akin to the opposite of normal, even though from outside it appears to be that way.
Anyway, this paper reminds me of an AI version of that: shining a light on distant associations in excruciating detail. Which is pretty much my favorite thing ever.
I love this. My personal completely unsupported theory is we’re cognitively capable of conceiving of less than 1% of what’s true/accurate about the universe, and cognitively capable of understanding 1% of that 1%
- art style
- a doctor might see indications of a health issue that Lisa has
- what it says about the society of that time
- geology of the landscape
- contemporary production of the color pigments
- what the picture does with you when you look at it
- and so on
Good Sci-Fi novels bring along some of these - aliens that have access to different concepts of reality, e.g. the dark forest hypothesis. The hard part is that you actually have to come up with that understanding for writing it, instead of just hand-waving it.
I think the way computers developed is kind of an interesting example of this in the real world. Babbage basically realized the idea of a modern computer, but tried to build a too-complicated 'supercomputer' (40 decimal digit precision!) because he didn't know how incredibly useful a much simpler, more limited computer could be. We had to wait another 75 years and spend an incredible amount of money to develop relay and vacuum-tube supercomputers, before realizing that the idea could be scaled down, and something like a PDP-8 / PLC / microcontroller was incredibly useful for industrial control and, with sufficient cleverness, even do a lot of scientific computing.
I think if you went back in time you could pay some power loom engineers to build you a pretty decent 4-8 bit microcontroller operating at 1-5 Hz (comparable to power looms at the time) and computer-controlled textiles would have paid for the computer revolution 75-100 years ahead of time.
Even after Babbage's failure to introduce us to the information age, there were a fair amount of competing approaches to how to build a computer. In a different world, a non-Von-Neumann architecture was selected.
Something like the original computers, Eniac etc, were actually generally useful. It was useful to very large enterprises that previously relied on an army of human "computers" but it was useful for them so it stimulated refinements like the PDP-8.
I think if you went back in time you could pay some power loom engineers to build you a pretty decent 4-8 bit microcontroller operating at 1-5 Hz (comparable to power looms at the time) and computer-controlled textiles would have paid for the computer revolution 75-100 years ahead of time.
Would it? You have to remember that the further you go back in time, the cheaper labor is relative to machinery. In the 19th century, you could pay a person for a multitude of customizations and so the market for custom cloth might not have been what you imagined.
There was an incredible amount of investment into high-speed programmable power looms, and there were a number of innovations (later rediscovered by computer architects) targeted at 'card-saving techniques' to reduce the labor required to card-punch the designs, and to reduce the down-time of the looms. 'Double Data Rate' card readers (with a design interleaved on two chains running at half frequency), SIMD-style one-reader/many-loom setups, 'cross-border' looms that switched between two separate card loops just like a 'branch' instruction, machines to copy chains of punch cards.
What you're saying might have been true in the 18th century or earlier, but things were highly mechanized by the 19th century, and there was a lot of motivation to keep the big expensive looms highly utilized and amortize investment in punch card sets across lots of production runs.
There's a plausible version where no matter what weird detours and dead-ends a human society takes on the tech tree, as long as there continue to be wars every few generations, the needed progress keeps getting made and the dead-ends or even just over-reaches get pruned afterward.
You can go down the wrong rabbit holes for a long while in peace-time (e.g. string theory), but war-time is a crucible for correct ideas.
I agree with you, though my own motivation is not centered on war, but (a long tail of) existential risks facing humanity, mostly centered around collective action problems.
Very cool - the Alien Science work actually had a precursor in image generation. Cultural Alien Sampler: Open-ended art generation balancing originality and coherence https://arxiv.org/abs/2510.20849
I guess it makes sense that art folks like creativity!
I'm very entertained by the hypothetical game descriptions hahah. Not sure how they would be as actual games though. They all feel like Kilgore Trout novel descriptions, which Vonnegut used as an outlet for all his sci-fi ideas he didn't want to actually turn into books.
A bit of a tangent, regarding innovation. It's interesting to think about counterfactuals in history. There are questions like what if solar panels became widespread 100 years earlier? Could we have avoided global warming and all the wars over coal and oil? (Not possible because lack of purity technology in materials and understanding of matter.) But are there other things where some easy discovery was waiting just around the corner but it wasn't discovered for some reason? Or something was discovered in one society but not in another for a long time, from similar starting conditions. Washing hands certainly is something that one could go back with, probably thousands of years, to bring immediate benefits. Some cultures have done it but some have absolutely refused it for a long time.
And then applying that kind of thinking to today. Looking back to 2026 from 2126, what would a historian facepalm over? Or what is already being done in one field or social scene but not in another, that would be greatly beneficial? The classic recent example of success is applying airline pilots' checklists to surgery. Airline pilots probably are a little bit like aliens, from surgeon point of view. How many people had already tried similar things, but failed with the institutions and people's inability to receive innovation? Maybe with AI tooling we can facilitate better innovation diffusion. It's not without its dangers of course.
Not at all a tangent. We run a backtest in section 5.2 with pre-2025 scores to recover 2025 directions.
We rank all 1.14M three-atom combinations and check them against the 2,424 that appeared in 2025 papers. Using coherence alone, the top 10k (under 1% of the space) recovers 31.4% of them, about 36x chance. Adding the availability score trades some of that for more genuinely new combinations, ones whose atoms had never appeared together before. Judge Decoding (ICLR 2025 oral) moves from rank ~1,489 to rank 1 of 1.14M.
To your point about diffusion, I agree that just having a good idea is not enough. Politics / institutional inertia is the primary determinant of policy.
After crawling out of the primordial ooze we send explorers in all directions. (Usually groups) As they get further from the source they spread out thinner. Someone will have to find the gaps eventually and they might even find things close to the source. If you add dimensions of exploration they might travel far leaving many large gaps back home.
I love the effort trying to squeeze wine from a mountain. I wonder if the generalisation also remedies unpopular topics as they are now reduced to simple data points. It should be fine to map forbidden grounds as long as you don't go there.
I'm on the other hand reminded of a certain Lem sub-story from A Perfect Vacuum, which was about geniuses whose discoveries were too divorced from the mainstream to be useful / worth following upon.
I've filtered the generative proposals for biology [0] because that's a field I'm a bit familiar with. I think it's a bunch of nonsense, tho the confident language really gaslights me.
Yea, so the way this was built causes regions of density (core machine learning topics) to be good and sparse areas like biology to be worse. There are some areas of intersection like Medical Coding, which anecdotally Joseph Paul Cohen (was a mentor of mine and a postdoc with Yoshua Bengio back in the day) reviewed while we were at COLM and he thought was good enough to share around his research lab. All that said, research proposals will be hit or miss.
The reason that biology is sparse is because we are processing full papers from open access venues. Machine learning papers have a long history of open access publication, whereas other areas still have a lot of concepts locked up. I suspect our method would probably not work very well if you only provided abstracts (like AI2 ScholarQA).
Our clusters and therefore our sampling of new ideas is based on density in concept space, and a lot of the key concepts don't end up in the abstract.
Go open science! All that said, the right partnership with a commercial publisher would probably make this work well in that regime.
Is there a way go finetune this and run it on my local academic pdf library (1-2k papers) ? I'd appreciate having a slightly crazy creative coach who can help me brainstorm new ideas!
edit: you can also certainly point a code agent at this paper and your library and have something interesting in a few hours. Or better yet, give your agent the Lacuna MCP and let it rip on the problem https://lacuna.tiptreesystems.com/mcp
The academic paper cost a few thousand bucks, max $10k in compute.
Lacuna is far more expensive and ~100x larger. If you knew what I know and started building it today (LLMs got cheaper).
Paper summaries cost ~$50k, concept extraction cost ~$25k, author profiles cost $25k, research proposals cost another $50k. So call it $150k for model calls.
It's also an online / updating daily so that costs too, plus infra, plus "spillage".
Co-author here - first paper from my postdoc accepted to NeurIPS main conference.
We built a way to sample diverse ideas in machine learning - ideas that make scientific sense, but that no existing research community is positioned to propose.
In short, the way it works is like this:
1. Take 16,068 LLM papers (NeurIPS, ICLR, ICML, ACL, EMNLP, 2017 to 2025) are broken into 273 "idea atoms". Every paper, and every candidate idea, becomes a set of atoms.
2. Train a Coherence Model: a 42.8M-parameter transformer trained on papers as sequences of atoms. Basically a language model whose tokens are ideas.
3. Train a Cognitive-Availability Model: a dual encoder trained on the publication histories of 39,791 researchers, which predicts where each person could plausibly go next.
4. Use those models to score sampled tuples using reciprocal rank fusion (you want ideas that go together well, but that are not cognitively available to anyone).
We tested it in a bunch of ways. One interesting one was that we took all the papers at the end of 2024, then checked which high-scoring tuples showed up in 2025 papers. It worked well - one examples is an ICLR 2025 oral, goes from rank 1,489 with coherence alone to the top 25 of 1.14M samples.
A funny one: if you ask Claude to come up with an ML research idea, it is often about interpretability. If you then run an autoresearch pipeline with some A100s on the idea, the result tends to be interpretability 8/10 times. Our Alien Sampling method stayed at 0 interpretability ideas even after letting claude cook on them.
We also paid 50 people with graduate degrees in CS/AI to judge the research reports and found that Alien’s reports had the best average quality rank and produced 3 of the top 5 reports.
Oh and then we scaled it way the hell up to a couple million papers and 10,000,000's of concept elements and generated 25,000 new research proposals that you can search over here: https://lacuna.tiptreesystems.com/explore?view=hypotheses
Code (MIT): https://github.com/alejandrohdez00/alien-space-of-science
I was really impressed by the scale of the lacuna tool you created, but then I dug into the niche topics (steganography, covert communication) that constitute my PhD research area and my expectations were tempered.
There’s a lot of noise in the research literature. Lots of papers get published that really don’t contribute much of anything to the field, but the existence of a bunch of papers often sends a signal that some concept is important and needs to be studied.
I find that often a big cluster of papers around a particular topic, none of which seems to move the needle much, is actually an indication that we need a new approach and writing more and more incremental papers just digs deeper into a local minimum.
We haven't done concrete measurements yet, but my intuition is that the quality of Lacuna proposals is a function of density of topics / research directions. Lacuna is most dense around AI topics; for neighboring ones (e.g. bio, physics), I'd join you in tempering expectations.
The one thing I could recommend however is rigging up the mcp (https://github.com/tiptreesystems/lacuna-research-mcp) to your code agent and trawling the database for ideas worth thinking about.
> I find that often a big cluster of papers around a particular topic, none of which seems to move the needle much, is actually an indication that we need a new approach and writing more and more incremental papers just digs deeper into a local minimum.
Yes, I think this can work. We use a similar idea to categorize what papers have been influential or foundational -- look at the ones that have changed the trajectory (in time) of a research direction significantly.
> big cluster of papers around a particular topic, none of which seems to move the needle much, is actually an indication that we need a new approach
I feel like when it comes to "Alien Spaces" or new approaches, there's no other way than to babble a lot of stuff that doesn't move the needle much but then leads through the implicit to language and communication mechanisms to a spike out of the local minimum. Didn't the ancient guard have hundreds more notes than ended up in their useful collections?
Nowadays you have so much research that nobody really cares to just ramble with attempts and approaches ... and instead skims the work that others certainly tried already ... distancing themselves from the possibility of new approaches ...
Thanks for checking it out! We're working on a collaborative discussion / editing feature mediated by Althea (our AI agent). That should provide a way for domain experts like yourself to contribute to improving it. Hopefully we'll have that soon!
The problem of course, is how to measure the "movement of the needle." It might help to weight papers and research in the corpus based on some notion of "significance". One metric I know people have used for this is based on citations, on the reasonable premise that influential work gets cited much more. IIRC the original PageRank paper was inspired by work on citation analysis as well.
We actually have a high-risk research idea for how to do this. It involves looking at the clustering as a vocabulary that compresses the concept space, and (one form) of significance as how much your paper / addition of concepts changes the clustering/vocabulary. I.e., if you change the optimal compression of the space, that's probably important.
Oooh, very interesting! I'm way out of my depth here but is the intuition that new concepts that by a) connecting other concepts across that space, or b) introducing entirely new dimensions, or c) some mix of both, change the distribution / clustering of concepts within that space, which impacts how well it compresses, which could be a measure of significance based on the change in compression?
It's like information theory for abstract concepts.
Yea pretty much. The picture I have in my head is of those arcade machines with the sort of snow shovel moving back and forth that sometimes pushes quarters off a shelf (called a coin pusher https://en.wikipedia.org/wiki/Coin_pusher).
You try to figure out where there’s lots of coins on the edge and drop your coin in to get it to push more coins out.
So some fields are ready to get pushed, and maybe density or patterns in concept space can show you that
I think velcro is the canonical example of this.
https://blog.karliner.net/posts/chatgpt-and-the-velcro-test/
> It will never, ever, come up with Velcro. If a concept does not exist on the Internet, or the corpus of training material, LLM’s will not invent it.
So this guy made up an imaginary test, provided a completely unjustified result to it- which casually confirms his ideas- and he's so proud he was right! What a good test he invented!
Not to mention that this test was created in 2023, when the best model we had was GPT 3.5!
I think if an LLM had read any discussions on how best to remove burrs from pet coats then it might eventually get to Velcro - I believe the inventor of Velcro got the idea from burrs?
> "I believe the inventor of Velcro got the idea from burrs?"
I think there's a provocative human-intelligence analog to be found there: "humans will never discover anything that doesn't already exist somewhere in nature. We can only recombine concepts we've seen—never invent truly new ones".
It's said that mathematics is "unreasonably effective" at describing the natural world. But what about the other way around? What if mathematics is unreasonably ineffective at escaping the convex-hull of concepts and patterns found in nature?
I'm working on a piece inspired in large part by William Thurston's "On proof and progress in mathematics"[1]. Should publish it next week.
Part of why math is effective is that its extraordinarily compressible. And you can take those ideas and apply them elsewhere (composition works well). The natural world has a lot of detail and is therefore hard to compress. That's why it's a good source of new ideas. Consider the relation between the McCulloch–Pitts neuron (MCP lol)[2], developed to model biological spiking neural networks, and an inspiration for modern artificial neural networks.
[1] https://arxiv.org/abs/math/9404236 [2] https://en.wikipedia.org/wiki/Artificial_neuron
So I though about "we can only recombine concepts that exist in nature - never invent truly new ones".
The best counterexample I could come up with was lasers. Yeah, all the pieces are there in quantum mechanics, but so far as I know, nothing in nature lases.
Semiconductor chips. Again, yes, all the physics is there, but you're going to be reaching pretty far to find something in nature that's like a CPU chip.
Dyson spheres.
Interrupt vector tables.
Riemannian geometry.
Databases.
I'm sure that people can poke holes in some of these (with varying degrees of far-fetchedness as to how directly the same concept is in nature). But I think there are enough counterexamples, where nature's "prior art" is indirect enough, to make the absolute version of the statement false.
"Humans rarely discover something that doesn't already exist somewhere in nature"? Yeah, that's probably true.
The wheel and axle is an often cited example. It was a big surprise when a bacterium was discovered with a wheel-like connection to its flagellum.
https://en.wikipedia.org/wiki/Rotating_locomotion_in_living_...
True. That mechanism is there.
And yet, whoever invented the wheel and axle, I'm pretty sure they didn't do so based on the bacterium's flagellum.
So, yes, the concept existed in nature. But it wasn't the inspiration for the human invention in any way.
> What if mathematics is unreasonably ineffective at escaping the convex-hull of concepts and patterns found in nature?
Interesting. My intuition here is quite the opposite, that math is vastly overpowered for physics. I don't think we invest a lot of work in figuring out math that isn't useful to describe the natural world.
Or if it ran a couple million soft-body simulations with the goal of creating a reusable temporary link between certain kinds of materials, I think it would probably get there. Just another argument from lack of imagination on the part of the author.
ya - I think my intuitions around velcro are firmly intertwined with my meat-space actuators and perceptual equipment.
In other words, I experience high-resolution multi-modal qualia all day and it's really noisy and I think a lot of new ideas and concepts come from obsessively exploring a small subset of those details. Maybe that's just me posthoc rationalizing my phd, though.
That's a bold claim. It was almost certainly true for models in 2023, when it was written. I think that the recent spate of mathematical discoveries at least significantly narrow the implications for even current models (one can argue that they are only re-mixing existing mathematical concepts to solve problems that humans initially generated, but in my opinion, at some point you are really narrowing what it means for an idea to be "novel"). And I would seriously consider taking odds against it for models within the next 3 years (since that's how long it's been since it was published).
We are basically already at the point where if what current frontier models can do isn't truly "novel", then the vast majority of humans have never had a novel idea in their lives. My three years from now guess is that if whatever it is that frontier models can do at that point isn't "novel", then no human ever will have had a novel idea.
LLMs assign non-zero probability to every output (unless you you sampling modifiers like top-k), so if you sampled enough times the model would definitely come up with velcro. It's the same as the proverbial monkey with a typewriter.
The real question is whether it would happen in a practical timeframe.
> The real question is whether it would happen in a practical timeframe.
How are we defining a practical timeframe? How many billions of man-years did it take for a human to come up with velcro? In a world without velcro I could easily imagine an llm proposing the idea long before a pure human team.
This is the same thing as "but AI can only repeat what it already saw" but the goalpost moved again.
Yann LeCun is right, sometimes
I dont read the argument as specifically AI being bad at the physical world but more that they can't be creative, physical world or not
It's ironic looking at a recent post on the same blog and immediately noticing the claudisms https://blog.karliner.net/posts/thingstudio-common-ground/
edit: clarification: the content itself is fine, in fact I'm curious!
[dead]
Thank you, that an excellent idea and an excellent execution!
But I wonder why when someone types 'medicine' there is only 6 proposals while when someone type 'physics' there is 447 proposals?
Thank you!
One part of the explanation is that we focus primarily on AI research, and a substantial chunk of physicists gravitate (heh) towards AI research, so we end up capturing more of their work.
Another part of the explanation could be that (AI +) physics papers are just easier to access, so we end up processing more of those.
Great question - the "universe" of venues we address is centered on peer-reviewed machine learning like ICML, ICLR, NeurIPS, COLM because (a) we have expertise in this area and so can therefore judge the quality and (b) it's all open access.
Therefore, the further you get away from the topic of ML, the less dense our coverage. Frankly, it's a tough to editorial choice to determine which research directions and therefore which proposals to include. ML is quite multi-disciplinary. And people who have a few papers in ML but many in (e.g.) medicine get upset when their profile is sparse. But we can't possibly produce good proposals in medicine due to its tendency to be closed-access.
Thanks; there is PMC, though it’s possible you may have already used it:
https://pmc.ncbi.nlm.nih.gov/
Hmm, interesting. Yea I think ML is a much smaller field, too. ArXiv + all the conference / applied ml stuff is ~150-250k papers per year. PubMed is 1.5-1.7 million new records per year, and like 40,000,000 total records, whereas in ML we only had to process a couple million. Plus, pretty much all of ML is open-access, whereas medical is ~50% closed.
If somebody gave us a $10-$20m grant for compute, and a few (dozen) helpful doctors showed up to help us QA / figure out how to approach it, it could probably be done.
Kudos. The proposals are actually interesting to browse through. Good paper name by the way.
Thanks! I wish we could take full credit for the paper name, but it's based on a line of work from James Evans (Chicago-based computational social scientist) investigating "Alien artificial intelligences" https://x.com/profjamesevans/highlights
Great stuff. Two questions:
- Are there any issues with vocabulary drift? I know the LLM field in 2017 is not super different, but is there no atoms from 2017 that ended up being irrelevant to 2026 and/or which are just not possible to combine with more recent atoms?
- What does the cost of this project look like? (Ignoring the research iterations, just the costs relevant to the final product).
Yes, vocab drift matters a lot. Most of the pipeline is basically compressing out any of the weird vocab/language stuff, so it doesn't have numbers and proper nouns and is in a sufficiently abstract concept space to cluster nicely. We use a density-based clustering method (HDBSCAN) which when you set min_samples=1 works as an online clustering (i.e., incrementally adding new things is the same as if you re-ran the clustering in batch mode).
It cost ballpark $250k
> "idea atoms"
I made it to the first figure in the paper before I raised an eyebrow at this term.
Then I thought of "Already Clean": https://already-clean.goodfire.com
I cant help but think of Plato's forms -
Oof - it's not like goodfire - we have quantitative results and a clearly defined method!
But yes, we might be bad at marketing. Internally the term we use more often is "concept elements".
The intuition is that if you take a complex treatment of a topic (like an academic paper), you should be able to break it down into some more abstract self-standing ideas. Then, critically, these units are composable and easily embeddable (i.e. they don't have proper names and numbers).
Here's a couple examples of these idea atoms pulled from our paper https://lacuna.tiptreesystems.com/work/the-alien-space-of-sc...: "Scientific ideation often conflates the logical coherence of a research direction with its likelihood of being discovered by the current community. This oversight makes it difficult to identify research paths that are scientifically sound yet unlikely to be discovered due to the existing distribution of expertise and collaboration networks."
Another example: "A generative model estimates the plausibility of concept combinations by treating conceptual units as tokens and calculating their length-normalized log-likelihood. By training on random orderings of these units, the model learns to identify which components are likely to co-occur in a valid research contribution regardless of their sequence."
> Oof - it's not like goodfire - we have quantitative results and a clearly defined method!
Goodfire is a discovery from research, an ahaha moment.
"concept elements" would have been a better descriptor in both cases. The V for the cats and the cars, and for your idea.
> 16,068 LLM ... The Coherence Model is a 42.8M-parameter transformer trained
My question is this: was this the smallest model you could train on the 16k papers? Isn't there an optimal small size to identify everything that is common?
Wouldn't it make more sense to use such a model to strip OUT the commonality? Get rid of everything where the "existing distribution of expertise and collaboration networks" are already optimal - and what's left is the novelty?
Ultimately I would be curious to see if your existing "atoms" happen to follow Zipfs law...
> I made it to the first figure in the paper before I raised an eyebrow at this term.
I came up with that name, and fair eyebrow. Think of an "atom" as a unit of a working vocabulary (a "token"), not a Platonic form. It's a way of compressing the corpus (paraphrases of the same idea across papers get merged into one symbol, a bit like a tokenizer's vocabulary).
We also talk about some limitations of this framing in page 11 (towards the top).
This site crashed my browser although I also wonder what exactly constitutes those arbitrarily picked "idea atom" sets.
Congratulations, this is a really cool approach and result. I lead a research team and I'm very interested in tooling for improved collaboration, especially between human and agent researchers.
Do you have any evidence that this approach scales to cross-domain research?
I'd say science is somewhat fractal in this sense. A research area contains topics, those topics have sub-topics, those sub-topics have competing lines that "talk" to each other, etc. So if it works in core ML that means it's working for RL and deep learning and computer vision and NLP -- which is kind of cross-domain.
That said - we believe the method does need to have sufficient data to model the conceptual density. So it'll not work in closed-science areas.
One other thing I will share: we have been working on a way to enable verified academics with sufficient topical domain expertise to engage in small group discussions that update these documents. You can think of it like Althea (our agent) being an area chair and moderating a discussion to determine how to update pages. The reason we think this will work is that you can have an anonymous discussion with the AI system (and be blunt), but we can weigh your expertise when updating the documents.
This sounds really useful. That concept of weighing contributions from different sources (especially in relation to their expertise in the domain of discussion) is very interesting. Congrats again on the paper and the great response!
Ok do you want to share any interesting ideas the method produced?
Sure! My personal favorite I found at COLM a couple weeks ago: https://lacuna.tiptreesystems.com/hypothesis/passing-uncerta...
The idea is to allocate academic reviewer time based on the estimated quality of a submitted paper. But critically, you should maintain uncertainty bounds in your estimate. It's kind of like an UCB (upper confidence bound) bandit approach but inspired by cascaded speech translation (where you really should not throw away information at any point in your pipeline).
We have generated a bunch here: https://lacuna.tiptreesystems.com/explore?view=hypotheses
Recommendation: rig up the mcp and have claude or codex trawl the corpus for the kinds of ideas you might care about.
``` claude mcp add --scope user lacuna-research -- uvx lacuna-research-mcp ```
or
``` codex mcp add lacuna-research -- uvx lacuna-research-mcp ```
I did that with Claude research. It collected tons of studies from two or more dozen platforms and crosschecked whether my hypothesis was already considered anywhere and it created this gigantic set of data where it showed me all kinds of fields where the many parts of my hypothesis--that Claude rephrased over and over based on contiguous expansions and elaborations in layman's terms--were already or actively researched and how much of it was proved in theory and/or practice. It turns out the specific connections I made were unique and the underlying theory existed, much of it peer-reviewed and in application. I did not check whether some of those applications, peer-reviews and several thousands of studies were challenged but people were applying the science and there was demand for more and lots of work to be done, requiring specialists from diverse fields.
I just quickly checked the 25,000 new research proposals and found a few that indeed touched my hypothesis and the underlying theories but no direct connection.
Honestly, as so often found in criticism of academia, your approach seems very technical and alienated--pun intended--from the people, their lives and interactions with each other and the world, including tech, and it thus seems that your objective is "estranged" from the very thing you are trying to find.
If Claude's and other LLMs surprise when they find rare occurrences of ideation and inferential trains of thought is worth anything, then that it's an appreciation of training sets lack of humanity and their individual thinking.
Here's some idea I likely will never get to: "linguistic conceptualization of psycho-social reverse pareidolia" or "when, why and how humans were taught to not identify meaningful connections". The "doesn't look like anything to me" equivalent so imperative to the narrative of the TV Show Westworld, created by Jonathan Nolan and Lisa Joy.
As much of a merry-go-round, cough, it is, science philosophy and personal narratives and perspectives is where scientific novelty resides, so getting LLMs to extract more out of the cattle, sorry, people, might not be such a bad idea, after all.
[dead]
I'm an engineer by trade, but spend quite a bit of time in pharmacology and molecular biology research in my free time.
One of the proposal papers caught my eye:
https://lacuna.tiptreesystems.com/hypothesis/why-receptor-hu...
When a ligand binds to the ligand binding pocket, the contact points it makes with the proteins inside (called "cofactors") and the strength of each contact point (measured in van der Waals force) modulate the distinct transcriptional effect.This is called "conformation": the shape it takes in the ligand binding pocket.
So it is not necessarily residence time and overall binding affinity that modulate transcriptional effect.
A very good example of this is the androgen Trestolone/MENT, which is an order of magnitude more potent than Testosterone but displays similar relative binding affinity for the androgen receptor:
https://en.wikipedia.org/wiki/Trestolone
This potency is due to the specific conformational shape it takes within the AR binding pocket.
Thanks for digging into one! I had a project back on this back in my Probabilistic Graphical Model days with Simon Lacost-Julien (I believe this was my worst-ever grade in a course)... so I’ll defer to you on the mechanism.
I do know you're right that affinity/residence time and efficacy are separate axes. Bio is one of our sparsest regions (due to closed access papers, and our focus on core ML), so proposals there are more likely to have issues like this.
We've got some ideas on how to improve these issues, but I really appreciate you helping us by pointing this one out!
It does seem like one of the best uses of AI -- digging through existing literature to make novel connections/ideas.
Curious if you've heard of Don Swanson's "Public Knowledge Problem" paper from the 1980's, which was roughly this line of thought?
I had not, and I find it deeply hilaritragic that his paper is paywalled by JSTOR. Thinking of you, Aaron Swartz.
Oof. Well, in any case: https://sci-hub.su/https://doi.org/10.1086/601720
Monotropism and neurodivergence make strange bedfellows.
I believe that I may be AuDHD, not just because it's trendy right now, but because I immediately see the most likely solution(s) to a problem (as if the branches of the search space fall away instantly), while simultaneously seeing more branches than most people seem to (so that I go down rabbit holes and get distracted).
I might compare it to having nearsightedness and farsightedness simultaneously, so that I can drive the ball hundreds of yards but my short game stinks because I'm thinking the trajectory of the putt is a random walk due to the grass. It used to take a lot of effort to step back and not overthink everything, until I had to silence my inner monologue completely to salvage my mental health from total annihilation in a neurotypical world that seems to be oblivious to nothing yet take everything at face value and somehow be productive.
Another analogy might be having depression and anxiety simultaneously, which results in something akin to the opposite of normal, even though from outside it appears to be that way.
Anyway, this paper reminds me of an AI version of that: shining a light on distant associations in excruciating detail. Which is pretty much my favorite thing ever.
I love this. My personal completely unsupported theory is we’re cognitively capable of conceiving of less than 1% of what’s true/accurate about the universe, and cognitively capable of understanding 1% of that 1%
I offer my support for that theory. There's so many facets of reality, many of which no one will ever explore or understand.
If you take a look at the https://en.wikipedia.org/wiki/Mona_Lisa , it can be seen from so many aspects
- art style - a doctor might see indications of a health issue that Lisa has - what it says about the society of that time - geology of the landscape - contemporary production of the color pigments - what the picture does with you when you look at it - and so on
Good Sci-Fi novels bring along some of these - aliens that have access to different concepts of reality, e.g. the dark forest hypothesis. The hard part is that you actually have to come up with that understanding for writing it, instead of just hand-waving it.
"The universe is under no obligation to make sense to you." -- Neil deGrasse Tyson
> we’re cognitively capable of conceiving of less than 1% of
If you can't conceive it, then you can't take a percentage of "it", since you've not conceived that "it".
Quick and fun read on this:
The Road not Taken by Harry Turtledove
https://pastebin.com/aJQfubrK
Fascinating story. Thanks for sharing it.
Makes one wonder if there were technologies that did something similar to humanity in the real world…
I wonder if all the investment in smartphones and LLM’s may have taken away from focus on other things…
I think the way computers developed is kind of an interesting example of this in the real world. Babbage basically realized the idea of a modern computer, but tried to build a too-complicated 'supercomputer' (40 decimal digit precision!) because he didn't know how incredibly useful a much simpler, more limited computer could be. We had to wait another 75 years and spend an incredible amount of money to develop relay and vacuum-tube supercomputers, before realizing that the idea could be scaled down, and something like a PDP-8 / PLC / microcontroller was incredibly useful for industrial control and, with sufficient cleverness, even do a lot of scientific computing.
I think if you went back in time you could pay some power loom engineers to build you a pretty decent 4-8 bit microcontroller operating at 1-5 Hz (comparable to power looms at the time) and computer-controlled textiles would have paid for the computer revolution 75-100 years ahead of time.
Even after Babbage's failure to introduce us to the information age, there were a fair amount of competing approaches to how to build a computer. In a different world, a non-Von-Neumann architecture was selected.
Something like the original computers, Eniac etc, were actually generally useful. It was useful to very large enterprises that previously relied on an army of human "computers" but it was useful for them so it stimulated refinements like the PDP-8.
I think if you went back in time you could pay some power loom engineers to build you a pretty decent 4-8 bit microcontroller operating at 1-5 Hz (comparable to power looms at the time) and computer-controlled textiles would have paid for the computer revolution 75-100 years ahead of time.
Would it? You have to remember that the further you go back in time, the cheaper labor is relative to machinery. In the 19th century, you could pay a person for a multitude of customizations and so the market for custom cloth might not have been what you imagined.
There was an incredible amount of investment into high-speed programmable power looms, and there were a number of innovations (later rediscovered by computer architects) targeted at 'card-saving techniques' to reduce the labor required to card-punch the designs, and to reduce the down-time of the looms. 'Double Data Rate' card readers (with a design interleaved on two chains running at half frequency), SIMD-style one-reader/many-loom setups, 'cross-border' looms that switched between two separate card loops just like a 'branch' instruction, machines to copy chains of punch cards.
What you're saying might have been true in the 18th century or earlier, but things were highly mechanized by the 19th century, and there was a lot of motivation to keep the big expensive looms highly utilized and amortize investment in punch card sets across lots of production runs.
There's a plausible version where no matter what weird detours and dead-ends a human society takes on the tech tree, as long as there continue to be wars every few generations, the needed progress keeps getting made and the dead-ends or even just over-reaches get pruned afterward.
You can go down the wrong rabbit holes for a long while in peace-time (e.g. string theory), but war-time is a crucible for correct ideas.
Necessity is the mother of invention.
I agree with you, though my own motivation is not centered on war, but (a long tail of) existential risks facing humanity, mostly centered around collective action problems.
A kind-of similar concept made the rounds yesterday but for games, instead of science.
https://x.com/re_skob/status/2113043120128483349
Very cool - the Alien Science work actually had a precursor in image generation. Cultural Alien Sampler: Open-ended art generation balancing originality and coherence https://arxiv.org/abs/2510.20849
I guess it makes sense that art folks like creativity!
I'm very entertained by the hypothetical game descriptions hahah. Not sure how they would be as actual games though. They all feel like Kilgore Trout novel descriptions, which Vonnegut used as an outlet for all his sci-fi ideas he didn't want to actually turn into books.
Damn, this was the killer app for semantic spaces I'd been waiting for since Word2vec.
A bit of a tangent, regarding innovation. It's interesting to think about counterfactuals in history. There are questions like what if solar panels became widespread 100 years earlier? Could we have avoided global warming and all the wars over coal and oil? (Not possible because lack of purity technology in materials and understanding of matter.) But are there other things where some easy discovery was waiting just around the corner but it wasn't discovered for some reason? Or something was discovered in one society but not in another for a long time, from similar starting conditions. Washing hands certainly is something that one could go back with, probably thousands of years, to bring immediate benefits. Some cultures have done it but some have absolutely refused it for a long time.
And then applying that kind of thinking to today. Looking back to 2026 from 2126, what would a historian facepalm over? Or what is already being done in one field or social scene but not in another, that would be greatly beneficial? The classic recent example of success is applying airline pilots' checklists to surgery. Airline pilots probably are a little bit like aliens, from surgeon point of view. How many people had already tried similar things, but failed with the institutions and people's inability to receive innovation? Maybe with AI tooling we can facilitate better innovation diffusion. It's not without its dangers of course.
[dead]
Not at all a tangent. We run a backtest in section 5.2 with pre-2025 scores to recover 2025 directions.
We rank all 1.14M three-atom combinations and check them against the 2,424 that appeared in 2025 papers. Using coherence alone, the top 10k (under 1% of the space) recovers 31.4% of them, about 36x chance. Adding the availability score trades some of that for more genuinely new combinations, ones whose atoms had never appeared together before. Judge Decoding (ICLR 2025 oral) moves from rank ~1,489 to rank 1 of 1.14M.
To your point about diffusion, I agree that just having a good idea is not enough. Politics / institutional inertia is the primary determinant of policy.
After crawling out of the primordial ooze we send explorers in all directions. (Usually groups) As they get further from the source they spread out thinner. Someone will have to find the gaps eventually and they might even find things close to the source. If you add dimensions of exploration they might travel far leaving many large gaps back home.
I love the effort trying to squeeze wine from a mountain. I wonder if the generalisation also remedies unpopular topics as they are now reduced to simple data points. It should be fine to map forbidden grounds as long as you don't go there.
this reminds of the short novel by Abbott: Flatland where the inhabitants live in a 2D world and are unaware of Space: the 3D environment.
I'm on the other hand reminded of a certain Lem sub-story from A Perfect Vacuum, which was about geniuses whose discoveries were too divorced from the mainstream to be useful / worth following upon.
Didn't that sort of happen to Heisenberg and Bohr? Einstein refused to accept it though he knew they were right (cuz the math worked).
I heard about https://idealab.ai/ in the Moonshots podcast. Sounds somewhat similar, but for business instead of science.
Humans could also benefit such a model of idea atoms to consciously explore the "alien space" of a field.
I've filtered the generative proposals for biology [0] because that's a field I'm a bit familiar with. I think it's a bunch of nonsense, tho the confident language really gaslights me.
[0] https://lacuna.tiptreesystems.com/explore?view=hypotheses&q=...
Yea, so the way this was built causes regions of density (core machine learning topics) to be good and sparse areas like biology to be worse. There are some areas of intersection like Medical Coding, which anecdotally Joseph Paul Cohen (was a mentor of mine and a postdoc with Yoshua Bengio back in the day) reviewed while we were at COLM and he thought was good enough to share around his research lab. All that said, research proposals will be hit or miss.
The reason that biology is sparse is because we are processing full papers from open access venues. Machine learning papers have a long history of open access publication, whereas other areas still have a lot of concepts locked up. I suspect our method would probably not work very well if you only provided abstracts (like AI2 ScholarQA).
Our clusters and therefore our sampling of new ideas is based on density in concept space, and a lot of the key concepts don't end up in the abstract.
Go open science! All that said, the right partnership with a commercial publisher would probably make this work well in that regime.
Is there a way go finetune this and run it on my local academic pdf library (1-2k papers) ? I'd appreciate having a slightly crazy creative coach who can help me brainstorm new ideas!
yea probably - to be followed :)
edit: you can also certainly point a code agent at this paper and your library and have something interesting in a few hours. Or better yet, give your agent the Lacuna MCP and let it rip on the problem https://lacuna.tiptreesystems.com/mcp
[dead]
Semantically similar papers: https://papermatch.me/?q=2603.01092
I'd like someone to do this with cuisine. Sampling Tasty but cognitively unavailable recipes.
Can you give some sort of breakdown of the costs involved to make this project? Once the methodology had stabilized.
The academic paper cost a few thousand bucks, max $10k in compute.
Lacuna is far more expensive and ~100x larger. If you knew what I know and started building it today (LLMs got cheaper).
Paper summaries cost ~$50k, concept extraction cost ~$25k, author profiles cost $25k, research proposals cost another $50k. So call it $150k for model calls.
It's also an online / updating daily so that costs too, plus infra, plus "spillage".
Brilliant.