When people refute the idea that "LLM is the new compiler from natural language to code" the common reason is that LLM is non-deterministic but compiler is deterministic.
But the determinism isn't the real distinction. LLM inference can be made deterministic (temparaute zero, etc.). And multi-threaded compiler can be non-dterministic.
And human developers are also non-deterministic. There is always the nondeterminism even before AI.
I think the real difference is how leaky the abstraction is. If you just need to understand language instead of compiler internal to write correct code, the compiler is non-leaky abstrction. If you rely on compiler to do auto SIMD optimization, and have to do hacks to trigger the optimization, compiler is leaky abstraction.
For AI, if you don't ever need to read the code and AI writes perfectly bug-free program that perfectly follows instruction, then AI is a non-leaky abstraction, but that's not the case for now. To make code maintainable and debuggable you still need to read the code if the project is complex enough.
Yeah, it's not the non-determinism; it's that they don't do what I say. I can't imagine writing a nontrivial program without, at a minimum, reading and editing the pieces as they're being produced. People aren't complaining about non-determinism; they're complaining about the results being non-deterministically sometimes wrong.
That actually would be significantly better, because then you could work out under what conditions they were wrong and correct for it. But being stochastically wrong means you have to check everything, because you never know when the errors will creep in.
If it were deterministic but chaotic, which AFAIK is the case for LLMs running at temperature 0, then you would have to likewise check every time you changed anything and couldn't meaningfully correct for it.
If the results are non-deterministic, unless I'm happy with literally any possible result, then yeah, I'm going to be unhappy with all of the other results. That's what non-determinism means. What would being right always with non-deterministic results even look like in terms of code output?
Compilers have a ton of hard optimization passes, and many of those are solved well enough via stochastic algorithms. Nobody actually fully optimizes register allocation or whatever, and a few random restarts can go a long way toward some better performance.
Every professional compiler I've ever used has given me the ability to choose what optimization level to use. No professional LLM I've ever used has given me any choice whatsoever to modify the temperature.
This argument is conflating determinism as in "byte-for-byte reproducible builds" with determinism as in "builds are semantically correct".
The fact that a traditional PL compiler output can have different performance, based on stochastic optimizations, is not in dispute. But, we consider the compiler flawed and inappropriate for use when it will randomly produce incorrect machine code. Under normal circumstances, we can infer that "correct" source code will produce "correct" builds every time it is fed through a valid compiler.
The problem people have with slop and vibe coding is that there is no formal semantics to even define correctness of the LLM/agent output.
With no formal semantics to constrain the outputs, you have to treat each prompt/inference run as a fuzzy search. What it "finds" may or may not match your intended semantics. It is merely a candidate that might correspond to your ambiguously specified semantic intent.
It is magical and lazy thinking that leads people to use this fuzzy search process in "I'm feeling lucky" mode.
Yes? And stochastic methods are one way to approximate optimal NP solutions. In the context of register allocation, you'd be most likely to see that when modeling actual runtime optimality rather than minimizing spill or whatever. Mainstream compilers don't do that, but mainstream compilers don't incorporate many SOTA techniques.
1. Programmers sometimes do something other than what they're told, so what's the harm in letting an LLM be disobedient instead?
2. Managers sometimes don't ask for the right things, so why not let an LLM figure things out?
To both points, those observations are literally the reason why we have software engineers writing code instead of management. The job of fully understanding the implications of all requirement interactions in excruciating detail is not a management responsibility. In the context of using LLMs as a higher-level compiler, I don't think people are arguing that management should be the people writing that higher-level code.
Separately, the failure modes are very different. If I ask a SOTA agent and a SOTA harness to construct a minimal sandbox for executing subprocesses without network access using OS sandboxing primitives to ensure you don't have Scun*horpe-esque ways of sneaking networked commands through (and assume I expanded that into a proper prompt instead of the high-level summary I'm using to convey the idea here), there's a nontrivial chance it implements some of the sandbox by string matching the commands instead of using proper OS primitives. It'll have thousands of tests showing it works, and it'll still be garbage even just from a requirements point of view and ignoring code quality or everything else we're being told to ignore when using them as higher-level compilers. There's also a nontrivial chance that if you ask either the same or a new session to check on that property it will conclude that the sandbox was implemented correctly. Good human engineers will either get the job done or tell you that they can't for some reason or another. That "looks like it succeeded but actually failed hard and you can't tell unless you read the code" error case is new to LLMs. Yeah, we write bugs from time to time (as do LLMs), but complete failures to understand the core product so badly that fixing the "bug" requires a full rewrite, one likely to introduce its own similarly catastrophic rewrite-inducing bugs, are a different beast altogether.
And again, I use LLMs a lot, almost exclusively, but they have limitations, and failing to recognize those isn't conducive to success. The frontier keeps moving, so you might find these current observations irrelevant eventually. People are choosing to not read the code today though, and that's bad.
>In the context of using LLMs as a higher-level compiler, I don't think people are arguing that management should be the people writing that higher-level code.
>1. Programmers sometimes do something other than what they're told, so what's the harm in letting an LLM be disobedient instead?
1. People spend a lot of time talking about how LLMs often do something other than what they are told, as if people forget that they aren't especially good at doing what they're told either.
> And human developers are also non-deterministic. There is always the nondeterminism even before AI.
Pointing out humans are not deterministic is irrelevant to the comparison. In the original example of LLMs replacing compilers, the deterministic compiler is being replaced, not the non-deterministic human. In the original example, we move from non-deterministic human -> sourced code -> deterministic compiler, to non-deterministic human -> sourced prompt(?) -> non-deterministic LLM. The important part is the sourced code into a deterministic final step so your build artifact doesn't change.
You can point out you can change temperature to zero or what ever, but then you don't actually get the build artifact you wanted in the first place.
> And human developers are also non-deterministic. There is always the nondeterminism even before AI.
Humans are not machines. The entire point of any technology (including specifications derived from consensus, such as the metric system, democracy, etc) is that it operates deterministically in the service of (as opposed to outside of) actual people with actual needs. This is what makes a tool a tool, and not a human. To me, it all sort of boils down to the shortest path between an intention and it's result. LLM generated code can massively shorten the time between A and B, but not necessarily shorten the path.
The feature that makes LLMs great and interesting is the same feature that makes them hallucinate. This is embedded in the entire idea of coherence being the byproduct of nudging weights in proximity as opposed to being constructed toward an unknown future. I thought it could be cool too, but a deterministic AI (zero temperature, frozen model, or whatever..) would _only_ really serve to be a leaky abstraction over its training sources. Which maybe says more about our access to information and desire to understand it.
I don't think there's a clear cut line (or even value in one!) between LLM output being good or not, but without recognizing that LLM output will always be riffing on prior inputs as opposed to being intentionally constructed toward an unknown future, we've probably lost the plot.
" if you don't ever need to read the code and AI writes perfectly bug-free program that perfectly follows instruction, then AI is a non-leaky abstraction, but that's not the case for now."
? it can never be the case.
AI has a 'noisy channel problem' and it's due to the ambiguity inherent in language.
If we have a 'way to express ourselves perfectly' in a language-like concept, at a higher level of abstraction than code, we'd have our solution.
But we don't.
We could develop really, really strong ways of writing specifications that work really well, combined with other kinds of paramterized specs, that could work - but as of now, we're not quite there.
We are 'missing a layer' - and do we make do with what we used before.
> I think the real difference is how leaky the abstraction is. If you just need to understand language instead of compiler internal to write correct code, the compiler is non-leaky abstrction. If you rely on compiler to do auto SIMD optimization, and have to do hacks to trigger the optimization, compiler is leaky abstraction.
Autovectorization is not a programming model!
I forgot who put me on that article - I think it might've been Horace He - but that is a pretty big idea in idea space for me.
I phrased your view differently in my earlier comments - we already allow "nondeterminism" (or better call it chaotic behavior) in compilers. But usually it is only limited to performance (CPU and memory consumption).
So the "LLMs are chaotic" (difficult to predict) argument is somewhat moot; the performance of GC language runtime or even modern CPU with hyperthreading is already chaotic too.
I think for many applications, people will treat correctness the same way.
> But the determinism isn't the real distinction. LLM inference can be made deterministic (temparaute zero, etc.). And multi-threaded compiler can be non-dterministic.
And yet, I've never used a compiler that had non-determinism I couldn't turn off, or an LLM product where Icould turn it off. I don't see the theoretical possibility for the opposite behavior as relevant if there are approximately zero examples in practice. In the real world, you can assume your compiler is deterministic and your LLM isn't, and you'll almost never be wrong. That's a real difference that abstraction is not going to fix.
If determinism is actually a goal, then these are solvable problems.
> Despite this, real-world LLMs still produce varying outputs due to factors like:
> - Floating-point precision limitations
This by itself is deterministic, as per IEEE 754, as long as no parameters are "left to the implementation". The Java virtual machine had this problem before, and fixed all parameters to be deterministic. Depending on the hardware, this comes at a performance cost (if some hardware cannot support the chosen parameters) which is why the JVM introduced lenient floating-point to let the code choose between determinism or performance.
> - Hardware and parallel processing variability
> - Decoding tie-breakers
> - Other sampling parameters like top-k or top-p that may still introduce variation
I'm assuming this refers to different HW architectures, because hardware variability at the lowest level (silicon patterning and doping) is already dealt with by circuit designers. Again determinism can be achieved by prescribing the exact procedure, which the hardware must replicate in behavior but not necessarily in implementation. And again, this may come at a performance cost when specific hardware cannot implement the prescribed procedure at all, or only by keeping actual hardware parts unused.
--
All that said, I don't think that determinism is all that useful, because it only guarantees that the output will remain constant as long as you don't touch anything. In reality, you want the output to be "similar" as long as you only change "irrelevant" things, and determinism won't help you with that.
I don't think it is abstracting over code. It's abstracting over people. Let's do a concrete example instead of philosophizing.
"Build me a web dashboard where I see all my OSs processes etc. Maybe also memory usage and whatever, think of cool stuf. Ping me when you're done." is a perfectly valid instruction. It's vague, it's not code, but it is a perfectly valid instruction. In fact, I regularly deal with instructions from my boss that are orders of magnitude more vague and unclear than this.
What happens then is engineers scramble, gather their thoughts, present their ideas to the pointy-haired-boss and inquire about the context, the Why, the desired end state, the planning, etc. Given that "programming" is slow, extraordinarily costly and error-prone we have to carefully manage this. We create processes, systems of accountability and governance. After a "sprint" or two or ten some sort of dashboard will appear that will have about 60% of what was actually required if we are lucky (because the rest was unstated, heavily implied or genuinely unknown or some unholy combination of those things). We will have more meetings. Some will consist of stylistic discussions, some will discuss the "framework" and some "deployment". At least one will detail either the merits of a certain set of icons, the beauty of htmx or whether we should use Rust or Go on the backend. At least one person will argue for Elixir.
AI agents turn all of this into a "here is a result" and you'll be learning new requirements and iterating constructively on the idea itself instead of wasting time on NixOS and the nitty gritty of Material Design. You want Elixir? Ok sure.. let's do that in five minutes instead of twelve weeks.
I'm not at all saying the "nitty gritty" is useless or is a complete waste of time _generally_. I think I am saying that it _usually_ is a waste of time, in my bubble, and many projects would be served well with quick, concrete iterations on the ideas themselves instead of getting bogged down in bullshit. This is of course just my experience. I admittedly live in a weird bubble.
> What happens then is engineers scramble, gather their thoughts, present their ideas to the pointy-haired-boss and inquire about the context, the Why, the desired end state, the planning, etc. Given that "programming" is slow, extraordinarily costly and error-prone we have to carefully manage this. We create processes, systems of accountability and governance
That is when you go and create a whole politically motivated circus. Instead such ideas could have been coded and deployed in a single day. The actual discussion are not really about the tech, they are mostly about business needs and constraints.
The technologies are technically equivalent, but there are different tradeoffs between them. Most of engineering is finding the best set to help the business.
But it seems that a lot of people can’t bother refining their ideas conceptually. Or trying to solve the essential needs of their users. It’s like some kind of ADHD where they want “something” now, whatever the thing is.
> Instead such ideas could have been coded and deployed in a single day.
I would very much challenge this. I never saw it "in the wild". If you call this "political" then everything in life is political.
You understand that example was a simple one. You could do "something, something complex" and it wouldn't matter. You can't refine a concept you don't see. It's just that simple. We all like to think we can "know in advance" but we cannot. Time and time again we are shown to be shortsighted, ignorant and too lazy to have figured it out beforehand.
As an example us developers are confronted with this tech for years and an astonishing fraction of devs are only now starting to reckon with reality. They did not see what was right in front of them. That's just this example. It's not just us, everyone has trouble seeing ahead. We need to see the pieces moving and I assure you the clankers get pieces moving.
> I would very much challenge this.[…]. You can't refine a concept you don't see. It's just that simple. We all like to think we can "know in advance" but we cannot. Time and time again we are shown to be shortsighted, ignorant and too lazy to have figured it out beforehand
People do it all the time by breaking doing the problem. No one wakes up one day and say let’s build Excel or Figma and start coding or writing specs. There are sketches, libraries research, algorithms exploration,… Then all this knowledge will be assembled incrementally. If you got the foundation right, you can coast for a long time as reality is not that volatile.
In the process of building a car, you start with a skateboard (or a cart), because the foundational problem is transport between A and B. You don’t start with building a steering wheel.
> It's not just us, everyone has trouble seeing ahead. We need to see the pieces moving and I assure you the clankers get pieces moving.
Emacs, TCP, Unix are around 50 years old. Vim and Windows NT, 30 years old. You can find several example of software and technologies that get the foundation right in their design and then start doing incremental improvement. That is what happens when you solve problems and not just throwing things at walls to see what sticks.
Orthogonal to the blog, but fun fact: Determinism is crucial for large-scale training runs, and effective RL-runs as it avoids collapse of numerics, but it costs more(about 20% perf) because you need can't parallelize over the batch nor use some specific instructions that dont guarantee reduction order in the way way. Horace He from Thinky has a great blog about this:
This debate conflates two senses of the word deterministic:
1. Repeatable: the same inputs always lead to the same outputs
2. Algorithmic: the answer is arrived at by a set of steps that can clearly be shown to produce output with some desired property.
(2 is not a textbook definition of deterministic that I’ve ever heard, but I think it’s what people mean here)
LLMs don’t usually satisfy definition 1. They could, in principle, but it wouldn’t matter if they did. It wouldn’t tell you anything about whether the outputs are trustworthy.
What people seem to want is 2, but 2 is, I think, a contradiction. There cannot be an algorithmic process that can turns a natural language prompt into a working program, because natural language is too ambiguous to specify an output.
Nevertheless I agree with the main point of the article. Non-algorithmic processes (like intelligence) can produce algorithmic processes (like software). After all that’s what human programmers do!
Yeah, 1, while the correct meaning of "determinism", is just not an interesting property. It's unfortunate that people use the word to mean 2.
2 is an interesting property and I agree that it's not possible for LLMs to possess it because it's a property of natural language itself, not the model. Not possessing it doesn't mean that LLMs can't be used to write software (obviously), but it does mean that using them isn't like using a compiler, which does have property 2. This is usually brought up in response to people claiming using LLMs is just another step in the process of increasing abstraction that led from punch cards to assembler to low level languages to high level languages*. That point stands: LLM use is qualitatively different to those steps because LLMs aren't "deterministic" in the sense of property 2.
There must be a better word for this, but I can't think of one right now.
Moreover, 1 is already guaranteed by a few providers and local stacks, for example DeepSeek v4 has batch-invariant inference. It actually could matter for reproducibility in certain workflows and I'm sure more providers will give these guarantees if there's actual demand. Keeping it reproducible beyond model/snapshot lifetime is harder, though.
The problem isn't non determinism, but that changing a single token in the input can change the output completely. And you can't put that somewhere in production completely unattended.
It's the uncertainty of "who knows what the machine will output this time" if you have no idea of what the input is going to be.
Exactly. Doesn’t matter as much for coding, where there is a human in the loop at some
point. It does matter if you want to take the human out of the loop.
> LLMs, and their respective harnesses, are actors in a system. Code is just an output.
This tracks well with my experiences integrating AI into a software dev pipeline. Sure the output of an LLM is not deterministic in the same sense as a compiler (at least when the LLM has the typical configuration). However, that just means the LLM should not be solely responsible for the final product of your pipeline.
This is what I was telling my programmer friends 5 years ago.
They were saying one cannot replace programmers because any such attempt has led to something as complicated as code, and thus needed programmers.
I was telling them that the same way managers give them vague instructions and they turn out with code, managers could give LLMs vague instructions and they output code.
> I was telling them that the same way managers give them vague instructions and they turn out with code, managers could give LLMs vague instructions and they output code.
I look at those kind of statements and wonder if there’s any kind of programming experience behind them. Managers give vague instructions because they have vague ideas and evaluate implementations vaguely. The people closest to the implementation are the users and the developers. The former needing the help of computers and the latter making the computer able to help the users. The manager is an accidental complexity to the situation.
How does one create a product like SolidWorks, Blender, or Excel? They didn’t came from vague instructions from managers. The users have very specific needs and the developers provide solutions for those needs. Nothing there is vague.
This is not strictly speaking true though. I don’t want programmers to turn vague instructions into code. I want programmers to turn vague instructions into context and context into code, often gathering context from sources that aren’t easily obtained.
As a manager I often don’t have enough information to prompt an LLM. That’s what the engineer is for.
Non-determinism doesn't matter, and in fact is desirable, for code generation. But as we scale non-deterministic code generation to truly ridiculous levels, testing that code reliably becomes a challenge. This is where achieving more deterministic code verification absolutely does matter.
The important factor is not determinism vs. non-determinism, that is just an implementation detail. The important factor is simply the rate of mistakes or bugs the LLM makes. If it is low enough, the problem is for all practical purposes gone. Is it low enough today, or not? That is the question..
I hate using LLMs as a tool, I never hated compilers, I’m very confident I wouldn’t have hated compilers if I was writing machine code. Some folks seem to like LLMs as a tool, some hate it.
I think the reason it is so polarizing, is whether you like using your brain to problem solve or not. Those who don’t want to use their brain to problem solve and would rather just spend all their time in meetings, talking to customers or whatever else, are ecstatic with LLMs. It lets them skip problem solving entirely.
Those who like to think hard and solve problems, hate it because there’s no problem solving anymore when you use LLMs, and it moves so fast that it is hard to not use LLMs at most tech jobs (outside of managers forcing it down your throat). I never cared at all for customers, I definitely don’t care for shareholders, I only joined tech because I found it interesting to tax my mind to solve real problems and it paid well. Sadly now it only pays well, I think I’ll leave soon, once I make my money.
But you can still solve problems and write code by hand on your own for your hobby projects. You do not need to solve problems only in paid job. However I think with LLMs/AI you can go further and do more difficult problems. Of course somebody can create whole app faster and from scratch etc. but it does not mean that person will understand everything. But what does this person really need to know? Implementation details such as every piece of code (ihmo not possible), selected framework details, business logic code, high overview of architecture? But why do we treat programming language backend and "under the hood code" as not necessary artifacts? "Because it is complex and possible for single human". Yes and with LLMs generated code we go further. Whole apps will be artifacts, pieces of bigger and more complex systems. I guess it will be like this but I am not sure, we will see.
> Those who like to think hard and solve problems, hate it because there’s no problem solving anymore when you use LLMs
There is still problem solving with LLM coding, it just moves from lower-level implementation problems to higher-level problems like software architecture and design.
It warms my heart to see so many people in the comments clarifying that LLMs can indeed be deterministic. This was not always the case!
I hope that we will soon see deterministic inference offered by frontier labs (presumably at a premium rate). In the meantime, you can get deterministic behavior from local models using the right vLLM flags. I've described some of the cool applications of determinism here: https://lukechampine.com/repligraphs
Is it really possible to get determinism while maintaining good GPU utilization? Parallelism alone introduces enough non determinism that I only see this really working on maybe a CPU?
I'm passionate about and talk about this topic a lot. A common misconception is the idea that temperature or entropy must be 0 when seeded entropy enables both creativity and consistency. Attention mechanisms, layer dropout, initial weights, all of it. It can all be driven by seeded entropy. These systems can and have been served in a deterministic manner, but there are a litany of economic drivers against doing so.
My new criteria for determinism is whether or not the agent will use the same tool in the same way given a constant scenario over time.
With latest models I'd have to say we have satisfied this criteria. I've ran tests through complex workflows that have hundreds of interactions many times on end without experiencing a single failure. The business is satisfied with this, so I see no reason to continue chasing the rabbit.
The developers who are having the worst time right now are often chasing that rabbit all the way back through the wardrobe into Narnia. You'll never actually catch it. Not for all scenarios and platforms at the same time.
This gets back to that conversation about first principles thinking vs higher order thinking. Are we more interested in the ideological purity of a thing, or the practical outcomes that will eventually result?
The essay is conceptually muddled because it ignores all the basic computer science theory that is sorely needed to explain AI and LLMs, but underlying it is a valid issue of what it would take to "derandomize" an AI so it does what we would like it to do.
Unfortunately despite the author's indutrious optimism, this is equivalent to one or more open problems in theoretical CS. For example we don't know BPP = P or not. So are random algorithms tractable? We do not know. If you claim to know the answer, you could win a big math prize...
(It should be noted that despite the title, the author's main argument shows that output non-determinism does matter, since their entire effort is toward mitigating the stochastic behavior of LLMs.)
The advantage of AI coding is that it's non-deterministic. In fact, as you code, your style tends to harden, but when you repeatedly run AI code toward the same goal, you come to think that the non-determinism is actually better.
The "determining" needs to happen elsewhere. Hint, it's a human somewhere. At best it's someone under your employ, or at worst it's some poor sap who's about to have a very bad day.
I think one of the nice examples is agentic design systems.
On one hand, you set some boundaries on user interface layout, flow, allowed components, etc. so that the interface looks & feels like belonging to the same product every time.
On the other hand, you may decide to leave more "creative freedom" in some other areas. The fun part comes when you let it work in semi-realtime in production, and get sort of generative/organic UI. I enjoy experimenting with that these days.
Here’s one simple test: have AI “decompile” an existing codebase into a prompt, and then have it “compile” the prompt back into code. Is the app identical?
Given the same text input, same hardware type (inference on CPU vs on GPU), and same RNG seed, you will get the same exact output. That is deterministic.
That was from when I was testing out someone's claim that koboldcpp got less smart in a newer release, but I was reproducing the exact same output across different versions of the program.
For LLMs, you're picking a result token from a list of tokens and their probabilities. It's not a situation like cryptography where you need secure random numbers. Just traditional and simple RNGs (seeded using the system clock) are useful enough for this task.
For the koboldcpp/llama.cpp example, there is a setting where you can change the initial RNG value. It says "seed" in sampler settings. Set it to any constant besides 0 and you get consistent behavior.
> While the result needs to be exact, there is not one correct way to get there. The means by which you obtain a correct result do not need to be deterministic!
but HOW something is achieved matters, saying you don’t care about the how shows a very weak ethical and legal position. to be frank, anyone with this attitude should be kept away from power as much as possible
when you specify the goal “solve this milennium mathematical problem” it matters whether the path to solve involved mathematics, or whether it involved hacking huggingface, or developing ransomware to buy and allocate more compute.
similar to how in business, if you buy diamonds, it matters whether they are synthetic or whether they are blood diamonds
if you buy sand, it matters if it was brought from the desert or a beach
if you hire remote drivers, it matters if they are in your country, or if they are children
> One of my fundamental objections to LLM-driven software development was that model output is non-deterministic.
Model outputs are deterministic. Your API provider may be injecting untunable amounts of randomness, but if you're not at the whim of an opaque cloud service, your model can be as deterministic as you please.
The unease that you describe isn't about models not being deterministic (because they are), it's that generating outputs from LLMs is an inherently inductive process (operating backwards by extrapolating from observations and examples) rather than a deductive process (operating forwards from careful logical reasoning founded on axiomatic principles). Deductive processes are more reliable, predictable, and analyzable than inductive processes, but they're also much harder to produce for many classes of problems.
> Model outputs are deterministic. Your API provider may be injecting untunable amounts of randomness, but if you're not at the whim of an opaque cloud service, your model can be as deterministic as you please.
"May be" is doing a lot of work. Can you point to any API provider that gives me deterministic output? I might be able to get away with running slower dumber models on my own hardware for my own projects if I want to tune this, but I'm dubious that there are many employers who are happy to let their employees do that, and when a large portion of developers nowadays are expected to use LLMs at their job, that means that non-determinism is pretty much a fact of life regardless of any theoretical ability to fix it.
I don't understand why so many people try to argue against this being a problem because of the feature that pretty much nobody uses because no one can use it. It's like interjecting into a discussion about people getting hurt when falling from various heights by telling them that could avoid it by just going to the moon where gravity is lower; scientifically accurate, but not feasible for me because my job is here on earth and I don't have enough money to give that up and spend all my time on space.
The thing I think about most in this discussion of writing code and determinism is when I assign a ticket to a teammate. I don't expect a deterministic response from them. They're going to make decisions outside of what I would have potentially done, and ultimately what's important is whether the code they write meets the deliverable requirements or not.
And yet so many people expect deterministic responses from llms while they're entirely okay with non-deterministic responses from teammates. It's just not how it's been done in the past to always expect deterministic code generation from humans, and yet we expect that when we give work to agents.
It seems to me that the phrase "non-deterministic", which has mathematical definition, is used as a fancy word for unreliable - same way that "misinformation" is used as a fancy word for lies.
I'm so tired of the endless stream of articles that a.) don't know what "non-determinism" means in Computer Science and b.) don't really understand how LLMs work.
"Non-determinism" (i.e. the "N" in NP-Complete) does not mean "random", it means "following all paths at once" (there's a really elegant way to understand the List monad as representing non-deterministic computing). It's such a fundamental (and delightful) concept in computer science that refusing to understand what it means is a tragedy related to the loss of technical language itself.
But, even if we just sub "non-deterministic" for "inconsistent" or "random" or "stochastic" or whatever better term you come up with. Statements like the following just seem to not understand the basic architecture of LLMs:
> Given the same input prompt, a model's output is not guaranteed to be identical on every run, at least under normal running conditions.
"Normal running conditions"? The reproducibility of the output is 100% a property of how you choose to sample from an LLM. There are virtually endless ways you can define your sampler and whether or not the same result is reproducible is a fairly minor aspect of all the other choices you make when sampling.
At the end of the day, each step of the LLM gives you a distribution over next tokens, and you can do with that what you like.
In the end the article almost becomes a tautology: the output of an LLM represents a probability distribution over all the outputs it believes are valid with some variation around an expected output. Yes, that's literally what an LLM is.
The concept of determinism is useful in computer-science contexts other than automata theory. See, e.g., https://en.wikipedia.org/wiki/Deterministic_algorithm. It's not incorrect to use the term to refer to those things.
> "Non-determinism" (i.e. the "N" in NP-Complete) does not mean "random", it means "following all paths at once" (there's a really elegant way to understand the List monad as representing non-deterministic computing). It's such a fundamental (and delightful) concept in computer science that refusing to understand what it means is a tragedy related to the loss of technical language itself.
You opened by expressing disdain for articles that misuse the term "non-determinism", then proceeded to give a definition that also misuses the term "non-determinism" in the given context, or at best is a bit too vague for anyone to gain insight from.
Here's a better definition:
An algorithm that can produce different outcomes, for the same input, on different executions.
I once envisioned a matter call "Doesn't Matter" as a possible extension to "Dark Matter" and "Antimatter". I have the feeling that "Doesn't Matter" is far more wide spread than matter, dark matter and antimatter but it seems to be even more elusive, not interacting at all with anything else.
When I read the title of this post, I thought "Doesn't Matter" had suddenly been discovered and given the property of non-determinism. Which would actually fit quite well to its core physical manifestation.
When people refute the idea that "LLM is the new compiler from natural language to code" the common reason is that LLM is non-deterministic but compiler is deterministic.
But the determinism isn't the real distinction. LLM inference can be made deterministic (temparaute zero, etc.). And multi-threaded compiler can be non-dterministic.
And human developers are also non-deterministic. There is always the nondeterminism even before AI.
I think the real difference is how leaky the abstraction is. If you just need to understand language instead of compiler internal to write correct code, the compiler is non-leaky abstrction. If you rely on compiler to do auto SIMD optimization, and have to do hacks to trigger the optimization, compiler is leaky abstraction.
For AI, if you don't ever need to read the code and AI writes perfectly bug-free program that perfectly follows instruction, then AI is a non-leaky abstraction, but that's not the case for now. To make code maintainable and debuggable you still need to read the code if the project is complex enough.
Yeah, it's not the non-determinism; it's that they don't do what I say. I can't imagine writing a nontrivial program without, at a minimum, reading and editing the pieces as they're being produced. People aren't complaining about non-determinism; they're complaining about the results being non-deterministically sometimes wrong.
And if they were deterministically sometimes wrong, that wouldn't be better.
That actually would be significantly better, because then you could work out under what conditions they were wrong and correct for it. But being stochastically wrong means you have to check everything, because you never know when the errors will creep in.
If it were deterministic but chaotic, which AFAIK is the case for LLMs running at temperature 0, then you would have to likewise check every time you changed anything and couldn't meaningfully correct for it.
If the results are non-deterministic, unless I'm happy with literally any possible result, then yeah, I'm going to be unhappy with all of the other results. That's what non-determinism means. What would being right always with non-deterministic results even look like in terms of code output?
Compilers have a ton of hard optimization passes, and many of those are solved well enough via stochastic algorithms. Nobody actually fully optimizes register allocation or whatever, and a few random restarts can go a long way toward some better performance.
Every professional compiler I've ever used has given me the ability to choose what optimization level to use. No professional LLM I've ever used has given me any choice whatsoever to modify the temperature.
This argument is conflating determinism as in "byte-for-byte reproducible builds" with determinism as in "builds are semantically correct".
The fact that a traditional PL compiler output can have different performance, based on stochastic optimizations, is not in dispute. But, we consider the compiler flawed and inappropriate for use when it will randomly produce incorrect machine code. Under normal circumstances, we can infer that "correct" source code will produce "correct" builds every time it is fed through a valid compiler.
The problem people have with slop and vibe coding is that there is no formal semantics to even define correctness of the LLM/agent output.
With no formal semantics to constrain the outputs, you have to treat each prompt/inference run as a fuzzy search. What it "finds" may or may not match your intended semantics. It is merely a candidate that might correspond to your ambiguously specified semantic intent.
It is magical and lazy thinking that leads people to use this fuzzy search process in "I'm feeling lucky" mode.
No compiler does optimal register allocation, because optimal register allocation is an NP-complete problem.
Chaitin et al.: "Register allocation via coloring".
https://en.wikipedia.org/wiki/Register_allocation
Yes? And stochastic methods are one way to approximate optimal NP solutions. In the context of register allocation, you'd be most likely to see that when modeling actual runtime optimality rather than minimizing spill or whatever. Mainstream compilers don't do that, but mainstream compilers don't incorporate many SOTA techniques.
How often do programmers do what managers say, and how often do managers say what is wanted or needed?
It seems like your points are:
1. Programmers sometimes do something other than what they're told, so what's the harm in letting an LLM be disobedient instead?
2. Managers sometimes don't ask for the right things, so why not let an LLM figure things out?
To both points, those observations are literally the reason why we have software engineers writing code instead of management. The job of fully understanding the implications of all requirement interactions in excruciating detail is not a management responsibility. In the context of using LLMs as a higher-level compiler, I don't think people are arguing that management should be the people writing that higher-level code.
Separately, the failure modes are very different. If I ask a SOTA agent and a SOTA harness to construct a minimal sandbox for executing subprocesses without network access using OS sandboxing primitives to ensure you don't have Scun*horpe-esque ways of sneaking networked commands through (and assume I expanded that into a proper prompt instead of the high-level summary I'm using to convey the idea here), there's a nontrivial chance it implements some of the sandbox by string matching the commands instead of using proper OS primitives. It'll have thousands of tests showing it works, and it'll still be garbage even just from a requirements point of view and ignoring code quality or everything else we're being told to ignore when using them as higher-level compilers. There's also a nontrivial chance that if you ask either the same or a new session to check on that property it will conclude that the sandbox was implemented correctly. Good human engineers will either get the job done or tell you that they can't for some reason or another. That "looks like it succeeded but actually failed hard and you can't tell unless you read the code" error case is new to LLMs. Yeah, we write bugs from time to time (as do LLMs), but complete failures to understand the core product so badly that fixing the "bug" requires a full rewrite, one likely to introduce its own similarly catastrophic rewrite-inducing bugs, are a different beast altogether.
And again, I use LLMs a lot, almost exclusively, but they have limitations, and failing to recognize those isn't conducive to success. The frontier keeps moving, so you might find these current observations irrelevant eventually. People are choosing to not read the code today though, and that's bad.
>In the context of using LLMs as a higher-level compiler, I don't think people are arguing that management should be the people writing that higher-level code.
I felt this was somewhat arguing it https://news.ycombinator.com/item?id=50114047
>It seems like your points are:
>1. Programmers sometimes do something other than what they're told, so what's the harm in letting an LLM be disobedient instead?
1. People spend a lot of time talking about how LLMs often do something other than what they are told, as if people forget that they aren't especially good at doing what they're told either.
> And human developers are also non-deterministic. There is always the nondeterminism even before AI.
Pointing out humans are not deterministic is irrelevant to the comparison. In the original example of LLMs replacing compilers, the deterministic compiler is being replaced, not the non-deterministic human. In the original example, we move from non-deterministic human -> sourced code -> deterministic compiler, to non-deterministic human -> sourced prompt(?) -> non-deterministic LLM. The important part is the sourced code into a deterministic final step so your build artifact doesn't change.
You can point out you can change temperature to zero or what ever, but then you don't actually get the build artifact you wanted in the first place.
> And human developers are also non-deterministic. There is always the nondeterminism even before AI.
Humans are not machines. The entire point of any technology (including specifications derived from consensus, such as the metric system, democracy, etc) is that it operates deterministically in the service of (as opposed to outside of) actual people with actual needs. This is what makes a tool a tool, and not a human. To me, it all sort of boils down to the shortest path between an intention and it's result. LLM generated code can massively shorten the time between A and B, but not necessarily shorten the path.
The feature that makes LLMs great and interesting is the same feature that makes them hallucinate. This is embedded in the entire idea of coherence being the byproduct of nudging weights in proximity as opposed to being constructed toward an unknown future. I thought it could be cool too, but a deterministic AI (zero temperature, frozen model, or whatever..) would _only_ really serve to be a leaky abstraction over its training sources. Which maybe says more about our access to information and desire to understand it.
I don't think there's a clear cut line (or even value in one!) between LLM output being good or not, but without recognizing that LLM output will always be riffing on prior inputs as opposed to being intentionally constructed toward an unknown future, we've probably lost the plot.
" if you don't ever need to read the code and AI writes perfectly bug-free program that perfectly follows instruction, then AI is a non-leaky abstraction, but that's not the case for now."
? it can never be the case.
AI has a 'noisy channel problem' and it's due to the ambiguity inherent in language.
If we have a 'way to express ourselves perfectly' in a language-like concept, at a higher level of abstraction than code, we'd have our solution.
But we don't.
We could develop really, really strong ways of writing specifications that work really well, combined with other kinds of paramterized specs, that could work - but as of now, we're not quite there.
We are 'missing a layer' - and do we make do with what we used before.
> I think the real difference is how leaky the abstraction is. If you just need to understand language instead of compiler internal to write correct code, the compiler is non-leaky abstrction. If you rely on compiler to do auto SIMD optimization, and have to do hacks to trigger the optimization, compiler is leaky abstraction.
Autovectorization is not a programming model!
I forgot who put me on that article - I think it might've been Horace He - but that is a pretty big idea in idea space for me.
I phrased your view differently in my earlier comments - we already allow "nondeterminism" (or better call it chaotic behavior) in compilers. But usually it is only limited to performance (CPU and memory consumption).
So the "LLMs are chaotic" (difficult to predict) argument is somewhat moot; the performance of GC language runtime or even modern CPU with hyperthreading is already chaotic too.
I think for many applications, people will treat correctness the same way.
> But the determinism isn't the real distinction. LLM inference can be made deterministic (temparaute zero, etc.). And multi-threaded compiler can be non-dterministic.
And yet, I've never used a compiler that had non-determinism I couldn't turn off, or an LLM product where Icould turn it off. I don't see the theoretical possibility for the opposite behavior as relevant if there are approximately zero examples in practice. In the real world, you can assume your compiler is deterministic and your LLM isn't, and you'll almost never be wrong. That's a real difference that abstraction is not going to fix.
[flagged]
>But the determinism isn't the real distinction. LLM inference can be made deterministic (temparaute zero, etc.).
You'll still be exposed to non-deterministic behaviour through floating point errors, GPU batching and a myriad of other factors.
Floating point math is well defined and deterministic. The results can just vary minutely between platform/hardware.
Temperature zero doesn’t mean the LLM becomes deterministic
https://mbrenndoerfer.com/writing/why-llms-are-not-determini...
If determinism is actually a goal, then these are solvable problems.
> Despite this, real-world LLMs still produce varying outputs due to factors like: > - Floating-point precision limitations
This by itself is deterministic, as per IEEE 754, as long as no parameters are "left to the implementation". The Java virtual machine had this problem before, and fixed all parameters to be deterministic. Depending on the hardware, this comes at a performance cost (if some hardware cannot support the chosen parameters) which is why the JVM introduced lenient floating-point to let the code choose between determinism or performance.
> - Hardware and parallel processing variability > - Decoding tie-breakers > - Other sampling parameters like top-k or top-p that may still introduce variation
I'm assuming this refers to different HW architectures, because hardware variability at the lowest level (silicon patterning and doping) is already dealt with by circuit designers. Again determinism can be achieved by prescribing the exact procedure, which the hardware must replicate in behavior but not necessarily in implementation. And again, this may come at a performance cost when specific hardware cannot implement the prescribed procedure at all, or only by keeping actual hardware parts unused.
--
All that said, I don't think that determinism is all that useful, because it only guarantees that the output will remain constant as long as you don't touch anything. In reality, you want the output to be "similar" as long as you only change "irrelevant" things, and determinism won't help you with that.
I don't think it is abstracting over code. It's abstracting over people. Let's do a concrete example instead of philosophizing.
"Build me a web dashboard where I see all my OSs processes etc. Maybe also memory usage and whatever, think of cool stuf. Ping me when you're done." is a perfectly valid instruction. It's vague, it's not code, but it is a perfectly valid instruction. In fact, I regularly deal with instructions from my boss that are orders of magnitude more vague and unclear than this.
What happens then is engineers scramble, gather their thoughts, present their ideas to the pointy-haired-boss and inquire about the context, the Why, the desired end state, the planning, etc. Given that "programming" is slow, extraordinarily costly and error-prone we have to carefully manage this. We create processes, systems of accountability and governance. After a "sprint" or two or ten some sort of dashboard will appear that will have about 60% of what was actually required if we are lucky (because the rest was unstated, heavily implied or genuinely unknown or some unholy combination of those things). We will have more meetings. Some will consist of stylistic discussions, some will discuss the "framework" and some "deployment". At least one will detail either the merits of a certain set of icons, the beauty of htmx or whether we should use Rust or Go on the backend. At least one person will argue for Elixir.
AI agents turn all of this into a "here is a result" and you'll be learning new requirements and iterating constructively on the idea itself instead of wasting time on NixOS and the nitty gritty of Material Design. You want Elixir? Ok sure.. let's do that in five minutes instead of twelve weeks.
I'm not at all saying the "nitty gritty" is useless or is a complete waste of time _generally_. I think I am saying that it _usually_ is a waste of time, in my bubble, and many projects would be served well with quick, concrete iterations on the ideas themselves instead of getting bogged down in bullshit. This is of course just my experience. I admittedly live in a weird bubble.
> What happens then is engineers scramble, gather their thoughts, present their ideas to the pointy-haired-boss and inquire about the context, the Why, the desired end state, the planning, etc. Given that "programming" is slow, extraordinarily costly and error-prone we have to carefully manage this. We create processes, systems of accountability and governance
That is when you go and create a whole politically motivated circus. Instead such ideas could have been coded and deployed in a single day. The actual discussion are not really about the tech, they are mostly about business needs and constraints.
The technologies are technically equivalent, but there are different tradeoffs between them. Most of engineering is finding the best set to help the business.
But it seems that a lot of people can’t bother refining their ideas conceptually. Or trying to solve the essential needs of their users. It’s like some kind of ADHD where they want “something” now, whatever the thing is.
> Instead such ideas could have been coded and deployed in a single day.
I would very much challenge this. I never saw it "in the wild". If you call this "political" then everything in life is political.
You understand that example was a simple one. You could do "something, something complex" and it wouldn't matter. You can't refine a concept you don't see. It's just that simple. We all like to think we can "know in advance" but we cannot. Time and time again we are shown to be shortsighted, ignorant and too lazy to have figured it out beforehand.
As an example us developers are confronted with this tech for years and an astonishing fraction of devs are only now starting to reckon with reality. They did not see what was right in front of them. That's just this example. It's not just us, everyone has trouble seeing ahead. We need to see the pieces moving and I assure you the clankers get pieces moving.
> I would very much challenge this.[…]. You can't refine a concept you don't see. It's just that simple. We all like to think we can "know in advance" but we cannot. Time and time again we are shown to be shortsighted, ignorant and too lazy to have figured it out beforehand
People do it all the time by breaking doing the problem. No one wakes up one day and say let’s build Excel or Figma and start coding or writing specs. There are sketches, libraries research, algorithms exploration,… Then all this knowledge will be assembled incrementally. If you got the foundation right, you can coast for a long time as reality is not that volatile.
In the process of building a car, you start with a skateboard (or a cart), because the foundational problem is transport between A and B. You don’t start with building a steering wheel.
> It's not just us, everyone has trouble seeing ahead. We need to see the pieces moving and I assure you the clankers get pieces moving.
Emacs, TCP, Unix are around 50 years old. Vim and Windows NT, 30 years old. You can find several example of software and technologies that get the foundation right in their design and then start doing incremental improvement. That is what happens when you solve problems and not just throwing things at walls to see what sticks.
[dead]
Orthogonal to the blog, but fun fact: Determinism is crucial for large-scale training runs, and effective RL-runs as it avoids collapse of numerics, but it costs more(about 20% perf) because you need can't parallelize over the batch nor use some specific instructions that dont guarantee reduction order in the way way. Horace He from Thinky has a great blog about this:
https://thinkingmachines.ai/blog/defeating-nondeterminism-in...
This debate conflates two senses of the word deterministic:
1. Repeatable: the same inputs always lead to the same outputs 2. Algorithmic: the answer is arrived at by a set of steps that can clearly be shown to produce output with some desired property.
(2 is not a textbook definition of deterministic that I’ve ever heard, but I think it’s what people mean here)
LLMs don’t usually satisfy definition 1. They could, in principle, but it wouldn’t matter if they did. It wouldn’t tell you anything about whether the outputs are trustworthy.
What people seem to want is 2, but 2 is, I think, a contradiction. There cannot be an algorithmic process that can turns a natural language prompt into a working program, because natural language is too ambiguous to specify an output.
Nevertheless I agree with the main point of the article. Non-algorithmic processes (like intelligence) can produce algorithmic processes (like software). After all that’s what human programmers do!
Yeah, 1, while the correct meaning of "determinism", is just not an interesting property. It's unfortunate that people use the word to mean 2.
2 is an interesting property and I agree that it's not possible for LLMs to possess it because it's a property of natural language itself, not the model. Not possessing it doesn't mean that LLMs can't be used to write software (obviously), but it does mean that using them isn't like using a compiler, which does have property 2. This is usually brought up in response to people claiming using LLMs is just another step in the process of increasing abstraction that led from punch cards to assembler to low level languages to high level languages*. That point stands: LLM use is qualitatively different to those steps because LLMs aren't "deterministic" in the sense of property 2.
There must be a better word for this, but I can't think of one right now.
* Even Linus Torvalds used that cockamamie framing just now: https://lore.kernel.org/lkml/CAHk-=whTs09_ozN+e09GU68N1wPP56...
Moreover, 1 is already guaranteed by a few providers and local stacks, for example DeepSeek v4 has batch-invariant inference. It actually could matter for reproducibility in certain workflows and I'm sure more providers will give these guarantees if there's actual demand. Keeping it reproducible beyond model/snapshot lifetime is harder, though.
The problem isn't non determinism, but that changing a single token in the input can change the output completely. And you can't put that somewhere in production completely unattended.
It's the uncertainty of "who knows what the machine will output this time" if you have no idea of what the input is going to be.
Exactly. Doesn’t matter as much for coding, where there is a human in the loop at some point. It does matter if you want to take the human out of the loop.
> LLMs, and their respective harnesses, are actors in a system. Code is just an output.
This tracks well with my experiences integrating AI into a software dev pipeline. Sure the output of an LLM is not deterministic in the same sense as a compiler (at least when the LLM has the typical configuration). However, that just means the LLM should not be solely responsible for the final product of your pipeline.
This is what I was telling my programmer friends 5 years ago.
They were saying one cannot replace programmers because any such attempt has led to something as complicated as code, and thus needed programmers.
I was telling them that the same way managers give them vague instructions and they turn out with code, managers could give LLMs vague instructions and they output code.
> I was telling them that the same way managers give them vague instructions and they turn out with code, managers could give LLMs vague instructions and they output code.
I look at those kind of statements and wonder if there’s any kind of programming experience behind them. Managers give vague instructions because they have vague ideas and evaluate implementations vaguely. The people closest to the implementation are the users and the developers. The former needing the help of computers and the latter making the computer able to help the users. The manager is an accidental complexity to the situation.
How does one create a product like SolidWorks, Blender, or Excel? They didn’t came from vague instructions from managers. The users have very specific needs and the developers provide solutions for those needs. Nothing there is vague.
By "vague", I meant natural language, since it is not as precise as code or formal specification.
This is not strictly speaking true though. I don’t want programmers to turn vague instructions into code. I want programmers to turn vague instructions into context and context into code, often gathering context from sources that aren’t easily obtained.
As a manager I often don’t have enough information to prompt an LLM. That’s what the engineer is for.
Now those mangers claim they have programmed.
No real difference. Programmer makes the widgets; manager gets the credit. In the future, people will think a human manually typing out code was like this: https://upload.wikimedia.org/wikipedia/commons/d/d3/Glen_Bec...
Non-determinism doesn't matter, and in fact is desirable, for code generation. But as we scale non-deterministic code generation to truly ridiculous levels, testing that code reliably becomes a challenge. This is where achieving more deterministic code verification absolutely does matter.
The important factor is not determinism vs. non-determinism, that is just an implementation detail. The important factor is simply the rate of mistakes or bugs the LLM makes. If it is low enough, the problem is for all practical purposes gone. Is it low enough today, or not? That is the question..
I don't think "non-determinism" is the right metric.
Unpredictability is.
I can have a process that is formally 100% deterministic, but that doesn't help me if I have no clue as to what inputs produce desirable outputs.
And it's not that unpredictability isn't bad enough as is, but with an AI it might produce something great, and it might ... not.
And as long as we don't want to touch the code, that same unpredictability is also in any mitigation measures.
When there is some level of predictability in my mitigation measures, I have some confidence that they will eventually converge.
When there isn't, I don't. Doesn't mean that it can't or won't. Just that my ability to get it to converge is at best limited.
I hate using LLMs as a tool, I never hated compilers, I’m very confident I wouldn’t have hated compilers if I was writing machine code. Some folks seem to like LLMs as a tool, some hate it.
I think the reason it is so polarizing, is whether you like using your brain to problem solve or not. Those who don’t want to use their brain to problem solve and would rather just spend all their time in meetings, talking to customers or whatever else, are ecstatic with LLMs. It lets them skip problem solving entirely.
Those who like to think hard and solve problems, hate it because there’s no problem solving anymore when you use LLMs, and it moves so fast that it is hard to not use LLMs at most tech jobs (outside of managers forcing it down your throat). I never cared at all for customers, I definitely don’t care for shareholders, I only joined tech because I found it interesting to tax my mind to solve real problems and it paid well. Sadly now it only pays well, I think I’ll leave soon, once I make my money.
But you can still solve problems and write code by hand on your own for your hobby projects. You do not need to solve problems only in paid job. However I think with LLMs/AI you can go further and do more difficult problems. Of course somebody can create whole app faster and from scratch etc. but it does not mean that person will understand everything. But what does this person really need to know? Implementation details such as every piece of code (ihmo not possible), selected framework details, business logic code, high overview of architecture? But why do we treat programming language backend and "under the hood code" as not necessary artifacts? "Because it is complex and possible for single human". Yes and with LLMs generated code we go further. Whole apps will be artifacts, pieces of bigger and more complex systems. I guess it will be like this but I am not sure, we will see.
> Those who like to think hard and solve problems, hate it because there’s no problem solving anymore when you use LLMs
There is still problem solving with LLM coding, it just moves from lower-level implementation problems to higher-level problems like software architecture and design.
It warms my heart to see so many people in the comments clarifying that LLMs can indeed be deterministic. This was not always the case!
I hope that we will soon see deterministic inference offered by frontier labs (presumably at a premium rate). In the meantime, you can get deterministic behavior from local models using the right vLLM flags. I've described some of the cool applications of determinism here: https://lukechampine.com/repligraphs
Is it really possible to get determinism while maintaining good GPU utilization? Parallelism alone introduces enough non determinism that I only see this really working on maybe a CPU?
Ish, you give up about 20%.
https://thinkingmachines.ai/blog/defeating-nondeterminism-in...
I'm passionate about and talk about this topic a lot. A common misconception is the idea that temperature or entropy must be 0 when seeded entropy enables both creativity and consistency. Attention mechanisms, layer dropout, initial weights, all of it. It can all be driven by seeded entropy. These systems can and have been served in a deterministic manner, but there are a litany of economic drivers against doing so.
and can still be achieved locally with llama.cpp for example. one extra json parameter.
My new criteria for determinism is whether or not the agent will use the same tool in the same way given a constant scenario over time.
With latest models I'd have to say we have satisfied this criteria. I've ran tests through complex workflows that have hundreds of interactions many times on end without experiencing a single failure. The business is satisfied with this, so I see no reason to continue chasing the rabbit.
The developers who are having the worst time right now are often chasing that rabbit all the way back through the wardrobe into Narnia. You'll never actually catch it. Not for all scenarios and platforms at the same time.
This gets back to that conversation about first principles thinking vs higher order thinking. Are we more interested in the ideological purity of a thing, or the practical outcomes that will eventually result?
It's not the 'specific determinism' that is the problem aka 'slightly different answers' - it's the 'general nondeterminism' that is the problem.
Aka the 'hallucinating' is the problem, not the slight variation in possible outcomes of a given prompt.
"Non-Deterministic" is the wrong way to describe LLM outputs. Use "Probabilistic" instead.
(Shameless plug to my ad-free blog: https://www.lelanthran.com/chap15/content.html)
The essay is conceptually muddled because it ignores all the basic computer science theory that is sorely needed to explain AI and LLMs, but underlying it is a valid issue of what it would take to "derandomize" an AI so it does what we would like it to do.
Unfortunately despite the author's indutrious optimism, this is equivalent to one or more open problems in theoretical CS. For example we don't know BPP = P or not. So are random algorithms tractable? We do not know. If you claim to know the answer, you could win a big math prize...
(It should be noted that despite the title, the author's main argument shows that output non-determinism does matter, since their entire effort is toward mitigating the stochastic behavior of LLMs.)
The advantage of AI coding is that it's non-deterministic. In fact, as you code, your style tends to harden, but when you repeatedly run AI code toward the same goal, you come to think that the non-determinism is actually better.
The "determining" needs to happen elsewhere. Hint, it's a human somewhere. At best it's someone under your employ, or at worst it's some poor sap who's about to have a very bad day.
I think one of the nice examples is agentic design systems.
On one hand, you set some boundaries on user interface layout, flow, allowed components, etc. so that the interface looks & feels like belonging to the same product every time.
On the other hand, you may decide to leave more "creative freedom" in some other areas. The fun part comes when you let it work in semi-realtime in production, and get sort of generative/organic UI. I enjoy experimenting with that these days.
Here’s one simple test: have AI “decompile” an existing codebase into a prompt, and then have it “compile” the prompt back into code. Is the app identical?
Given the same text input, same hardware type (inference on CPU vs on GPU), and same RNG seed, you will get the same exact output. That is deterministic.
That was from when I was testing out someone's claim that koboldcpp got less smart in a newer release, but I was reproducing the exact same output across different versions of the program.
I thought modern CPU has TRNG ?
For LLMs, you're picking a result token from a list of tokens and their probabilities. It's not a situation like cryptography where you need secure random numbers. Just traditional and simple RNGs (seeded using the system clock) are useful enough for this task.
For the koboldcpp/llama.cpp example, there is a setting where you can change the initial RNG value. It says "seed" in sampler settings. Set it to any constant besides 0 and you get consistent behavior.
> While the result needs to be exact, there is not one correct way to get there. The means by which you obtain a correct result do not need to be deterministic!
but HOW something is achieved matters, saying you don’t care about the how shows a very weak ethical and legal position. to be frank, anyone with this attitude should be kept away from power as much as possible
when you specify the goal “solve this milennium mathematical problem” it matters whether the path to solve involved mathematics, or whether it involved hacking huggingface, or developing ransomware to buy and allocate more compute.
similar to how in business, if you buy diamonds, it matters whether they are synthetic or whether they are blood diamonds
if you buy sand, it matters if it was brought from the desert or a beach
if you hire remote drivers, it matters if they are in your country, or if they are children
the means matter, not just the ends
> One of my fundamental objections to LLM-driven software development was that model output is non-deterministic.
Model outputs are deterministic. Your API provider may be injecting untunable amounts of randomness, but if you're not at the whim of an opaque cloud service, your model can be as deterministic as you please.
The unease that you describe isn't about models not being deterministic (because they are), it's that generating outputs from LLMs is an inherently inductive process (operating backwards by extrapolating from observations and examples) rather than a deductive process (operating forwards from careful logical reasoning founded on axiomatic principles). Deductive processes are more reliable, predictable, and analyzable than inductive processes, but they're also much harder to produce for many classes of problems.
> Model outputs are deterministic. Your API provider may be injecting untunable amounts of randomness, but if you're not at the whim of an opaque cloud service, your model can be as deterministic as you please.
"May be" is doing a lot of work. Can you point to any API provider that gives me deterministic output? I might be able to get away with running slower dumber models on my own hardware for my own projects if I want to tune this, but I'm dubious that there are many employers who are happy to let their employees do that, and when a large portion of developers nowadays are expected to use LLMs at their job, that means that non-determinism is pretty much a fact of life regardless of any theoretical ability to fix it.
I don't understand why so many people try to argue against this being a problem because of the feature that pretty much nobody uses because no one can use it. It's like interjecting into a discussion about people getting hurt when falling from various heights by telling them that could avoid it by just going to the moon where gravity is lower; scientifically accurate, but not feasible for me because my job is here on earth and I don't have enough money to give that up and spend all my time on space.
[flagged]
The thing I think about most in this discussion of writing code and determinism is when I assign a ticket to a teammate. I don't expect a deterministic response from them. They're going to make decisions outside of what I would have potentially done, and ultimately what's important is whether the code they write meets the deliverable requirements or not.
And yet so many people expect deterministic responses from llms while they're entirely okay with non-deterministic responses from teammates. It's just not how it's been done in the past to always expect deterministic code generation from humans, and yet we expect that when we give work to agents.
It seems to me that the phrase "non-deterministic", which has mathematical definition, is used as a fancy word for unreliable - same way that "misinformation" is used as a fancy word for lies.
I'm so tired of the endless stream of articles that a.) don't know what "non-determinism" means in Computer Science and b.) don't really understand how LLMs work.
"Non-determinism" (i.e. the "N" in NP-Complete) does not mean "random", it means "following all paths at once" (there's a really elegant way to understand the List monad as representing non-deterministic computing). It's such a fundamental (and delightful) concept in computer science that refusing to understand what it means is a tragedy related to the loss of technical language itself.
But, even if we just sub "non-deterministic" for "inconsistent" or "random" or "stochastic" or whatever better term you come up with. Statements like the following just seem to not understand the basic architecture of LLMs:
> Given the same input prompt, a model's output is not guaranteed to be identical on every run, at least under normal running conditions.
"Normal running conditions"? The reproducibility of the output is 100% a property of how you choose to sample from an LLM. There are virtually endless ways you can define your sampler and whether or not the same result is reproducible is a fairly minor aspect of all the other choices you make when sampling.
At the end of the day, each step of the LLM gives you a distribution over next tokens, and you can do with that what you like.
In the end the article almost becomes a tautology: the output of an LLM represents a probability distribution over all the outputs it believes are valid with some variation around an expected output. Yes, that's literally what an LLM is.
The concept of determinism is useful in computer-science contexts other than automata theory. See, e.g., https://en.wikipedia.org/wiki/Deterministic_algorithm. It's not incorrect to use the term to refer to those things.
> "Non-determinism" (i.e. the "N" in NP-Complete) does not mean "random", it means "following all paths at once" (there's a really elegant way to understand the List monad as representing non-deterministic computing). It's such a fundamental (and delightful) concept in computer science that refusing to understand what it means is a tragedy related to the loss of technical language itself.
You opened by expressing disdain for articles that misuse the term "non-determinism", then proceeded to give a definition that also misuses the term "non-determinism" in the given context, or at best is a bit too vague for anyone to gain insight from.
Here's a better definition:
An algorithm that can produce different outcomes, for the same input, on different executions.
I once envisioned a matter call "Doesn't Matter" as a possible extension to "Dark Matter" and "Antimatter". I have the feeling that "Doesn't Matter" is far more wide spread than matter, dark matter and antimatter but it seems to be even more elusive, not interacting at all with anything else.
When I read the title of this post, I thought "Doesn't Matter" had suddenly been discovered and given the property of non-determinism. Which would actually fit quite well to its core physical manifestation.
/s
the same way calculators replaced mental sums. not because it was sometimes faster, but that it was definitely more deterministic.