It may surprise some people here to see that Andrew is warming up to using LLMs to discover bugs (inspired by results from SQLlite) and considers it a tool on the path to getting to bug free software.
> It may surprise some people here to see that Andrew is warming up to using LLMs to discover bugs (inspired by results from SQLlite) and considers it a tool on the path to getting to bug free software.
Someone in the thread below says their bug was closed because of mentioning that they use AI to confirm the bug they had encountered. Is he going to go back and reopen all of those now that he learned what pretty much everyone else already knew?
> Someone in the thread below says their bug was closed because of mentioning that they use AI to confirm the bug they had encountered
I don't know about this particular case, but if I saw someone report a bug and as evidence claim they had X Y and Z LLMs verify it I would be pretty upset. If you're going to use an LLM to make a replication, just do that and give me the replication, don't point to your notoriously error-prone tools as though they lend your report credence.
It's in a similar vein to people who reply to questions with "well Claude says: <chat transcript dump>"
I feel very torn as a maintainer on this, since the only way to respond to the increased noise from AI has been to have AI do the research for me to extract all the links and line numbers I used to have to find by hand to explain why the PR needs more effort to be completed. But I also will be highly dismissive of any submitter who just posts AI text without cleaning it up first. It feels hypocritical, but the alternative is that I just can’t respond to most people instead due to limited bandwidth. I mark if something comes directly from the LLM though and to try to express my degree of confidence in its claims.
Seems like we might need to go to different issue tracking model for AI reports. Such reports must not only provide a bug report/issue but must provide a fix as well. That fix must then be reasoned back to why it solves the bug, why it is minimal, why it is unlikely to silently cause new bugs. It can't introduce new capabilities/side-effects. The platform verifies the claims in the report using another LLM. Then the tests are run, which are no longer publicly accessible to avoid slipping malicious code through the cracks. When okay the patch can go into staging. Perhaps a special version containing only accumulated AI fixes.
Human reproductions are notoriously error prone whether ai assisted or not right? I’m not sure what the analogy is between “llms are error prone” and “thoughtlessly copy-pasting something from Claude” is.
Human reproductions don't have to be bad. It feels just as justified to push back on a bad bug report whether human or AI and say "I don't have enough to go on here".
If a report is improved and becomes actionable, that's great.
He specifically says in that presentation that they are open to LLMs helping them get to bug free, but because the language is still in flux, they would rather prioritise bugs users actually find rather than those found by LLMs, essentially with the intent of unblocking people rather than wasting time fixing things that may need to be fixed again or be wasted work come the next update.
My understanding of the situation is that the user did find a bug themselves, because it literally broke on their own code, and then they just had the LLM help them verify their theory of the bug that they already had.
Shouldn't everybody already know that while using AI to find bugs for oneself is amazingly efficient, using AI to submit bug reports for others is quite the opposite? Burden of verification and all.
Exactly, the main issue isn't the LLM creating or confirming the report. Rather, the maintainer has no idea how it was prompted, and the results may be completely wrong. Some LLMs also have a bad habit of trying to please the user, confirming their biases.
I don't care how it was prompted any more than I care about what exact thoughts went through your head before you realized it was a bug. The problem isn't usually a lack of information, but the abundance of non-information. Chatbots and the people who use them to pen bug reports frequently seem to have had "no time to make it shorter", as Blaise Pascal put it.
That you used an LLM to "confirm" the bug is an example of such non-information. The reproduction testcase and/or the reproduction steps and description of the expected outcome confirm the bug.
Neither of those are my understanding of what happened: a human found a bug, and then had an LLM verify their theory of the issue. To me, that's just extra due diligence and pretty weird to use as grounds to ignore.
> that I had every LLM check it to confirm it's a bug
I'm primarily objecting the way he phrased it, as opposed to just saying "I've tested and confirmed and reproduced the bugs". Instead, it sounds uncertain and detached, like he asked the LLMs if it's indeed a bug without further verification.
They want to focus first on bugs hitting people's actual code to unblock them, not theoretical bugs that LLMs can hit by coming up with some contrived code, regardless of the quality of the bug report.
A lot of bug reports aren’t valid, or handle a case that can’t realistically happen or realistically be handled (eg what do you do if you detect a crash while the prior crash is crashing and the logging pipe is throwing errors)
Sure but this is true regardless of the model. Either it found a bug or it didn't—who cares about the intermediary steps or tools used so long as the reporter can reproduce it?
That’s assuming the reporter reproduced it. I’ve seen so much slop these days. Sometimes people post whole conversations in which the final answer from the AI is that the initial report was impossible to fit to the data after reproducing it, then continues in circles arguing that proves the original reported issue was real and the refuted cause isn’t actually refuted (eg it said there was a bug in function A, in code which never used function A — and that example was with the top tier models just released last month)
Nobody refuses penicillin because it came from mold in a dish. If an AI found a cure for a disease, people would ask one question: does it work?
Bug fixes should get the same treatment. A patch is either correct or it isn't. Projects that ban AI-written fixes outright are asking "who wrote this?" instead of "is this right?", and users live with the bug in the meantime.
I get why maintainers are fed up. Review time is scarce, and they're drowning in plausible-looking garbage. But that's a problem with low-quality submissions, not with AI as such. Require tests, require a human who vouches for the patch and will answer for it, and ban repeat offenders. Then hold every patch to that same bar, whoever or whatever wrote it.
But if I called you in the middle of the night to offer you something that will make every 9th sneeze on every second tuesday of March in a leap year less irritating your first reaction wouldn't be to thank me.
If I were making a product to try to eliminate sneezes, and someone said "I noticed an issue with it, and after I looked into your manufacturing process, I think I figured out what's causing it, so I double checked using an AI tool and it added further evidence that the defect I found is legitimate and would be solved by the solution I had come up with", it would be bizarre for me to say "I would have accepted this if you didn't spend the extra time at the end to verify your existing findings with an AI"
I personally think using AI is of no problem for certain cases. It is absolutely pissing off when someone tries to generate slops that too verbose to review only to increase the complexity of the codebase meaninglessly.
I’ve written software for a living in JS, C, Pascal, and Go, and I’ve tried many, many more languages. After working on a project in Zig for a year, I’m convinced that Zig is the best-designed language I’ve tried so far. Haskell comes close. At least, it’s the best-designed language for humans.
It’s not for everyone yet. It’s still unstable, and its ecosystem is small. However, both are improving.
I'd like to learn more about the connection between Zig and Haskell as they seem almost opposite in design and philosophy. I'm pretty sure you just meant they're both "good languages" but if there are parallels I'm missing I'd love to know!
I'm not the person you asked, but I see them both as languages that try to get a lot of mileage out of a few features. Both of them try to have small cores, instead of taking a "maximalist" approach like C++.
> It’s not for everyone yet. It’s still unstable, and its ecosystem is small. However, both are improving.
To be fair, I've heard this for years now, yet the hype keeps mounting. I wasted half an hour this morning debugging a broken Zig project, all because I used a release a few months too new that removed some options used in build.zig
Releasing stable software and caring about backward compatibility is a skill many open-source maintainers don't ever want to engage with. It's so easy saying "our code is not stable, if it breaks good luck to you", but at some point it starts to smell like fear of commitment to running a serious project that people depend on.
Why such high expectations from a 0.x project? They explicitly break backwards compatibility now so they don't have to do it once 1.0 comes. There's nothing wrong with waiting a few years before trying Zig.
My point is that at some point one needs to recognize their project is used for production stuff and make stability the focus, rather than hiding behind the 0.x excuse for years. Call it the Peter Pan syndrome of open source.
For comparison, Elixir started in 2012, and got its 1.0 in 2014, two years later. Zig stable has been ‘a few years’ for literally a decade.
Congratulations to zig team. Good to know they are trying pragmatic approach to LLM now. I left the zig eco-system due to zig core members hostile behaviour toward humans not just LLM, so good to see the change they are becoming pragmatic. For me I am slowly porting the same work to odin language [1].
Background is I created issue and one pull request to fix them in zig compiler version 0.16.1 issue numbers 36812, 36811 (you cannot access them as my account is banned can see my fork at [2]). Respecting the community’s no AI stand. For these specific issues I wrote the issue and code myself and not let AI write it. Spend a lot of time on it. Subsequently without any notice my account was banned because my projects on github using zig uses LLM. This was done without message or any information. My account was banned on zig repository.
I can now understand the other side of coin how bun team might have been treated with disdain when they used LLM.
I wrote an email and left the zig community, have many work in zig but slowly moving them to odin.
I feel personal disdain should not be spilled on to people who are pragmatic on using LLM. I was a very big evangelist of zig for their no LLM stand and promoted them among my community, but with poor treatment by community I just left. You can still see projects I wrote in zig [3].
I have worked with postgreql community since 1997 and python community since 1998. Never felt such hostile community. So all the best and I wish zig continue its progress
> because my projects on github using zig uses LLM
To be clear, we do not block people for merely having LLM-related projects. Obviously we have opinions about LLMs in a broader context, but in terms of rules enforcement, we only care about LLM usage taking place within official Zig spaces.
It's possible you were blocked in error. LLM detection is not foolproof, so unless it's an open-and-shut case, our usual approach is to just unblock if people reach out to us by email.
I would understand if its by error but based on how it was banned it did feel deliberate. Also I have written email to core member before making a decision to leave the community. I think a community can only thrive when they are nice to humans. Here it felt like zig is worse then a oligopoly where few core members can block anyone without any reason and don't even have a courtesy to send a message. I worked with python community since its beginning, same worked with linux community since 1992, never seen such attitude even though both of the projects were driven by benevolent dictators, but they did care about the community around it. I still have one major work [1] in zig, which I will still release but after this won't work on zig. Zig community felt like a hostile ologipoly of few core members without proper due process, when it comes to treating humans.
Well of course the block itself was deliberate. By "in error" I meant that we might have misjudged the text as being LLM.
Who did you email, when, and did you get a response? We've all been very busy leading up to the release, and most of the team is also currently traveling for SYCL.
Regarding sending a message, consider our side: We're tired of having our time wasted on slop, so we don't also want to have to write formal block notifications to people we block on suspicion of using LLMs. I'm fully aware that this can lead to an unfortunate situation like this, however. For what it's worth, this is why we want to move to an invite tree for Zig development; it'll allow us to create a high-trust environment where this kind of suspicion is unnecessary.
I don't think the zig community is necessarily hostile to LLMs, they just don't want it in their "maintained by 10-ish core people not even full time" language impl. Mitchell Hashimoto, a big zig contributor (both money and effort), for example, uses LLMs a lot and no particular shade is thrown.
This is from my own first hand experience both on ziggit forums as well as raising bugs in zig repository. My account on codeberg is still banned on zig repository without any single comment, my issues or pull request raised on that project just went in ether without being accessible, if you see the issue yourself could have made the decision if it was warranted to ban the account, but that right also taken by the core contributor out of spite and disdain for LLM.
I have been in open source world with linux kernel since 1992, never every had seen a community so hostile, especially towards humans who spend time and efforts just out of curiosity, inquisitiveness and trying to support some simple alternative when odds are already against them.
Facing hostile behavior from inside the community made me switch to odin language where they also do not use LLM but are pragmatic about people using it. Cannot comment on Mitchell because I do not contribute with money.
> my account was banned because my projects on github using zig uses LLM.
Its not Zig. Its codeberg. Codeberg has a strict no-AI policy. Codeberg is ALL about the community and not the lone hacker. I think its a good policy to prevent overload and clearly distinguish project ecosystems from each other.
I mean I pr'd something (and it was not accepted) but the grounds have nothing to do with LLMs. I'm pretty vocal about using LLMs to write zig code. Obviously I did not use LLMs in my PR
I would be fine if they reject the PR, given I was first time raising it to fix POSIX related issue. It was just few lines of code. But my account was banned all issues removed my own PR not visible and looked like tyrannical approach not a benevolent dictator approach. I think being nice to another human being is the basis for building community. Here felt like if you are among the few who dictate the terms there is no process just my way or highway.
It is not codeberg, I can work on other repository properly on it. Its only zig repository where my account is blocked. My initial thought was same as you, only later realized it. Also on ziggit these people did the same even though based on policy when I put the post i clearly mentioned use of LLM. This all without any proper communication. After working on zig for over 6 months, now I am moving slowly to odin [1] language. This decision I did not take lightly, I had spend significant resources in zig earlier and now moving away.
don't let people gaslight you here. i mean, look at how the creator treated a person who was a zig evangelist and was donating tens of thousands of dollars to the project:
"Jarred was already writing slop well before he had access to LLMs."
"Jarred was a stinky manager. Poor communication, unrealistic expectations, low empathy, no experience. Just a total shit show"
"When Jarred announced the Rust rewrite, we were ecstatic. It seemed too good to be true. I have to admit, I didn't think the technology was there, to pull off this stunt. But he did it, and now I'm metaphorically sipping delicious tea from a mug that says "It Tastes Like It's Not My Problem Anymore"." [1]
or what about when he called people "monkeys" because he didn't like their product [2]
or when their vp of "community" called github employees clowns [3]
so don't let people tell you that there isn't a pattern here. and when this kind of behaviour comes from the top it shapes the rest of the community
I’m actually going to be a little bit sad when Zig does hit 1.0 and slows down with the rewrites of core functionality. It’s not for everyone, if you have SLAs on production I can see you getting sick of it quickly, but no programming has ever felt as alive for me as right now in Zig. It’s like the various JavaScript heydays of Bluebird, Node, Babel, ES6 etc all rolled into one.
They seem to be doing fine. Tigerbeetle and Ghostty are two projects that continue to do well and are written in Zig. The foundation’s funding also seems to be doing well. They are continuing to make releases.
Strange in what way? Mitchell has also stated that he reviews every line of code that goes into Ghostty so I don't think its quite the same as the Bun rewrite.
He can say that … but after reviewing the 1000th PR from a LLM and only finding minor issues that really go down to subjective taste, I think he may start to reconsider that.
Really, even after a LLM has gone through a review and fixed issues found? I almost never see anything relevant since around Opus 5 came out, and now with Fable and Opus 5.5 whatever I think is wrong is almost certainly right.
On my small personal projects I seldom find any mistakes. I use Chinese models on those. GLM, DeepSeek, MiMo.
At work I use Claude, but it is a considerably larger and quite complex codebase. I use LLMs a lot, but it is a common weekly occurrence for me to correct misconceptions, bugs, or overengineering from Opus/Sonnet (Opus plans and reviews, Sonnet implements).
Filed a compiler bug related to dwarf tables that screws up debugging and line of code coverage that they completely ignored, just because I mentioned that I had every LLM check it to confirm it's a bug, since all I know is what kcov and every coverage tool generates incorrect coverage data for my repo, for 100% certain.
Reading their comment it sounds like they wanted to confirm the bug existed with AI, not sure they ever said they had AI write the code. Why have such a dumb policy?
The things you've quoted and your conclusion feel at odds. They just don't want AI contributions, and they, like a lot of the world, are bored of hearing about AI. Is it really too much to ask?
If there's an edict that no one is allowed to bring up the topic, how can someone change this part of the code of conduct?
Asking non-rhetorically. It seems like one position "ai in any circumstance = bad" is being enforced. The commenter above didn't even understand why he was ignored.
Interesting to hear! We maintain the project Antfly (entirely zig) and have been nervous about bringing issues to the zig folks or asking questions because of our ai usage.
We’re quite knowledgeable and thoughtful folks fwiw
I think I’ve seen you are a core team member or contributor? I remember your tag?
So for instance because of the size of our codebase our project has pushed Zig to some of the edges, specifically we end up hitting a bug when using llvm and zig on arm64 (Mac and Linux) where it seems to be caused by some configuration Zig passes through to LLVM. I’ve used codex and claude to help me diagnose and find the bug (we use nix’s glibc zig to circumvent the problem now). I now understand the root cause but am not sure what the proper fix would be. But I’ve not known whether or not even raising the issue would break the terms of contributing? Would raising the issue break the implicit agreement?
I would suggest just starting by filling out the bug report template, i.e. steps to reproduce, expected behavior, and observed behavior. If for some reason you can't provide a reasonable reproduction, the symptoms on their own can sometimes be enough for us to make an educated guess at what's going wrong.
Regarding whether you should post the root cause analysis: Per the current policy, the answer would have to be no.
I do personally have more nuanced thoughts on this, and I started typing them out... but then I realized that my reply was getting dangerously close to blog post length, so I decided to restrain myself and commit to turning it into an actual blog post later. In a nutshell, though, the problem is that even if there is such a thing as responsible use of LLMs for bug analysis, the only way we can currently be confident that someone possesses the required qualities for that is by working with them for a while.
I understand, maybe one day I'll have the chance to hear the blog posts' worth of thought on the matter!
https://codeberg.org/ziglang/zig/issues/37060 I opened the issue and can give you the agent generated RCA on the matter if you all want it in issue 604 on antfly's github but I understand that's against policy and totally respect that.
Appreciate all the work you guys do and have been following the whole Zig project since inception fwiw!
The second part affirms that it is centralized. There is nothing wrong with that except saying that because you can leave the community and go elsewhere, it is ipso facto decentralized.
I’ve not found the zig folks to ban dissent, they engage in a lot of thoughtful dialog. Just because they’ve made a different decision for how they take contributions than other people agree with doesn’t make them a cult?
>Even in projects where I use LLM liberally, I would rather not read or have to engage with your LLM output.
I don't want to read LLM output on a project either. But you may have misread the quoted section, it isn't saying "you can't post LLM output," it is saying you can't ask an LLM to advise/critique your own comment before posting it.
>>If you use a chatbot to give you advice on a comment on the issue tracker, that comment is unwelcome.
Wow I thought surely they wouldn't object to using AI to confirm bugs, but they really do.
Tbf I guess as a popular open source project not using AI to fix bugs, they probably already have more open bugs than they can ever fix so it doesn't really help them for people to find more.
I would imagine his bug was actually ignored just because Zig has 2700 open bugs, rather than some AI policy violation.
Gotta admit it's weird that they haven't set up Claude to fix them! Tbf my threshold for "AI now writes good enough code" was Astra or Opus 5.5 and they only just released that.
They don’t, at least not any more. Andrew Kelley has explicitly stated that he sees the value of using LLMs to uncover bugs. Inspired by sqllite project.
They may just be taking the slow route of rejecting by default until they can be sure that the usage of LLMs provides long term value. I don’t see anything wrong with that. If you’re writing robust software, using LLMs at this stage is a bit of a gamble. We don’t fully know the long term effects on code quality yet.
He sees the value of using LLMs... only after performing NASA-level verification and fully comprehensive fuzzing using traditional tools. Since Zig has not done those things, LLM-generated bugs are still banned.
Project maintainers know what level of AI use is acceptable for themselves, and quantifying it and enforcing it for random contributors is very difficult.
It's clearly a hobby project (constant breakages, the maintainer getting into politics, rejecting some safety mechanisms, the anti-LLM crusade, a strange focus on esoteric targets with little to no commercial significance), but the maintainer does not admit that it is a hobby project.
Encountering something that seems incorrect, and proving something is a bug and not just YOUR user error, AND reproducing it minimally - turns out to not be easy when it's a low-level compiler issue, and YOU'RE not an expert, especially when it relates to Dwarf tables...
Typically, any time you think you've found a compiler error, you're using it wrong...
If it's a real bug, you can just report what you know and leave it at that. You don't need to embellish it with random guesses about a root cause.
Okay: "I compile this input and the linker crashes."
Not okay: "I compile this input and the linker crashes. Also here's 10 paragraphs of slop about dwarf tables, which I can't even evaluate the accuracy of since I'm not an expert."
> Typically, any time you think you've found a compiler error, you're using it wrong...
Yes, if you can't figure out if it's a bug, the bug tracker is the wrong place to get help. Ask in a community forum instead.
> Not okay: "I compile this input and the linker crashes. Also here's 10 paragraphs of slop about dwarf tables, which I can't even evaluate the accuracy of since I'm not an expert."
It's a 20-line file with a few commands to run it to reproduce it. Not 10 paragraphs of slop.
Presumably that is much more helpful than - here's my gigantic repo, good luck running my tests, also good luck finding the bug.
> here's my gigantic repo, good luck running my tests
What about my comment made you think I was suggesting not to give repro steps?
> also good luck finding the bug
But yes actually, this half is true. It's better to give them no extra information, than to give too much information that you have no idea if it's true or not.
I reported a bug and waited for a year. one day it was merged. the thing is that there are a looot of bugs and the team is small, so your turn might not have come up yet
Seems like "parallel construction" where you show you actually understand and can explain the issue independent of an LLM would be the way to go. No need to mention how you found the bug as long as you can explain what the bug is, why it matters, and how to repro.
Lying to get around a project policy you disagree with is immature. They have the right to run things the way they want, either accept their rules or leave the project be.
If they really insist on no LLM involvement at all then they're going to fall behind and lose out. Your LLM use case seems really very conservative - you didn't write any code with it, you just use it to confirm the bug. There are many OSS projects that are also taking a similar hardline against LLMs - people are going to fork them and then move on. I've had LLMs fix bugs/add features to a couple of projects like this and, well, they're missing out on the fixes/added features that I'm using locally.
It's remarkable how easily these idiots can be conditioned into parroting the usual FOMO lines without question. Same people would've had me think I'm a fucking moron for not buying ugly monkey pictures during the NFT bubble.
I'm thinking their lives must be absolutely dreadful for this constant fear of being left behind.
How do you evaluate this? Have you tried looking for a job? Or compared your project with a similar project where people are using AI and you are not? Or you’re just staying put and waiting to see what happens?
Can you offer any specifics? The ToC looks quite extensive, which to me indicates a lot of design decisions needed to be made (probably involving many people) and unclear how that process might be accelerated by AI.
Unless you're suggesting the language design should also be vibed together?
> Unless you're suggesting the language design should also be vibed together?
Sounds like a fun little project, have a bunch of AI pushers fork Zig and see if they can do a better job. I want to see results, not snarky HN comments. After all this progress, ChatGPT should be able to one-shot a better language since AI is so good now... right?
One-shot, no, but there are a bunch of people pretty much solo-building their personal ideal language with AI and it's going quite well. You need to know just enough about language design to be dangerous, but you don't need to be a seasoned pro.
It seems like such a strange thing to do, building a language that you aren't going to write by hand. It's guaranteed to perform worse at higher cost, fill up a lot more of the context window, and burn a ton more reasoning tokens.
If you're using AI, a language with a large training set is going to win.
> If you're using AI, a language with a large training set is going to win
Not necessarily? What if the training set contains an overwhelming amount if bad code written by neophytes? I imagine Python quality by the LLM suffers from this, for example.
What if the language has extremely confusing syntax constructs (like early php) or bad or no conventions (suppose the standard library has somecollection.put(key, value) sometimes and othercollection.put(value, key) other times), and individual code authors just pick what they want adhoc
There's a lot of RL that goes into this, you're not just training on bad code. Python is one of the programming languages that LLMs consistently perform best at.
I guess it depends on whether you will write everything in an AI assisted fashion or not. There's benefits to languages that are quick and easy to read by the author even if the LLM is doing the writing, because most code still benefits from human review above and beyond the review that agents provide. Language popularity certainly helps but it seems like for moderately popular languages [1] the cost you pay for a lack of popularity is quite modest.
A language you just created isn't going to be moderately popular, so it's just going to put you at a disadvantage -- and you're not even going to be writing in it, so why the self-kneecapping?
I mean what does "put you at a disadvantage" even mean concretely? To use a less popular language, it means you need to load up context related to the semantics of your language, load context on how to invoke tools to make sure the syntax with your language is correct, load up context related to each tool call you make (which will be more numerous in a niche language), and load up context on architectural decisions that might be specific to your language. All of this is simply a token cost. By forcing a model to load an initial amount of context per harness turn you also effectively shorten the max context window beyond which the model becomes stupid (which itself is much shorter than the max context length.)
Obviously it's not like people are specifically trimming each and every prompt they give a model to tokenmax their models to get the best output / input prompt, we instead live in a spectrum of how many tokens of input and context we're willing to provide to a model to make progress. If the cost of those tokens is low enough for the problem domain you're working in, then it's fine. For some the readability of a personal language may outstrip any of the token costs that one needs to pay to use it. Alternatively maybe you want something like an array language (J, K, APL, etc) which allows array programming and optimizations that conventional PLs just can't do. Maybe you want your language to compile to a target that is highly portable. There's actually a lot of stuff out there that previously wasn't feasible but with LLMs-as-force-multiplier absolutely is.
I also suspect the space is a continuum. There may be pareto optimal points, such as DSLs built atop languages, that are both highly readable but also fairly token efficient.
It's both a token cost and a performance cost; there's only so much that documentation can do, compared to a ton of RL on top of millions of lines of examples. The space is a continuum, but the more you stray from the trained path the higher the cost you pay.
> the more you stray from the trained path the higher the cost you pay.
Note the LLMs are trained on language semantics far beyond the mainstream ones, so language design can become quite exotic without straying too much from the training. You really do have to measure these things, I don’t see how you can make a confident assertion without data.
I don't think those assertions about AI development necessarily hold. And I think it'd be a depressing future if we can't ever have anything new or better that wasn't popular in the training set as of November 2025.
But you’re very clearly wrong. Just go ahead and try it: write your own language or just a DSL, write a brief description of how it works , let a LLM use it. Many of us have done this and everyone knows that it works extremely well, to most people’s surprise.
I'd suggest benchmarking the outcomes. It's going to "work", but at a higher cost with worse ontcomes compared to an existing language with a large training set.
I compare one algorithm across several programming languages and backends. There is no training data for Mech in the LLM yet it beats most other implementations in perf, which were optimized by LLM.
Maybe human performance engineers trained in these languages could write better implementations. But to answer the question of whether the LLM could write more performant code in languages it’s trained in versus languages it’s not, this comparison is at least illustartive.
I don't think it's particularly sarcastic. Its bitter, of course, because the world we're working towards kind of sucks. I'd like to opt out of it, but it's being forced on me.
The best I can do is understand its edges and try to find some advantage that leaves me well off enough to stave off the worst effects.
Been thinking about doing this myself (did some PL in grad school but it's been a long time), but I find myself wanting to reach for a Scheme (using macros to grow the language I want) and customize it or build something atop Janet.
Curious why you wanted a more ML / Rust / Scala inspired syntax. (Personal preference here is totally valid btw, just curious.)
None of these systems can. They need enormous training. They need alignment and reinforcement. They need harnesses. And most importantly they need a human that knows how to write and develop a C compiler.
The ISO specifications are not sufficient. Neither are the System V guidelines. Not even spec tests and compcert.
Claude can one-shot a C compiler just fine. It just won't produce great code. But with all the C code out there in the wild, it won't have a shortage of things to test on, which is mostly what coding agents need to complete a task.
You're not going to one-shot it, but over the course of a couple of months of evenings you could come up with something usable/interesting if you manage the LLM well.
Yep, it's true, but the reality of what it would take to actually fork and commence with healthy use of AI alongside with the elusive soft/hard stance needed to lead technically and socially is probably a bigger challenge than anyone is willing to take on! Soo.. slow and (rather?) well it goes for Zig.
This is getting really close to 1.0 now? I guess may be 2 more release and 1 more RC? That is 2028. I am looking forward to stabilising the language and the core team has previously indicated that exporting ir will be a supported feature.
Once that is done, @dnautics is working on memory safety for zig [1]. A Checker for Lifetimes and other Refinement types.
Then we can start the Rust vs Zig Debate again. Hopefully not too late.
Given the efforts we are already seeing where people post "I rewrote X in Rust in 24 hours using the latest agental harness", I don't think there is any kind of "too late".
Either Zig gets good enough to be a worthy re-write target, or it doesn't. But I suspect that isn't even what they want.
I imagine languages are going to bifurcate into ones that go all in on agent experience and ones that go all in on human experience. And Zig seems to be in the latter group. I am confident that there will always be some niche market for languages lovingly crafted for humans. But the criteria we will use to define success within these groups of languages will be different.
The fix could not be backported to LLVM 22 because it changed the LLVM library ABI. We also could not skip straight to LLVM 23 because that would make life harder for distro package maintainers.
Was loop vectorization by chance enabled in 0.15.2 and disabled since 0.16.0? One of my projects got 14% slower after upgrading to 0.16. I didn't dig in yet but I assumed the Io vtable was the cause (it does a lot of file io). But if the versions line up maybe it's actually loop vectorization.
I just noticed it today upgrading to 0.17.0 from 0.15.2, while doing bitwise operations a lot of large integers. I haven't looked at a debugger yet, but my guess is that would benefit a lot from loop vectorization.
Nope, but why would it matter? Nobody is stopping you from writing Zig code with an LLM, the policy you are referring to is only relevant to the compiler source code.
Not the parent, but some people may feel like it's betting on a bad horse. Zig already has a small ecosystem, so pushing culture that may make it even smaller while having core development stagnate in an ever-accelerating software world may be seen as a tougher sell.
Back in a more adult age, the way that this was handled was "We've benefited from our past collaboration, but we believe a new direction is needed going forward."
For some reason--COVID brain rot, poorly-socialized people coming online, general increase in viciousness in the population, who knows!--people have forgotten the utility and purpose of boring polite manners and communication.
Would recommend checking out the Zig state of the (tagged) union as well.
https://youtu.be/zwi5b5xSsKA?is=PTjJJjSnVMdRuZag
It may surprise some people here to see that Andrew is warming up to using LLMs to discover bugs (inspired by results from SQLlite) and considers it a tool on the path to getting to bug free software.
> It may surprise some people here to see that Andrew is warming up to using LLMs to discover bugs (inspired by results from SQLlite) and considers it a tool on the path to getting to bug free software.
Someone in the thread below says their bug was closed because of mentioning that they use AI to confirm the bug they had encountered. Is he going to go back and reopen all of those now that he learned what pretty much everyone else already knew?
> Someone in the thread below says their bug was closed because of mentioning that they use AI to confirm the bug they had encountered
I don't know about this particular case, but if I saw someone report a bug and as evidence claim they had X Y and Z LLMs verify it I would be pretty upset. If you're going to use an LLM to make a replication, just do that and give me the replication, don't point to your notoriously error-prone tools as though they lend your report credence.
It's in a similar vein to people who reply to questions with "well Claude says: <chat transcript dump>"
I feel very torn as a maintainer on this, since the only way to respond to the increased noise from AI has been to have AI do the research for me to extract all the links and line numbers I used to have to find by hand to explain why the PR needs more effort to be completed. But I also will be highly dismissive of any submitter who just posts AI text without cleaning it up first. It feels hypocritical, but the alternative is that I just can’t respond to most people instead due to limited bandwidth. I mark if something comes directly from the LLM though and to try to express my degree of confidence in its claims.
Seems like we might need to go to different issue tracking model for AI reports. Such reports must not only provide a bug report/issue but must provide a fix as well. That fix must then be reasoned back to why it solves the bug, why it is minimal, why it is unlikely to silently cause new bugs. It can't introduce new capabilities/side-effects. The platform verifies the claims in the report using another LLM. Then the tests are run, which are no longer publicly accessible to avoid slipping malicious code through the cracks. When okay the patch can go into staging. Perhaps a special version containing only accumulated AI fixes.
> It's in a similar vein to people who reply to questions with "well Claude says: <chat transcript dump>"
or substantially worse: "<chat transcript dump>"
Human reproductions are notoriously error prone whether ai assisted or not right? I’m not sure what the analogy is between “llms are error prone” and “thoughtlessly copy-pasting something from Claude” is.
Human reproductions don't have to be bad. It feels just as justified to push back on a bad bug report whether human or AI and say "I don't have enough to go on here".
If a report is improved and becomes actionable, that's great.
Hey, we're all the result of human reproductions.
Might be my favorite comment ever now.
He specifically says in that presentation that they are open to LLMs helping them get to bug free, but because the language is still in flux, they would rather prioritise bugs users actually find rather than those found by LLMs, essentially with the intent of unblocking people rather than wasting time fixing things that may need to be fixed again or be wasted work come the next update.
My understanding of the situation is that the user did find a bug themselves, because it literally broke on their own code, and then they just had the LLM help them verify their theory of the bug that they already had.
Shouldn't everybody already know that while using AI to find bugs for oneself is amazingly efficient, using AI to submit bug reports for others is quite the opposite? Burden of verification and all.
Exactly, the main issue isn't the LLM creating or confirming the report. Rather, the maintainer has no idea how it was prompted, and the results may be completely wrong. Some LLMs also have a bad habit of trying to please the user, confirming their biases.
I don't care how it was prompted any more than I care about what exact thoughts went through your head before you realized it was a bug. The problem isn't usually a lack of information, but the abundance of non-information. Chatbots and the people who use them to pen bug reports frequently seem to have had "no time to make it shorter", as Blaise Pascal put it.
That you used an LLM to "confirm" the bug is an example of such non-information. The reproduction testcase and/or the reproduction steps and description of the expected outcome confirm the bug.
> Is he going to go back and reopen all of those now that he learned what pretty much everyone else already knew?
In the state of the tagged video he says still not accepting AI submissions until a certain set of preconditions is met. So... No?
So "warming up to" is moving at a glacial pace, in both senses of the word
Fine but you could have checked the primary source.
Note the distinction:
- using the LLM to find (possible) bugs and a human confirms it by testing, reviewing, etc.
- using the LLM to find and confirm the bug without the human confirming it
Neither of those are my understanding of what happened: a human found a bug, and then had an LLM verify their theory of the issue. To me, that's just extra due diligence and pretty weird to use as grounds to ignore.
You are referring to this comment right? https://news.ycombinator.com/item?id=49938732
> that I had every LLM check it to confirm it's a bug
I'm primarily objecting the way he phrased it, as opposed to just saying "I've tested and confirmed and reproduced the bugs". Instead, it sounds uncertain and detached, like he asked the LLMs if it's indeed a bug without further verification.
I suspect they can't easily tell the difference between a high quality LLM based contribution and a low quality one.
They want to focus first on bugs hitting people's actual code to unblock them, not theoretical bugs that LLMs can hit by coming up with some contrived code, regardless of the quality of the bug report.
There's a difference between Opus 4.5 and Astra 6
With regards to whether when it finds a legitimate bug, it should be given weight?
With regards to whether a blanket ban has a payoff
Why would the difference matter? A bug report is valid or it isn't.
A lot of bug reports aren’t valid, or handle a case that can’t realistically happen or realistically be handled (eg what do you do if you detect a crash while the prior crash is crashing and the logging pipe is throwing errors)
Sure but this is true regardless of the model. Either it found a bug or it didn't—who cares about the intermediary steps or tools used so long as the reporter can reproduce it?
That’s assuming the reporter reproduced it. I’ve seen so much slop these days. Sometimes people post whole conversations in which the final answer from the AI is that the initial report was impossible to fit to the data after reproducing it, then continues in circles arguing that proves the original reported issue was real and the refuted cause isn’t actually refuted (eg it said there was a bug in function A, in code which never used function A — and that example was with the top tier models just released last month)
citation needed
Scroll down: https://news.ycombinator.com/item?id=49938732
Nobody refuses penicillin because it came from mold in a dish. If an AI found a cure for a disease, people would ask one question: does it work?
Bug fixes should get the same treatment. A patch is either correct or it isn't. Projects that ban AI-written fixes outright are asking "who wrote this?" instead of "is this right?", and users live with the bug in the meantime.
I get why maintainers are fed up. Review time is scarce, and they're drowning in plausible-looking garbage. But that's a problem with low-quality submissions, not with AI as such. Require tests, require a human who vouches for the patch and will answer for it, and ban repeat offenders. Then hold every patch to that same bar, whoever or whatever wrote it.
But if I called you in the middle of the night to offer you something that will make every 9th sneeze on every second tuesday of March in a leap year less irritating your first reaction wouldn't be to thank me.
If I were making a product to try to eliminate sneezes, and someone said "I noticed an issue with it, and after I looked into your manufacturing process, I think I figured out what's causing it, so I double checked using an AI tool and it added further evidence that the defect I found is legitimate and would be solved by the solution I had come up with", it would be bizarre for me to say "I would have accepted this if you didn't spend the extra time at the end to verify your existing findings with an AI"
Warming up to LLMS... explicitly _after_ full branch-coverage and fuzzing.
He'll reevaluate the LLM bug-finding situation in a few years.
https://youtu.be/zwi5b5xSsKA?si=w6zZN6AtIvJS9MxP&t=2084
I personally think using AI is of no problem for certain cases. It is absolutely pissing off when someone tries to generate slops that too verbose to review only to increase the complexity of the codebase meaninglessly.
I’ve written software for a living in JS, C, Pascal, and Go, and I’ve tried many, many more languages. After working on a project in Zig for a year, I’m convinced that Zig is the best-designed language I’ve tried so far. Haskell comes close. At least, it’s the best-designed language for humans.
It’s not for everyone yet. It’s still unstable, and its ecosystem is small. However, both are improving.
Zig couldn't be further away from Haskell in features and community culture.
I'd like to learn more about the connection between Zig and Haskell as they seem almost opposite in design and philosophy. I'm pretty sure you just meant they're both "good languages" but if there are parallels I'm missing I'd love to know!
I'm not the person you asked, but I see them both as languages that try to get a lot of mileage out of a few features. Both of them try to have small cores, instead of taking a "maximalist" approach like C++.
> It’s not for everyone yet. It’s still unstable, and its ecosystem is small. However, both are improving.
To be fair, I've heard this for years now, yet the hype keeps mounting. I wasted half an hour this morning debugging a broken Zig project, all because I used a release a few months too new that removed some options used in build.zig
Releasing stable software and caring about backward compatibility is a skill many open-source maintainers don't ever want to engage with. It's so easy saying "our code is not stable, if it breaks good luck to you", but at some point it starts to smell like fear of commitment to running a serious project that people depend on.
Why such high expectations from a 0.x project? They explicitly break backwards compatibility now so they don't have to do it once 1.0 comes. There's nothing wrong with waiting a few years before trying Zig.
My point is that at some point one needs to recognize their project is used for production stuff and make stability the focus, rather than hiding behind the 0.x excuse for years. Call it the Peter Pan syndrome of open source.
For comparison, Elixir started in 2012, and got its 1.0 in 2014, two years later. Zig stable has been ‘a few years’ for literally a decade.
I mean. Zig is a really freaking good language for LLMs (in my hands) too
What LLM? I used claude and it just cracks up on zig 0.16. Terrible.
Claude. I have several things written in zig:
- wasm lalign utility (integrated in opengenepool.vidalalabs.com, not obvious how to trigger it but you can inspect the wasm package)
- real-time DNA gel lane assignment
- proprietary (Claude also used zig to reverse engineer the usb wire protocol) industrial camera driver
Last two:
https://x.com/DNAutics/status/2099583335940936175?s=20
If you want weirder shit:
https://github.com/ityonemo/2b4m
I'm always impressed by Zig's targets support. I'm starting to think that it might be the only language that competes with C on this regard.
I'm looking forward to see what the new build integration can unlock on the tooling side.
What I'm looking for the most for the next release(s):
- New stackless coroutine IO implementation
- First class fuzzer tooling
> competes with C
Outcompetes C even? I'm especially exited for SpirV. Would be great to use Zig for both CPU and GPU programming.
Especially in WebGPU, where WGSL tooling is very early.
Absolutely
Congratulations to zig team. Good to know they are trying pragmatic approach to LLM now. I left the zig eco-system due to zig core members hostile behaviour toward humans not just LLM, so good to see the change they are becoming pragmatic. For me I am slowly porting the same work to odin language [1].
Background is I created issue and one pull request to fix them in zig compiler version 0.16.1 issue numbers 36812, 36811 (you cannot access them as my account is banned can see my fork at [2]). Respecting the community’s no AI stand. For these specific issues I wrote the issue and code myself and not let AI write it. Spend a lot of time on it. Subsequently without any notice my account was banned because my projects on github using zig uses LLM. This was done without message or any information. My account was banned on zig repository.
I can now understand the other side of coin how bun team might have been treated with disdain when they used LLM.
I wrote an email and left the zig community, have many work in zig but slowly moving them to odin.
I feel personal disdain should not be spilled on to people who are pragmatic on using LLM. I was a very big evangelist of zig for their no LLM stand and promoted them among my community, but with poor treatment by community I just left. You can still see projects I wrote in zig [3].
I have worked with postgreql community since 1997 and python community since 1998. Never felt such hostile community. So all the best and I wish zig continue its progress
[1] https://github.com/insanai/sqlodin
[2] https://codeberg.org/vyomtech/zig
[3] https://github.com/insanai/zenfmt
> because my projects on github using zig uses LLM
To be clear, we do not block people for merely having LLM-related projects. Obviously we have opinions about LLMs in a broader context, but in terms of rules enforcement, we only care about LLM usage taking place within official Zig spaces.
It's possible you were blocked in error. LLM detection is not foolproof, so unless it's an open-and-shut case, our usual approach is to just unblock if people reach out to us by email.
I would understand if its by error but based on how it was banned it did feel deliberate. Also I have written email to core member before making a decision to leave the community. I think a community can only thrive when they are nice to humans. Here it felt like zig is worse then a oligopoly where few core members can block anyone without any reason and don't even have a courtesy to send a message. I worked with python community since its beginning, same worked with linux community since 1992, never seen such attitude even though both of the projects were driven by benevolent dictators, but they did care about the community around it. I still have one major work [1] in zig, which I will still release but after this won't work on zig. Zig community felt like a hostile ologipoly of few core members without proper due process, when it comes to treating humans.
[1] https://github.com/insanai/sibuna
Well of course the block itself was deliberate. By "in error" I meant that we might have misjudged the text as being LLM.
Who did you email, when, and did you get a response? We've all been very busy leading up to the release, and most of the team is also currently traveling for SYCL.
Regarding sending a message, consider our side: We're tired of having our time wasted on slop, so we don't also want to have to write formal block notifications to people we block on suspicion of using LLMs. I'm fully aware that this can lead to an unfortunate situation like this, however. For what it's worth, this is why we want to move to an invite tree for Zig development; it'll allow us to create a high-trust environment where this kind of suspicion is unnecessary.
> Good to know they are trying pragmatic approach to LLM now.
What's this new approach they're trying?
I don't think the zig community is necessarily hostile to LLMs, they just don't want it in their "maintained by 10-ish core people not even full time" language impl. Mitchell Hashimoto, a big zig contributor (both money and effort), for example, uses LLMs a lot and no particular shade is thrown.
This is from my own first hand experience both on ziggit forums as well as raising bugs in zig repository. My account on codeberg is still banned on zig repository without any single comment, my issues or pull request raised on that project just went in ether without being accessible, if you see the issue yourself could have made the decision if it was warranted to ban the account, but that right also taken by the core contributor out of spite and disdain for LLM.
I have been in open source world with linux kernel since 1992, never every had seen a community so hostile, especially towards humans who spend time and efforts just out of curiosity, inquisitiveness and trying to support some simple alternative when odds are already against them.
Facing hostile behavior from inside the community made me switch to odin language where they also do not use LLM but are pragmatic about people using it. Cannot comment on Mitchell because I do not contribute with money.
> My account on codeberg is still banned
> my account was banned because my projects on github using zig uses LLM.
Its not Zig. Its codeberg. Codeberg has a strict no-AI policy. Codeberg is ALL about the community and not the lone hacker. I think its a good policy to prevent overload and clearly distinguish project ecosystems from each other.
https://blog.codeberg.org/protecting-our-floss-commons-from-...
They said that their projects on GitHub use LLM, not on Codeberg.
I mean I pr'd something (and it was not accepted) but the grounds have nothing to do with LLMs. I'm pretty vocal about using LLMs to write zig code. Obviously I did not use LLMs in my PR
I would be fine if they reject the PR, given I was first time raising it to fix POSIX related issue. It was just few lines of code. But my account was banned all issues removed my own PR not visible and looked like tyrannical approach not a benevolent dictator approach. I think being nice to another human being is the basis for building community. Here felt like if you are among the few who dictate the terms there is no process just my way or highway.
Well, it's Codeberg. They ban people just for using LLMs.
It is not codeberg, I can work on other repository properly on it. Its only zig repository where my account is blocked. My initial thought was same as you, only later realized it. Also on ziggit these people did the same even though based on policy when I put the post i clearly mentioned use of LLM. This all without any proper communication. After working on zig for over 6 months, now I am moving slowly to odin [1] language. This decision I did not take lightly, I had spend significant resources in zig earlier and now moving away.
[1] https://odin-lang.org/
You sound really fun to collaborate with.
don't let people gaslight you here. i mean, look at how the creator treated a person who was a zig evangelist and was donating tens of thousands of dollars to the project:
"Jarred was already writing slop well before he had access to LLMs."
"Jarred was a stinky manager. Poor communication, unrealistic expectations, low empathy, no experience. Just a total shit show"
"When Jarred announced the Rust rewrite, we were ecstatic. It seemed too good to be true. I have to admit, I didn't think the technology was there, to pull off this stunt. But he did it, and now I'm metaphorically sipping delicious tea from a mug that says "It Tastes Like It's Not My Problem Anymore"." [1]
or what about when he called people "monkeys" because he didn't like their product [2]
or when their vp of "community" called github employees clowns [3]
so don't let people tell you that there isn't a pattern here. and when this kind of behaviour comes from the top it shapes the rest of the community
[1] https://archive.is/wNLqY
[2] https://web.archive.org/web/20251127021007/https://ziglang.o...
[3] https://bsky.app/profile/kristoff.it/post/3lwa2whcrj22c
> Jarred was already writing slop well before he had access to LLMs.
> Jarred was a stinky manager. Poor communication, unrealistic expectations, low empathy, no experience. Just a total shit show
Are these claims false, though?
I’m actually going to be a little bit sad when Zig does hit 1.0 and slows down with the rewrites of core functionality. It’s not for everyone, if you have SLAs on production I can see you getting sick of it quickly, but no programming has ever felt as alive for me as right now in Zig. It’s like the various JavaScript heydays of Bluebird, Node, Babel, ES6 etc all rolled into one.
How is Zig doing as a project? I remember they took a hard line against AI
They seem to be doing fine. Tigerbeetle and Ghostty are two projects that continue to do well and are written in Zig. The foundation’s funding also seems to be doing well. They are continuing to make releases.
Ghostty is a strange example though because Mitchell is very keen on agentic engineering.
Strange in what way? Mitchell has also stated that he reviews every line of code that goes into Ghostty so I don't think its quite the same as the Bun rewrite.
He can say that … but after reviewing the 1000th PR from a LLM and only finding minor issues that really go down to subjective taste, I think he may start to reconsider that.
One day I may get there, but even for minor PRs I constantly find important issues that I need to intervene.
Really, even after a LLM has gone through a review and fixed issues found? I almost never see anything relevant since around Opus 5 came out, and now with Fable and Opus 5.5 whatever I think is wrong is almost certainly right.
On my small personal projects I seldom find any mistakes. I use Chinese models on those. GLM, DeepSeek, MiMo.
At work I use Claude, but it is a considerably larger and quite complex codebase. I use LLMs a lot, but it is a common weekly occurrence for me to correct misconceptions, bugs, or overengineering from Opus/Sonnet (Opus plans and reviews, Sonnet implements).
It looks like they're spending 5 months what should be a 1 month job these days.
Filed a compiler bug related to dwarf tables that screws up debugging and line of code coverage that they completely ignored, just because I mentioned that I had every LLM check it to confirm it's a bug, since all I know is what kcov and every coverage tool generates incorrect coverage data for my repo, for 100% certain.
Sounds like you violated the clearly stated project rules and admitted to it. Are you surprised that they're not engaging with you?
Reading their comment it sounds like they wanted to confirm the bug existed with AI, not sure they ever said they had AI write the code. Why have such a dumb policy?
It's a violation of project rules to confirm bugs with AI?
Read it yourself: https://ziglang.org/code-of-conduct/
>If you use a chatbot to give you advice on a comment on the issue tracker, that comment is unwelcome.
>No LLMs for finding bugs.
>No talking about use of chatbot/LLM services.
I've said it before and I'll say it again- it's a cult that bans dissent
The things you've quoted and your conclusion feel at odds. They just don't want AI contributions, and they, like a lot of the world, are bored of hearing about AI. Is it really too much to ask?
For the record, there isn't even uniformity of thought on LLMs within the core team, let alone the broader Zig community.
If there's an edict that no one is allowed to bring up the topic, how can someone change this part of the code of conduct?
Asking non-rhetorically. It seems like one position "ai in any circumstance = bad" is being enforced. The commenter above didn't even understand why he was ignored.
> If there's an edict that no one is allowed to bring up the topic, how can someone change this part of the code of conduct?
To clarify, do you mean someone who isn't part of the core team?
Interesting to hear! We maintain the project Antfly (entirely zig) and have been nervous about bringing issues to the zig folks or asking questions because of our ai usage.
We’re quite knowledgeable and thoughtful folks fwiw
I think I’ve seen you are a core team member or contributor? I remember your tag?
We simply ask that Zig bug reports be written by humans, for humans. Whether you use LLMs on your project is immaterial to us.
Yes, I'm a core team member.
Thanks for the clarification here!
So for instance because of the size of our codebase our project has pushed Zig to some of the edges, specifically we end up hitting a bug when using llvm and zig on arm64 (Mac and Linux) where it seems to be caused by some configuration Zig passes through to LLVM. I’ve used codex and claude to help me diagnose and find the bug (we use nix’s glibc zig to circumvent the problem now). I now understand the root cause but am not sure what the proper fix would be. But I’ve not known whether or not even raising the issue would break the terms of contributing? Would raising the issue break the implicit agreement?
I would suggest just starting by filling out the bug report template, i.e. steps to reproduce, expected behavior, and observed behavior. If for some reason you can't provide a reasonable reproduction, the symptoms on their own can sometimes be enough for us to make an educated guess at what's going wrong.
Regarding whether you should post the root cause analysis: Per the current policy, the answer would have to be no.
I do personally have more nuanced thoughts on this, and I started typing them out... but then I realized that my reply was getting dangerously close to blog post length, so I decided to restrain myself and commit to turning it into an actual blog post later. In a nutshell, though, the problem is that even if there is such a thing as responsible use of LLMs for bug analysis, the only way we can currently be confident that someone possesses the required qualities for that is by working with them for a while.
I understand, maybe one day I'll have the chance to hear the blog posts' worth of thought on the matter!
https://codeberg.org/ziglang/zig/issues/37060 I opened the issue and can give you the agent generated RCA on the matter if you all want it in issue 604 on antfly's github but I understand that's against policy and totally respect that.
Appreciate all the work you guys do and have been following the whole Zig project since inception fwiw!
> The Zig community is decentralized. Anyone is free to start and maintain their own community, which is not subject to these rules.
The second part affirms that it is centralized. There is nothing wrong with that except saying that because you can leave the community and go elsewhere, it is ipso facto decentralized.
I figure they're either using a loose interpretation of "decentralized" or an honest interpretation of "community".
I’ve not found the zig folks to ban dissent, they engage in a lot of thoughtful dialog. Just because they’ve made a different decision for how they take contributions than other people agree with doesn’t make them a cult?
Not sure I see the problem there.
Even in projects where I use LLM liberally, I would rather not read or have to engage with your LLM output.
I use LLMs already, I can make do without yours.
>Even in projects where I use LLM liberally, I would rather not read or have to engage with your LLM output.
I don't want to read LLM output on a project either. But you may have misread the quoted section, it isn't saying "you can't post LLM output," it is saying you can't ask an LLM to advise/critique your own comment before posting it.
>>If you use a chatbot to give you advice on a comment on the issue tracker, that comment is unwelcome.
The cult are the people forcing their LLM addiction on other people.
Yes ... pretty damn stupid.
"NO LLMs for finding bugs" is easier written as "Don't use the tools your adversaries will"
I've written code a long time and that's probably the dumbest rule I've seen.
Can a programming language have an adversary?
Given all more modern programming language implementations include a standard library that will be attacked by people: yes.
Pointing out something that is not good doesn't require surprise at the fact that it happened
Wow I thought surely they wouldn't object to using AI to confirm bugs, but they really do.
Tbf I guess as a popular open source project not using AI to fix bugs, they probably already have more open bugs than they can ever fix so it doesn't really help them for people to find more.
I would imagine his bug was actually ignored just because Zig has 2700 open bugs, rather than some AI policy violation.
Claude Code has 13.4k open bugs: https://github.com/anthropics/claude-code/issues
Show me a popular open source project that doesn't have a large number of open issues and I'll show you one that has a triage bot auto-close them.
Some have both, despite using an auto-close bot! OC is at 4.6K open and 24K closed issues. https://github.com/anomalyco/opencode/issues
Gotta admit it's weird that they haven't set up Claude to fix them! Tbf my threshold for "AI now writes good enough code" was Astra or Opus 5.5 and they only just released that.
They don’t, at least not any more. Andrew Kelley has explicitly stated that he sees the value of using LLMs to uncover bugs. Inspired by sqllite project.
https://youtu.be/zwi5b5xSsKA?is=PTjJJjSnVMdRuZag
They may just be taking the slow route of rejecting by default until they can be sure that the usage of LLMs provides long term value. I don’t see anything wrong with that. If you’re writing robust software, using LLMs at this stage is a bit of a gamble. We don’t fully know the long term effects on code quality yet.
He sees the value of using LLMs... only after performing NASA-level verification and fully comprehensive fuzzing using traditional tools. Since Zig has not done those things, LLM-generated bugs are still banned.
Ah, so yet another language in the long line of "I'm smart enough to use this but you the user are not"
Project maintainers know what level of AI use is acceptable for themselves, and quantifying it and enforcing it for random contributors is very difficult.
This is because they got so many shit admissions from people using LLMs they were wasting time reviewing them.
The popular LLM projects have infinitely many more bugs than the zig project does this is a ridiculous message.
Yeah, the maintainers of a DOA hobby project should certainly be allowed to run it however they want imo
That's honestly my biggest problem with Zig.
It's clearly a hobby project (constant breakages, the maintainer getting into politics, rejecting some safety mechanisms, the anti-LLM crusade, a strange focus on esoteric targets with little to no commercial significance), but the maintainer does not admit that it is a hobby project.
It makes me respect the Rust community even more.
If it's generating detectably incorrect code, why do you need an LLM to tell you it's a bug?
Encountering something that seems incorrect, and proving something is a bug and not just YOUR user error, AND reproducing it minimally - turns out to not be easy when it's a low-level compiler issue, and YOU'RE not an expert, especially when it relates to Dwarf tables...
Typically, any time you think you've found a compiler error, you're using it wrong...
If it's a real bug, you can just report what you know and leave it at that. You don't need to embellish it with random guesses about a root cause.
Okay: "I compile this input and the linker crashes."
Not okay: "I compile this input and the linker crashes. Also here's 10 paragraphs of slop about dwarf tables, which I can't even evaluate the accuracy of since I'm not an expert."
> Typically, any time you think you've found a compiler error, you're using it wrong...
Yes, if you can't figure out if it's a bug, the bug tracker is the wrong place to get help. Ask in a community forum instead.
> Not okay: "I compile this input and the linker crashes. Also here's 10 paragraphs of slop about dwarf tables, which I can't even evaluate the accuracy of since I'm not an expert."
It's a 20-line file with a few commands to run it to reproduce it. Not 10 paragraphs of slop.
Presumably that is much more helpful than - here's my gigantic repo, good luck running my tests, also good luck finding the bug.
> here's my gigantic repo, good luck running my tests
What about my comment made you think I was suggesting not to give repro steps?
> also good luck finding the bug
But yes actually, this half is true. It's better to give them no extra information, than to give too much information that you have no idea if it's true or not.
I reported a bug and waited for a year. one day it was merged. the thing is that there are a looot of bugs and the team is small, so your turn might not have come up yet
> breaks the project rules
> gets ignored
Who could have forseen this.
Their policy is immature. Clicking little upvote and downvote buttons is immature. Better to just lie and duplicate the issue.
Seems like "parallel construction" where you show you actually understand and can explain the issue independent of an LLM would be the way to go. No need to mention how you found the bug as long as you can explain what the bug is, why it matters, and how to repro.
Sending a repro and saying you don't understand why it doesn't work is valid too.
Lying to get around a project policy you disagree with is immature. They have the right to run things the way they want, either accept their rules or leave the project be.
If they really insist on no LLM involvement at all then they're going to fall behind and lose out. Your LLM use case seems really very conservative - you didn't write any code with it, you just use it to confirm the bug. There are many OSS projects that are also taking a similar hardline against LLMs - people are going to fork them and then move on. I've had LLMs fix bugs/add features to a couple of projects like this and, well, they're missing out on the fixes/added features that I'm using locally.
People keep promising me that I'll be left behind, but somehow it never seems to happen.
It's remarkable how easily these idiots can be conditioned into parroting the usual FOMO lines without question. Same people would've had me think I'm a fucking moron for not buying ugly monkey pictures during the NFT bubble.
I'm thinking their lives must be absolutely dreadful for this constant fear of being left behind.
How do you evaluate this? Have you tried looking for a job? Or compared your project with a similar project where people are using AI and you are not? Or you’re just staying put and waiting to see what happens?
They remind me of the "have fun staying poor" people from crypto
I'd really love to figure out how to stay behind, but the people who keep insisting I'll be left behind are doing their level best to prevent it.
TBF, most people did stay poor, they just didn't really get richer.
They are doing fine, all this "left behind" talk has no basis other than cope.
Can you offer any specifics? The ToC looks quite extensive, which to me indicates a lot of design decisions needed to be made (probably involving many people) and unclear how that process might be accelerated by AI.
Unless you're suggesting the language design should also be vibed together?
> Unless you're suggesting the language design should also be vibed together?
Sounds like a fun little project, have a bunch of AI pushers fork Zig and see if they can do a better job. I want to see results, not snarky HN comments. After all this progress, ChatGPT should be able to one-shot a better language since AI is so good now... right?
One-shot, no, but there are a bunch of people pretty much solo-building their personal ideal language with AI and it's going quite well. You need to know just enough about language design to be dangerous, but you don't need to be a seasoned pro.
I'm doing it myself: https://zena-lang.dev/
It seems like such a strange thing to do, building a language that you aren't going to write by hand. It's guaranteed to perform worse at higher cost, fill up a lot more of the context window, and burn a ton more reasoning tokens.
If you're using AI, a language with a large training set is going to win.
> If you're using AI, a language with a large training set is going to win
Not necessarily? What if the training set contains an overwhelming amount if bad code written by neophytes? I imagine Python quality by the LLM suffers from this, for example.
What if the language has extremely confusing syntax constructs (like early php) or bad or no conventions (suppose the standard library has somecollection.put(key, value) sometimes and othercollection.put(value, key) other times), and individual code authors just pick what they want adhoc
Large training set ain't gonna save you.
There's a lot of RL that goes into this, you're not just training on bad code. Python is one of the programming languages that LLMs consistently perform best at.
Near bottom on auto coder benchmark from last year, so clearly claiming "consistently" is false or misleading. Did you make that up?
I guess it depends on whether you will write everything in an AI assisted fashion or not. There's benefits to languages that are quick and easy to read by the author even if the LLM is doing the writing, because most code still benefits from human review above and beyond the review that agents provide. Language popularity certainly helps but it seems like for moderately popular languages [1] the cost you pay for a lack of popularity is quite modest.
[1]: https://danluu.com/pl-tokens/
A language you just created isn't going to be moderately popular, so it's just going to put you at a disadvantage -- and you're not even going to be writing in it, so why the self-kneecapping?
I mean what does "put you at a disadvantage" even mean concretely? To use a less popular language, it means you need to load up context related to the semantics of your language, load context on how to invoke tools to make sure the syntax with your language is correct, load up context related to each tool call you make (which will be more numerous in a niche language), and load up context on architectural decisions that might be specific to your language. All of this is simply a token cost. By forcing a model to load an initial amount of context per harness turn you also effectively shorten the max context window beyond which the model becomes stupid (which itself is much shorter than the max context length.)
Obviously it's not like people are specifically trimming each and every prompt they give a model to tokenmax their models to get the best output / input prompt, we instead live in a spectrum of how many tokens of input and context we're willing to provide to a model to make progress. If the cost of those tokens is low enough for the problem domain you're working in, then it's fine. For some the readability of a personal language may outstrip any of the token costs that one needs to pay to use it. Alternatively maybe you want something like an array language (J, K, APL, etc) which allows array programming and optimizations that conventional PLs just can't do. Maybe you want your language to compile to a target that is highly portable. There's actually a lot of stuff out there that previously wasn't feasible but with LLMs-as-force-multiplier absolutely is.
I also suspect the space is a continuum. There may be pareto optimal points, such as DSLs built atop languages, that are both highly readable but also fairly token efficient.
It's both a token cost and a performance cost; there's only so much that documentation can do, compared to a ton of RL on top of millions of lines of examples. The space is a continuum, but the more you stray from the trained path the higher the cost you pay.
> the more you stray from the trained path the higher the cost you pay.
Note the LLMs are trained on language semantics far beyond the mainstream ones, so language design can become quite exotic without straying too much from the training. You really do have to measure these things, I don’t see how you can make a confident assertion without data.
I don't think those assertions about AI development necessarily hold. And I think it'd be a depressing future if we can't ever have anything new or better that wasn't popular in the training set as of November 2025.
I wrote about some of my thoughts with Zena and AI here: https://zena-lang.dev/blog/2026/09/languages-for-the-ai-era/
Yes, we're building a sad world. I'm glad you noticed. Until we get LLMs with online learning, training data will dominate.
But you’re very clearly wrong. Just go ahead and try it: write your own language or just a DSL, write a brief description of how it works , let a LLM use it. Many of us have done this and everyone knows that it works extremely well, to most people’s surprise.
I'd suggest benchmarking the outcomes. It's going to "work", but at a higher cost with worse ontcomes compared to an existing language with a large training set.
Here are some benchmarks I’ve collected recently:
https://mech-lang.org/iros-r4r-2026/index.html#5805406811462...
I compare one algorithm across several programming languages and backends. There is no training data for Mech in the LLM yet it beats most other implementations in perf, which were optimized by LLM.
Maybe human performance engineers trained in these languages could write better implementations. But to answer the question of whether the LLM could write more performant code in languages it’s trained in versus languages it’s not, this comparison is at least illustartive.
> Yes, we're building a sad world. I'm glad you noticed
What is the point of sarcastic, passive aggressive comments like this?
I don't think it's particularly sarcastic. Its bitter, of course, because the world we're working towards kind of sucks. I'd like to opt out of it, but it's being forced on me.
The best I can do is understand its edges and try to find some advantage that leaves me well off enough to stave off the worst effects.
This assumes the goal is to create a productive language.
When I design my own languages (I have written several, all terrible!) it's typically to learn about language design.
Been thinking about doing this myself (did some PL in grad school but it's been a long time), but I find myself wanting to reach for a Scheme (using macros to grow the language I want) and customize it or build something atop Janet.
Curious why you wanted a more ML / Rust / Scala inspired syntax. (Personal preference here is totally valid btw, just curious.)
The syntax is TypeScript inspired because I like it and my bet is that the familiarity helps human and AI coders learn the language.
Claude can’t one-shot a C compiler.
None of these systems can. They need enormous training. They need alignment and reinforcement. They need harnesses. And most importantly they need a human that knows how to write and develop a C compiler.
The ISO specifications are not sufficient. Neither are the System V guidelines. Not even spec tests and compcert.
Claude can one-shot a C compiler just fine. It just won't produce great code. But with all the C code out there in the wild, it won't have a shortage of things to test on, which is mostly what coding agents need to complete a task.
You're not going to one-shot it, but over the course of a couple of months of evenings you could come up with something usable/interesting if you manage the LLM well.
Nobody is one-shotting big projects. Even humans can't do that.
Fabrice Bellard might be an exception.
Ha funny but no.
Yep, it's true, but the reality of what it would take to actually fork and commence with healthy use of AI alongside with the elusive soft/hard stance needed to lead technically and socially is probably a bigger challenge than anyone is willing to take on! Soo.. slow and (rather?) well it goes for Zig.
Slow and steady wins the race
Anthropic levels of slowness (pacing)?
Parables aren't reality.
in tech? really?
Language design and implementation is much more in the realm of actual computer science than it is "tech"
In programming languages, usually. Building software on top of a rapidly-changing language is not often a recipe for success.
This is why Java deliberately is the last mover. Letting other languages experiment first.
Yes, the Lindy effect applies in tech too.
And apparently ignores legitimate compiler bugs on the way that someone happened to ask an LLM to double check their diagnosis
Im really enjoying using it, works great tbh
Seems to be C++ing which is disappointing. Looking more at C3 and Odin for cases where I'd hoped Zig to land.
Odin scratches this “stick shift” itch for me better than anything else. Highly recommended.
In what sense are the C++ing? Too complicated or what?
can you elaborate on what you mean by "C++ing"?
C3 is honestly so good and a bit underrated.
It's my favorite of the new round of languages by far. I just wish it had a bigger ecosystem.
This is getting really close to 1.0 now? I guess may be 2 more release and 1 more RC? That is 2028. I am looking forward to stabilising the language and the core team has previously indicated that exporting ir will be a supported feature.
Once that is done, @dnautics is working on memory safety for zig [1]. A Checker for Lifetimes and other Refinement types.
Then we can start the Rust vs Zig Debate again. Hopefully not too late.
[1] https://github.com/ityonemo/clr/
Given the efforts we are already seeing where people post "I rewrote X in Rust in 24 hours using the latest agental harness", I don't think there is any kind of "too late".
Either Zig gets good enough to be a worthy re-write target, or it doesn't. But I suspect that isn't even what they want.
I imagine languages are going to bifurcate into ones that go all in on agent experience and ones that go all in on human experience. And Zig seems to be in the latter group. I am confident that there will always be some niche market for languages lovingly crafted for humans. But the criteria we will use to define success within these groups of languages will be different.
Curious as to the state of evented IO / io_uring in this version.
Answering my own question. Andrew says "we almost had it but ripped it out" ... here: https://youtu.be/zwi5b5xSsKA?t=2711.
https://ziglang.org/news/0.17.0-released/
Although I understand that upgrading LLVM version is a lot of work, it's a bit frustating that loop vectorization is still disabled.
The fix could not be backported to LLVM 22 because it changed the LLVM library ABI. We also could not skip straight to LLVM 23 because that would make life harder for distro package maintainers.
Was loop vectorization by chance enabled in 0.15.2 and disabled since 0.16.0? One of my projects got 14% slower after upgrading to 0.16. I didn't dig in yet but I assumed the Io vtable was the cause (it does a lot of file io). But if the versions line up maybe it's actually loop vectorization.
Thanks for your work btw.
Yes, 0.16.0 and 0.17.0 are the only two releases that have loop vectorization disabled. (And it will be re-enabled in 0.18.0.)
I just noticed it today upgrading to 0.17.0 from 0.15.2, while doing bitwise operations a lot of large integers. I haven't looked at a debugger yet, but my guess is that would benefit a lot from loop vectorization.
You should not worry. They said they would make a short release cycle for updating to 23.
Any change to the AI policy?
Nope, but why would it matter? Nobody is stopping you from writing Zig code with an LLM, the policy you are referring to is only relevant to the compiler source code.
Not the parent, but some people may feel like it's betting on a bad horse. Zig already has a small ecosystem, so pushing culture that may make it even smaller while having core development stagnate in an ever-accelerating software world may be seen as a tougher sell.
> Please consider a recurring donation so that we can offer more billable hours to our core team members.
Just don't be surprised if the project BFDL talks shit about you or your company later.
(Still a good language though, credit where credit is due.)
I wouldn't consider donating until the project leader has demonstrated considerably more personal maturity.
would you rather the reverse to be true? if someone donates enough that's sufficient for the BDFL to silence their opinions?
yes.
A simple, "this entity is a sponsor, and therefore there is a conflict of interest and we will not comment on recent controversy" is enough.
If it's big enough, refuse to take further contributions.
I know it's not entertaining, but that's why we have video games.
Back in a more adult age, the way that this was handled was "We've benefited from our past collaboration, but we believe a new direction is needed going forward."
For some reason--COVID brain rot, poorly-socialized people coming online, general increase in viciousness in the population, who knows!--people have forgotten the utility and purpose of boring polite manners and communication.
Not sure how it's new. Torvalds and plenty like him have been more abrasive for much longer.
I'd rather them be frank than use corpospeak.
I wish C had similar dynamics.