AI now writes most of the code at frontier AI developers. Anthropic says Claude leads 26% of its R&D, and staff there and at OpenAI describe losing track of what the models built. Every proposal for pacing self-improvement gates on the AI (capability evals, compute caps, a lag before a model does AI R&D). None gates on whether the people responsible can still explain the work.
In comprehension audits independent auditors are embedded at an AI developer and watch its workstream. They pick a contribution: a meaningful unit of R&D output such as a completed experiment, a new training recipe, or integration of new data. They study the work and prepare, then call a short notice audit meeting with the responsible people. In that meeting they ask questions to test whether the R&D staff understand what the contribution does and how it works, not model internals. The meeting is like a thesis defense or design review, with staff answering without AI assistance and conducted in a blameless manner.
Across 2000+ leading open source AI R&D projects we see human review comments dropped about 50% per LOC from early 2024 to mid 2026.
I'd love feedback on the approach and how developers are keeping track of agentic output.
Author here. Preprint at https://arxiv.org/abs/2610.10064
AI now writes most of the code at frontier AI developers. Anthropic says Claude leads 26% of its R&D, and staff there and at OpenAI describe losing track of what the models built. Every proposal for pacing self-improvement gates on the AI (capability evals, compute caps, a lag before a model does AI R&D). None gates on whether the people responsible can still explain the work. In comprehension audits independent auditors are embedded at an AI developer and watch its workstream. They pick a contribution: a meaningful unit of R&D output such as a completed experiment, a new training recipe, or integration of new data. They study the work and prepare, then call a short notice audit meeting with the responsible people. In that meeting they ask questions to test whether the R&D staff understand what the contribution does and how it works, not model internals. The meeting is like a thesis defense or design review, with staff answering without AI assistance and conducted in a blameless manner. Across 2000+ leading open source AI R&D projects we see human review comments dropped about 50% per LOC from early 2024 to mid 2026. I'd love feedback on the approach and how developers are keeping track of agentic output.