Yeah — i agree based on my own experience. It was that the loop only works because if you have a rubruc to measure againt.
I’ve had similar issues but not in any decompiler mode. Agents can be deceptive in explaining something so well (and I was too lazy to carefully read it verify it myself). Where I got burned was when a beta tester pointed out some citations that didnt exist. So i put a check in place. agent can propose freely, unconfirmed stuff fails closed, my rubric checks get the veto. Same shape as what you’re describing. The agent can refine; the rubric metrics is what keeps it from just becoming fluent drift. Without that human in the loop, autonomous improvement is mostly relies on luck. On the other hand, I am seeing more and more the newer model's reasoning and having another llm eval (provided that llm has rubrics to measure against) does lessen the falseness.
It would be cool to see agentic interpretation of function and variable names. It can see and track the flow of data a lot faster than a human can so if given some context, maybe it could synthesize names for them.
I've used Claude with Ghidra MCP and had good success. It correctly identified and named MD5 functions, file IO functions, and the whole chain from a UI dialogue function. And when prompted it named all the interesting variables too.
Well, that's too bad; as the main developer of https://github.com/nneonneo/ghidra-wasm-plugin it would have been really nice to get better support for all the new features that have been added into the spec over the last few years.
I also think that, for larger projects, it's still useful to have real decompilation support; for example, Il2CppDumper works in tandem with ghidra-wasm-plugin to enable decompilation of Unity Il2Cpp projects, which are often enormous (100MB wasm files are not uncommon) and for which the symbols + types are invaluable.
this is really amazing work thank you and in my opinion (as its explained) a very good example of how to use AI powered development and research to advance the tools we have. decompilation is really hard, and better algorithms for it are very valuable contributions for many areas of tech.
Not sure why someone would want to use an LLM to build a new decompiler instead of training an LLM to BE a decompiler.
Especially given the fact that you have an infinite training set to train that LLM from (compilers can generate as much training data as you could possibly want).
I read this less as “build a decompiler with an LLM instead of training one to decompile,” and more as a different loop: you still get an artifact you can measure and improve against IDA / Ghidra / angr. Training on infinite compiler data is powerful for the first path; this post is mostly about the second one (i think, but i could wrong).
Yeah — i agree based on my own experience. It was that the loop only works because if you have a rubruc to measure againt. I’ve had similar issues but not in any decompiler mode. Agents can be deceptive in explaining something so well (and I was too lazy to carefully read it verify it myself). Where I got burned was when a beta tester pointed out some citations that didnt exist. So i put a check in place. agent can propose freely, unconfirmed stuff fails closed, my rubric checks get the veto. Same shape as what you’re describing. The agent can refine; the rubric metrics is what keeps it from just becoming fluent drift. Without that human in the loop, autonomous improvement is mostly relies on luck. On the other hand, I am seeing more and more the newer model's reasoning and having another llm eval (provided that llm has rubrics to measure against) does lessen the falseness.
It would be cool to see agentic interpretation of function and variable names. It can see and track the flow of data a lot faster than a human can so if given some context, maybe it could synthesize names for them.
I've used Claude with Ghidra MCP and had good success. It correctly identified and named MD5 functions, file IO functions, and the whole chain from a UI dialogue function. And when prompted it named all the interesting variables too.
I'm not sure you even need the MCP tbh, I just have Claude use pyghidra scripting.
I was working on adding better WASM support to ghidra, but now I can just point Claude to binaryen
Well, that's too bad; as the main developer of https://github.com/nneonneo/ghidra-wasm-plugin it would have been really nice to get better support for all the new features that have been added into the spec over the last few years.
I also think that, for larger projects, it's still useful to have real decompilation support; for example, Il2CppDumper works in tandem with ghidra-wasm-plugin to enable decompilation of Unity Il2Cpp projects, which are often enormous (100MB wasm files are not uncommon) and for which the symbols + types are invaluable.
I wonder, how many such interesting and universally beneficial developments did emergence of tools like Claude Code kill?
this is really amazing work thank you and in my opinion (as its explained) a very good example of how to use AI powered development and research to advance the tools we have. decompilation is really hard, and better algorithms for it are very valuable contributions for many areas of tech.
Not sure why someone would want to use an LLM to build a new decompiler instead of training an LLM to BE a decompiler.
Especially given the fact that you have an infinite training set to train that LLM from (compilers can generate as much training data as you could possibly want).
LLMs are famously inefficient, expensive and unreliable.
Yeah, fair question.
I read this less as “build a decompiler with an LLM instead of training one to decompile,” and more as a different loop: you still get an artifact you can measure and improve against IDA / Ghidra / angr. Training on infinite compiler data is powerful for the first path; this post is mostly about the second one (i think, but i could wrong).