The three properties mentioned, I think require full AGI, I will shorten them for this comment.
> It should consistently result in functionally correct code [...]
> It should accomplish [...] business requirements in natural language.
> It should allow the operator to modify the code without requiring knowledge of implementation [...]
These properties are exactly what current human software engineers are doing. It's why you hire someone and offload all the knowledge and responsibilities unto them. AI will have to match a human level intelligence, and when they do, since they already have all the knowledge on the Internet, it will basically be above AGI and closer to ASI.
My overall point here is that the current hype, and why there will probably be a market adjustment if not a bubble, is that people underestimate how hard it is to build this type of human intelligence. People are extrapolating from a very capable pattern matching system to something that has self-awareness, agency, and a sense of place in a world of humans.
I think it's mainly the "business requirements in natural language" part that would require full AGI. In practice though, most requirements don't require that much intelligence to accomplish as illustrated by the recent success of coding agents.
It also makes sense that the language would necessitate having some algorithms as (pseudo-)code embedded in the specification when the requirement is complex enough.
I still think we don't really know how well they work because there is a lot of noise. I also think we're underestimating this small things that humans do. It's Moravec's Paradox but for business requirements (what is easy for humans is hard for machines), and I think this is a blind spot. We are assuming it's easy but there are so many intangible things to consider, i.e. thinking about customers, thinking about deadlines, etc., and they all factor in to decision making when working on something.
The three properties mentioned, I think require full AGI, I will shorten them for this comment.
> It should consistently result in functionally correct code [...]
> It should accomplish [...] business requirements in natural language.
> It should allow the operator to modify the code without requiring knowledge of implementation [...]
These properties are exactly what current human software engineers are doing. It's why you hire someone and offload all the knowledge and responsibilities unto them. AI will have to match a human level intelligence, and when they do, since they already have all the knowledge on the Internet, it will basically be above AGI and closer to ASI.
My overall point here is that the current hype, and why there will probably be a market adjustment if not a bubble, is that people underestimate how hard it is to build this type of human intelligence. People are extrapolating from a very capable pattern matching system to something that has self-awareness, agency, and a sense of place in a world of humans.
I think it's mainly the "business requirements in natural language" part that would require full AGI. In practice though, most requirements don't require that much intelligence to accomplish as illustrated by the recent success of coding agents.
It also makes sense that the language would necessitate having some algorithms as (pseudo-)code embedded in the specification when the requirement is complex enough.
I still think we don't really know how well they work because there is a lot of noise. I also think we're underestimating this small things that humans do. It's Moravec's Paradox but for business requirements (what is easy for humans is hard for machines), and I think this is a blind spot. We are assuming it's easy but there are so many intangible things to consider, i.e. thinking about customers, thinking about deadlines, etc., and they all factor in to decision making when working on something.