>"Consider another core part of the C abstract machine’s memory model: flat memory. This hasn’t been true for more than two decades. A modern processor often has three levels of cache in between registers and main memory, which attempt to hide latency.
The cache is, as its name implies, hidden from the programmer and so is not visible to C. Efficient use of the cache is one of the most important ways of making code run quickly on a modern processor, yet this is completely hidden by the abstract machine..."
separation of the algorithm being implemented from its execution schedule
, i.e. code specifying the loop nesting, parallelization, loop unrolling and vector instruction.[3] These two are usually interleaved together and experimenting with changing the schedule requires the programmer to rewrite large portions of the algorithm with every change.[4] With Halide, changing the schedule does not require any changes to the algorithm, allowing the programmer to experiment with scheduling.[5][6]"
Observation: Most Computer Languages at this point in time -- do not separate algorithm code from implementation / parallelization / multiple levels of memory cache hierarchy management code.
Future computing languages should address separating these concerns.
i.e., the separation of What is being ran (the algorithm) from How it is being run (where is memory allocated/cached in the memory hierarchy, how many nodes/grids/cores/warps/threads/processes/cores/units of execution are being used, what's the data flow, etc., etc.)
So that's the problem -- of which Halide would seem to be one possible interesting, potential solution!
CUDA, ROCm, Vulkan, and anything that else deals with massive computational parallelism are also contenders!
In the future, I'm guessing that at least a few future general purpose computer programming languages will attempt to address this problem as well...
https://queue.acm.org/doi/10.1145/3212477.3212479
>"Consider another core part of the C abstract machine’s memory model: flat memory. This hasn’t been true for more than two decades. A modern processor often has three levels of cache in between registers and main memory, which attempt to hide latency.
The cache is, as its name implies, hidden from the programmer and so is not visible to C. Efficient use of the cache is one of the most important ways of making code run quickly on a modern processor, yet this is completely hidden by the abstract machine..."
https://en.wikipedia.org/wiki/Halide_(programming_language)
>"The main innovation Halide brings is the
separation of the algorithm being implemented from its execution schedule
, i.e. code specifying the loop nesting, parallelization, loop unrolling and vector instruction.[3] These two are usually interleaved together and experimenting with changing the schedule requires the programmer to rewrite large portions of the algorithm with every change.[4] With Halide, changing the schedule does not require any changes to the algorithm, allowing the programmer to experiment with scheduling.[5][6]"
Observation: Most Computer Languages at this point in time -- do not separate algorithm code from implementation / parallelization / multiple levels of memory cache hierarchy management code.
Future computing languages should address separating these concerns.
i.e., the separation of What is being ran (the algorithm) from How it is being run (where is memory allocated/cached in the memory hierarchy, how many nodes/grids/cores/warps/threads/processes/cores/units of execution are being used, what's the data flow, etc., etc.)
So that's the problem -- of which Halide would seem to be one possible interesting, potential solution!
CUDA, ROCm, Vulkan, and anything that else deals with massive computational parallelism are also contenders!
In the future, I'm guessing that at least a few future general purpose computer programming languages will attempt to address this problem as well...
Anyway, Halide looks really interesting!