Under this framework, Rust's async fns would not be colored, since any function in the callstack could be blocking on an executor. if you consider "spawn point" a part of the callstack, many other languages are similar.
This doesn't feel like a useful distinction, but i don't think "function coloring" in general is a useful framework.
I think this is a fine distinction to make, though it's worth noting that function arguments _can be_ colors as the author describes if the arguments have to come from a specific place, e.g. in languages where only `main` receives certain arguments relating to system functionality, or a capability framework where only the top-level function receives the capability token.
I don’t quite understand why the author says Context doesn’t count here. I’ve certainly experienced a top-down version where because a function starts accepting a Context, and therefore needs to be cancelable, lots of functions it calls now need to take a Context too, and check for cancellation. Perhaps the color here is cancelability, not the Context per se. (Which may be even worse, as a kind of “invisible color”.)
How often does a conceptually incremental change correspond to a proportionally incremental code diff, vs. a surprisingly whole-world-upside-down rearchitect?
Do the code patterns naturally leave enough room for you to incrementally shift/transpose how things are organized?
When the answers to these are unfavorable, I often find what effects to "coloring" in the framework, library, or language, sometimes all at once.
IMO it is a syntax thing. If you need to use a special keyword (eg. func vs async func) then it is colored. If it is a semantics thing (eg. parameters) is is not colored.
I'm not familiar with how Rust's async works. What if Rust used the function argument approach, how would that work? Would people pass around a reference to an async executor as arguments (or similar)?
> What would it even look like for the top-most function to have to know “everything” that all the child functions end up ever needing? For every function parameter change in the entire program to somehow force a change all the way up the entire call stack? Clearly this does not happen. It so thoroughly does not happen that you may be having difficulty even imagining what it is I am talking about.
Really? In my experience, it certainly does happen and I'm generally happy (but not thrilled) when it does. It means the type system is doing its job, forcing me to provide all the dependencies that a function needs. If 38 layers deep something now needs a connection string that it didn't have before, that had better bubble up to the top-most function. Unless of course the top-most function has already provided it to an intermediate layer which can break the chain, but that is very different than the context.Background() example.
Yeah, in my experience it absolutely can happen, and the way it generally looks is that the dependencies get roughly grouped into larger structs which are then being passed down the call stack mostly as-is, until they hit a certain abstraction level/barrier where the parts of those structs start being passed down.
It's somewhat tedious, that's why modularity becomes even more important: with correctly chosen borders, the grouping of dependencies becomes very clean and self-evident, the whole data flow just flows. Chose them wrong, and you spend most of your time adding and removing those pesky extra arguments.
If you're using it diligently enough to where some functions can be statically proven "pure", then yes. You can't call an impure function from a pure one.
A subset of function arguments are function colors because async/await is isomorphic to requiring and passing a `callback` arg around. It was named 'callback hell` because there wasn't an escape hatch and it affected every function in the callstack.
I'd venture to guess that there's a more abstract way of formalizing this in the sense that monad instances are required to be threaded through the callstack in particular cases(would love to see this spelled out formally) where IO and Cont(?) have this requirement but State like the author points out, does not.
I think this is a case where you want to avoid architecture astronautics and deal with languages as they are presented. Yes, in the end it's all continuation passing, or depending on the cut of your jib, it's all just assembler in the end, but meanwhile, down in the trenches, there are real, practical differences in color that change how you program in those languages, and that's the topic color addresses.
>> JS... Because our strawman is a modern (shitty) language
This is an attitude I dislike from all these niche language communities. They give each other pats on the back for shitting on the most popular, versatile, compatible and adaptable programming language of all time.
I started using JS back in 2004 and, since then, I have watched it constantly being shat on by theoretically-minded folks. Before Node.js, people would keep pointing out "JS is a scripting language, not a programming language." Then, it became a programming language and went through the biggest adoption growth phase of any language in the history of mankind. I literally saw hardware manufacturers adopting JavaScript and even building custom JavaScript engines for their custom hardware, all while their academically-minded peers simultaneously shat on the language. They shat on it wave after wave, while it kept getting better and better and proved itself to be more adaptable than their own pet languages.
People are so hard-headed and biased against JS that the community literally had to invent a new label/wrapper; 'TypeScript' as a Trojan horse to get those stubborn folks to use it.
My view now is that JS critics are all talk. JS has been doing the walk; trampling all over their theories and abstractions and leading the way in many regards. Yes, it has bad parts, but all of these parts are avoidable; none of these parts are fundamental to the language. The matrix showing of how JS comparison operators down-cast everything to a string (leading to un-intuitive results) keeps being brought up but nobody asks the question "Why would anyone use a casting comparison operator to compare variables of two different types in the first place?". The fundamentals are excellent and proven as such.
"Function colors" are just effect systems, which are not a new concept while also being extremely useful and super nifty. I hope someday we can stop wringing our hands in a panic over "colored functions" just because some ancient version of Javascript had a frustrating implementation of one specific effect. It reminds me of how ages ago certain people acquired an instinctual aversion to static typing attributable entirely to ancient versions of Java and its attendant verbosity and relative anemia.
No. Function colors (see criterion 4 in the original article) deal with the idea of "how much of a pain in the ass is it to deal with" which is not really a concept that is a PL theory concept.
The article only lists 3 criteria, not 4, but if you're referring to the final one:
"3. The change may require a change to all functions in the stack."
That's a potential feature of effect systems. An effect generally indicates a restriction of some kind, and depending on your desired semantics it could be totally counterproductive to paper over it with abstraction. And there's a variety of different knobs to twist that aren't necessarily the same semantics that any given implementation of async uses. To use Rust as an example, its `const `effect imposes restrictions down the callstack, and cannot be discharged (const functions can only call other const functions, period), while its `unsafe` effect imposes restrictions up the call stack and can be discharged (by a non-unsafe function that uses an `unsafe` block internally.
Under this framework, Rust's async fns would not be colored, since any function in the callstack could be blocking on an executor. if you consider "spawn point" a part of the callstack, many other languages are similar.
This doesn't feel like a useful distinction, but i don't think "function coloring" in general is a useful framework.
I think this is a fine distinction to make, though it's worth noting that function arguments _can be_ colors as the author describes if the arguments have to come from a specific place, e.g. in languages where only `main` receives certain arguments relating to system functionality, or a capability framework where only the top-level function receives the capability token.
I don’t quite understand why the author says Context doesn’t count here. I’ve certainly experienced a top-down version where because a function starts accepting a Context, and therefore needs to be cancelable, lots of functions it calls now need to take a Context too, and check for cancellation. Perhaps the color here is cancelability, not the Context per se. (Which may be even worse, as a kind of “invisible color”.)
My personal yardsticks:
How often does a conceptually incremental change correspond to a proportionally incremental code diff, vs. a surprisingly whole-world-upside-down rearchitect?
Do the code patterns naturally leave enough room for you to incrementally shift/transpose how things are organized?
When the answers to these are unfavorable, I often find what effects to "coloring" in the framework, library, or language, sometimes all at once.
IMO it is a syntax thing. If you need to use a special keyword (eg. func vs async func) then it is colored. If it is a semantics thing (eg. parameters) is is not colored.
I'm not familiar with how Rust's async works. What if Rust used the function argument approach, how would that work? Would people pass around a reference to an async executor as arguments (or similar)?
> What would it even look like for the top-most function to have to know “everything” that all the child functions end up ever needing? For every function parameter change in the entire program to somehow force a change all the way up the entire call stack? Clearly this does not happen. It so thoroughly does not happen that you may be having difficulty even imagining what it is I am talking about.
Really? In my experience, it certainly does happen and I'm generally happy (but not thrilled) when it does. It means the type system is doing its job, forcing me to provide all the dependencies that a function needs. If 38 layers deep something now needs a connection string that it didn't have before, that had better bubble up to the top-most function. Unless of course the top-most function has already provided it to an intermediate layer which can break the chain, but that is very different than the context.Background() example.
Yeah, in my experience it absolutely can happen, and the way it generally looks is that the dependencies get roughly grouped into larger structs which are then being passed down the call stack mostly as-is, until they hit a certain abstraction level/barrier where the parts of those structs start being passed down.
It's somewhat tedious, that's why modularity becomes even more important: with correctly chosen borders, the grouping of dependencies becomes very clean and self-evident, the whole data flow just flows. Chose them wrong, and you spend most of your time adding and removing those pesky extra arguments.
Is `const`-ness (in the rust function keyword sense) a function color?
If you're using it diligently enough to where some functions can be statically proven "pure", then yes. You can't call an impure function from a pure one.
Generally when talking about colored functions, I think of an example like "async function" in javascript, those are definitely different colored.
I don't know about what would constitute as colored function in go as I generally don't consider arguments to be function colors.
Maybe methods in go would be different colored functions when they are defined on different struct types.
I don't think there's a way to color functions in Go. That's not terribly special, several languages have no colors in them.
A subset of function arguments are function colors because async/await is isomorphic to requiring and passing a `callback` arg around. It was named 'callback hell` because there wasn't an escape hatch and it affected every function in the callstack.
I'd venture to guess that there's a more abstract way of formalizing this in the sense that monad instances are required to be threaded through the callstack in particular cases(would love to see this spelled out formally) where IO and Cont(?) have this requirement but State like the author points out, does not.
I think this is a case where you want to avoid architecture astronautics and deal with languages as they are presented. Yes, in the end it's all continuation passing, or depending on the cut of your jib, it's all just assembler in the end, but meanwhile, down in the trenches, there are real, practical differences in color that change how you program in those languages, and that's the topic color addresses.
>> JS... Because our strawman is a modern (shitty) language
This is an attitude I dislike from all these niche language communities. They give each other pats on the back for shitting on the most popular, versatile, compatible and adaptable programming language of all time.
I started using JS back in 2004 and, since then, I have watched it constantly being shat on by theoretically-minded folks. Before Node.js, people would keep pointing out "JS is a scripting language, not a programming language." Then, it became a programming language and went through the biggest adoption growth phase of any language in the history of mankind. I literally saw hardware manufacturers adopting JavaScript and even building custom JavaScript engines for their custom hardware, all while their academically-minded peers simultaneously shat on the language. They shat on it wave after wave, while it kept getting better and better and proved itself to be more adaptable than their own pet languages.
People are so hard-headed and biased against JS that the community literally had to invent a new label/wrapper; 'TypeScript' as a Trojan horse to get those stubborn folks to use it.
My view now is that JS critics are all talk. JS has been doing the walk; trampling all over their theories and abstractions and leading the way in many regards. Yes, it has bad parts, but all of these parts are avoidable; none of these parts are fundamental to the language. The matrix showing of how JS comparison operators down-cast everything to a string (leading to un-intuitive results) keeps being brought up but nobody asks the question "Why would anyone use a casting comparison operator to compare variables of two different types in the first place?". The fundamentals are excellent and proven as such.
"Function colors" are just effect systems, which are not a new concept while also being extremely useful and super nifty. I hope someday we can stop wringing our hands in a panic over "colored functions" just because some ancient version of Javascript had a frustrating implementation of one specific effect. It reminds me of how ages ago certain people acquired an instinctual aversion to static typing attributable entirely to ancient versions of Java and its attendant verbosity and relative anemia.
No. Function colors (see criterion 4 in the original article) deal with the idea of "how much of a pain in the ass is it to deal with" which is not really a concept that is a PL theory concept.
The article only lists 3 criteria, not 4, but if you're referring to the final one:
"3. The change may require a change to all functions in the stack."
That's a potential feature of effect systems. An effect generally indicates a restriction of some kind, and depending on your desired semantics it could be totally counterproductive to paper over it with abstraction. And there's a variety of different knobs to twist that aren't necessarily the same semantics that any given implementation of async uses. To use Rust as an example, its `const `effect imposes restrictions down the callstack, and cannot be discharged (const functions can only call other const functions, period), while its `unsafe` effect imposes restrictions up the call stack and can be discharged (by a non-unsafe function that uses an `unsafe` block internally.
I don’t know, “the change is a major pain in the ass” might be a good addition as criterion 4.