Fascinating perspective on slow requests causing unwanted or less useful concurrency. I am always trying to get frameworks and so on to enable high concurrency at low costs. It drives me nuts that so many web tier things always end up scaling like one core per simultaneous request, at best. To be fair they have a lot of parsing and formatting which to this day are slow. But booking latency as a consumer of that concurrent capacity is a whole nother approach. I usually attack latency as an enemy of deterministic behaviors, this is an interesting line of thought.
That's a very neat observation. I've long argued that "for capacity planning, focus on the mean; for user experience, focus on the high percentiles."
But as the author says, under a heavy-tailed distribution (which service times typically have) the mean is dominated by the high percentiles!
Fascinating perspective on slow requests causing unwanted or less useful concurrency. I am always trying to get frameworks and so on to enable high concurrency at low costs. It drives me nuts that so many web tier things always end up scaling like one core per simultaneous request, at best. To be fair they have a lot of parsing and formatting which to this day are slow. But booking latency as a consumer of that concurrent capacity is a whole nother approach. I usually attack latency as an enemy of deterministic behaviors, this is an interesting line of thought.