> The amount of things a computer can do in a single thread are amazing, and computers now have a dozen or more threads to do work. If developers cared about performance, things would easily be 20x as performant as they are today.
Why? A good portion of programs are still single-threaded, and often that's the correct choice. Even in games a single-threaded main thread or logic thread may be the only choice. Where multi-threading makes sense it should be employed, but it's difficult to do well.
Otherwise, it's up to the OS to balance threads appropriately. All major OSes do this well today.
I think what the author wanted to say is that because computers are very fast today developers have no incentive of writing optimized code.
Nowadays you just "scale horizontally" by the magic of whatever orchestration platform you happen to use, which is the modern approach of throwing hardware at the problem in the vertical scaling days.
It’s not about programs being multithreaded. It’s about computers running multiple programs at once on different threads and they all perform well.
One can write software that uses the CPU cache in non-dumb ways no matter how many threads your program has. You can craft your structs so that they take less space in RAM, meaning you can fit more in cache at once. You can have structs of arrays instead of arrays of structs if that helps your application. Few people think of things like this today, they just go for the most naive implementation possible so that the branch predictor can’t work well and everything needs to be fetched from RAM every time instead of building things so that the branch predictor and the cache are helping you instead of impeding you. People just do the bare minimum so that the PM says the card is complete and they never think of it again. It’s depressing.
The tools to write fast software are at our fingertips, already installed on our computers. And I have had zero success in getting people to believe that they should develop with performance in mind.
So your assertion is that developers should get in a big huddle to decide how they’re going to consume L1 between applications? Which of course no dev has control over since the OS determines what runs and when.
You can make your time in the CPU more efficient by thinking of the cache and the branch predictor, or you can say “nothing I do matters because the OS schedules things how it wants.” Up to you I guess, but I know which of those approaches performs significantly better.
Why? A good portion of programs are still single-threaded, and often that's the correct choice. Even in games a single-threaded main thread or logic thread may be the only choice. Where multi-threading makes sense it should be employed, but it's difficult to do well.
Otherwise, it's up to the OS to balance threads appropriately. All major OSes do this well today.