Boot speeds on the timescales in the video end up having little to do with horsepower once you get to having "some sort of SSD to load the boot image from" and "some sort of CPU that can at least decompress that in a reasonable time". It's more the "Wait here for 1 second for this hardware to probe", "Wait here for 5 seconds to see if the user wants to go into setup", "Wait here for network to timeout" type configuration items. Having a ton of high end hardware just makes this worse, more cores to initialize, more memory to initialize, more devices to probe.
I picked the low end fanless box as example because of this - it boots significantly faster than my high end box after all. It still loses out to how little a VM has to do while booting but not by much. It defaults to assuming a very short "wait on user" time for BIOS/UEFI, it has a mostly static and very simple hardware topology so doesn't need to dynamically load much during BIOS/UEFI, and systemd isn't told to do all those boot time loads/delays the Debian default config seems to have. For whatever you have "slimmed down" on it you're still trying to load nvidia GPUs, run cloud-init scripts each boot, and whatever networked service you mentioned thinks it needs to wait for network before letting the box finish booting is. Those kinds of things are what make the Debian VM boot slow, not systemd or necessarily the size of the kernel until the more noticeable things are fixed. Systemd is just what Debian uses to do those boot time things and if you did the same things with OpenRC or SysVinit or your own init system without changing what was being done it'd be a pretty slow boot too.
As for VPS if "not systemd" and small/fast/static are high or your list I'd highly recommend Alpine Linux over Oasis. It's known as the gold standard for containers but it makes a nice lightweight VM box too. I don't have anything against systemd but I did run Alpine as my standard VM image for a number of years and was very happy with it on the server side plus you get to keep convenient things like a package manager if you so desire. Their "Virtual" pre-built image will have the kernel image and firmware files slimmed down to the point you might not even care about trying to roll your own changes into it.
I picked the low end fanless box as example because of this - it boots significantly faster than my high end box after all. It still loses out to how little a VM has to do while booting but not by much. It defaults to assuming a very short "wait on user" time for BIOS/UEFI, it has a mostly static and very simple hardware topology so doesn't need to dynamically load much during BIOS/UEFI, and systemd isn't told to do all those boot time loads/delays the Debian default config seems to have. For whatever you have "slimmed down" on it you're still trying to load nvidia GPUs, run cloud-init scripts each boot, and whatever networked service you mentioned thinks it needs to wait for network before letting the box finish booting is. Those kinds of things are what make the Debian VM boot slow, not systemd or necessarily the size of the kernel until the more noticeable things are fixed. Systemd is just what Debian uses to do those boot time things and if you did the same things with OpenRC or SysVinit or your own init system without changing what was being done it'd be a pretty slow boot too.
As for VPS if "not systemd" and small/fast/static are high or your list I'd highly recommend Alpine Linux over Oasis. It's known as the gold standard for containers but it makes a nice lightweight VM box too. I don't have anything against systemd but I did run Alpine as my standard VM image for a number of years and was very happy with it on the server side plus you get to keep convenient things like a package manager if you so desire. Their "Virtual" pre-built image will have the kernel image and firmware files slimmed down to the point you might not even care about trying to roll your own changes into it.