I haven't played around with hypervisors much but the whole point of k8s is not just isolation but all the primitives the control plane gives you which you don't need to implement. Things like StatefulSet, ReplicaSet, Volumes, HorizontalPodAutoscaler, Service, DNS, ConfigMaps, Secrets, Accounts, Roles, Permissions etc.

Also the container runtime which is containerd by default I believe can be switched out for micro vms like Firecracker (never done this though - not sure how painful it is).