Another advantage of IPv6, if implemented well by the hosting provider (i.e. they assign you a /64 or larger), is that you can pick a random IP address from a pool of billions to host your SSH server on. There's a tiny chance of accidentally conflicting with another service if you're provisioning your addressing using SLAAC, but that chance is low enough that I'm willing to risk it. Scanning the entire IPv6 internet isn't very feasible for automated tools because of how large the IP space is.

This approach does require some client side hacking, though either in the form of SSH config, or in the form of a split horizon DNS so you can easily access your server, but that's no different from alternatives such as port knocking or simply altering the SSH port.