A less finnicky approach would be binding to 0.0.0.0, then configuring incoming firewall policy. By default deny, then allow SSH through wireguard network. Or by default allow then deny SSH through public network.
if you want to get fancy and/or over engineered you would use systemd templated units to setup sshd@.service and a ListenAddress in the config listening on %i. Then you could bring up sshd@(expectedip).service for each expected IP
.. but that doesn't gain all that much tbh.if anything the only hesitation I'd have on listening to * and relying on firewall rules is if the service comes up before its configured. but exposing sshd isn't even that bad
