If you are using the yubico-authenticator app then you are using TOTP, just with the seeds stored on your yubikey. This is still vulnerable to phishing.
I hope what you meant to say is that you are switching to using WebAuthn with your yubikey on all sites that support it, and then using your yubikey for TOTP on sites that don't support WebAuthn yet. WebAuthn is the thing that gives you actual protection against phishing.
I hope what you meant to say is that you are switching to using WebAuthn with your yubikey on all sites that support it, and then using your yubikey for TOTP on sites that don't support WebAuthn yet. WebAuthn is the thing that gives you actual protection against phishing.