Micro QR codes are a thing however they cannot store enough data for even a basic URL (https://google.com)

All QR codes are using some amount of error correction plus the locating squares in the corners which uses up space but is needed for people to scan them quickly and reliably.

The fundamental issue is that you are displaying binary data in physical space. My Google url is 18 characters so 18 bytes ASCII and 144 bits. At minimum that's 12x12 already. Then add the location squares and error correction. Then realise that your link is probably longer than that. Then realise that a standard is going to need to support links that are longer than yours also.

Now the resulting code needs to be physically large enough that someone's with an old phone with a bad camera can see it in sufficient detail without needing to get so close that the camera cannot focus.

You could always just use an NFC tag instead if that works. Buy them for pennies, load the link onto the tag and then they just tap their phone.

No its a dynamic link that changes over time and with each user.

NFC is still possible. You just need something you can program over USB

If some encoders are a bit inefficent, sometimes the wrong form of encoding an url is chosen and very rarely the data can't be encoded more efficiently because of the data.

https://example.com : 152 bits of raw data

HTTPS://EXAMPLE.COM : 105 bits of raw data

Some QR codes are really tiny. I think people just make them large so you can see and focus on the easily.