Well the big difference is that keys can't be accessed in a hardware wallet. If you accidentally plug in the USB drive into some untrusted box, you could potentially have everything stolen.
> some rando thumb drive that no one knows the location of or knows exists
You still have this option with a hardware wallet. eg:
> some rando hardware wallet that no one knows the location of or knows exists
> Well the big difference is that keys can't be accessed in a hardware wallet.
In theory.
> If you accidentally plug in the USB drive into some untrusted box, you could potentially have everything stolen.
This is like saying that no one should own a gun because you might point it straight at your own head and pull the trigger.
> You still have this option with a hardware wallet. eg:
Yes, that is true, but a hardware wallet is an extra piece of hardware that needs to be actively used to provide HD wallet keys for transactions. If all a person wants to do is store BTC for the very long term, then this approach is more complicated and hazardous than storing a non-HD wallet key pair on some physical medium; this is because a hardware wallet failure can result in loss of the ability to sign transactions, and chances are you're storing the seed phrase separately. Again, just for long term storage, there's next to no advantage between storing a seed phrase for an HD wallet and storing private and public keys. Depending on what a person intends to do with their money, a hardware wallet may provide zero value over the least complicated approach. Unless someone is paranoid over other people or the government knowing their balance, a single public address is more straight forward than managing an HD wallet.
> some rando thumb drive that no one knows the location of or knows exists
You still have this option with a hardware wallet. eg:
> some rando hardware wallet that no one knows the location of or knows exists