If the price I have to pay is that people like you think I'm an idiot, I'm sure I'll manage.
I am curious though, how do you distinguish between people "talking out loud to their phones", and people talking out loud to other people using their phones (ie making a phone call).
There are a lot of situations where people talking on the phone to other people is annoying and rude. I expect there's substantial overlap with the situations where people talking out loud to Siri is annoying and rude.
To me, people talking out loud to their phones in public is akin to bluetooth headsets.
The technology might provide an easier way to let you do something, but you look like an idiot.