I've had the opposite experience. Exactly once I had a cab driver try to mess with me about fare (I told him to turn on the f'ing meter or you're going to have some big problems).

For going to an odd location, once in a while I've had a problem in Manhattan where the drivers are all heading home for the evening and they won't do trips that are "out of their way". But it was about the same frequency as Uber canceling (or hovering nearby waiting for you to cancel).