It's not convenient and you need to reboot your console to go back to running commercial apps, but it is present. At least it's a one-time payment to get the developer account.
Would anyone who's calling for iPhone sideloading be upset about Xbox sideloading? I wouldn't. By all means, let's regulate both Microsoft and Apple. The more the merrier.
I don't think I've ever looked at a video game console and thought, "I'm sure glad that I can't sideload games onto this." This is a slippery slope that I am happy to go down.