Arial actually looks much better onscreen at small sizes, since that's what it was designed for. It bothers me to no end when "designers" use Helvetica or Helvetica Neue on websites with small body copy. Guys: Arial is Helvetica, but optimized for the screen. Use it.
As for why not Helvetica or Times New Roman, it's because they're boring. Using either of these typefaces is a non-choice. It's like not using any color in your design. If you do it right, your design can look timeless, transparent, minimal. If you do it wrong, you look lazy. Arial is a great choice for somebody like Google who wants to position their brand as if it's the very substance of the internet. Before the recent re-branding, they looked lazy (because they were). Now they look timeless. But if you want your brand to have a unique character, or you have interesting typographical problems to solve, you should look into other typefaces.
Arial, believe it or not, wasn't originally designed for the screen. It's not an optimized Helvetica, it's a generic looking franken-Grotesque that was billed as a low-rent Helvetica. And that's pretty much what it is.
Some of the other 'Microsoft core fonts' (Arial was originally designed by Monotype in 1982, not Microsoft) were actually designed for low resolution screen display, like Georgia and Verdana. Arial is better hinted than some of the versions of Helvetica out there, especially some of the lousy ones more common to older PCs. It has a high x-height and was designed to be readable at smaller sizes—in print. But so was Helvetica.
In fact, one of the virtues of Helvetica is that it's pretty legible as body text. Like any typeface, it can be abused, but it's not a crime to use it for body copy. Just make sure not to track it too tightly.
As high resolution displays become more commonplace, concerns about hinted type should lessen. And having to design fat, wide letters for crude raster grids will hopefully become a thing of the past—like old terminal displays.
One last thing: Helvetica's a pretty sweet looking typeface; contrary to its reputation, not at all 'neutral', it has midcentury charm and character without looking too dated. Give it its due!
Wow, I stand corrected. I had recalled reading an article that claimed this, but according to the Wikipedia entry, it's not true.
I just did a quick comparison of Helvetica and Arial at 13px and, while they are very nearly identical at that size, enough of the superiority of Helvetica shows through (the q and R look nicer, for instance) that I'm willing to completely reverse my opinion.
Helvetica Neue on the other hand still looks awful at 13px on a standard resolution screen due to the tracking being too tight.
I really like Helvetica Neue, but I think you're right -- it doesn't work too well when it's small, at least on screen. At a browser's standard 16px it seems to be pretty good.
But in five years we'll all have high-DPI displays, right? (For certain values of "high-DPI" and "all.")
So I want people to concentrating on what I write, not how the letters look. Am I missing the point though? Every time I've looked at custom fonts I haven't been able to find any that look more "readable" for large amounts of body text than what already ships on everyone's devices. They either look basically the same (so why bother) or they look gimmicky, and as I said, I want people to looking at my writing, not at my writing, if you get my drift.
As for why not Helvetica or Times New Roman, it's because they're boring. Using either of these typefaces is a non-choice. It's like not using any color in your design. If you do it right, your design can look timeless, transparent, minimal. If you do it wrong, you look lazy. Arial is a great choice for somebody like Google who wants to position their brand as if it's the very substance of the internet. Before the recent re-branding, they looked lazy (because they were). Now they look timeless. But if you want your brand to have a unique character, or you have interesting typographical problems to solve, you should look into other typefaces.