AWS makes sense if your needs are elastic. Netflix's needs are elastic, their usage grows and falls over time and even over the hours during the day. Most sites' aren't.

If you know you need a server or two with a certain amount of RAM and HDD space, then rent a server or two at a high-quality DC like SoftLayer or Rackspace. It'll be cheaper, and you might be surprised how many "instances" worth of performance you get out of a single real box. You can treat those servers the same as virtual servers in every way -- they just take 2 hours to "boot a new instance" because someone has to build and provision it to your account at the DC.

You don't need to buy your own hardware and colocate unless you have specialized hardware needs.

I agree with you, but I'd like to remind us of something. Netflix doesn't use AWS for their primary thing: streaming videos. So their AWS workload is probably just not that demanding. What runs on AWS is just a bunch of scripts for selecting which videos to stream. Since the costs of hosting these scripts are negligible compared to the cost of streaming videos, Netflix just doesn't care about the fact that they are overpaying. If something is responsible for 0.1% of your costs, the fact that you're paying 2 to 10 times more for it is easy to ignore. That's the reason why they still use AWS. Their needs are not particularly elastic, in my opinion. Yes, people probably access Netflix more often in the evenings and weekends, but that is true of just about every entertainment website or blog, and it doesn't make AWS cost-effective.

Check out http://daveyeu.com/ops-for-little-babies/ -- the transitions make it a bit slow to get through, but the content is worth it.

His argument is: start on Heroku until you realize you can get 10x to 100x better performance running on AWS directly, move to AWS, but then you have to do your own monitoring, logging, deployment, upgrades, load balancing, database maintenance, etc.

As far as non-AWS, there's no need to be scared of "real" servers. You'll often get stronger hardware (minus those crazy 100+ GB VMs), cheaper transit (100TB included in a $200/month server), cheap dedicated IPs, and custom IPv6/VLANs/ACLs if needed.

At this point, Ruby is like AWS. People just do it because everybody just does it. "Why is everybody using Ruby when it's slow, has an implementation dictated by implementation instead of specification, and has multiple competing runtimes in an attempt to fix flaws in the primary implementation?" People latch on to the hype and forget to evaluate reality after a while.

If you expect your needs to fluctuate wildly, then AWS is the thing to go; if they fluctuate mildly, then metal will be cheaper but has other issues, such as all the administration tasks are different.

AWS allows you try to run a large setup and then just drop it if the project doesn't work out; AWS allows you to cheaply run large but infrequent tasks quickly - you're not paying for idle servers.

Also, you may want to choose dedicated hosted servers instead of "in-house metal", that's a solid middleground between that and AWS.

What I don't get is that the administration is pretty much equivalent between the 2. I spent a ton of time learning how to run recipes for EngineYard, and how to restart my processes, and how to kick off SOLR as soon as I deployed... For all the hype that these cloud/PAAS gets, you'd think it's as easy as clicking a button, checking a few boxes, etc.

Ever since I moved to managing my own servers, it got arguably easier. I wrote a Capistrano script that builds and deploys everything to production.

The issue with managing your own servers is that you are tied 1:1 to specific hardware. Something fails? You have no server. Need an upgrade? It's a financial outlay. You may lease the hardware and you may also lease the hardware maintenance. In an environment like AWS you wouldn't really care about the hardware. Your instance would have died and you'd restart it and hope for a better hardware. You do trade ease on the hardware front with battling with the unknowns of the cloud. You just need to pick your battles. Hardware battles normally require more work and understanding of the hardware and a lot more menial work (RMA of an AWS instance is far simpler than RMA of a server).