You can find essays from AWS users who swear it would be prohibitive--financially and mentally--to ever switch.

And you can find essays of people who run their own rack who swear the same thing about moving to AWS.

I worked at an all-AWS bay area startup that ran a $100k/mo Amazon bill, and now I work at a bay area startup that runs all its own hardware.

It's about tradeoffs. It's not just about cost. (Though there are many cases where Amazon is cheaper and it has little to do with your monthly spend, and certainly not an arbitrary threshold like $3k/mo.)

Writing this at the close of my employer's Open Enrollment period, I'd say this: Comparing AWS with running your own hardware is like trying to compare two health insurance plans. Each brings its own seemingly impossible tradeoffs. All you can do is bring your experience and judgment to identify core issues, make the best call you can, and suppress the urge to dream of the road not taken when you're knee deep in whatever hell you're experiencing that would be a non-issue if only you had went the other way.

Though if you loathe the AWS "fanboyism" so much then maybe it's the company you're keeping: Few currencies in a startup are as valuable as flexibility. The younger the startup, the truer that is.

If I knew how to setup OpenVZ and create < 10ms latency VPNs across multiple datacenters, then I would be a lot better at ops than I am. While I'm much better at ops now than I was when I co-founded awe.sm, I am still primarily a developer.

Our ops guy is much better than me, but his time is better spent working on higher-stack stuff like deployment automation, monitoring and efficiency tuning than on re-inventing a virtualization stack to save a few thousand dollars every month.

If we were bigger, it would be more worth the time and money spent. But without doing the math, we would have to be quite a lot bigger, I think.

I am not trying to be sarcastic, I am just tired of some of the AWS fanboi's (not you) who act as if AWS solves all of your problems immediately for $50 a month.

There are real costs no matter which way you go.

PS, would be very surprised if you had even 45ms latency between AWS-east in Virginia and any of their facilities on the west coast...

Much of the attraction that I see from people comes from the flexibility. Once you get to any significant size in an organization, it's not unusual for people to have to wait for _months_ to get servers rolled out.

The ability to whip out your credit card, and fire up new servers at AWS in minutes, becomes very attractive in that scenario.

Ghshephard nailed it, as usual.

I wasn't taking you as sarcastic, nor was I attempting snark in my reply: there are real costs either way, and it's our current judgement that learning how to do a virtualized, distributed stack would be more expensive than getting AWS to do it for us. (But, if I'm being honest, we haven't done any math)

And the < 10ms latency I'm talking about is between zones within us-east; latency to the west coast is a lot worse, but we only have emergency failover capacity in west.

> If I knew how to setup OpenVZ and create < 10ms latency VPNs across multiple datacenters, then I would be a lot better at ops than I am.

I've got the Nobel Committee for Physics on line 2 should you accomplish that trick for DCs more than 30 km (18.6 mi) distant.

Edit: Nevermind (misplaced decimal).

10 msec * c = ~3000 km or ~1860 miles.

Of course light doesn't travel that fast through glass and there is latency at the hardware on either end. ;)

Units foulup. I'd used 0.10ms, not 10ms.