And then there are places like Mountain View and Cupertino that were happy to have companies headquarter there but then refused to allow new housing to be built for the workers, putting even more pressure on SF to pick up the slack.

I was not aware the numbers were that bleak. Is that total housing units or just rental units?

Well, no one forced said companies to set ground there. Especially when a ton of their employees don't even want to live near Mountain View, Cupertino or Menlo Park and thus have to be bused for an hour+ from SF. Jus' saying ;)

This

The US is vast, but of course, let's cram all the technology workers in a tight corner of a bay and densify that to the max.

Grow Out. Of course, SF is a "city", not the kind of city that's part of an urban sprawl like the others. Maybe that's what other cities should start doing. That and quality public transport (not only Google Bus)

I dream of building a city in an otherwise-empty part of the U.S., one built around electric bike paths (and hell Segways lol), with bicycle and pedestrian paths and lightweight rail moving around the heavy stuff. Cars and trucks and trains can be on the perimeter and in several large arteries.

Looks like total units: http://www.bayareacensus.ca.gov/counties/SanFranciscoCounty....

Though I was actually surprised it was so high. That's units, not bedrooms, and it's only a city of ~800k people. I guess there's a lot of 1-person households to bring down the average despite all the families and twenty-somethings in 2-4 bedroom units with at least as many inhabitants as bedrooms.

As a new immigrant to US and Bay Area I am totally amused to see that there is so much vacant land everywhere and yet so little housing. Dont see why cupertino, MV, Sunnyvale cant have 10 floored housing complexes.

It's cars, mostly.

Many of California's cities have laws requiring massive, incredibly wasteful amounts of parking to be built even in supposedly high-density or transit-oriented communities. In addition, there are rules saying houses have to be set back a certain distance from the road, which takes a good deal of space. Finally, most places also require that roads meet a certain (high) "Level Of Service", meaning lots of cars can move through very quickly. Nevermind the fact that by building things like this you are forcing them to go much, much FARTHER. Have you noticed that driving through LA (or San Jose for that matter) is mostly driving through a sea of parking lots and wide streets? See http://en.wikipedia.org/wiki/Level_of_service for more.

There is also a cultural preference for lots of land; I'm bemused by how many people mock "McMansions" for having small yards but never spend any time in their own yards.

This is less true in the bay (and not at all in SF) than other parts of the state (looking at you LA) but it remains a problem. Also, areas where the majority of the population are homeowners tend to vote against new housing construction. This increases the value of their own homes at the cost of strangling newcomers to the area and making the cost of living exorbitant. Of course, without new housing those areas will continue to consist primarily of homeowners, so the cycle continues. Even in supposedly lefty places you see people oppose construction that would help increase housing supply, presumably because it's bad for the environment or will damage neighborhood character. I'm sympathetic to those claims, but making people commute from 40 miles away (if they're LUCKY it'll be on BART or Caltrain, but probably not) can't be good for those things either.

I wonder if it's time we reconsidered whether it makes sense for highly interdependent neighboring communities to be separate cities. Would we have a more integrated and efficient transportation, housing, and office infrastructure if the bay from San Mateo up to Mill Valley and east to 680 were the same city? Quite possibly. It would at least mean, to some small extent, that they couldn't say "we'll take your companies but housing the plebs is somebody else's problem - only huge single-family homes for us, thank you!".

there is also a lot of, we like our town and we don't want others here. Hence throw up building codes, regulations, and such, to keep housing from being built. You can use all sorts of PC buzzwords to make it palatable, from environment to maintaining historic sites. When large scale housing is built it usually is where people don't want to be.

Throwing up obstacles to building housing is often called NIMBY-ism or "PC", but it is also what owners of homes would be expected to do it they were maximizing their own economic self-interest

I was not aware that other home owners get a say into what is being built in the city. That should explain anything. It is very much like our Indian community where people on H1B often demand immigration reforms but once they have it they demand stricter immigration laws. :P

It explains some of the enclaves of low density you see even in large cities. Santa Monica and Beverly Hills come to mind as good examples of this; those areas would normally expect to draw more building but the local residents fight tooth and nail against it. Even small developments spend years and years being fought over.

Source: I lived in SM and attended city council meetings that discussed the matter because I wanted to see more transit-oriented development and improved bike infrastructure. Can't say I'm sad I left; socal is dysfunctional with regards to transportation. I do miss the beer, though.

I should have clarified that SM and BH are independent cities within Los Angeles. LA itself has some of the stranger borders of any city I know.

New construction is being built taller than previous (but not necessarily 10 stories).

The problem is that getting around in that part of the Bay Area still requires a car, so units with no available parking are much less desirable. Having 1 parking spot per unit, and 1 level of parking, effectively limits the height to ~4 floors of marketable residential units (a second level of parking would be more expensive than the first, and allow for ~8 floors of marketable units).

There is a fear of earthquakes destroying property and lives in California.

Please point Californians who think this at a photo of Tokyo.

Fears are often not rational, and such fears do play a minor role in feelings against certain structures in California. Talk to some people for a while and it comes up.

Having said that, it's not the main driver preventing development in the Bay Area. Citizens with money, influence, and established interests want lower-density development, and so that's what exists.

Or photos of the very much intact skyscrapers in downtown San Francisco after the 1989 earthquake.

Fwiw, they don't actually think this.

For whatever the reasons - the Japanese government decided that opportunities of building taller buildings outweighs the risks involved.

Both California and Japan have a long history of earthquakes and know the danger involved. When you look at the numbers, more people have died due to earthquakes in Japan than in California.

Prime example: CA 1994 Northridge earthquake (6.7 magnitude, 70 people died, $20B in damage) vs Hanshin earthquake (6.8 magnitude, 6400+ died, $100 billion in damage).

Are the fears irrational? Each culture has a different approach.

I agree with other comments though - fear of earthquakes is a minor footnote in the grand scheme of property development in California.

Most deaths in the Hanshin earthquake were caused by collapse of older buildings; modern highrises were largely unaffected.

I don't know how strong that fear is, considering that 12% of the US population lives in California.

> Only 376k housing units? WTF!

In a city with a population of 840K, is that so bad? That works out to roughly 2.23 people per unit.

In NYC, in 2011, there were 3.35M housing units, and population of NYC was 8.273M. That works out to 2.47 people per unit.

So why is NYC's situation not a clusterfuck, but SF's is?

First, NYC is also kind of a clusterfuck; rents there are terrible. But while NYC needs to continue building more, it at least has the excuse that it's built up a lot already, and its mayor is actively encouraging more development.

People don't cram into arbitrarily small spaces, so one thing to remember is the cause and effect: way more than 840K people want to live in SF proper, but there's no room. So 376K housing units effectively sets the population cap. But because a larger pool of people want to live there, the prices all go up. Rent control fixes that for incumbent residents but worsens the problem for everyone else. The only way to alleviate that is to create more houses so that a larger percentage of the people who want to live there actually can.

The subways in New York help the common case - minimizing how much is needed to spend on housing, while being well-connected to Manhattan. This is how people can live in a relatively cheap borough like Queens, and still have 15-35 minute commute to their job.

There's also a massive amount of rental apartment stock at any time. This means that you have the liquidity to make any tradeoffs necessary, by balancing your budget, commute distance, number of roommates, and the amount of space you need.

New York is also connected to several public transit systems (NJTransit, Metro North, PATH), so you can often go way cheaper, by leaving the city bounds itself.

It's not all sunshine and roses, though. Rent is going up in a lot of the previously-cheap neighborhoods in Brooklyn. This hasn't really affected me, but my sister has been priced out of multiple apartments in the 5 years she has lived in the city.