It seems to me that it is even more simple. Economics 101: scarcity of a highly demanded product drives up costs. There's just no room for significant expansion, the bay area is surrounded on 3 sides by water. Furthermore, a good majority of the San Francisco peninsula is undevelopable since it's state park.