It’s a different situation computationally. Transformers are asymmetric: hard to train but easy to run.

There is no such thing as an open source Google because Google’s value is in its vast data centers. Search is hard to train and hard to run.

GPT4 is not that big. It’s about 220B parameters, if you believe geohot, or perhaps more if you don’t.

One hard drive.