I feel like neural networks are increasingly going to look like code.

The next big innovation will be whoever figures out how to convert MOE style models into something like function calls.