I don't read that as necessarily meaning the president or political power. There are a lot of decision making processes people are already trying to offload to language models or other black box AI products. Things like hiring, medical diagnoses, insurance claims, etc.

There is good reason to be concerned about the potential societal consequences of deploying this at scale while AI explainability, training data bias, and alignment remain unresolved.