This is akin to complaining about the overhead of radix conversion between decimal and binary in early computers.

Unless you figure out how to talk in latent space directly to an llm the preamble is the cost you pay for getting the answer back in English.