What makes you believe LLMs will stick to the rules of your formalized language? It's likely. There are only probabilistic guarantees, which kind of obviates the benefits of a formal spec.
If you need formalism, wrap it into a formal language for the formal parts. But assuming that a formal language spec prompt avoids probabilistic outcomes? Yes, that is missing it by a million miles. You're missing the strength of LLMs. (Which is unstructured input->semi structured output)
Unit tests. SudoLang has a port of the Riteway unit testing framework, and programs can be stepped through in debug mode to show intermediate processes. Such debugging sessions could be used to fine train future models on algorithmic thinking.
This stuff is experimental and not 100% yet, but it does mostly work today, just days after SudoLang was specified. And you don't even need to paste the spec: the whole language is inferable by GPT-4.
What makes you believe LLMs will stick to the rules of your formalized language? It's likely. There are only probabilistic guarantees, which kind of obviates the benefits of a formal spec.
If you need formalism, wrap it into a formal language for the formal parts. But assuming that a formal language spec prompt avoids probabilistic outcomes? Yes, that is missing it by a million miles. You're missing the strength of LLMs. (Which is unstructured input->semi structured output)