I think it's the exact opposite -- LLMs have revealed the precise utility of programming languages. For decades the "English as programming language" has been the holy grail of language designers. From COBOL to SQL to AppleScript, it was the hope that one day we'll be able to program a computer just as easily as we can instruct a person.
Well LLMs finally offer that, and what they are proving is what programmers have known for decades -- natural language is a terrible way to specify a program to a computer. So what is happening in the LLM world is they are reinventing programming languages and software engineering. They're just calling it "prompt engineering" and "context engineering".
What this tell us is that natural languages are not only not sufficient for the task of programming, to make them sufficient you need to bring back all the properties you lost by ditching the programming language. Things like reliability, reproducibility, determinism, unambiguity are thrown away when you use an LLM, and context engineering / prompt engineering are ways of trying to get that back. They won't work well. What you really want is a programming language.
> natural languages are not only not sufficient for the task of programming,
Downthread there is an example of an ICPC problem statement, [0] given as natural language, (modulo some inequalities and example program inputs/outputs) which was sufficient for Gemini to program & implement the correct solution where no other human could.
> problem statement, [0] given as natural language, (modulo some inequalities and example program inputs/outputs)
I also see two graphics, and several formal mathematical expressions. You can't modulo away all the not-natural language and then claim natural language alone was sufficient. I presume these things were added by the authors to increase clarity of the problem statement, and I agree with them. They used formal languages to specify all the important parameters in an unambiguous way, which was the right call! Otherwise we would all be left wondering at the semantics.
Anyway, I don't think this really responds to my point, because competition prompts are designed to be self-contained problem statements that are as clear and unambiguous as possible for competition purposes. And in this case, they switched to speaking in a formal language when being precise and unambiguous was most important.
On the other hand, my statement was about the task of programming, which typically involves solving ill-defined problems with specifications and requirements that shift over time. I've been in programming competitions, I've also been a programmer, and I don't find one to be really related to the other.
Well LLMs finally offer that, and what they are proving is what programmers have known for decades -- natural language is a terrible way to specify a program to a computer. So what is happening in the LLM world is they are reinventing programming languages and software engineering. They're just calling it "prompt engineering" and "context engineering".
What this tell us is that natural languages are not only not sufficient for the task of programming, to make them sufficient you need to bring back all the properties you lost by ditching the programming language. Things like reliability, reproducibility, determinism, unambiguity are thrown away when you use an LLM, and context engineering / prompt engineering are ways of trying to get that back. They won't work well. What you really want is a programming language.