The counter-intuitive nature of LLMs is so simultaneously interesting and frustrating. Overloading a single prompt definitely can create challenge remarkably similar to human short-term memory and attentional drift.
LLMs gain so much knowledge and capability from absorbing the symbolic relationships embedded in human language but in doing so, inevitably absorb many of the human foibles, sensitivities and weaknesses reflected in our languages.
It could be the best written, most informative article they've ever read, but anti-ai folks would dismiss it as slop the moment someone told them it was written by ai.
This is very reminiscent of the "everyone's a Russian bot" era of social media, where everyone would just lob that accusation at people without any real proof.
reply