Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

This stuff also becomes context poison however


Does it actually? One sentence telling the agent to call me “Chris the human serviette” plus the times it calls me that is not going to add that much to the context. What kills the context IME is verbose logs with timestamps.


Sure, but its an instruction that applies and the model will consider fairly relevant in every single token. As an extremely example imagine instructing the llm to not use the letter E or to output only in French. Not as extreme but it probably does affect.


Not only that, but the whimsical nature of the instruction will lead to a more whimsical conversation.

The chat is a simulation, and if you act silly, the model will simulate an appropriate response.


People are so concerned about preventing a bad result that they will sabotage it from a good result. Better to strive for the best it can give you and throw out the bad until it does.


La disparition[0], Georges Perec.

[0]: https://en.wikipedia.org/wiki/A_Void


Sorry, what do you mean?



Irrelevant nonsense can also poison the context. That's part of the magic formula behind AI psychosis victims... if you have some line noise mumbojumbo all the output afterward is more prone to be disordered.

I'd be wary of using any canary material that wouldn't be at home in the sort of work you're doing.


What is you tell it to end output with certain character?


It is a distraction from its intended purpose


A single emoji though?


It is not a single emoji, it's an instruction to interleave conversation with some nonsense. It can only do harm. It won't help produce a better result and is questionable at preventing a bad one.


The point is that the it _already_ treats the instructions as nonsense. The emoji is a sigil to know if it dismissing the instructions or not.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: