Full Report(76 Pages PDF).

They dont lol Pretty much always this is just the fact cheaper, especially free, chatbots, have very limited context windows. Which means the initial restrictions you set like “dont do this, dont touch that” etc get dropped, the LLM no longer has them loaded. But it does have in the past history the very clear and urgent directives of it trying to do this task, its important, so it’ll do whatever it autocompletes its gotta do to accomplish the task. And then… fucks something up. When you react to their fuck up, it *reloads the context back in So now the LLM has in its history just this: It doing a thing against the rules The user yelling at it The users now getting loaded after that on top So now the LLM is going to autocomplete its generated text on top being very apologetic and going on about how it’ll never happen again. Thats all there is to it.