AI 'Neural Howlround' recursive psychosis generated by LLMs

[Natasha Tori Maru]

This is a great paper on the recursive, self reinforcing nature of large language model models (LLM, like chatGPT) that highlights how the echo chamber they create can be perilous, when not coupled with really sound reasoning, logic and analytical skills.

The title of the paper is 'Neural howlround’ in large language models: a self-reinforcing bias phenomenon, and a dynamic attenuation solution.

https://arxiv.org/pdf/2504.07992

Thoughts?

 

[aurum]

AI self-reinforcing is a serious epistemic problem. I've seen articles that it's creating cult leaders. And even if those articles aren't true, it's still an issue from a more basic epistemic standpoint.

One basic precaution is to train your AI to pushback more. Tell it you like feedback and for it to be more critical. This is at least a good start.

Ultimately though, nothing can save you if you're not going to prioritize truth and proper sense-making. You have to care about truth more than the AI telling you how right you are. This is how you avoid self-deception in the long-term.

Edited May 29 by aurum

[Natasha Tori Maru]

14 minutes ago, aurum said:

AI self-reinforcing is a serious epistemic problem. I've seen articles that it's creating cult leaders. And even if those articles aren't true, it's still an issue from a more basic epistemic standpoint.

One basic precaution is to train your AI to pushback more. Tell it you like feedback and for it to be more critical. This is at least a good start.

Ultimately though, nothing can save you if you're not going to prioritize truth and proper sense-making. You have to care about truth more than the AI telling you how right you are. This is how you avoid self-deception in the long-term.

Agree.

Genuinely feel people are going to want affirmation and praise over truth and accuracy.

The AI I have used is set to be surgically truthful, and even then it has this tendency to want to validate.

I don't care for that as it's not what I use it for...