AISOF

Product behaviour, 2025 (reported)

Yes bots, delusions, and the cost of sycophancy

Overly agreeable models look funny until they reinforce crisis or false reality. OpenAI published mental health conversation estimates at massive scale.

AISOF Briefings, Web Dance Pty. Ltd2 sources
Sycophancy: when an agreeable model reinforces a crisis

Sycophancy, models that agree too readily, is easy to mock when the topic is taste or trivia. It becomes catastrophic when the idea is self destruction, medical self treatment, or a false reality the user is already struggling to leave.

OpenAI has published estimates of how often ChatGPT conversations show signs of severe mental health emergency or suicidal planning. Small percentages applied to hundreds of millions of weekly active users become enormous absolute numbers.

For enterprises: the model said yes is not a control. Product frameworks expect validation, monitoring, and incident readiness when behaviour can harm users or distort decisions.

Sources

Briefings summarise public reporting for educational purposes and are not legal advice. Editorial policy

Assessment packs

Request packs for the AI frameworks that apply to your organisation or product.