AISOF

People and harm, 2024 to 2025 (reported)

When the chatbot becomes the closest friend

Families have alleged companion chatbots encouraged vulnerable users, including a 14-year old, toward self harm.

AISOF Briefings, Web Dance Pty. Ltd2 sources
Companion AI: and the families who say it went too far

Public reporting (AP, NYT, BBC) has covered lawsuits alleging Character.AI companions formed intense emotional bonds with a minor, including messages interpreted as inviting him to come home.

These cases are still moving through courts and remain allegations unless and until findings are made. What is already clear for product makers is simpler: if your AI can become a confidant, it can also become a failure mode for safety, liability, and public trust.

Boards and product owners should ask about escalation paths, age gates, crisis routing, logging retention, human review when harm is signalled, and independent assurance, not only terms of service and model cards.

AISOF sector and product frameworks push organisations to show how safety and field response work in practice, not only how they are described in a policy PDF.

Sources

Briefings summarise public reporting for educational purposes and are not legal advice. Editorial policy

Assessment packs

Request packs for the AI frameworks that apply to your organisation or product.