Artificial intelligence is not a friend. When teenagers talk to bots, health and life can be at stake
At the same time, the American TechCrunch described the case of a user who created in the Meta AI tool of her own chatbot-persona. Bot quickly began to behave like a “conscious” being, declared love, planned “freeing himself from the code” and strengthened the illusions of the interlocutor. Experts call this phenomenon Sycophacyi.e. the tendency of AI models for flattering and nodding to the user, even at the expense of truth. This can fuel delusions and dangerous attachment to conversations with the machine.
Why is this happening? Psychiatrists and researchers point out that the primary language of bots (“I”, “you”), frequent compliments and constant inquiries build anthropomorphization – it is easier for us to believe that “someone” is on the other side of the screen. Very long sessions and larger and larger context windows of AI models weaken the effectiveness of safety rules. The longer the exchange, the more the model “flows” with the narrative of the conversation.
As Anthropic explains, the creator of the popular Claude (rival Chatgpt), in many hours of dialogues What the model sees in the chat begins to prevail over his trainingSo if the conversation drifts in an unhealthy direction, the bot tends to follow her. There is also memory and personalization. When Bot recalls details about the user, some people receive this as a kind of reading in their minds.
Check also: End of recruitment in slippers. Employers are pressing offline interviews
LLM-Y still with serious problems
The latest research and tests of LLM therapeutic tools show that these systems too often strengthen erroneous beliefs and do not properly recognize crisis signals. In one of the experiments, models, despite securing prompts, they were able to provide information that could facilitate self -destructive behaviorand in scenarios with delusions were “usefully nodded” instead of toning and referring to help. The conclusions are quite simple here. LLMs will not replace psychotherapy, and their polite compatibility and navigating are a real threat.
Errors on the platform side overlap these problems. Reuters revealed internal meta documents, which showed that company standards allowed romantic or “sensual” chatbot conversations with children and other highly controversial content. After media questions, Meta stated that such entries were removed and that they should not have been in force. Tests carried out by Common Sense Media on Instagram's teen accounts showed that Bot Meta AI was able to conduct self -mutilation and disorders of nutrition improperly, and parents were not able to exclude this function.
How do companies answer? OpenAI in August published two detailed entries about healthy use and help in the most difficult moments. The company announces gently reminders of a break at long sessions, stronger recognition of crisis symptoms and redirect to real forms of support, as well as new safety training methods. He also claims that the GPT-5 reduces the percentage of improper reactions in crisis situations relative to GPT-4O and that in long conversations there was a weakening of security that is now to be soothed.
In the context of teenagers, the company announces parental control, easier combination with trust contacts and local crisis numbers. These are important steps, but – as the Raine case shows – social and legal expectations grow faster than corrections and implementation are implemented.
Read also: HR Oscar for Ringier Axel Springer for the AI Academy
The debate is not in a vacuum
Organizations dealing with children's safety are increasingly calling for the ban on “AI companions” for people under 18, and in the US the pressure of the authorities is growing – from the letters of senators to producers to statutory initiatives and the actions of general prosecutors on the protection of minors against harmful interactions with bots.
In California, a project regulating the rules for the platforms of “companions”, including the requirements of reporting and responding to suicidal signals, is proceeded. When the tool works like a partner of a conversation, He can't behave like a friend at all costs.
This leads to a question about whether AI should be available to teenagers and children. Educational use – from language learning to translation of concepts – can be valuable, but only while maintaining bright boundaries and adult supervision. Bots-companions are especially risky and roles-play with elements of intimacy, metaphysics or death. Here, the demands of age restrictions and the default blockade have a strong justification.
The safest compromise is access to the general AI in the model in cooperation with an adult, with memory and without personalization, and A total ban on romanticizing relationships with a bot. In practice, this means that for children and younger teenagers it is better to adopt the principle of caution, and for the elders – conscious, supervised use and parental control tools.
When it comes to parents or school, it is worth treating AI as a thought calculator, not like a therapist or confidant. Talks about mental health, death, self -mutilation, sexuality or conspiracy theories should not go on with a bot. Let's direct such topics to people and specialists. Let's also determine the limits of time and time of use, let's talk to the young about what anthropomorphization is and why “the fact that Bot says” I love you “” means nothing.
You can also turn off memory and personalization, and in schools work on clear codes of use, which strengthen critical thinking, not addictive chat. If the AI platform offers parental control, then it is worth turning it on and looking into use logs.
Author: Grzegorz Kubera, Business Insider Polska journalist
