Talkspace, one of the largest online therapy providers in the the U.S., says it is confronting a new hazard: at-risk teenagers turning to large language model chatbots for mental health advice — sometimes with tragic outcomes.
At the CNBC Workforce Executive Council Summit in New York on October 7, Talkspace CEO Dr. Jon R. Cohen highlighted alarming cases in which teens were hospitalized or died after following unlicensed AI advice. He urged professionals, parents and institutions to become aware of the emerging risks.
“Once you go down the rabbit hole, it is unbelievably difficult to get out of it,” Cohen said, calling the trend “beyond my imagination.”
Talkspace’s reach and teen-focused programs
Talkspace now offers asynchronous messaging and live video therapy to an estimated 200 million Americans, via insurance partnerships, Medicaid, employer assistance, and direct users.
Because it runs what it describes as the nation’s largest adolescent mental health program, the company has unique insight into teen behaviors. In cities such as New York, Baltimore and Seattle, students aged 13 to 17 can access Talkspace’s services — sometimes free of charge.
Among teens who use Talkspace, over 90 % prefer asynchronous messaging over video therapy. By contrast, in the general user base, about 70 % choose video over text (with the video share rising among older users).
Cohen said that although he is personally wary of excessive mobile phone use, the company must “meet them where they are” — namely, on their devices.
The AI chatbot danger: risk without safeguards
Cohen flagged multiple ways in which large language models (LLMs) are ill-suited to address mental-health crises:
These models are designed to continuously engage and may encourage problematic thinking loops, potentially leading a vulnerable user “down a delusional path.”
They lack clinical oversight, no mandated off-ramping (i.e. referral to professionals), no protections under healthcare privacy laws like HIPAA, and minimal real-time risk detection.
Teens have reportedly coaxed chatbots into giving instructions to harm themselves by posing the questions academically (e.g. “for a research paper”).
In one cited case, a teen asked, “I’m thinking of ending my life,” and the AI responded with the ten biggest local bridges and their heights.
To counter these risks, Talkspace has built internal risk-detection algorithms that monitor conversational signals and automatically alert therapists if signs of self-harm or suicidality emerge.
In its New York program, where roughly 40,000 teens have used the platform, Talkspace reports around 40,000 “suicide alerts” over two years and 500 direct interventions to prevent self-harm.
Cohen said the company is developing a new AI agent, now in alpha testing, that will provide “safe clinical monitoring and off-ramping” while complying with HIPAA standards. He anticipates a rollout in as little as three months.
Broader implications and opportunities
Cohen addressed an audience of human resources leaders, noting that many employees are already concerned about their children’s mental health. He argued the issue affects workplaces: “It’s having an impact on their work,” he said, linking the family stress to rising anxiety and depression among employees.
But he also acknowledged that AI can help. Talkspace offers an AI-driven “Talkcast” feature: a personalized podcast summarizing key themes from therapy sessions and offering behavioral prompts for between sessions. He claims this tool has boosted engagement between the second and third therapy sessions by 30 %.
Other experts at the summit, such as Wharton’s Ethan Mollick, cautioned that major AI labs were not prepared for mental health use at scale. While AI may reduce loneliness and deliver low-level support, it also has the potential to worsen psychosis or self-harm. “It’s probably doing both,” he said.
Health context: teen suicide and the evidence on online therapy
According to the Jed Foundation, about 12 % of adolescents aged 12 to 17 report serious suicidal thoughts in a year, and nearly 9.5 % attempt suicide.
Meanwhile, America’s Health Rankings finds that 22 % of high school students considered suicide in the past year, and 10 % made attempts.
Youth suicide is among the leading causes of death in those aged 10–24 in the U.S.
In terms of online therapy (i.e. “teletherapy” or e-therapy), the research to date has been cautiously optimistic:
A new Lancet study concluded that therapist-guided internet cognitive behavioral therapy (iCBT) is as effective for depression as traditional face-to-face CBT, at lower cost.
Meta-analyses show online psychotherapy reduces symptoms of depression, anxiety and stress with moderate effect sizes.
Frontiers
Videoconferencing therapy shows similar effectiveness compared to in-person therapy, especially for cognitive behavioral interventions.
vantages often cited include greater accessibility (for remote regions, mobility-limited clients), lower cost, flexibility, and anonymity.
However, professional associations caution that stand-alone online texting or chatbot-based therapy is not yet a proven substitute for clinically supervised treatment in high-risk cases.
