Our mission to make business better is fueled by readers like you. To enjoy unlimited access to our journalism, subscribe today.
It’s been a traumatic 12 months for the thousands of queer teens who have contacted the Trevor Project, a mental-health crisis and suicide-prevention center.
First there was the coronavirus pandemic, which blocked many people from physically socializing with friends or attending schools in person. Then there was the police killing of Black Minneapolis resident George Floyd, which became a tipping point in the fight against racism.
Adding to the turmoil recently were the Capitol riots, particularly for young people of color, explained Tia Dole, the Trevor Project’s chief clinical operations officer.
“It is such a shock to see the differential treatment of the people who took the Capitol versus what happened with [Black Lives Matter] protesters or other protests over the summer,” she said. “It was reinforcement of the belief that they are not equal citizens in this country.”
For crisis centers like the Trevor Project, the coronavirus pandemic and recent political upheaval have proved to be major challenges for counselors and volunteers. The Trevor Project, like other organizations, had to scramble in March to ensure their employees could work from home. But given the nature of the nonprofit’s work, the stakes were much higher if its systems went offline.
“There is no pause button for suicide,” said John Callery, the Trevor Project’s vice president of technology. “We couldn’t afford a minute of downtime.”
He added that calls from distressed youth were “nearly double our pre-COVID volume.”
The challenges of a mental health crisis
Although the U.S. is experiencing a mental health crisis, some experts say, not everyone who needs help is seeking it. Becky Stoll, vice president for crisis and disaster management at nonprofit Centerstone, said that calls to her crisis center were relatively flat versus the previous year. “People tend to rally” during disasters, she said, by focusing on merely “surviving” the catastrophe: “It’s almost like mental health issues are put to the back burner.”
Travis Atkinson, a behavioral health crisis systems expert and consultant for health care firm TBD Solutions, added that for some people, COVID-19 and shelter-in-place rules actually had a positive impact on their mental health. “For some people who are chronically suicidal and who struggle with depression, having family and support around because of mandated stay-at-home orders is actually helping them,” he said.
But for children and young people living with abusive families, shelter-in-place rules have made their already difficult lives more distressing. Jonathan Goldfinger, CEO of Didi Hirsch Mental Health Services, said his nonprofit started testing a new service in August that lets people contact his organization via text messages instead of phone calls. Many children, worried that their parents may overhear them talking about their problems to counselors, have used the texting service.
For example, Goldfinger said an 11-year-old texted the firm about ingesting a bottle of her mom’s prescription drugs out of despair. A few weeks later, a 14-year-old texted the center saying that she was holding a loaded gun and that she was “distraught over her parent’s failing marriage.” In both cases, the center was able to counsel the children over text message and dispatched emergency personnel.
Goldfinger said his nonprofit expects that child abuse has increased during the pandemic in many households. But because many children aren’t going to school or seeing pediatricians in person, teachers and doctors aren’t able to see the warning signs.
At the Trevor Project, Dole said that many children are texting the organization as well because of similar reasons. Last year, a 5-year-old texted the crisis hotline, Dole said, declining to discuss the case.
“Five-year-olds are suicidal sometimes,” Dole said.
How A.I. can help crisis centers
This year, some centers like the Trevor Project and the Crisis Text Line have started using A.I. to keep up with the influx of emergency texts and calls. They have found machine learning to be a good tool for triaging cases, by analyzing the words in a chat to determine who is more likely to harm themselves.
“One of the models allows us to assess who is at the highest risk of suicide so that those folks get at the top of the queue,” Dole said.
Lili Torok, a Crisis Text Line senior data researcher, said that her firm’s machine-learning software tries to deduce when a person is at “imminent risk,” meaning they have expressed “suicidal threats.” In such cases, counselors are flagged to review the texts so that they can take quick action, including contacting a health care firm that can quickly dispatch an ambulance, Torok said.
Despite advances in A.I., however, all of the crisis centers Fortune talked to said that A.I. is no replacement for human counselors. The technology is useful for preliminary screenings and triaging, but it’s no substitute for professionals who are trained to develop a rapport and counsel those in distress.
As Goldfinger said, if a child on a crisis call says, “Oh, I’m fine,” a human counselor may recognize a change in the child’s voice, indicating something is wrong. A.I. is not capable of noticing nuances yet, he believes.
Goldfinger noted that some vendors, which he declined to name, are pitching their A.I. services for the mental health industry as more capable than they actually are.
Said Goldfinger: “When you get underneath the hood and you ask, How many lives have you saved? What is the actual risk reduction? Stuff like that is not necessarily being calculated in what I would say is the most responsible and equitable manner.”
More must-read tech coverage from Fortune:
- Tesla recalls roughly 135,000 vehicles
- Everything to know about buzzy social app Clubhouse
- Elon vs. nature: Tesla is building its new Berlin factory on an endangered reptile habitat—will it matter?
- Meet Airbnb’s host whisperer
- Why companies are thinking twice about using artificial intelligence
