Pressing need for ethical and regulatory oversight of therapeutic voice AI, SFU expert urges

media release

Pressing need for ethical and regulatory oversight of therapeutic voice AI, SFU expert urges

As voice artificial intelligence (AI) speeds toward use in clinical settings, a researcher from Simon Fraser University is highlighting the urgent need for ethical, legal, and social oversight-especially in therapeutic care.

Voice AI analyzes vocal patterns to detect signs of physical, cognitive, and mental health conditions based on vocal qualities like pitch and jitter or fluency and specific words people use. Some tech companies have even dubbed it "the new blood" of healthcare because of its potential to act as a biomarker, but SFU health sciences researcher Zoha Khawaja urges caution.

In her thesis paper, Khawaja, a member of the Bridge2AI Voice Consortium, explores the potential and the perils of voice-based AI apps in the mental health field.

Khawaja's study used structured, multi-round surveys to gather insights from 13 stakeholders, including clinicians, ethicists, and patients. While 77 per cent of participants supported using voice AI to improve patient outcomes, 92 per cent agreed that governance models should be established by healthcare or governmental organizations to oversee its integration.

"Voice AI holds real promise as an objective tool in the mental health field, which has always relied on subjective diagnostics like self-reporting and interviews," says Khawaja. "But the entrepreneurial speed of the tech is outpacing regulatory oversight in such a high-stakes environment like healthcare."

Some companies already offer apps that analyze short voice samples to assess mental fitness. However, Khawaja warns that these tools often operate in a "wellness" gray zone-avoiding classification as medical devices and sidestepping privacy protections.

"There's a real risk of therapeutic misconception, where people may believe these apps are providing clinical diagnoses or treatment, when in fact they're not," Khawaja explains. "That's particularly dangerous for vulnerable users who may not have access to traditional care."

Key concerns raised by participants included algorithmic bias, lack of transparency, erosion of human connection in care, and unclear accountability. The study advocates for a digital compassionate care approach, where AI tools support-not replace-human relationships in therapy.

"Patients might feel safer talking to a chatbot than a person," Khawaja says. "But that can lead to overreliance and isolation. These tools should strengthen the clinician-patient bond, not undermine it."

She also recommends a shared responsibility model among developers, regulators, and healthcare providers to prevent ethics dumping-the unfair shifting of ethical burdens onto clinicians. Notably, 83 per cent of participants agreed that healthcare practitioners should be held accountable for adverse events resulting from the use of voice AI tools.

"But clinicians are already overburdened," Khawaja says. "Expecting them to bear the ultimate responsibility of these technologies is unrealistic."

Clinical trials to validate voice as a biomarker are currently underway in the U.S., where regulatory sandboxes-controlled environments for testing new technologies-are being proposed to anticipate ethical challenges and inform policy before voice AI enters clinical practice.

"I'm not saying we shouldn't use voice AI or virtual chatbots in mental healthcare," Khawaja says. "But we must use them safely, responsibly, and with compassion. We need a framework that balances innovation with ethics, technology with humanity."

SFU expert available

ZOHA KHAWAJA, researcher, health sciences [email protected] Expertise: responsible therapeutic AI support, ethical design and usage of voice AI virtual conversational agents in digital mental healthcare

Contact

ROBYN STUBBS, SFU Communications & Marketing 604.376.0971 | [email protected]

Simon Fraser University
Communications & Marketing | SFU Media Experts Directory
778.782.3210

ABOUT SIMON FRASER UNIVERSITY
Who We Are
SFU is a leading research university, advancing an inclusive and sustainable future. Over the past 60 years, SFU has been recognized among the top universities worldwide in providing a world-class education and working with communities and partners to develop and share knowledge for deeper understanding and meaningful impact. Committed to excellence in everything we do, SFU fosters innovation to address global challenges and continues to build a welcoming, inclusive community where everyone feels a sense of belonging. With campuses in British Columbia's three largest cities-Burnaby, Surrey and Vancouver-SFU has ten faculties that deliver 368 undergraduate degree programs and 149 graduate degree programs for more than 37,000 students each year. The university boasts more than 200,000 alumni residing in 145+ countries.