The human voice is unique for each individual and contains lots of acoustic data. A friend can tell something is not right if your voice sounds a little off. Voice is also widely used to verify a person’s identity. Do you know the frequency, speed of voice and even the way a person coughs can offer clues to health? Clinicians and researchers at Weill Cornell Medicine and their collaborators hope to harness the voice data to detect signs of diseases and train AI algorithm to improve the accuracy of diagnoses and predict disease progression.
Leveraging Multidisciplinary Collaboration
Dr. Anais Rameau
Dr. Anaïs Rameau, an associate professor of otolaryngology, has been involved in using AI-assisted acoustic analysis to screen patients for difficulties in swallowing and pneumonia risk. When the National Institutes of Health launched the Bridge2AI program, which aims to create large-scale health care databases for precision medicine, Dr. Rameau connected Dr. Yaël Bensoussan, an assistant professor of otolaryngology at Morsani College of Medicine at the University of South Florida, with the Englander Institute for Precision Medicine to form the Bridge2AI Voice consortium. This collaborative relationship, led by the University of South Florida and funded by a $14 million four-year grant, expanded to include clinicians and scientists from several universities in the United States and Canada.
Dr. Rameau is not the only innovator who is interested in the potential of this NIH program to establish a flagship clinical-grade dataset for better diagnosis, prognosis and patient care. Dr. Alexandros Sigaras, an assistant professor of research in physiology and biophysics and director of AI and Extended Reality at the Englander Institute, also sees the opportunities this project offers.
As a scientist developing software solutions for precision medicine, Dr. Sigaras has always wondered why a clinical voice dataset hasn’t been established. After all, recording a patient’s voice is much less invasive and costly than collecting other biomarkers, he said. More importantly, the voice biomarker could serve as an indicator of a broad array of diseases including respiratory disorders, depression, Alzheimer’s and Parkinson’s diseases, autism and cancer.
Dr. Alexandros Sigaras
Dr. Olivier Elemento, director of the Englander Institute for Precision Medicine and a professor of physiology and biophysics at Weill Cornell Medicine, co-leads the project with Dr. Bensoussan and stresses the importance of breaking down traditional research silos. “The future of precision medicine lies at the intersection of diverse disciplines,” he said, “where we pair the clinical insights of physicians like Dr. Rameau with our computational and AI specialists to tackle challenges that no single group could solve alone.”
Creating a First-of-Its-Kind Voice Biomarker Bank
Bridge2AI Voice is a data generation project designed to assess the possibilities and set precedence for correctly applying AI’s power to analyze voice data and make clinical work more efficient. The team had to tackle several challenges during data collection, such as how to generate good quality voice data, what to record and how to guarantee data security and compliance with federal patient privacy laws.
Pantelis Zisimopoulos, senior software engineer in Dr. Elemento’s lab, worked closely with participating clinicians and listened to their needs at the beginning of the project. Zisimopoulos and his colleagues then created an app that met those needs and functioned in a standard format. As the team went through iterations with different patient cohorts, they improved on the data collection process. Tasks to be completed were condensed to smaller subsets targeting what the clinicians wanted to capture, which also reduced patients’ time commitment for reporting.
Software egineering team: Jeff Tang (left) and Pantelis Zisimopoulos
The voice data comprised patients’ tones and sounds of coughing and breathing, in addition to speech. Bridge2AI Voice researchers complemented voice data with clinically validated questionnaires, which included clinical diagnoses. Voice samples were compared with those diagnoses. In some cases, longitudinal questionnaires were collected as well.
To date, Bridge2AI Voice has collected nearly 30,000 recordings at clinical sites in five different diseases categories: respiratory disorders like COPD and others, mood and major depressive disorders like anxiety and bipolar disorder, neurodegenerative disorders like Alzheimer's and Parkinson's, autism and speech disorders.
Dr. Sigaras noted that the team put in significant amount of time and effort to ensure their tool delivers on a multitude of data and is accessible by design, encompassing vision, hearing, speech, mobility and cognitive features from the get-go. In the next step, the researchers want participants to be able to enroll in the project remotely, securely and contribute to the voice biomarker bank in the comfort of their homes. In other words, “everyone’s voice can be heard—with the pun intended,” Dr. Sigaras said.
Making Voice Data Accessible and Sustainable
As Bridge2AI Voice passed the halfway mark of the four-year, NIH-funded program, its team recently shared with researchers over 20,000 recordings of voice data, which included clinical annotations and all questionnaires with patients’ personally identifiable information removed. Dr. Sigaras anticipates launching remote data collection soon and making the voice biomarker bank publicly available with strict stipulations protecting patient identity and privacy. He believes the demand for biobanks like theirs will grow exponentially. Well-trained AI algorithm can process data and detect signs of diseases much faster than a human being, which could lead to early diagnosis, better treatment and free up clinicians’ time.
“Institutions like the NIH should continue to push the boundaries for this,” he said. “We're just scratching the tip of the iceberg. What if we combine AI in health care? And this is literally the first part that we are starting to reap the benefits.”
Dr. Rameau also emphasizes protecting patients while innovating with patient data. As one of the data acquisition leads for Bridge2AI Voice, her role entails helping generate protocols for data collection as well as manage different teams with their data acquisition process in different sites. She considers it essential to sustaining the project. She hopes to continue with this project which already taught them a lot about how to utilize AI in health.
“There's been a lot of buzz about AI in health. The reality is that you need to have collaboration at scale to really tackle these complex challenges,” said Dr. Rameau. “We’re hoping that many people will be able to use this dataset to develop new tools that we may not even think about right now and can then use those tools to potentially monitor disease or for other applications.”