First Wearable Device to Detect Vocal Fatigue

Researchers from Northwestern University have created the first intelligent wearable device that continuously monitors how much people use their voices, warning them when they are using their voices excessively before vocal fatigue and potential harm set in.

For professional singers, professors, politicians, contact center agents, trainers, and anyone else who depends on their voices to effectively communicate and earn a career, the first-of-its-kind, battery-powered, cordless device, and supporting algorithms may transform the game. Also, it might make it easier for clinicians to follow up on voice issues with patients remotely as they receive treatment.

The study behind the new technology, which was created by an interdisciplinary team of materials scientists, biomedical engineers, opera singers, and a speech-language pathologist, will be released in the Proceedings of the National Academy of Sciences during the week of February 20.

The postage stamp-sized, soft, flexible device securely attaches to the upper chest to detect the minute vibrations caused by speaking and singing. From there, the recorded information is instantly sent via Bluetooth to the user’s tablet or smartphone so they can track their daily vocal activities in real time and calculate their overall vocal usage. Specialized machine learning algorithms can discriminate between singing and speaking, allowing vocalists to keep track of each activity independently.

Users of the program can customize their vocal thresholds. Their smartphone, smartwatch, or companion device is worn on the wrist and sends a real-time haptic feedback alert as they approach that threshold. Before going too far, they can rest their voices at that point.

“The device precisely measures the amplitude and frequency for speaking and singing,” said Northwestern’s John A. Rogers, a bioelectronics pioneer who led the device’s development. “Those two parameters are most important in determining the overall load that’s occurring on the vocal folds. Being aware of those parameters, both at a given instant and cumulatively over time is essential for managing healthy patterns of vocalization.”

“It’s easy for people to forget how much they use their voice,” said Northwestern’s Theresa Brancaccio, a voice expert who co-led the study. “Seasoned classical singers tend to be more aware of their vocal usage because they have lived and learned. But some people—especially singers with less training or people, like teachers, politicians, and sports coaches, who must speak a lot for their jobs—often don’t realize how much they are pushing it. We want to give them greater awareness to help prevent injury.”

Professor of materials science and engineering, biomedical engineering, and neurological surgery at Northwestern University’s Feinberg School of Medicine, Rogers holds the Louis Simpson and Kimberly Querrey Chair. Moreover, he is the Querrey Simpson Institute for Bioelectronics’ director. Mezzo-soprano and renowned opera singer Brancaccio teaches voice and vocal pedagogy at Northwestern University’s Bienen School of Music.

Oblivious to Overuse

Vocal fatigue is a continual, looming worry for the millions of Americans who rely on speaking or singing for their living. The typical issue happens when overused vocal folds enlarge, losing endurance and producing a raspy voice. Singers in particular are adversely affected by vocal fatigue, which reduces their capacity to sing clearly or reach the same notes as their healthy voice can. A singer’s intentions can, at best, be momentarily derailed by one brief episode of voice exhaustion. At worst, it might cause enough harm to wreck a career.

The root cause of the issue is ignorance. Rarely do people consider the relationship between vocal activity and how that activity affects their voices.

Although one in 13 U.S. adults has experienced vocal fatigue, most people don’t notice they are overusing their voices until hoarseness already has set in.

“What leads people into trouble is when events stack up,” Brancaccio said. “They might have rehearsals, teach lessons, talk during class discussions, and then go to a loud party, where they have to yell over the background noise. Then, throw a cold or illness into the mix. People have no idea how much they are coughing or clearing their throats. When these events stack up for days, that can put major stress on the voice.”

Inter-disciplinary Relationship

Brancaccio has spent decades looking for methods to keep her pupils conscious of how frequently they use their voices as a vocal health advocate. She gave her kids the task of maintaining a paper budget in 2009, which required them to physically record every time they sang, spoke, drank water, and did other activities. Almost ten years later, she transformed the program into Singer Savvy, an app that gives each user a tailored vocal budget and aids them in adhering to it.

Separately, Rogers had created a wireless wearable gadget to monitor swallowing and speech in stroke patients in association with researchers at the Shirley Ryan AbilityLab. The bandage-like sensor tracks speech patterns and swallowing abilities to track stroke patients’ recovery.

Rogers’ team adjusted the equipment to monitor coughing, a crucial COVID-19 pandemic symptom, in the early weeks of the outbreak.

“I wanted to gather more data and make our tracking system more precise and more accurate,” Brancaccio said. “So, I reached out to John to see if his sensors could help us gather more information.”

“I thought it was a great opportunity for us to extend our technologies beyond our very important, but narrowly targeted, uses in health care to something that might capture a broader population of users,” Rogers said. “Anyone who uses their voice extensively could benefit.”

The pair also partnered with a speech pathologist and voice expert Aaron M. Johnson to explore how the devices could be used to evaluate and monitor treatment for patients with vocal disorders. Johnson, who co-directs the NYU Langone’s Voice Center, said the small, wireless device could help track patients’ voices in the real world—outside of a clinical setting.

“A key part of voice therapy is helping people change how—and how much—they use their voice,” said Johnson, study co-author and associate professor in the department of otolaryngology at NYU Grossman School of Medicine. “This device will enable patients and their clinicians to understand voice use patterns and make adjustments in vocal demand to reduce vocal fatigue and speed recovery from voice disorders. Generalizing vocal techniques and exercises from therapy sessions into daily life is one of the most challenging aspects of voice therapy, and this device could greatly enhance that process.”

Trained by Singer Algorithms

The group altered Rogers’ already-existing tools to precisely gauge vocal burden as it changed. This covers the time of day, frequency, volume, amplitude, and duration. The new gadget recognizes vibrations as well, just as Rogers’ earlier COVID-19 and stroke patient devices, but it doesn’t record sounds. As a result, the device can identify the user’s vocal activity specifically rather than background noise.

The hardest issue was creating algorithms that could tell singing from speech. Brancaccio enlisted voice and opera students to participate in a range of singing exercises to train the machine-learning algorithms in order to solve this difficulty.

A group of classical singers, whose vocal ranges ranged from bass to soprano, used the gadgets while reading, humming, singing staccato scales, and other activities. Each artist produced 2,500 windows of singing and 2,500 windows of speaking, each lasting one second.

With greater than 95% accuracy, the developed system can distinguish between singing and speaking. Also, the device only records data from the wearer while it is being utilized in a choir environment, not sounds from singers nearby.

“Prolonged talking is one of the most fatiguing activities for people who are training to become professional singers,” Brancaccio said. “By separating singing and speaking, it can help people develop more awareness around how much they are speaking. There is evidence that even brief 15- to 20-minute periods of total silence interspersed throughout the day can help vocal fold tissues recover and repair.”

How To Use It

The wearer merely attaches the device to the sternum, which is below the neck and syncs it with the companion app to use it. The Rogers team is now developing a technique to tailor vocal budgeting for each user. Here, users will tap a button in the app if they feel vocal discomfort at any time throughout the day, thereby collecting the current instantaneous and cumulative vocal load. This information can be used to create a unique vocal fatigue threshold. A haptic gadget will vibrate to warn the user when they approach or pass their chosen threshold.

Similar in size and form to a wristwatch, this haptic device includes multiple motors that can activate in different patterns and with varying levels of intensity to convey different messages. Users also can monitor a graphical display within the app, which splits information into speaking and singing categories.

“It uses Bluetooth, so it can talk to any device that has a haptic motor embedded,” Rogers said. “So, you don’t have to use our wristband. You could just leverage a standard smartwatch for haptic feedback.

Although other vocal-monitoring devices do exist, those use big collars, tethering wires and bulky equipment. Some also use embedded microphones to capture audible vocal data, leading to privacy concerns.

“Those don’t work for continuous monitoring in a real environment,” Brancaccio said. “Instead of wearing cumbersome, wired equipment, I can stick on this soft, wearable device. Once it’s on, I don’t even notice it. It’s super light and easy.”

What’s next

Because Rogers’ previous devices capture body temperature, heart rates, and respiratory activity, the researchers included those capabilities in the vocal-monitoring device. They believe these extra data will help to explore fundamental research questions concerning vocal fatigue.

“This is more speculative, but it might be interesting to see how physical activity affects vocal fatigue,” Rogers said. “If someone is dancing while singing, is that more stressful on the vocal folds compared to someone who is not physically exerting themselves? Those are the kinds of questions we can ask and quantitatively answer.”

In the meantime, Brancaccio is excited for her students to have a tool that can help prevent injury. She hopes others—including non-singers—will see the benefit to keeping their vocal cords healthy.

“Your voice is part of your identity—whether you are a singer or not,” she said. “It’s integral to daily life, and it’s worth protecting.”

The study, “Closed-loop network of skin-interfaced wireless devices for quantifying vocal fatigue and providing user feedback,” was supported by the Querrey Simpson Institute for Bioelectronics at North-western University.

Source Link