Artificial Intelligence gave a Paralyzed Woman her Voice back

Artificial Intelligence allowed a paralyzed woman who lost her ability to speak after a brain stem stroke to communicate through a digital avatar.

Pat Bennett’s advice is a little more complicated than “Take a couple of aspirins and call me in the morning.” However, a quartet of baby-aspirin-sized sensors implanted in her brain are targeted at correcting an issue that has irritated her and others: the inability to communicate clearly. The implants send signals from Bennett’s brain’s speech-related regions to cutting-edge software, which decodes her brain activity and translates it to text shown on a computer screen.

Bennett, 68, is a retired human resources director and former equestrian who jogged every day. She was diagnosed with amyotrophic lateral sclerosis in 2012, a progressive neurological illness that destroys the neurons that control movement, resulting in physical weakness and, eventually, paralysis.

“When you think of ALS, you think of arm and leg impact,” Bennett wrote in an interview conducted by email. “But in a group of ALS patients, it begins with speech difficulties. I am unable to speak.”

ALS typically emerges initially at the body’s periphery—arms and legs, hands and fingers. Bennett’s degeneration began in her brain stem rather than her spinal cord, as is customary. She can still move about, dress herself, and type, though with growing difficulty. However, she is unable to use the muscles of her lips, tongue, larynx, and jaws to clearly pronounce the phonemes—or sound units, such as “sh”—that are the building blocks of speech.

Bennett’s brain can still generate those phonemes, but her muscles are unable to carry out the commands.

Instead of teaching the artificial intelligence to recognize complete words, the researchers developed a method that decodes words from phonemes. These are the speech subunits that combine to produce spoken words in the same manner that letters combine to form written words. “Hello” comprises four phonemes: “HH,” “AH,” “L,” and “OW.”

Using this method supported by artificial intelligence, the computer only needed to learn 39 phonemes to understand every English word. This improved the system’s accuracy while also making it three times faster.

On March 29, 2022, a Stanford Medicine neurosurgeon implanted two tiny sensors in two separate regions of Bennett’s brain, both of which are involved in speech production. The sensors are part of an intracortical brain-computer interface, often known as iBCI. They’re supposed to translate the brain activity associated with efforts at speech into words on a screen when combined with cutting-edge decoding software.

A month following her operation, a group of Stanford researchers began twice-weekly study sessions to train the software that was deciphering her voice. Bennett’s attempted utterances were transformed into words on a computer screen at 62 words per minute after four months, more than three times faster than the previous record for BCI-assisted communication.

“These initial results have proven the concept, and eventually this artificial intelligence technology will catch up to make it easily accessible to people who cannot speak,” Bennett wrote. “For those who are nonverbal, this means they can stay connected to the bigger world, perhaps continue to work, maintain friends and family relationships.”

Approaching the Pace of Speech

Bennett’s pace begins to approach the about 160-word-per-minute rate of natural conversation among English speakers, according to Jaimie Henderson, MD, the surgeon who conducted the surgery.

“We’ve shown you can decode intended speech by recording activity from a very small area on the brain’s surface,” Henderson said.

Henderson, the John and Jean Blume-Robert and Ruth Halperin Professor of Neurosurgery, is a co-senior author of an article explaining the findings, which was published in Nature on August 23.

Krishna Shenoy, Ph.D., professor of electrical engineering and bioengineering, died before the study could be published.

The study’s principal author is Frank Willett, Ph.D., a Howard Hughes Medical Institute staff scientist affiliated with Henderson and Shenoy’s Neural Prosthetics Translational Lab, which Henderson and Shenoy co-founded in 2009.

Henderson, Shenoy, and Willett were co-authors of a study published in Nature in 2021 that described their success in converting a paralyzed person’s imagined handwriting into text on a screen using an iBCI at a speed of 90 characters, or 18 words, per minute—a world record for an iBCI-related methodology until now.

Bennett learns of Henderson and Shenoy’s work in 2021. She contacted Henderson and volunteered to take part in the clinical trial.

How does it work?

Henderson placed square arrays of microscopic silicon electrodes in Bennett’s cerebral cortex, the brain’s outermost layer. Each array has 64 electrodes that are organized in eight by eight grids and separated by about half the thickness of a credit card. The electrodes penetrate the cerebral cortex to about the depth of two stacked quarters.

The arrays are attached to fine gold wires that escape through pedestals placed into the skull and are then connected to a computer via cable.

An artificial-intelligence program receives and decodes electronic information from Bennett’s brain, eventually teaching itself to recognize the different brain activity associated with her attempts to articulate each of the 39 phonemes that comprise spoken English.

It inputs its best guess about the sequence of Bennett’s attempted phonemes into a language model, which is effectively a sophisticated autocorrect system that translates streams of phonemes into the sequence of words they represent.

“This system is trained to know what words should come before other ones, and which phonemes make what words,” Willett explained. “If some phonemes were wrongly interpreted, it can still take a good guess.”

Perfect practice makes perfect

Bennett engaged in about 25 training sessions, each lasting about four hours, to teach the algorithm to recognize which brain-activity patterns were associated with which phonemes. During these sessions, she attempted to repeat sentences chosen at random from a large data set consisting of samples of phone conversations.

An example: “It’s only been that way in the last five years.” Another: “I left right in the middle of it.”

Bennett’s brain activity, translated by the decoder into a phoneme stream and then constructed into words by the autocorrect system, would be displayed on the screen below the original as she tried to recite each sentence. The screen would then display a fresh sentence.

Bennett practiced between 260 and 480 sentences per training session. The system as a whole improved as it became acquainted with Bennett’s brain activity throughout her speech efforts.

The iCBI’s capacity to translate intended speech was tested using sentences other than those used in the training sessions. The translation system’s error rate was 9.1% when the sentences and the word-assembling language model were constrained to a 50-word vocabulary (in which case the sentences used were taken from a predefined list).

When the vocabulary was enlarged to 125,000 words (enough to say practically anything), the mistake rate increased to 23.8%—far from perfect, but a significant improvement over the previous state of the art.

“This is a scientific proof of concept, not an actual device people can use in everyday life,” Willett said. “But it’s a big advance toward restoring rapid communication to people with paralysis who can’t speak.”

“Imagine,” Bennett wrote, “how different conducting everyday activities like shopping, attending appointments, ordering food, going into a bank, talking on a phone, expressing love or appreciation—even arguing—will be when nonverbal people can communicate their thoughts in real time.”

The gadget reported in this article is solely licensed for research purposes and is not commercially available. The registered clinical trial was carried out under the auspices of BrainGate, a multi-institution consortium dedicated to advancing the use of BCIs in prosthetic applications, and was led by study co-author Leigh Hochberg, MD, Ph.D., a neurologist and researcher affiliated with Massachusetts General Hospital, Brown University, and the VA Providence (Rhode Island) Health care System.

Source Link