A Step Toward Talking With Words and Hands Again for People With Paralysis

Losing your voice takes more than words. It takes the shrug, the open palm and the thumbs up that tell a listener whether you are joking, agreeing or simply glad they came. For people paralyzed by a brainstem stroke or by amyotrophic lateral sclerosis, both channels can go quiet at once, and brain-computer interfaces have mostly given back only one of them at a time, decoding either speech or hand movement.

A team at the University of California, San Francisco has now shown that a single implant resting on the surface of the brain can carry both together. Two people with paralysis attempted a phrase and a gesture at the same time, and neural network decoders turned their brain activity into speech and movement for a personalized full-body avatar in real time. The study was published in Nature Neuroscience on September 14, 2026. It is a laboratory proof of concept, not something a hospital can offer yet.

What the volunteers did

Three people took part, all in the BRAVO clinical trial (ClinicalTrials.gov NCT03698149), a phase I single-center early-feasibility study. All three had severe paralysis of the vocal tract. Two had brainstem strokes; one had ALS. Each had a high-density electrocorticography array, 253 electrodes at 3-millimeter spacing, placed on the surface of the left hemisphere over sensorimotor cortex, the strip of brain that plans movement.

In all three, the recordings could tell an attempted arm or hand movement apart from an attempted movement of the mouth and face. Then two of them took on the hard part: words and a communicative gesture together. That is genuinely tricky, because the paper notes that upper-limb and orofacial activity partially overlap in sensorimotor cortex. Two messages, one crowded neighborhood.

Across five blocks of a conversation task, one participant’s gestures were decoded with 85.0 percent average accuracy and speech with 75.0 percent, where chance was 9.1 percent for each decoder. The other reached a median of 100 percent for both across three blocks, where chance was 16.7 percent for speech and 20.0 percent for gestures. For one participant, the decoders also handled phrase and gesture pairings they had never seen in training about as well as familiar ones: 66.7 versus 65.8 percent for gestures and 80.0 versus 79.3 percent for speech, differences that were not statistically significant.

One lesson is lovely in its plainness. Decoders trained only on isolated speech or isolated gestures did not automatically handle the two attempted together. Training on both isolated and simultaneous attempts improved performance across contexts. If you want a system to understand someone talking with their hands, you have to show it someone talking with their hands.

Who this could reach, and how far away it is

The people this is for are easy to picture: someone after a brainstem stroke, someone living with ALS, who still has plenty to say and people around them who want to hear all of it, tone included. This work suggests a future device would not have to make them choose between a word and a gesture.

There is a practical reason that matters. Each implant carries surgical risk and covers limited cortical real estate, so getting two kinds of communication from one device is a real design gain. The authors write that their findings show one cortical implant can support “multi-effector” control, a step toward interfaces that allow more natural communication for people with paralysis.

The distance is real, too. The avatar is a laboratory system working from a fixed menu of phrases and gestures, and a Nature Research press release on the paper says testing in more participants and with larger sets of words and gestures will be needed before the technology can be used in practice. The BRAVO trial is listed on ClinicalTrials.gov as recruiting. The next chapter will be written the way this one was: by volunteers with paralysis lending their own brain activity so that people like them might one day talk with their words and their hands again.