A UCSF team showed that a single high-density electrocorticography implant can support simultaneous decoding of speech and gestures in people with paralysis, letting participants control a personalized virtual avatar by attempting speech and gestures together or separately.
Researchers had suspected the simultaneous signal would just be the two patterns added together, but the data from simultaneous expressions looked very different from the signals for speech or gestures alone. Scale matters too: participants practiced up to 10 gestures and 10 words or phrases. Eye-tracking, the current standard of care, is slow, limited in expression and physically exhausting, so even a small vocabulary points at something better.