I’m at a Loss for Words

The most natural way for people to communicate with one another is through speech, so when that ability is lost, it creates significant barriers to communication and generally leads to a reduced quality of life. Neurological conditions like amyotrophic lateral sclerosis or brain lesions can result in a loss of the ability to speak, as can traumatic brain injuries. Loss of speech is particularly impactful for those with concomitant movement disorders that restrict or prevent one’s ability to use sign language, write, or use a keyboard. Advances have been made in this area in the form of brain-machine interfaces, typically consisting of a brain implant that measures brain activity, and some type of system that interprets that brain activity as speech.

Existing systems mostly require that the user attempt to speak, or mime speech, to achieve a sufficiently high signal-to-noise ratio for their downstream decoding pipelines. But when it comes to internal speech — or just the thought of what one intends to say — little progress has been made towards the development of an accurate system. Recent work done by a team at the California Institute of Technology and the University of Southern California has moved the state of the art for decoding internal speech significantly forward. Their technique can classify internal speech with an accuracy of up to 91%.

Right about now, you may be asking yourself if this is actually a good idea — a machine that can read your inner thoughts? What could possibly go wrong, aside from everything? Not so fast, say the researchers. They point out that their system needs to be trained on each user individually, and that it can only recognize words when a person focuses on them, much to the dismay of totalitarian regimes everywhere. Personally, I’ll dust off my tin foil hat, but am not going to put it on just yet. Technologies do have a tendency to improve over time, after all, so it is worthwhile to consider the implications of where this could go in the future.

As it is currently implemented, the team’s approach relies on microelectrode arrays implanted in the supramarginal gyrus and primary somatosensory cortex within the brain. The signals were fed into a support vector machine classifier model that was trained to recognize six words (spoon, python, battlefield, cowboy, swimming, telephone) and two pseudowords (Bindip, Nifzig). By using machine learning, the researchers did not need to understand how to translate brain activity into internal speech, but rather were able to let the model gain a generalized understanding by showing it labeled example data.

A very small validation study was performed with a single tetraplegic individual. After collecting data from this individual and training the model, the participant was shown a word on a computer screen and asked to internally speak that word. The highest classification accuracy achieved was 91%, which required the participant to repeat their internal speech sixteen times for each word. With eight repetitions, the top accuracy falls to 72% on the eight-word vocabulary.

Exactly where this will go in the future is anyone’s guess, but at this point in time, the system is far from a mind reader. Aside from requiring a brain implant to operate, it is also necessary for a user of the system to concentrate on a word, and repeat that word up to 16 times. Moreover, the tiny eight-word vocabulary tested does not give a good representation of how this device might operate when tasked with decoding every word in a language. It may prove to be too difficult to distinguish accurately between thousands of words. For now, we will have to sit back and wait for further studies to refine the technology and answer the open questions.