Hybrid CNN-LSTM Machine Learning Model Proves Ideal for sEMG Control of Prosthetic Limbs
Combining the best of two network types, this hybrid model can run on embedded hardware and achieve high accuracy recognition.
Researchers at the Shenyang University of Technology and the University of Electro-Communications Tokyo have published a paper detailing a hybrid machine learning approach that, they say, can improve muscle gesture recognition for prosthetic limb control.
"Convolutional neural networks were after all conceived with image recognition in mind, not control of prostheses," explains Dianchun Bai, professor of electrical engineering, of the need for a hybrid approach. "We needed to couple CNN with a technique that could deal with the dimension of time, while also ensuring feasibility in the physical device that the user must wear."
The solution: Combining a convolutional neural network model with a long short-term memory (LSTM) model, bringing the benefits of the latter's ability to process fitful sequences of data over time with the latter's ability to generate small models capable of running at the edge on embedded hardware with limited resources using surface electromyogram (sEMG) sensors as input.
"The experimental results show that under the same network structure, the sEMG signal with fast Fourier transform and root mean square as feature data processing has a good recognition rate, and the recognition accuracy of complex gestures is 91.40%, with the size of 1 MB," the researchers note. "The model can still control the artificial hand accurately when the model is small and the precision is high."
In testing, the team's hybrid approach was able to allow 10 non-amputee subjects to control prosthetics across 16 different gestures — including holding a phone, a pen, and a glass of water, pointing, and pinching — with results, which beat out traditional single-method machine learning approaches.
In future work, the team has said it will be investigating difficulties the model encountered in recognizing pinching gestures and also expanding their experiments to a much larger subject group — while improving the algorithm to boost accuracy while reducing model size.
The team's work has been published under open-access terms in the journal Cyborg and Bionic Systems.