Researchers develop new training model for limb prostheses
Researchers from Shenyang University of Technology and the University of Electro-Communications in Tokyo are trying to figure out how to make prosthetic hands respond to hand movements.
Over the past decade, scientists have tried to figure out how to use surface electromyography (EMG) signals to control prosthetic limbs. EMG signals are the electrical signals that cause our muscles to contract. They can be recorded by inserting electrode needles into a muscle. Surface EMGs are recorded using electrodes placed on the skin over the muscles.
Over the past decade, researchers have begun to explore the potential use of surface EMG signals to control prostheses for people with amputated limbs, especially regarding the complexity of movements and gestures required of prosthetic hands, in order to make devices smoother, more responsive and more intuitive than is currently possible.
Surface EMGs can indeed be used to make prosthetic limbs react faster and move more naturally. However, interruptions such as electrode displacement can make it difficult for the device to recognize these signals. One way to overcome this is to train surface EMG signals. Training can be a long and sometimes difficult process for people with amputated limbs.
That is why many researchers have turned to machine learning. Using machine learning, a prosthesis can learn to distinguish between muscle movements that signal gestures and movements of the electrodes.
The authors of the study, published in the journal Cyborg and Bionic Systems, developed a unique machine learning method that combines a convolutional neural network (CNN) and a long short-term memory (LSTM) artificial neural network. They settled on these two methods because of their complementary advantages.
"Deep learning for gesture recognition based on surface electromyography is playing an increasingly important role in human-computer interaction. To ensure high accuracy of deep learning in recognizing multi-state muscle movements and to guarantee that the learning model can be applied in an embedded chip with small memory, this paper presents a method for building and optimizing a functional model based on a multichannel sEMG amplification unit," the study says.
CNNs capture the spatial dimensions of surface EMG signals well and understand how they relate to hand gestures. They struggle with time. Gestures occur over time, but CNNs ignore timing information during continuous muscle contractions. CNNs are typically used for image recognition.
LSTM is typically used for handwriting and speech recognition. This neural network processes, classifies and makes predictions well based on data sequences over time. The problem here is that, although researchers have achieved better classification of sEMG gestures, the size of the required computational model is a serious challenge.
The microprocessor that must be used is limited. Using something more powerful would be too expensive. And finally, although such deep learning models work with computers in the lab, they are difficult to apply with the embedded hardware found in a prosthetic device.
“Ultimately, convolutional neural networks were designed for image recognition, not for controlling prostheses. We needed to combine CNNs with a technique that could handle the time dimension, while also providing a physical device that the user could wear”
The research team created a hybrid model that combines the spatial awareness of CNNs with the temporal awareness of LSTM. In the end, they reduced the size of the deep learning model while maintaining high accuracy and high resistance to interference.
The system was tested on ten subjects without amputated limbs with a series of 16 different hand gestures. It had a recognition accuracy of over 80%. The system performed well on most gestures, such as grasping a phone, a pen, pointing, pinching and gripping a cup of water. However, the hybrid approach failed to accurately recognize two pinch gestures: a pinch using the middle finger and one using the index finger.
Overall, according to the team, the results outperformed traditional learning methods. In future efforts, the researchers want to optimize the algorithm and further improve its accuracy while keeping the training model small so it can be used in prosthetic hardware. They also want to find out what causes difficulties in recognizing pinch gestures and expand their experiments to a much larger number of subjects.
Ultimately, the researchers want to develop a prosthetic hand that will be as flexible and reliable as the user's original limb.
