5 years ago, James Barnes was amputated right arm above the elbow after a severe electric injury. However, nevertheless some muscles have partially retained the functions of controlling the fingers. This was used by a team of researchers from the Georgia Institute of Technology, led by Gil Weinberg. They developed a prosthesis, through which James got the opportunity to play the piano.
Electromyographic sensors, found in both conventional and advanced prostheses, are used in the prosthesis. These sensors with the help of electrodes fix electrical signals from the muscles. However, according to Gil Weinberg, they are not very accurate.
“The sensors fix the movement of the muscles,” he explains, “but the signal is too” noisy “, which makes it impossible to pinpoint which of the fingers the person is going to move. We were trying to improve the EMG (Electro-Magnetic Generator) prosthesis manufactured specifically for James, but we were not able to achieve precise control. “
To improve the result, Weinberg joined forces with colleagues from Georgia Tech. They decided to attach an ultrasound probe to the muscles of the hand, which is usually used for ultrasound.
The work of the muscles observed in this way makes it possible to distinguish the movements pertaining to each finger. These templates were included in the machine learning algorithm, by which the system learned to guess which finger Barnes was going to move and even what effort he was going to spend on it.
By combining ultrasonic sensors and algorithms, Weinberg and his colleagues created a prosthesis, which makes it possible to control each finger separately. As a result, James Barnes was able to perform on the piano a song from the series “Star Wars”, thereby paying tribute to the bionic hand of Luke Skywalker.