Researchers develop AI system to control robots with thoughts
Two research groups at EPFL (the Swiss Federal Institute of Technology in Lausanne) joined forces to develop a machine learning program that can be connected to the human brain and used to control a robot. The program adjusts the robot's movements based on the brain's electrical signals. There is hope that with this invention, patients with tetraplegia will be able to perform more everyday tasks on their own.
People with tetraplegia are prisoners of their own bodies, unable to speak or make even the slightest movements. For years, researchers have worked on developing systems that can help these patients perform some tasks on their own.
“People with spinal cord injuries often suffer from permanent neurological deficits and severe motor impairments that prevent them from performing even the simplest tasks, such as grasping an object. Robotic assistance can help these people partially regain lost dexterity, since the robot can perform tasks in their place”
Professor Billard conducted the study with Professor José Millán, who at the time headed the Brain-Machine Interface Laboratory at EPFL but has since moved to the University of Texas.
The two research groups developed a computer program that can control a robot using the electrical signals emitted by the patient's brain. No voice control or sensory function is required; patients can move the robot simply with their thoughts. The study was published in Communications Biology from Nature Portfolio.
To build their system, the researchers started with a robotic arm developed several years ago. The arm can move forward and backward, left and right, move objects in front of it, and avoid objects in its path.
“In our study, we programmed the robot to avoid obstacles, but we could choose any other type of task, such as filling a glass with water, pushing or pulling an object”
The engineers began by improving the robot's obstacle-avoidance mechanism to make it more precise. Billard's laboratory PhD student Caroline Gaspar said: "At first, the robot chose a path that was too wide for some obstacles, taking it too far away, and not wide enough for others, keeping it too close. Since the goal of our robot was to help paralyzed patients, we had to find a way for users to communicate with it without using voice or movements."
This involved developing an algorithm that could adjust the robot's movements based only on the patient's thoughts. The algorithm was connected to a headset fitted with electrodes to perform an electroencephalogram (EEG) scan of the patient's brain activity.
To use the system, the patient simply needs to look at the robot. If the robot makes a wrong move, the patient's brain sends an "error message" through a clearly identifiable signal, as if the patient had said: "No, not that way." The robot then understands that what it is doing is wrong, but at first it will not know exactly why.
For example, did it get too close to the object or too far from it? To help the robot find the right answer, the error message is fed into an algorithm that uses inverse reinforcement learning to determine what the patient wants and what actions the robot should take.
This is done by trial and error, with the robot trying different movements to see which one is correct. The process is fairly fast: the robot usually needs just three to five attempts to find the right answer and fulfill the patient's wishes.
“The robot's AI program can learn quickly, but you have to tell it when it makes a mistake so it can correct its behavior. Developing technology to detect error signals was one of the biggest technical challenges we faced”
Lead author of the study Iason Batzianoulis noted: "In our study, it was especially difficult to link the patient's brain activity to the robot's control system, or in other words, to 'translate' the patient's brain signals into actions performed by the robot. We did this by using machine learning to connect a given brain signal with a specific task. We then linked the tasks to individual robot controls so that the robot does what the patient intends."
The researchers hope to eventually use their algorithm to control wheelchairs.
“Many engineering obstacles still need to be overcome. Wheelchairs present a completely new set of problems, since both the patient and the robot are in motion”
The research team also plans to use its algorithm with a robot that can read several different types of signals and coordinate data from the brain with data from visuomotor functions.
