A virtual drone was piloted through an obstacle course by a person imagining moving their fingers
Willsey et al.
A man with paralysis who had electrodes implanted in his brain can pilot a virtual drone through an obstacle course simply by imagining moving his fingers. His brain signals are interpreted by an AI model and then used to control a simulated drone.
Brain-computer interface (BCI) research has made huge strides in recent years, allowing people with paralysis to precisely control a mouse cursor and dictate speech to computers by imagining writing words with a pen. But so far, they haven’t yet shown great promise in complex applications with multiple inputs.
Now, Matthew Willsey at the University of Michigan and his colleagues have created an algorithm that allows a user to trigger four discrete signals by imagining moving their fingers and thumb.
The anonymous man who tried the technology has tetraplegia due to a spinal cord injury. He had already been fitted with a BCI from Blackrock Neurotech made up of 192 electrodes, implanted in the area of the brain that controls hand motion.
An AI model was used to map the complex neural signals received by the electrodes to the user’s thoughts. The participant learned how to think of the first two fingers of one hand moving, creating an electrical signal that can be made stronger or weaker. Another signal was generated by the second two fingers and another two by the thumb.
These were sufficient to allow the user to control a virtual drone by thought alone, and with practice he could skilfully pilot it through an obstacle course. Willsey says the experiment could have been done using a real drone, but was kept virtual for ease and safety.
“The goal of doing the quadcopter was really kind of shared between our lab and the participant,” says Willsey. “For him, it was the realisation of kind of a dream that he thought was lost once he suffered his injury. He had a passion and a dream for flying. He seemed very empowered and enabled; he would have us take videos and send it to friends.”
Although the results are impressive, there is still much to be done before BCIs can be reliably used for complex tasks, says Willsey. Firstly, AI is needed to interpret signals from the electrodes, and this relies on individual training for every user. Secondly, this training needs to be repeated over time as functionality declines, which may be due to electrodes shifting slightly in the brain or changes in the brain itself.
Topics:
