For the first time, three people with spinal injury have controlled a robot and seen what it sees.
Helen Thomson

New Scientist
27 Oct 2016 - 12:01 PM  UPDATED 27 Oct 2016 - 12:02 PM

In the 2009 Bruce Willis movie Surrogates, people live their lives by embodying themselves as robots. They meet people, go to work, even fall in love, all without leaving the comfort of their own home. Now, for the first time, three people with severe spinal injuries have taken the first steps towards that vision by controlling a robot thousands of kilometres away, using thought alone.

The idea is that people with spinal injuries will be able to use robot bodies to interact with the world. It is part of the European Union-backed VERE project, which aims to dissolve the boundary between the human body and a surrogate, giving people the illusion that their surrogate is in fact their own body.

In 2012, an international team went some way to achieving this by taking fMRI scans of the brains of volunteers while they thought about moving their hands or legs. The scanner measured changes in blood flow to the brain area responsible for such thoughts. An algorithm then passed these on as instructions to a robot. 

Brain implant lets paralysed man move his hand with his thoughts
A breakthrough mind-reading device developed by US researchers could one day lead to paralysed patients regaining control of their limbs.

The volunteers could see what the robot was looking at via a head-mounted display. When they thought about moving their left or right hand, the robot moved 30 degrees to the left or right. Imagining moving their legs made the robot walk forward.

Now, a second team has tested a similar set-up in people who are paralysed from the neck or trunk down. To make the technology cheaper, more comfortable and more portable, the team swapped the fMRI scanner for an electroencephalogram (EEG) cap, which records electrical activity in the brain using electrodes attached to the scalp.

Each of the three volunteers in Italy donned the cap plus a head-mounted display that showed what a robot – in a lab in Tsukuba, Japan – was looking at. To move the robot, they had to concentrate on arrows superimposed across the display, each flashing at a different frequency. A computer could detect which arrow a participant was staring at using the EEG readings that each frequency provoked. It then sent the corresponding movement to the robot.

The set-up allowed the volunteers to control the robot in near real time. They were able to pick up a drink, move across the room and put the drink on a table. “It took just 6 minutes of training to start using the technology,’ says Emmanuele Tidoni at the University of Rome. “The feeling of actually embodying the robot was good, although needless to say, the sensation varied over time,” said Alessandro, one of the volunteers living with spinal cord injury. “When the robot was stationary the feeling of embodiment was low, but the moment I gave the first command or changed direction, there was this feeling of control and increased embodiment.”

The team also tried to boost this feeling using auditory feedback. While controlling the robot, both able-bodied volunteers and people with spinal cord injury managed to place a bottle closer to a target location when they heard footsteps as they walked, rather than a beep or no noise at all. The improved control suggests they feel more in tune with the robot itself, says Tidoni. 

The project also studied how this technology could be used in social interactions, by getting people in Italy to play a chess-like game via the robot against an opponent located in Munich, Germany. The results will be published later this year.

Alessandro is excited about the potential. “It’s a sensitive and important issue, but will certainly have a major impact on the way we all can communicate to each other,” he says.

But he hopes that the implications for mental health will also be looked at. “What will happen to a person who cannot move in real life after they use this technology intensively? Will they still feel isolated or lonely?” he asks. “Any developments also need to study the impact that these technologies may have on the psychological well-being of people with various degrees of disability.”

Although we’re not yet at Surrogates-level immersion, the technology could one day dramatically improve the lives of people with paralysis, says Noel Sharkey at the University of Sheffield, UK. “This is a very long way off, but getting towards that for people who otherwise can’t move would be astounding.” 

SBS Insight: Trusting Robots
How much should we hand over to robots?

Check out Real Humans, a ten-part Swedish drama series set in a parallel present day in which robots have become so human it is barely possible to distinguish them from real people.

This article was originally published in New Scientist© All Rights reserved. Distributed by Tribune Content Agency.