sponsored links
TEDGlobal 2014

Miguel Nicolelis: Brain-to-brain communication has arrived. How we did it

October 15, 2014

You may remember neuroscientist Miguel Nicolelis — he built the brain-controlled exoskeleton that allowed a paralyzed man to kick the first ball of the 2014 World Cup. What’s he working on now? Building ways for two minds (rats and monkeys, for now) to send messages brain to brain. Watch to the end for an experiment that, as he says, will go to "the limit of your imagination."

Miguel Nicolelis - Neuroscientist
Miguel Nicolelis explores the limits of the brain-machine interface. Full bio

sponsored links
Double-click the English subtitles below to play the video.
On June 12, 2014, precisely at 3:33
00:12
in a balmy winter afternoon
in São Paulo, Brazil,
00:18
a typical South American winter afternoon,
00:23
this kid, this young man
that you see celebrating here
00:26
like he had scored a goal,
00:29
Juliano Pinto, 29 years old,
accomplished a magnificent deed.
00:31
Despite being paralyzed
00:38
and not having any sensation
from mid-chest to the tip of his toes
00:40
as the result of a car crash six years ago
that killed his brother
00:45
and produced a complete spinal cord lesion
that left Juliano in a wheelchair,
00:49
Juliano rose to the occasion,
and on this day did something
00:57
that pretty much everybody that saw him
in the six years deemed impossible.
01:02
Juliano Pinto delivered the opening kick
01:09
of the 2014 Brazilian
World Soccer Cup here
01:14
just by thinking.
01:20
He could not move his body,
01:22
but he could imagine the movements
needed to kick a ball.
01:24
He was an athlete before the lesion.
He's a para-athlete right now.
01:29
He's going to be in the Paralympic Games,
I hope, in a couple years.
01:32
But what the spinal cord lesion
did not rob from Juliano
01:36
was his ability to dream.
01:40
And dream he did that afternoon,
for a stadium of about 75,000 people
01:43
and an audience of close to a billion
watching on TV.
01:49
And that kick crowned, basically,
30 years of basic research
01:54
studying how the brain,
02:00
how this amazing universe
that we have between our ears
02:02
that is only comparable to universe
that we have above our head
02:06
because it has about 100 billion elements
02:10
talking to each other
through electrical brainstorms,
02:13
what Juliano accomplished
took 30 years to imagine in laboratories
02:16
and about 15 years to plan.
02:21
When John Chapin and I,
15 years ago, proposed in a paper
02:24
that we would build something
that we called a brain-machine interface,
02:28
meaning connecting a brain to devices
02:33
so that animals and humans
could just move these devices,
02:36
no matter how far they are
from their own bodies,
02:40
just by imagining what they want to do,
02:42
our colleagues told us that
we actually needed professional help,
02:44
of the psychiatry variety.
02:49
And despite that,
a Scot and a Brazilian persevered,
02:52
because that's how we were raised
in our respective countries,
02:57
and for 12, 15 years,
03:02
we made demonstration after demonstration
suggesting that this was possible.
03:04
And a brain-machine interface
is not rocket science,
03:08
it's just brain research.
03:11
It's nothing but using sensors
03:13
to read the electrical brainstorms
that a brain is producing
03:16
to generate the motor commands
03:19
that have to be downloaded
to the spinal cord,
03:21
so we projected sensors that can read
03:24
hundreds and now thousands
of these brain cells simultaneously,
03:26
and extract from these electrical signals
03:30
the motor planning
that the brain is generating
03:33
to actually make us move into space.
03:36
And by doing that, we converted
these signals into digital commands
03:39
that any mechanical, electronic,
or even a virtual device can understand
03:42
so that the subject can imagine
what he, she or it wants to make move,
03:47
and the device obeys that brain command.
03:54
By sensorizing these devices
with lots of different types of sensors,
03:57
as you are going to see in a moment,
04:01
we actually sent messages
back to the brain to confirm
04:03
that that voluntary motor will
was being enacted, no matter where --
04:07
next to the subject, next door,
or across the planet.
04:11
And as this message gave feedback
back to the brain,
04:16
the brain realized its goal:
to make us move.
04:19
So this is just one experiment
that we published a few years ago,
04:23
where a monkey, without moving its body,
04:27
learned to control the movements
of an avatar arm,
04:29
a virtual arm that doesn't exist.
04:33
What you're listening to
is the sound of the brain of this monkey
04:35
as it explores three different
visually identical spheres
04:39
in virtual space.
04:43
And to get a reward,
a drop of orange juice that monkeys love,
04:45
this animal has to detect,
select one of these objects
04:49
by touching,
04:52
not by seeing it, by touching it,
04:54
because every time this virtual hand
touches one of the objects,
04:56
an electrical pulse goes back
to the brain of the animal
04:59
describing the fine texture
of the surface of this object,
05:02
so the animal can judge what is
the correct object that he has to grab,
05:06
and if he does that, he gets a reward
without moving a muscle.
05:10
The perfect Brazilian lunch:
05:16
not moving a muscle
and getting your orange juice.
05:18
So as we saw this happening,
05:22
we actually came and proposed the idea
that we had published 15 years ago.
05:25
We reenacted this paper.
05:29
We got it out of the drawers,
05:31
and we proposed that perhaps we could get
a human being that is paralyzed
05:33
to actually use the brain-machine
interface to regain mobility.
05:37
The idea was that if you suffered --
05:42
and that can happen to any one of us.
05:44
Let me tell you, it's very sudden.
05:46
It's a millisecond of a collision,
05:48
a car accident that
transforms your life completely.
05:50
If you have a complete lesion
of the spinal cord,
05:53
you cannot move because your brainstorms
cannot reach your muscles.
05:56
However, your brainstorms
continue to be generated in your head.
06:00
Paraplegic, quadriplegic patients
dream about moving every night.
06:04
They have that inside their head.
06:08
The problem is how
to get that code out of it
06:10
and make the movement be created again.
06:13
So what we proposed was,
let's create a new body.
06:16
Let's create a robotic vest.
06:19
And that's exactly why Juliano could
kick that ball just by thinking,
06:21
because he was wearing
the first brain-controlled robotic vest
06:26
that can be used by paraplegic,
quadriplegic patients to move
06:30
and to regain feedback.
06:34
That was the original idea, 15 years ago.
06:35
What I'm going to show you is how
156 people from 25 countries
06:38
all over the five continents
of this beautiful Earth,
06:44
dropped their lives,
dropped their patents,
06:47
dropped their dogs, wives,
kids, school, jobs,
06:50
and congregated to come to Brazil
for 18 months to actually get this done.
06:54
Because a couple years after Brazil
was awarded the World Cup,
06:59
we heard that the Brazilian government
wanted to do something meaningful
07:03
in the opening ceremony
07:07
in the country that reinvented
and perfected soccer
07:08
until we met the Germans, of course.
07:12
(Laughter)
07:14
But that's a different talk,
07:16
and a different neuroscientist
needs to talk about that.
07:18
But what Brazil wanted to do
is to showcase
07:22
a completely different country,
07:24
a country that values science
and technology,
07:25
and can give a gift to millions,
25 million people around the world
07:28
that cannot move any longer
because of a spinal cord injury.
07:32
Well, we went to the Brazilian government
and to FIFA and proposed,
07:35
well, let's have the kickoff
of the 2014 World Cup
07:39
be given by a Brazilian paraplegic
07:42
using a brain-controlled exoskeleton
that allows him to kick the ball
07:44
and to feel the contact of the ball.
07:49
They looked at us,
thought that we were completely nuts,
07:52
and said, "Okay, let's try."
07:54
We had 18 months to do everything
from zero, from scratch.
07:57
We had no exoskeleton, we had no patients,
08:01
we had nothing done.
08:04
These people came all together
08:06
and in 18 months, we got eight patients
in a routine of training
08:08
and basically built from nothing this guy,
08:12
that we call Bra-Santos Dumont 1.
08:16
The first brain-controlled
exoskeleton to be built
08:21
was named after the most famous
Brazilian scientist ever,
08:24
Alberto Santos Dumont,
08:28
who, on October 19, 1901,
created and flew himself
08:30
the first controlled airship on air
in Paris for a million people to see.
08:36
Sorry, my American friends,
08:43
I live in North Carolina,
08:45
but it was two years
before the Wright Brothers flew
08:47
on the coast of North Carolina.
08:52
(Applause)
08:54
Flight control is Brazilian.
(Laughter)
08:57
So we went together with these guys
09:02
and we basically put
this exoskeleton together,
09:05
15 degrees of freedom,
hydraulic machine
09:08
that can be commanded by brain signals
09:11
recorded by a non-invasive technology
called electroencephalography
09:14
that can basically allow the patient
to imagine the movements
09:18
and send his commands
to the controls, the motors,
09:22
and get it done.
09:26
This exoskeleton was covered
with an artificial skin
09:27
invented by Gordon Cheng,
one of my greatest friends, in Munich,
09:30
to allow sensation from the joints moving
and the foot touching the ground
09:34
to be delivered back to the patient
through a vest, a shirt.
09:39
It is a smart shirt
with micro-vibrating elements
09:43
that basically delivers the feedback
and fools the patient's brain
09:46
by creating a sensation that it is not
a machine that is carrying him,
09:51
but it is he who is walking again.
09:55
So we got this going,
and what you'll see here
09:58
is the first time one of our patients,
Bruno, actually walked.
10:01
And he takes a few seconds
because we are setting everything,
10:06
and you are going to see a blue light
cutting in front of the helmet
10:08
because Bruno is going to imagine
the movement that needs to be performed,
10:12
the computer is going to analyze it,
Bruno is going to certify it,
10:16
and when it is certified,
10:19
the device starts moving
under the command of Bruno's brain.
10:21
And he just got it right,
and now he starts walking.
10:24
After nine years without
being able to move,
10:29
he is walking by himself.
10:32
And more than that --
10:34
(Applause) --
10:36
more than just walking,
10:39
he is feeling the ground,
10:41
and if the speed of the exo goes up,
10:43
he tells us that he is walking again
on the sand of Santos,
10:45
the beach resort where he used to go
before he had the accident.
10:50
That's why the brain is creating
a new sensation in Bruno's head.
10:54
So he walks, and at the end of the walk --
I am running out of time already --
10:57
he says, "You know, guys,
11:01
I need to borrow this thing from you
when I get married,
11:04
because I wanted to walk to the priest
11:06
and see my bride and actually
be there by myself.
11:09
Of course, he will have it
whenever he wants.
11:13
And this is what we wanted to show
during the World Cup, and couldn't,
11:16
because for some mysterious reason,
FIFA cut its broadcast in half.
11:20
What you are going to see very quickly
is Juliano Pinto in the exo doing the kick
11:25
a few minutes before we went to the pitch
11:33
and did the real thing
in front of the entire crowd,
11:35
and the lights you are going to see
just describe the operation.
11:38
Basically, the blue lights pulsating
indicate that the exo is ready to go.
11:41
It can receive thoughts
and it can deliver feedback,
11:46
and when Juliano
makes the decision to kick the ball,
11:49
you are going to see
two streams of green and yellow light
11:52
coming from the helmet
and going to the legs,
11:55
representing the mental commands
that were taken by the exo
11:58
to actually make that happen.
12:02
And in basically 13 seconds,
12:04
Juliano actually did.
12:07
You can see the commands.
12:09
He gets ready,
the ball is set, and he kicks.
12:11
And the most amazing thing is,
12:15
10 seconds after he did that,
and looked at us on the pitch,
12:17
he told us, celebrating as you saw,
12:20
"I felt the ball."
12:22
And that's priceless.
12:25
(Applause)
12:27
So where is this going to go?
12:29
I have two minutes to tell you
12:31
that it's going to the limits
of your imagination.
12:33
Brain-actuating technology is here.
12:36
This is the latest: We just
published this a year ago,
12:38
the first brain-to-brain interface
12:41
that allows two animals
to exchange mental messages
12:43
so that one animal that sees something
coming from the environment
12:46
can send a mental SMS,
a torpedo, a neurophysiological torpedo,
12:50
to the second animal,
12:56
and the second animal performs
the act that he needed to perform
12:57
without ever knowing what
the environment was sending as a message,
13:01
because the message came
from the first animal's brain.
13:05
So this is the first demo.
13:09
I'm going to be very quick
because I want to show you the latest.
13:11
But what you see here
is the first rat getting informed
13:15
by a light that is going to show up
on the left of the cage
13:21
that he has to press the left cage
to basically get a reward.
13:24
He goes there and does it.
13:27
And the same time,
he is sending a mental message
13:29
to the second rat
that didn't see any light,
13:31
and the second rat,
in 70 percent of the times
13:35
is going to press the left lever
and get a reward
13:37
without ever experiencing
the light in the retina.
13:41
Well, we took this
to a little higher limit
13:45
by getting monkeys to collaborate
mentally in a brain net,
13:49
basically to donate their brain activity
13:54
and combine them to move
the virtual arm that I showed you before,
13:56
and what you see here is the first time
the two monkeys combine their brains,
14:00
synchronize their brains perfectly
to get this virtual arm to move.
14:05
One monkey is controlling the x dimension,
14:09
the other monkey
is controlling the y dimension.
14:12
But it gets a little more interesting
when you get three monkeys in there
14:14
and you ask one monkey to control x and y,
14:19
the other monkey to control y and z,
14:22
and the third one to control x and z,
14:25
and you make them all
play the game together,
14:28
moving the arm in 3D into a target
to get the famous Brazilian orange juice.
14:30
And they actually do.
14:36
The black dot is the average
of all these brains working
14:38
in parallel, in real time.
14:43
That is the definition
of a biological computer,
14:46
interacting by brain activity
and achieving a motor goal.
14:49
Where is this going?
14:53
We have no idea.
14:55
We're just scientists.
14:58
(Laughter)
14:59
We are paid to be children,
15:01
to basically go to the edge
and discover what is out there.
15:03
But one thing I know:
15:07
One day, in a few decades,
15:09
when our grandchildren
surf the Net just by thinking,
15:11
or a mother donates her eyesight
to an autistic kid who cannot see,
15:15
or somebody speaks because
of a brain-to-brain bypass,
15:19
some of you will remember
that it all started on a winter afternoon
15:22
in a Brazilian soccer field
with an impossible kick.
15:28
Thank you.
15:32
(Applause)
15:34
Thank you.
15:43
Bruno Giussani: Miguel,
thank you for sticking to your time.
15:59
I actually would have given you
a couple more minutes,
16:03
because there are a couple of points
we want to develop, and, of course,
16:05
clearly it seems that we need connected
brains to figure out where this is going.
16:08
So let's connect all this together.
16:12
So if I'm understanding correctly,
16:14
one of the monkeys
is actually getting a signal
16:16
and the other monkey
is reacting to that signal
16:18
just because the first one is receiving it
and transmitting the neurological impulse.
16:21
Miguel Nicolelis:
No, it's a little different.
16:25
No monkey knows of the existence
of the other two monkeys.
16:27
They are getting a visual feedback in 2D,
16:30
but the task they have
to accomplish is 3D.
16:33
They have to move an arm
in three dimensions.
16:36
But each monkey is only getting
the two dimensions on the video screen
16:38
that the monkey controls.
16:42
And to get that thing done,
16:44
you need at least two monkeys
to synchronize their brains,
16:47
but the ideal is three.
16:49
So what we found out is that
when one monkey starts slacking down,
16:51
the other two monkeys
enhance their performance
16:55
to get the guy to come back,
16:57
so this adjusts dynamically,
16:59
but the global synchrony remains the same.
17:02
Now, if you flip
without telling the monkey
17:06
the dimensions that each brain
has to control,
17:09
like this guy is controlling x and y,
17:11
but he should be controlling now y and z,
17:13
instantaneously, that animal's brain
forgets about the old dimensions
17:16
and it starts concentrating
on the new dimensions.
17:20
So what I need to say is
that no Turing machine,
17:23
no computer can predict
what a brain net will do.
17:26
So we will absorb technology
as part of us.
17:30
Technology will never absorb us.
17:33
It's simply impossible.
17:35
BG: How many times have you tested this?
17:38
And how many times
have you succeeded versus failed?
17:41
MN: Oh, tens of times.
17:44
With the three monkeys?
Oh, several times.
17:46
I wouldn't be able to talk about this here
unless I had done it a few times.
17:48
And I forgot to mention, because of time,
17:53
that just three weeks ago,
a European group
17:55
just demonstrated the first
man-to-man brain-to-brain connection.
17:59
BG: And how does that play?
18:04
MN: There was one bit of information --
big ideas start in a humble way --
18:06
but basically the brain activity
of one subject
18:10
was transmitted to a second object,
all non-invasive technology.
18:17
So the first subject got a message,
like our rats, a visual message,
18:21
and transmitted it to the second subject.
18:25
The second subject received
a magnetic pulse in the visual cortex,
18:28
or a different pulse,
two different pulses.
18:32
In one pulse, the subject saw something.
18:36
On the other pulse,
he saw something different.
18:38
And he was able to verbally indicate
18:40
what was the message
the first subject was sending
18:42
through the Internet across continents.
18:45
Moderator: Wow.
Okay, that's where we are going.
18:48
That's the next TED Talk
at the next conference.
18:51
Miguel Nicolelis, thank you.
MN: Thank you, Bruno. Thank you.
18:53

sponsored links

Miguel Nicolelis - Neuroscientist
Miguel Nicolelis explores the limits of the brain-machine interface.

Why you should listen

At the Nicolelis Laboratory at Duke University, Miguel Nicolelis is best known for pioneering studies in neuronal population coding, Brain Machine Interfaces (BMI) and neuroprosthetics in human patients and non-human primates.His lab's work was seen, famously though a bit too briefly, when a brain-controlled exoskeleton from his lab helped Juliano Pinto, a paraplegic man, kick the first ball at the 2014 World Cup.

But his lab is thinking even bigger. They've developed an integrative approach to studying neurological disorders, including Parkinsons disease and epilepsy. The approach, they hope, will allow the integration of molecular, cellular, systems and behavioral data in the same animal, producing a more complete understanding of the nature of the neurophysiological alterations associated with these disorders. He's the author of the books Beyond Boundaries and The Relativistic Brain.

Miguel was honored as one of Foreign Policy's 2015 Global Thinkers.

sponsored links

If you need translations, you can install "Google Translate" extension into your Chrome Browser.
Furthermore, you can change playback rate by installing "Video Speed Controller" extension.

Data provided by TED.

This website is owned and operated by Tokyo English Network.
The developer's blog is here.