TEDxJaffa 2013

Guy Hoffman: Robots with "soul"

Filmed:

What kind of robots does an animator / jazz musician / roboticist make? Playful, reactive, curious ones. Guy Hoffman shows demo film of his family of unusual robots -- including two musical bots that like to jam with humans.

- Roboticist
Can robots and humans interact the way that human beings interact with each other? Guy Hoffman researches embodied cognition and intelligence in robots. Full bio

My job is to design, build and study
00:12
robots that communicate with people.
00:15
But this story doesn't start with robotics at all,
00:17
it starts with animation.
00:18
When I first saw Pixar's "Luxo Jr.,"
00:20
I was amazed by how much emotion
00:23
they could put into something
00:25
as trivial as a desk lamp.
00:27
I mean, look at them -- at the end of this movie,
00:29
you actually feel something for two pieces of furniture.
00:31
(Laughter)
00:33
And I said, I have to learn how to do this.
00:35
So I made a really bad career decision.
00:37
And that's what my mom was like when I did it.
00:41
(Laughter)
00:43
I left a very cozy tech job in Israel
00:45
at a nice software company and I moved to New York
00:48
to study animation.
00:50
And there I lived
00:51
in a collapsing apartment building
in Harlem with roommates.
00:52
I'm not using this phrase metaphorically,
00:55
the ceiling actually collapsed one day
00:57
in our living room.
00:59
Whenever they did those news stories
about building violations in New York,
01:00
they would put the report in front of our building.
01:03
As kind of like a backdrop
to show how bad things are.
01:05
Anyway, during the day I went to school and at night
01:08
I would sit and draw frame by frame
of pencil animation.
01:10
And I learned two surprising lessons --
01:14
one of them was that
01:16
when you want to arouse emotions,
01:19
it doesn't matter so much how something looks,
01:21
it's all in the motion -- it's in the timing
01:22
of how the thing moves.
01:25
And the second, was something
one of our teachers told us.
01:26
He actually did the weasel in Ice Age.
01:30
And he said:
01:32
"As an animator you are not
a director, you're an actor."
01:33
So, if you want to find the
right motion for a character,
01:37
don't think about it, go use your body to find it --
01:40
stand in front of a mirror, act it out
01:42
in front of a camera -- whatever you need.
01:44
And then put it back in your character.
01:45
A year later I found myself at MIT
01:48
in the robotic life group, it was one of the first groups
01:50
researching the relationships
between humans and robots.
01:52
And I still had this dream to make
01:55
an actual, physical Luxo Jr. lamp.
01:57
But I found that robots didn't move at all
02:00
in this engaging way that I was used to
02:01
for my animation studies.
02:03
Instead, they were all --
02:04
how should I put it, they were all kind of robotic.
02:06
(Laughter)
02:09
And I thought, what if I took whatever
I learned in animation school,
02:11
and used that to design my robotic desk lamp.
02:14
So I went and designed frame by frame
02:17
to try to make this robot
02:19
as graceful and engaging as possible.
02:21
And here when you see the robot interacting with me
02:24
on a desktop.
02:26
And I'm actually redesigning the robot so,
02:27
unbeknownst to itself,
02:30
it's kind of digging its own grave by helping me.
02:31
(Laughter)
02:34
I wanted it to be less of a mechanical structure
02:36
giving me light,
02:38
and more of a helpful, kind of quiet apprentice
02:39
that's always there when you need
it and doesn't really interfere.
02:42
And when, for example, I'm looking for a battery
02:46
that I can't find,
02:47
in a subtle way, it will show me where the battery is.
02:49
So you can see my confusion here.
02:53
I'm not an actor.
02:56
And I want you to notice how the same
03:00
mechanical structure can at one point,
03:02
just by the way it moves seem gentle and caring --
03:04
and in the other case, seem
violent and confrontational.
03:07
And it's the same structure,
just the motion is different.
03:10
Actor: "You want to know something?
Well, you want to know something?
03:19
He was already dead!
03:25
Just laying there, eyes glazed over!"
03:26
(Laughter)
03:30
But, moving in graceful ways is just one
building block of this whole structure
03:31
called human-robot interaction.
03:34
I was at the time doing my Ph.D.,
03:36
I was working on human robot teamwork;
03:38
teams of humans and robots working together.
03:40
I was studying the engineering,
03:42
the psychology, the philosophy of teamwork.
03:43
And at the same time I found myself
03:46
in my own kind of teamwork situation
03:47
with a good friend of mine who is actually here.
03:49
And in that situation we can easily imagine robots
03:51
in the near future being there with us.
03:54
It was after a Passover seder.
03:56
We were folding up a lot of folding chairs,
03:58
and I was amazed at how quickly
we found our own rhythm.
04:00
Everybody did their own part.
04:03
We didn't have to divide our tasks.
04:04
We didn't have to communicate verbally about this.
04:06
It all just happened.
04:08
And I thought,
04:10
humans and robots don't look at all like this.
04:10
When humans and robots interact,
04:12
it's much more like a chess game.
04:14
The human does a thing,
04:15
the robot analyzes whatever the human did,
04:16
then the robot decides what to do next,
04:18
plans it and does it.
04:20
And then the human waits, until it's their turn again.
04:21
So, it's much more like a chess game
04:23
and that makes sense because chess is great
04:24
for mathematicians and computer scientists.
04:27
It's all about information analysis,
04:28
decision making and planning.
04:30
But I wanted my robot to be less of a chess player,
04:33
and more like a doer
04:37
that just clicks and works together.
04:39
So I made my second horrible career choice:
04:41
I decided to study acting for a semester.
04:44
I took off from a Ph.D. I went to acting classes.
04:47
I actually participated in a play,
04:50
I hope theres no video of that around still.
04:52
And I got every book I could find about acting,
04:55
including one from the 19th century
04:58
that I got from the library.
05:00
And I was really amazed because my
name was the second name on the list --
05:01
the previous name was in 1889. (Laughter)
05:04
And this book was kind of waiting for 100 years
05:07
to be rediscovered for robotics.
05:09
And this book shows actors
05:12
how to move every muscle in the body
05:13
to match every kind of emotion
that they want to express.
05:16
But the real revelation was
05:18
when I learned about method acting.
05:20
It became very popular in the 20th century.
05:22
And method acting said, you don't have
to plan every muscle in your body.
05:24
Instead you have to use your body
to find the right movement.
05:27
You have to use your sense memory
05:29
to reconstruct the emotions and kind of
05:31
think with your body to find the right expression.
05:33
Improvise, play off yor scene partner.
05:36
And this came at the same time
as I was reading about this trend
05:38
in cognitive psychology called embodied cognition.
05:41
Which also talks about the same ideas --
05:45
We use our bodies to think,
05:46
we don't just think with our brains
and use our bodies to move.
05:48
but our bodies feed back into our brain
05:50
to generate the way that we behave.
05:52
And it was like a lightning bolt.
05:55
I went back to my office.
05:56
I wrote this paper -- which I never really published
05:57
called "Acting Lessons for Artificial Intelligence."
06:00
And I even took another month
06:03
to do what was then the first theater play
06:04
with a human and a robot acting together.
06:07
That's what you saw before with the actors.
06:08
And I thought:
06:12
How can we make an artificial intelligence model --
06:13
computer, computational model --
06:16
that will model some of these ideas of improvisation,
06:18
of taking risks, of taking chances,
06:20
even of making mistakes.
06:23
Maybe it can make for better robotic teammates.
06:24
So I worked for quite a long time on these models
06:27
and I implemented them on a number of robots.
06:30
Here you can see a very early example
06:32
with the robots trying to use this
embodied artificial intelligence,
06:34
to try to match my movements
as closely as possible,
06:38
sort of like a game.
06:40
Let's look at it.
06:42
You can see when I psych it out, it gets fooled.
06:47
And it's a little bit like what you might see actors do
06:51
when they try to mirror each other
06:54
to find the right synchrony between them.
06:55
And then, I did another experiment,
06:58
and I got people off the street
to use the robotic desk lamp,
07:00
and try out this idea of embodied
artificial intelligence.
07:04
So, I actually used two kinds
of brains for the same robot.
07:08
The robot is the same lamp that you saw,
07:12
and I put in it two brains.
07:14
For one half of the people,
07:15
I put in a brain that's kind of the traditional,
07:17
calculated robotic brain.
07:20
It waits for its turn, it analyzes everything, it plans.
07:21
Let's call it the calculated brain.
07:24
The other got more the stage actor, risk taker brain.
07:26
Let's call it the adventurous brain.
07:29
It sometimes acts without knowing
everything it has to know.
07:31
It sometimes makes mistakes and corrects them.
07:34
And I had them do this very tedious task
07:36
that took almost 20 minutes
07:39
and they had to work together.
07:41
Somehow simulating like a factory job
07:42
of repetitively doing the same thing.
07:44
And what I found was that people actually loved
07:47
the adventurous robot.
07:49
And they thought it was more intelligent,
07:50
more committed, a better member of the team,
07:52
contributed to the success of the team more.
07:54
They even called it 'he' and 'she,'
07:56
whereas people with the calculated brain called it 'it.'
07:58
And nobody ever called it 'he' or 'she'.
08:01
When they talked about it after the task
08:03
with the adventurous brain,
08:05
they said, "By the end, we were good
friends and high-fived mentally."
08:07
Whatever that means.
08:11
(Laughter) Sounds painful.
08:12
Whereas the people with the calculated brain
08:16
said it was just like a lazy apprentice.
08:18
It only did what it was supposed
to do and nothing more.
08:21
Which is almost what people expect robots to do,
08:23
so I was surprised that people
had higher expectations
08:26
of robots, than what anybody in robotics
thought robots should be doing.
08:28
And in a way, I thought, maybe it's time --
08:34
just like method acting changed the way
08:36
people thought about acting in the 19th century,
08:38
from going from the very calculated,
08:40
planned way of behaving,
08:42
to a more intuitive, risk-taking,
embodied way of behaving.
08:43
Maybe it's time for robots
08:47
to have the same kind of revolution.
08:48
A few years later,
08:52
I was at my next research job
at Georgia Tech in Atlanta,
08:53
and I was working in a group
08:55
dealing with robotic musicians.
08:56
And I thought, music, that's the perfect place
08:58
to look at teamwork, coordination,
09:00
timing, improvisation --
09:03
and we just got this robot playing marimba.
09:05
Marimba, for everybody who was like me,
09:07
it was this huge, wooden xylophone.
09:09
And, when I was looking at this,
09:12
I looked at other works in
human-robot improvisation --
09:15
yes, there are other works in
human-robot improvisation --
09:17
and they were also a little bit like a chess game.
09:20
The human would play,
09:22
the robot would analyze what was played,
09:23
would improvise their own part.
09:26
So, this is what musicians called
09:28
a call and response interaction,
09:30
and it also fits very well, robots
and artificial intelligence.
09:31
But I thought, if I use the same ideas I used
09:35
in the theater play and in the teamwork studies,
09:37
maybe I can make the robots jam together
09:40
like a band.
09:42
Everybody's riffing off each other,
nobody is stopping it for a moment.
09:44
And so, I tried to do the same
things, this time with music,
09:48
where the robot doesn't really know
09:51
what it's about to play.
09:52
It just sort of moves its body
09:53
and uses opportunities to play,
09:54
And does what my jazz teacher
when I was 17 taught me.
09:56
She said, when you improvise,
09:59
sometimes you don't know what you're doing
10:00
and you're still doing it.
10:02
And so I tried to make a robot that doesn't actually
10:03
know what it's doing, but it's still doing it.
10:05
So let's look at a few seconds
from this performance.
10:07
Where the robot listens to the human musician
10:09
and improvises.
10:12
And then, look at how the human musician also
10:14
responds to what the robot is doing, and picking up
10:17
from its behavior.
10:18
And at some point can even be surprised
by what the robot came up with.
10:21
(Music)
10:26
(Applause)
11:12
Being a musician is not just about making notes,
11:17
otherwise nobody would ever go see a live show.
11:19
Musicians also communicate with their bodies,
11:21
with other band members, with the audience,
11:23
they use their bodies to express the music.
11:25
And I thought, we already have
a robot musician on stage,
11:27
why not make it be a full-fledged musician.
11:30
And I started designing a socially expressive head
11:33
for the robot.
11:35
The head does't actually touch the marimba,
11:37
it just expresses what the music is like.
11:38
These are some napkin sketches
from a bar in Atlanta,
11:40
that was dangerously located exactly halfway
11:43
between my lab and my home. (Laughter)
11:46
So I spent, I would say on average,
11:47
three to four hours a day there.
11:49
I think. (Laughter)
11:52
And I went back to my animation
tools and tried to figure out
11:54
not just what a robotic musician would look like,
11:57
but especially what a robotic
musician would move like.
12:00
To sort of show that it doesn't like
what the other person is playing --
12:02
and maybe show whatever beat it's feeling
12:06
at the moment.
12:08
So we ended up actually getting the money
to build this robot, which was nice.
12:10
I'm going to show you now the
same kind of performance,
12:14
this time with a socially expressive head.
12:17
And notice one thing --
12:19
how the robot is really showing us
12:21
the beat it's picking up from the human.
12:23
We're also giving the human a sense
that the robot knows what it's doing.
12:24
And also how it changes the way it moves
12:28
as soon as it starts its own solo.
12:30
(Music)
12:32
Now it's looking at me to make sure I'm listening.
12:36
(Music)
12:39
And now look at the final chord of the piece again,
13:00
and this time the robot communicates with its body
13:04
when it's busy doing its own thing.
13:07
And when it's ready
13:09
to coordinate the final chord with me.
13:11
(Music)
13:14
(Applause)
13:27
Thanks. I hope you see how much this totally not --
13:33
how much this part of the body
that doesn't touch the instrument
13:37
actually helps with the musical performance.
13:40
And at some point, we are in Atlanta,
so obviously some rapper
13:43
will come into our lab at some point.
13:46
And we had this rapper come in
13:48
and do a little jam with the robot.
13:50
And here you can see the robot
13:53
basically responding to the beat and --
13:56
notice two things. One, how irresistible it is
13:57
to join the robot while it's moving its head.
14:00
and you kind of want to move
your own head when it does it.
14:02
And second, even though the rapper
is really focused on his iPhone,
14:04
as soon as the robot turns to him, he turns back.
14:08
So even though it's just in
the periphery of his vision --
14:11
it's just in the corner of his eye --
it's very powerful.
14:13
And the reason is that we can't ignore
14:15
physical things moving in our environment.
14:18
We are wired for that.
14:19
So, if you have a problem with maybe your partners
14:21
looking at the iPhone too much
or their smartphone too much,
14:24
you might want to have a robot there
14:27
to get their attention. (Laughter)
14:29
(Music)
14:30
(Applause)
14:50
Just to introduce the last robot
14:57
that we've worked on,
14:59
that came out of something kind
of surprising that we found:
15:01
At some point people didn't care anymore
about the robot being so intelligent,
15:03
and can improvise and listen,
15:06
and do all these embodied intelligence
things that I spent years on developing.
15:08
They really liked that the robot was
enjoying the music. (Laughter)
15:12
And they didn't say that the
robot was moving to the music,
15:16
they said that the robot was enjoying the music.
15:18
And we thought, why don't we take this idea,
15:20
and I designed a new piece of furniture.
15:22
This time it wasn't a desk
lamp; it was a speaker dock.
15:25
It was one of those things you
plug your smartphone in.
15:27
And I thought, what would happen
15:31
if your speaker dock didn't
just play the music for you,
15:32
but it would actually enjoy it too. (Laughter)
15:35
And so again, here are some animation tests
15:37
from an early stage. (Laughter)
15:39
And this is what the final product looked like.
15:44
("Drop It Like It's Hot")
15:59
So, a lot of bobbing head.
16:21
(Applause)
16:24
A lot of bobbing heads in the audience,
16:27
so we can still see robots influence people.
16:29
And it's not just fun and games.
16:32
I think one of the reasons I care so much
16:35
about robots that use their body to communicate
16:37
and use their body to move --
16:39
and I'm going to let you in on a little
secret we roboticists are hiding --
16:41
is that every one of you is
going to be living with a robot
16:44
at some point in their life.
16:47
Somewhere in your future there's
going to be a robot in your life.
16:49
And if not in yours, then in your children's lives.
16:51
And I want these robots to be --
16:53
to be more fluent, more engaging, more graceful
16:55
than currently they seem to be.
16:59
And for that I think that maybe robots
17:00
need to be less like chess players
17:02
and more like stage actors and more like musicians.
17:04
Maybe they should be able to
take chances and improvise.
17:06
And maybe they should be able to
anticipate what you're about to do.
17:09
And maybe they need to be able to make mistakes
17:12
and correct them,
17:14
because in the end we are human.
17:16
And maybe as humans, robots
that are a little less than perfect
17:18
are just perfect for us.
17:21
Thank you.
17:23
(Applause)
17:24

▲Back to top

About the Speaker:

Guy Hoffman - Roboticist
Can robots and humans interact the way that human beings interact with each other? Guy Hoffman researches embodied cognition and intelligence in robots.

Why you should listen

As co-director of the IDC Media Invention Lab, Guy Hoffman researches robots with soul. He explores the humanity of robots -- how they think, feel, act, and move, as they interact with humans. He and his team staged the world’s first human-robot theater piece, as well as the first human-robot jazz duet, improv and all.

Hoffman’s work has been named one of TIME Magazine’s Best Inventions of the Year, and in 2010 and 2012, he was listed as one of Israel’s most promising researchers under 40.