16:47
TED2012

Vijay Kumar: Robots that fly ... and cooperate

Filmed:

In his lab at Penn, Vijay Kumar and his team build flying quadrotors, small, agile robots that swarm, sense each other, and form ad hoc teams -- for construction, surveying disasters and far more.

- Roboticist
As the dean of the University of Pennsylvania's School of Engineering and Applied Science, Vijay Kumar studies the control and coordination of multi-robot formations. Full bio

Good morning.
00:20
I'm here today to talk
00:22
about autonomous, flying beach balls.
00:24
No, agile aerial robots like this one.
00:27
I'd like to tell you a little bit about the challenges in building these
00:31
and some of the terrific opportunities
00:34
for applying this technology.
00:36
So these robots
00:38
are related to unmanned aerial vehicles.
00:40
However, the vehicles you see here are big.
00:43
They weigh thousands of pounds,
00:46
are not by any means agile.
00:48
They're not even autonomous.
00:50
In fact, many of these vehicles
00:52
are operated by flight crews
00:54
that can include multiple pilots,
00:56
operators of sensors
00:59
and mission coordinators.
01:01
What we're interested in is developing robots like this --
01:03
and here are two other pictures --
01:05
of robots that you can buy off the shelf.
01:07
So these are helicopters with four rotors
01:10
and they're roughly a meter or so in scale
01:13
and weigh several pounds.
01:17
And so we retrofit these with sensors and processors,
01:19
and these robots can fly indoors
01:22
without GPS.
01:24
The robot I'm holding in my hand
01:26
is this one,
01:28
and it's been created by two students,
01:30
Alex and Daniel.
01:33
So this weighs a little more
01:35
than a tenth of a pound.
01:37
It consumes about 15 watts of power.
01:39
And as you can see,
01:41
it's about eight inches in diameter.
01:43
So let me give you just a very quick tutorial
01:45
on how these robots work.
01:48
So it has four rotors.
01:50
If you spin these rotors at the same speed,
01:52
the robot hovers.
01:54
If you increase the speed of each of these rotors,
01:56
then the robot flies up, it accelerates up.
01:59
Of course, if the robot were tilted,
02:02
inclined to the horizontal,
02:04
then it would accelerate in this direction.
02:06
So to get it to tilt, there's one of two ways of doing it.
02:09
So in this picture
02:12
you see that rotor four is spinning faster
02:14
and rotor two is spinning slower.
02:16
And when that happens
02:18
there's moment that causes this robot to roll.
02:20
And the other way around,
02:23
if you increase the speed of rotor three
02:25
and decrease the speed of rotor one,
02:28
then the robot pitches forward.
02:30
And then finally,
02:33
if you spin opposite pairs of rotors
02:35
faster than the other pair,
02:37
then the robot yaws about the vertical axis.
02:39
So an on-board processor
02:41
essentially looks at what motions need to be executed
02:43
and combines these motions
02:46
and figures out what commands to send to the motors
02:48
600 times a second.
02:51
That's basically how this thing operates.
02:53
So one of the advantages of this design
02:55
is, when you scale things down,
02:57
the robot naturally becomes agile.
02:59
So here R
03:02
is the characteristic length of the robot.
03:04
It's actually half the diameter.
03:06
And there are lots of physical parameters that change
03:09
as you reduce R.
03:12
The one that's the most important
03:14
is the inertia or the resistance to motion.
03:16
So it turns out,
03:18
the inertia, which governs angular motion,
03:20
scales as a fifth power of R.
03:23
So the smaller you make R,
03:26
the more dramatically the inertia reduces.
03:28
So as a result, the angular acceleration,
03:31
denoted by Greek letter alpha here,
03:34
goes as one over R.
03:36
It's inversely proportional to R.
03:38
The smaller you make it the more quickly you can turn.
03:40
So this should be clear in these videos.
03:43
At the bottom right you see a robot
03:45
performing a 360 degree flip
03:48
in less than half a second.
03:50
Multiple flips, a little more time.
03:52
So here the processes on board
03:55
are getting feedback from accelerometers
03:57
and gyros on board
03:59
and calculating, like I said before,
04:01
commands at 600 times a second
04:03
to stabilize this robot.
04:05
So on the left, you see Daniel throwing this robot up into the air.
04:07
And it shows you how robust the control is.
04:10
No matter how you throw it,
04:12
the robot recovers and comes back to him.
04:14
So why build robots like this?
04:18
Well robots like this have many applications.
04:20
You can send them inside buildings like this
04:23
as first responders to look for intruders,
04:26
maybe look for biochemical leaks,
04:29
gaseous leaks.
04:32
You can also use them
04:34
for applications like construction.
04:36
So here are robots carrying beams, columns
04:38
and assembling cube-like structures.
04:42
I'll tell you a little bit more about this.
04:45
The robots can be used for transporting cargo.
04:48
So one of the problems with these small robots
04:51
is their payload carrying capacity.
04:54
So you might want to have multiple robots
04:56
carry payloads.
04:58
This is a picture of a recent experiment we did --
05:00
actually not so recent anymore --
05:02
in Sendai shortly after the earthquake.
05:04
So robots like this could be sent into collapsed buildings
05:07
to assess the damage after natural disasters,
05:10
or sent into reactor buildings
05:12
to map radiation levels.
05:15
So one fundamental problem
05:19
that the robots have to solve if they're to be autonomous
05:21
is essentially figuring out
05:24
how to get from point A to point B.
05:26
So this gets a little challenging
05:28
because the dynamics of this robot are quite complicated.
05:30
In fact, they live in a 12-dimensional space.
05:33
So we use a little trick.
05:35
We take this curved 12-dimensional space
05:37
and transform it
05:40
into a flat four-dimensional space.
05:42
And that four-dimensional space
05:44
consists of X, Y, Z and then the yaw angle.
05:46
And so what the robot does
05:49
is it plans what we call a minimum snap trajectory.
05:51
So to remind you of physics,
05:55
you have position, derivative, velocity,
05:57
then acceleration,
05:59
and then comes jerk
06:01
and then comes snap.
06:03
So this robot minimizes snap.
06:05
So what that effectively does
06:08
is produces a smooth and graceful motion.
06:10
And it does that avoiding obstacles.
06:12
So these minimum snap trajectories in this flat space
06:15
are then transformed back
06:18
into this complicated 12-dimensional space,
06:20
which the robot must do
06:22
for control and then execution.
06:24
So let me show you some examples
06:26
of what these minimum snap trajectories look like.
06:28
And in the first video,
06:30
you'll see the robot going from point A to point B
06:32
through an intermediate point.
06:34
So the robot is obviously capable
06:42
of executing any curve trajectory.
06:44
So these are circular trajectories
06:46
where the robot pulls about two G's.
06:48
Here you have overhead motion capture cameras on the top
06:52
that tell the robot where it is 100 times a second.
06:56
It also tells the robot where these obstacles are.
06:59
And the obstacles can be moving.
07:02
And here you'll see Daniel throw this hoop into the air,
07:04
while the robot is calculating the position of the hoop
07:07
and trying to figure out how to best go through the hoop.
07:09
So as an academic,
07:13
we're always trained to be able to jump through hoops to raise funding for our labs,
07:15
and we get our robots to do that.
07:18
(Applause)
07:21
So another thing the robot can do
07:27
is it remembers pieces of trajectory
07:29
that it learns or is pre-programmed.
07:32
So here you see the robot
07:34
combining a motion
07:36
that builds up momentum
07:38
and then changes its orientation and then recovers.
07:40
So it has to do this because this gap in the window
07:43
is only slightly larger than the width of the robot.
07:46
So just like a diver stands on a springboard
07:50
and then jumps off it to gain momentum,
07:53
and then does this pirouette, this two and a half somersault through
07:55
and then gracefully recovers,
07:58
this robot is basically doing that.
08:00
So it knows how to combine little bits and pieces of trajectories
08:02
to do these fairly difficult tasks.
08:05
So I want change gears.
08:09
So one of the disadvantages of these small robots is its size.
08:11
And I told you earlier
08:14
that we may want to employ lots and lots of robots
08:16
to overcome the limitations of size.
08:18
So one difficulty
08:21
is how do you coordinate lots of these robots?
08:23
And so here we looked to nature.
08:26
So I want to show you a clip
08:28
of Aphaenogaster desert ants
08:30
in Professor Stephen Pratt's lab carrying an object.
08:32
So this is actually a piece of fig.
08:35
Actually you take any object coated with fig juice
08:37
and the ants will carry them back to the nest.
08:39
So these ants don't have any central coordinator.
08:42
They sense their neighbors.
08:45
There's no explicit communication.
08:47
But because they sense the neighbors
08:49
and because they sense the object,
08:51
they have implicit coordination across the group.
08:53
So this is the kind of coordination
08:56
we want our robots to have.
08:58
So when we have a robot
09:01
which is surrounded by neighbors --
09:03
and let's look at robot I and robot J --
09:05
what we want the robots to do
09:07
is to monitor the separation between them
09:09
as they fly in formation.
09:12
And then you want to make sure
09:14
that this separation is within acceptable levels.
09:16
So again the robots monitor this error
09:18
and calculate the control commands
09:21
100 times a second,
09:23
which then translates to the motor commands 600 times a second.
09:25
So this also has to be done
09:28
in a decentralized way.
09:30
Again, if you have lots and lots of robots,
09:32
it's impossible to coordinate all this information centrally
09:34
fast enough in order for the robots to accomplish the task.
09:38
Plus the robots have to base their actions
09:41
only on local information,
09:43
what they sense from their neighbors.
09:45
And then finally,
09:47
we insist that the robots be agnostic
09:49
to who their neighbors are.
09:51
So this is what we call anonymity.
09:53
So what I want to show you next
09:56
is a video
09:58
of 20 of these little robots
10:00
flying in formation.
10:03
They're monitoring their neighbors' position.
10:05
They're maintaining formation.
10:08
The formations can change.
10:10
They can be planar formations,
10:12
they can be three-dimensional formations.
10:14
As you can see here,
10:16
they collapse from a three-dimensional formation into planar formation.
10:18
And to fly through obstacles
10:21
they can adapt the formations on the fly.
10:23
So again, these robots come really close together.
10:27
As you can see in this figure-eight flight,
10:30
they come within inches of each other.
10:32
And despite the aerodynamic interactions
10:34
of these propeller blades,
10:37
they're able to maintain stable flight.
10:39
(Applause)
10:41
So once you know how to fly in formation,
10:48
you can actually pick up objects cooperatively.
10:50
So this just shows
10:52
that we can double, triple, quadruple
10:54
the robot strength
10:57
by just getting them to team with neighbors, as you can see here.
10:59
One of the disadvantages of doing that
11:01
is, as you scale things up --
11:04
so if you have lots of robots carrying the same thing,
11:06
you're essentially effectively increasing the inertia,
11:08
and therefore you pay a price; they're not as agile.
11:11
But you do gain in terms of payload carrying capacity.
11:14
Another application I want to show you --
11:17
again, this is in our lab.
11:19
This is work done by Quentin Lindsey who's a graduate student.
11:21
So his algorithm essentially tells these robots
11:23
how to autonomously build
11:26
cubic structures
11:28
from truss-like elements.
11:30
So his algorithm tells the robot
11:33
what part to pick up,
11:35
when and where to place it.
11:37
So in this video you see --
11:39
and it's sped up 10, 14 times --
11:41
you see three different structures being built by these robots.
11:43
And again, everything is autonomous,
11:46
and all Quentin has to do
11:48
is to get them a blueprint
11:50
of the design that he wants to build.
11:52
So all these experiments you've seen thus far,
11:56
all these demonstrations,
11:59
have been done with the help of motion capture systems.
12:01
So what happens when you leave your lab
12:04
and you go outside into the real world?
12:06
And what if there's no GPS?
12:09
So this robot
12:12
is actually equipped with a camera
12:14
and a laser rangefinder, laser scanner.
12:16
And it uses these sensors
12:19
to build a map of the environment.
12:21
What that map consists of are features --
12:23
like doorways, windows,
12:26
people, furniture --
12:28
and it then figures out where its position is
12:30
with respect to the features.
12:32
So there is no global coordinate system.
12:34
The coordinate system is defined based on the robot,
12:36
where it is and what it's looking at.
12:39
And it navigates with respect to those features.
12:42
So I want to show you a clip
12:45
of algorithms developed by Frank Shen
12:47
and Professor Nathan Michael
12:49
that shows this robot entering a building for the very first time
12:51
and creating this map on the fly.
12:55
So the robot then figures out what the features are.
12:58
It builds the map.
13:01
It figures out where it is with respect to the features
13:03
and then estimates its position
13:05
100 times a second
13:07
allowing us to use the control algorithms
13:09
that I described to you earlier.
13:11
So this robot is actually being commanded
13:13
remotely by Frank.
13:15
But the robot can also figure out
13:17
where to go on its own.
13:19
So suppose I were to send this into a building
13:21
and I had no idea what this building looked like,
13:23
I can ask this robot to go in,
13:25
create a map
13:27
and then come back and tell me what the building looks like.
13:29
So here, the robot is not only solving the problem,
13:32
how to go from point A to point B in this map,
13:35
but it's figuring out
13:38
what the best point B is at every time.
13:40
So essentially it knows where to go
13:42
to look for places that have the least information.
13:45
And that's how it populates this map.
13:47
So I want to leave you
13:50
with one last application.
13:52
And there are many applications of this technology.
13:54
I'm a professor, and we're passionate about education.
13:57
Robots like this can really change the way
13:59
we do K through 12 education.
14:01
But we're in Southern California,
14:03
close to Los Angeles,
14:05
so I have to conclude
14:07
with something focused on entertainment.
14:09
I want to conclude with a music video.
14:11
I want to introduce the creators, Alex and Daniel,
14:13
who created this video.
14:16
(Applause)
14:18
So before I play this video,
14:25
I want to tell you that they created it in the last three days
14:27
after getting a call from Chris.
14:30
And the robots that play the video
14:32
are completely autonomous.
14:34
You will see nine robots play six different instruments.
14:36
And of course, it's made exclusively for TED 2012.
14:39
Let's watch.
14:43
(Music)
15:19
(Applause)
16:23

▲Back to top

About the Speaker:

Vijay Kumar - Roboticist
As the dean of the University of Pennsylvania's School of Engineering and Applied Science, Vijay Kumar studies the control and coordination of multi-robot formations.

Why you should listen

At the General Robotics, Automation, Sensing and Perception (GRASP) Lab at the University of Pennsylvania, flying quadrotor robots move together in eerie formation, tightening themselves into perfect battalions, even filling in the gap when one of their own drops out. You might have seen viral videos of the quads zipping around the netting-draped GRASP Lab (they juggle! they fly through a hula hoop!). Vijay Kumar headed this lab from 1998-2004. He's now the dean of the School of Engineering and Applied Science at the University of Pennsylvania in Philadelphia, where he continues his work in robotics, blending computer science and mechanical engineering to create the next generation of robotic wonders.

More profile about the speaker
Vijay Kumar | Speaker | TED.com