TED2012
Vijay Kumar: Robots that fly ... and cooperate
Filmed:
Readability: 4.5
5,188,706 views
In his lab at Penn, Vijay Kumar and his team build flying quadrotors, small, agile robots that swarm, sense each other, and form ad hoc teams -- for construction, surveying disasters and far more.
Vijay Kumar - Roboticist
As the dean of the University of Pennsylvania's School of Engineering and Applied Science, Vijay Kumar studies the control and coordination of multi-robot formations. Full bio
As the dean of the University of Pennsylvania's School of Engineering and Applied Science, Vijay Kumar studies the control and coordination of multi-robot formations. Full bio
Double-click the English transcript below to play the video.
00:20
Good morning.
0
5000
2000
00:22
I'm here today to talk
1
7000
2000
00:24
about autonomous, flying beach balls.
2
9000
3000
00:27
No, agile aerial robots like this one.
3
12000
4000
00:31
I'd like to tell you a little bit about the challenges in building these
4
16000
3000
00:34
and some of the terrific opportunities
5
19000
2000
00:36
for applying this technology.
6
21000
2000
00:38
So these robots
7
23000
2000
00:40
are related to unmanned aerial vehicles.
8
25000
3000
00:43
However, the vehicles you see here are big.
9
28000
3000
00:46
They weigh thousands of pounds,
10
31000
2000
00:48
are not by any means agile.
11
33000
2000
00:50
They're not even autonomous.
12
35000
2000
00:52
In fact, many of these vehicles
13
37000
2000
00:54
are operated by flight crews
14
39000
2000
00:56
that can include multiple pilots,
15
41000
3000
00:59
operators of sensors
16
44000
2000
01:01
and mission coordinators.
17
46000
2000
01:03
What we're interested in is developing robots like this --
18
48000
2000
01:05
and here are two other pictures --
19
50000
2000
01:07
of robots that you can buy off the shelf.
20
52000
3000
01:10
So these are helicopters with four rotors
21
55000
3000
01:13
and they're roughly a meter or so in scale
22
58000
4000
01:17
and weigh several pounds.
23
62000
2000
01:19
And so we retrofit these with sensors and processors,
24
64000
3000
01:22
and these robots can fly indoors
25
67000
2000
01:24
without GPS.
26
69000
2000
01:26
The robot I'm holding in my hand
27
71000
2000
01:28
is this one,
28
73000
2000
01:30
and it's been created by two students,
29
75000
3000
01:33
Alex and Daniel.
30
78000
2000
01:35
So this weighs a little more
31
80000
2000
01:37
than a tenth of a pound.
32
82000
2000
01:39
It consumes about 15 watts of power.
33
84000
2000
01:41
And as you can see,
34
86000
2000
01:43
it's about eight inches in diameter.
35
88000
2000
01:45
So let me give you just a very quick tutorial
36
90000
3000
01:48
on how these robots work.
37
93000
2000
01:50
So it has four rotors.
38
95000
2000
01:52
If you spin these rotors at the same speed,
39
97000
2000
01:54
the robot hovers.
40
99000
2000
01:56
If you increase the speed of each of these rotors,
41
101000
3000
01:59
then the robot flies up, it accelerates up.
42
104000
3000
02:02
Of course, if the robot were tilted,
43
107000
2000
02:04
inclined to the horizontal,
44
109000
2000
02:06
then it would accelerate in this direction.
45
111000
3000
02:09
So to get it to tilt, there's one of two ways of doing it.
46
114000
3000
02:12
So in this picture
47
117000
2000
02:14
you see that rotor four is spinning faster
48
119000
2000
02:16
and rotor two is spinning slower.
49
121000
2000
02:18
And when that happens
50
123000
2000
02:20
there's moment that causes this robot to roll.
51
125000
3000
02:23
And the other way around,
52
128000
2000
02:25
if you increase the speed of rotor three
53
130000
3000
02:28
and decrease the speed of rotor one,
54
133000
2000
02:30
then the robot pitches forward.
55
135000
3000
02:33
And then finally,
56
138000
2000
02:35
if you spin opposite pairs of rotors
57
140000
2000
02:37
faster than the other pair,
58
142000
2000
02:39
then the robot yaws about the vertical axis.
59
144000
2000
02:41
So an on-board processor
60
146000
2000
02:43
essentially looks at what motions need to be executed
61
148000
3000
02:46
and combines these motions
62
151000
2000
02:48
and figures out what commands to send to the motors
63
153000
3000
02:51
600 times a second.
64
156000
2000
02:53
That's basically how this thing operates.
65
158000
2000
02:55
So one of the advantages of this design
66
160000
2000
02:57
is, when you scale things down,
67
162000
2000
02:59
the robot naturally becomes agile.
68
164000
3000
03:02
So here R
69
167000
2000
03:04
is the characteristic length of the robot.
70
169000
2000
03:06
It's actually half the diameter.
71
171000
3000
03:09
And there are lots of physical parameters that change
72
174000
3000
03:12
as you reduce R.
73
177000
2000
03:14
The one that's the most important
74
179000
2000
03:16
is the inertia or the resistance to motion.
75
181000
2000
03:18
So it turns out,
76
183000
2000
03:20
the inertia, which governs angular motion,
77
185000
3000
03:23
scales as a fifth power of R.
78
188000
3000
03:26
So the smaller you make R,
79
191000
2000
03:28
the more dramatically the inertia reduces.
80
193000
3000
03:31
So as a result, the angular acceleration,
81
196000
3000
03:34
denoted by Greek letter alpha here,
82
199000
2000
03:36
goes as one over R.
83
201000
2000
03:38
It's inversely proportional to R.
84
203000
2000
03:40
The smaller you make it the more quickly you can turn.
85
205000
3000
03:43
So this should be clear in these videos.
86
208000
2000
03:45
At the bottom right you see a robot
87
210000
3000
03:48
performing a 360 degree flip
88
213000
2000
03:50
in less than half a second.
89
215000
2000
03:52
Multiple flips, a little more time.
90
217000
3000
03:55
So here the processes on board
91
220000
2000
03:57
are getting feedback from accelerometers
92
222000
2000
03:59
and gyros on board
93
224000
2000
04:01
and calculating, like I said before,
94
226000
2000
04:03
commands at 600 times a second
95
228000
2000
04:05
to stabilize this robot.
96
230000
2000
04:07
So on the left, you see Daniel throwing this robot up into the air.
97
232000
3000
04:10
And it shows you how robust the control is.
98
235000
2000
04:12
No matter how you throw it,
99
237000
2000
04:14
the robot recovers and comes back to him.
100
239000
4000
04:18
So why build robots like this?
101
243000
2000
04:20
Well robots like this have many applications.
102
245000
3000
04:23
You can send them inside buildings like this
103
248000
3000
04:26
as first responders to look for intruders,
104
251000
3000
04:29
maybe look for biochemical leaks,
105
254000
3000
04:32
gaseous leaks.
106
257000
2000
04:34
You can also use them
107
259000
2000
04:36
for applications like construction.
108
261000
2000
04:38
So here are robots carrying beams, columns
109
263000
4000
04:42
and assembling cube-like structures.
110
267000
3000
04:45
I'll tell you a little bit more about this.
111
270000
3000
04:48
The robots can be used for transporting cargo.
112
273000
3000
04:51
So one of the problems with these small robots
113
276000
3000
04:54
is their payload carrying capacity.
114
279000
2000
04:56
So you might want to have multiple robots
115
281000
2000
04:58
carry payloads.
116
283000
2000
05:00
This is a picture of a recent experiment we did --
117
285000
2000
05:02
actually not so recent anymore --
118
287000
2000
05:04
in Sendai shortly after the earthquake.
119
289000
3000
05:07
So robots like this could be sent into collapsed buildings
120
292000
3000
05:10
to assess the damage after natural disasters,
121
295000
2000
05:12
or sent into reactor buildings
122
297000
3000
05:15
to map radiation levels.
123
300000
3000
05:19
So one fundamental problem
124
304000
2000
05:21
that the robots have to solve if they're to be autonomous
125
306000
3000
05:24
is essentially figuring out
126
309000
2000
05:26
how to get from point A to point B.
127
311000
2000
05:28
So this gets a little challenging
128
313000
2000
05:30
because the dynamics of this robot are quite complicated.
129
315000
3000
05:33
In fact, they live in a 12-dimensional space.
130
318000
2000
05:35
So we use a little trick.
131
320000
2000
05:37
We take this curved 12-dimensional space
132
322000
3000
05:40
and transform it
133
325000
2000
05:42
into a flat four-dimensional space.
134
327000
2000
05:44
And that four-dimensional space
135
329000
2000
05:46
consists of X, Y, Z and then the yaw angle.
136
331000
3000
05:49
And so what the robot does
137
334000
2000
05:51
is it plans what we call a minimum snap trajectory.
138
336000
4000
05:55
So to remind you of physics,
139
340000
2000
05:57
you have position, derivative, velocity,
140
342000
2000
05:59
then acceleration,
141
344000
2000
06:01
and then comes jerk
142
346000
2000
06:03
and then comes snap.
143
348000
2000
06:05
So this robot minimizes snap.
144
350000
3000
06:08
So what that effectively does
145
353000
2000
06:10
is produces a smooth and graceful motion.
146
355000
2000
06:12
And it does that avoiding obstacles.
147
357000
3000
06:15
So these minimum snap trajectories in this flat space
148
360000
3000
06:18
are then transformed back
149
363000
2000
06:20
into this complicated 12-dimensional space,
150
365000
2000
06:22
which the robot must do
151
367000
2000
06:24
for control and then execution.
152
369000
2000
06:26
So let me show you some examples
153
371000
2000
06:28
of what these minimum snap trajectories look like.
154
373000
2000
06:30
And in the first video,
155
375000
2000
06:32
you'll see the robot going from point A to point B
156
377000
2000
06:34
through an intermediate point.
157
379000
2000
06:42
So the robot is obviously capable
158
387000
2000
06:44
of executing any curve trajectory.
159
389000
2000
06:46
So these are circular trajectories
160
391000
2000
06:48
where the robot pulls about two G's.
161
393000
3000
06:52
Here you have overhead motion capture cameras on the top
162
397000
4000
06:56
that tell the robot where it is 100 times a second.
163
401000
3000
06:59
It also tells the robot where these obstacles are.
164
404000
3000
07:02
And the obstacles can be moving.
165
407000
2000
07:04
And here you'll see Daniel throw this hoop into the air,
166
409000
3000
07:07
while the robot is calculating the position of the hoop
167
412000
2000
07:09
and trying to figure out how to best go through the hoop.
168
414000
4000
07:13
So as an academic,
169
418000
2000
07:15
we're always trained to be able to jump through hoops to raise funding for our labs,
170
420000
3000
07:18
and we get our robots to do that.
171
423000
3000
07:21
(Applause)
172
426000
6000
07:27
So another thing the robot can do
173
432000
2000
07:29
is it remembers pieces of trajectory
174
434000
3000
07:32
that it learns or is pre-programmed.
175
437000
2000
07:34
So here you see the robot
176
439000
2000
07:36
combining a motion
177
441000
2000
07:38
that builds up momentum
178
443000
2000
07:40
and then changes its orientation and then recovers.
179
445000
3000
07:43
So it has to do this because this gap in the window
180
448000
3000
07:46
is only slightly larger than the width of the robot.
181
451000
4000
07:50
So just like a diver stands on a springboard
182
455000
3000
07:53
and then jumps off it to gain momentum,
183
458000
2000
07:55
and then does this pirouette, this two and a half somersault through
184
460000
3000
07:58
and then gracefully recovers,
185
463000
2000
08:00
this robot is basically doing that.
186
465000
2000
08:02
So it knows how to combine little bits and pieces of trajectories
187
467000
3000
08:05
to do these fairly difficult tasks.
188
470000
4000
08:09
So I want change gears.
189
474000
2000
08:11
So one of the disadvantages of these small robots is its size.
190
476000
3000
08:14
And I told you earlier
191
479000
2000
08:16
that we may want to employ lots and lots of robots
192
481000
2000
08:18
to overcome the limitations of size.
193
483000
3000
08:21
So one difficulty
194
486000
2000
08:23
is how do you coordinate lots of these robots?
195
488000
3000
08:26
And so here we looked to nature.
196
491000
2000
08:28
So I want to show you a clip
197
493000
2000
08:30
of Aphaenogaster desert ants
198
495000
2000
08:32
in Professor Stephen Pratt's lab carrying an object.
199
497000
3000
08:35
So this is actually a piece of fig.
200
500000
2000
08:37
Actually you take any object coated with fig juice
201
502000
2000
08:39
and the ants will carry them back to the nest.
202
504000
3000
08:42
So these ants don't have any central coordinator.
203
507000
3000
08:45
They sense their neighbors.
204
510000
2000
08:47
There's no explicit communication.
205
512000
2000
08:49
But because they sense the neighbors
206
514000
2000
08:51
and because they sense the object,
207
516000
2000
08:53
they have implicit coordination across the group.
208
518000
3000
08:56
So this is the kind of coordination
209
521000
2000
08:58
we want our robots to have.
210
523000
3000
09:01
So when we have a robot
211
526000
2000
09:03
which is surrounded by neighbors --
212
528000
2000
09:05
and let's look at robot I and robot J --
213
530000
2000
09:07
what we want the robots to do
214
532000
2000
09:09
is to monitor the separation between them
215
534000
3000
09:12
as they fly in formation.
216
537000
2000
09:14
And then you want to make sure
217
539000
2000
09:16
that this separation is within acceptable levels.
218
541000
2000
09:18
So again the robots monitor this error
219
543000
3000
09:21
and calculate the control commands
220
546000
2000
09:23
100 times a second,
221
548000
2000
09:25
which then translates to the motor commands 600 times a second.
222
550000
3000
09:28
So this also has to be done
223
553000
2000
09:30
in a decentralized way.
224
555000
2000
09:32
Again, if you have lots and lots of robots,
225
557000
2000
09:34
it's impossible to coordinate all this information centrally
226
559000
4000
09:38
fast enough in order for the robots to accomplish the task.
227
563000
3000
09:41
Plus the robots have to base their actions
228
566000
2000
09:43
only on local information,
229
568000
2000
09:45
what they sense from their neighbors.
230
570000
2000
09:47
And then finally,
231
572000
2000
09:49
we insist that the robots be agnostic
232
574000
2000
09:51
to who their neighbors are.
233
576000
2000
09:53
So this is what we call anonymity.
234
578000
3000
09:56
So what I want to show you next
235
581000
2000
09:58
is a video
236
583000
2000
10:00
of 20 of these little robots
237
585000
3000
10:03
flying in formation.
238
588000
2000
10:05
They're monitoring their neighbors' position.
239
590000
3000
10:08
They're maintaining formation.
240
593000
2000
10:10
The formations can change.
241
595000
2000
10:12
They can be planar formations,
242
597000
2000
10:14
they can be three-dimensional formations.
243
599000
2000
10:16
As you can see here,
244
601000
2000
10:18
they collapse from a three-dimensional formation into planar formation.
245
603000
3000
10:21
And to fly through obstacles
246
606000
2000
10:23
they can adapt the formations on the fly.
247
608000
4000
10:27
So again, these robots come really close together.
248
612000
3000
10:30
As you can see in this figure-eight flight,
249
615000
2000
10:32
they come within inches of each other.
250
617000
2000
10:34
And despite the aerodynamic interactions
251
619000
3000
10:37
of these propeller blades,
252
622000
2000
10:39
they're able to maintain stable flight.
253
624000
2000
10:41
(Applause)
254
626000
7000
10:48
So once you know how to fly in formation,
255
633000
2000
10:50
you can actually pick up objects cooperatively.
256
635000
2000
10:52
So this just shows
257
637000
2000
10:54
that we can double, triple, quadruple
258
639000
3000
10:57
the robot strength
259
642000
2000
10:59
by just getting them to team with neighbors, as you can see here.
260
644000
2000
11:01
One of the disadvantages of doing that
261
646000
3000
11:04
is, as you scale things up --
262
649000
2000
11:06
so if you have lots of robots carrying the same thing,
263
651000
2000
11:08
you're essentially effectively increasing the inertia,
264
653000
3000
11:11
and therefore you pay a price; they're not as agile.
265
656000
3000
11:14
But you do gain in terms of payload carrying capacity.
266
659000
3000
11:17
Another application I want to show you --
267
662000
2000
11:19
again, this is in our lab.
268
664000
2000
11:21
This is work done by Quentin Lindsey who's a graduate student.
269
666000
2000
11:23
So his algorithm essentially tells these robots
270
668000
3000
11:26
how to autonomously build
271
671000
2000
11:28
cubic structures
272
673000
2000
11:30
from truss-like elements.
273
675000
3000
11:33
So his algorithm tells the robot
274
678000
2000
11:35
what part to pick up,
275
680000
2000
11:37
when and where to place it.
276
682000
2000
11:39
So in this video you see --
277
684000
2000
11:41
and it's sped up 10, 14 times --
278
686000
2000
11:43
you see three different structures being built by these robots.
279
688000
3000
11:46
And again, everything is autonomous,
280
691000
2000
11:48
and all Quentin has to do
281
693000
2000
11:50
is to get them a blueprint
282
695000
2000
11:52
of the design that he wants to build.
283
697000
4000
11:56
So all these experiments you've seen thus far,
284
701000
3000
11:59
all these demonstrations,
285
704000
2000
12:01
have been done with the help of motion capture systems.
286
706000
3000
12:04
So what happens when you leave your lab
287
709000
2000
12:06
and you go outside into the real world?
288
711000
3000
12:09
And what if there's no GPS?
289
714000
3000
12:12
So this robot
290
717000
2000
12:14
is actually equipped with a camera
291
719000
2000
12:16
and a laser rangefinder, laser scanner.
292
721000
3000
12:19
And it uses these sensors
293
724000
2000
12:21
to build a map of the environment.
294
726000
2000
12:23
What that map consists of are features --
295
728000
3000
12:26
like doorways, windows,
296
731000
2000
12:28
people, furniture --
297
733000
2000
12:30
and it then figures out where its position is
298
735000
2000
12:32
with respect to the features.
299
737000
2000
12:34
So there is no global coordinate system.
300
739000
2000
12:36
The coordinate system is defined based on the robot,
301
741000
3000
12:39
where it is and what it's looking at.
302
744000
3000
12:42
And it navigates with respect to those features.
303
747000
3000
12:45
So I want to show you a clip
304
750000
2000
12:47
of algorithms developed by Frank Shen
305
752000
2000
12:49
and Professor Nathan Michael
306
754000
2000
12:51
that shows this robot entering a building for the very first time
307
756000
4000
12:55
and creating this map on the fly.
308
760000
3000
12:58
So the robot then figures out what the features are.
309
763000
3000
13:01
It builds the map.
310
766000
2000
13:03
It figures out where it is with respect to the features
311
768000
2000
13:05
and then estimates its position
312
770000
2000
13:07
100 times a second
313
772000
2000
13:09
allowing us to use the control algorithms
314
774000
2000
13:11
that I described to you earlier.
315
776000
2000
13:13
So this robot is actually being commanded
316
778000
2000
13:15
remotely by Frank.
317
780000
2000
13:17
But the robot can also figure out
318
782000
2000
13:19
where to go on its own.
319
784000
2000
13:21
So suppose I were to send this into a building
320
786000
2000
13:23
and I had no idea what this building looked like,
321
788000
2000
13:25
I can ask this robot to go in,
322
790000
2000
13:27
create a map
323
792000
2000
13:29
and then come back and tell me what the building looks like.
324
794000
3000
13:32
So here, the robot is not only solving the problem,
325
797000
3000
13:35
how to go from point A to point B in this map,
326
800000
3000
13:38
but it's figuring out
327
803000
2000
13:40
what the best point B is at every time.
328
805000
2000
13:42
So essentially it knows where to go
329
807000
3000
13:45
to look for places that have the least information.
330
810000
2000
13:47
And that's how it populates this map.
331
812000
3000
13:50
So I want to leave you
332
815000
2000
13:52
with one last application.
333
817000
2000
13:54
And there are many applications of this technology.
334
819000
3000
13:57
I'm a professor, and we're passionate about education.
335
822000
2000
13:59
Robots like this can really change the way
336
824000
2000
14:01
we do K through 12 education.
337
826000
2000
14:03
But we're in Southern California,
338
828000
2000
14:05
close to Los Angeles,
339
830000
2000
14:07
so I have to conclude
340
832000
2000
14:09
with something focused on entertainment.
341
834000
2000
14:11
I want to conclude with a music video.
342
836000
2000
14:13
I want to introduce the creators, Alex and Daniel,
343
838000
3000
14:16
who created this video.
344
841000
2000
14:18
(Applause)
345
843000
7000
14:25
So before I play this video,
346
850000
2000
14:27
I want to tell you that they created it in the last three days
347
852000
3000
14:30
after getting a call from Chris.
348
855000
2000
14:32
And the robots that play the video
349
857000
2000
14:34
are completely autonomous.
350
859000
2000
14:36
You will see nine robots play six different instruments.
351
861000
3000
14:39
And of course, it's made exclusively for TED 2012.
352
864000
4000
14:43
Let's watch.
353
868000
3000
15:19
(Music)
354
904000
10000
16:23
(Applause)
355
968000
17000
ABOUT THE SPEAKER
Vijay Kumar - RoboticistAs the dean of the University of Pennsylvania's School of Engineering and Applied Science, Vijay Kumar studies the control and coordination of multi-robot formations.
Why you should listen
At the General Robotics, Automation, Sensing and Perception (GRASP) Lab at the University of Pennsylvania, flying quadrotor robots move together in eerie formation, tightening themselves into perfect battalions, even filling in the gap when one of their own drops out. You might have seen viral videos of the quads zipping around the netting-draped GRASP Lab (they juggle! they fly through a hula hoop!). Vijay Kumar headed this lab from 1998-2004. He's now the dean of the School of Engineering and Applied Science at the University of Pennsylvania in Philadelphia, where he continues his work in robotics, blending computer science and mechanical engineering to create the next generation of robotic wonders.
Vijay Kumar | Speaker | TED.com