TED2011
Dennis Hong: Making a car for blind drivers
Filmed:
Readability: 4.3
923,134 views
Using robotics, laser rangefinders, GPS and smart feedback tools, Dennis Hong is building a car for drivers who are blind. It's not a "self-driving" car, he's careful to note, but a car in which a non-sighted driver can determine speed, proximity and route -- and drive independently.
Dennis Hong - Roboticist
Dennis Hong is the founder and director of RoMeLa -- a Virginia Tech robotics lab that has pioneered several breakthroughs in robot design and engineering. Full bio
Dennis Hong is the founder and director of RoMeLa -- a Virginia Tech robotics lab that has pioneered several breakthroughs in robot design and engineering. Full bio
Double-click the English transcript below to play the video.
00:15
Many believe driving is an activity
0
0
3000
00:18
solely reserved for those who can see.
1
3000
2000
00:20
A blind person driving a vehicle safely and independently
2
5000
3000
00:23
was thought to be an impossible task, until now.
3
8000
3000
00:26
Hello, my name is Dennis Hong,
4
11000
2000
00:28
and we're bringing freedom and independence to the blind
5
13000
2000
00:30
by building a vehicle for the visually impaired.
6
15000
3000
00:33
So before I talk about this car for the blind,
7
18000
3000
00:36
let me briefly tell you about another project that I worked on
8
21000
2000
00:38
called the DARPA Urban Challenge.
9
23000
2000
00:40
Now this was about building a robotic car
10
25000
2000
00:42
that can drive itself.
11
27000
2000
00:44
You press start, nobody touches anything,
12
29000
2000
00:46
and it can reach its destination fully autonomously.
13
31000
3000
00:49
So in 2007, our team won half a million dollars
14
34000
3000
00:52
by placing third place in this competition.
15
37000
2000
00:54
So about that time,
16
39000
2000
00:56
the National Federation of the Blind, or NFB,
17
41000
2000
00:58
challenged the research committee
18
43000
2000
01:00
about who can develop a car
19
45000
2000
01:02
that lets a blind person drive safely and independently.
20
47000
2000
01:04
We decided to give it a try,
21
49000
2000
01:06
because we thought, "Hey, how hard could it be?"
22
51000
2000
01:08
We have already an autonomous vehicle.
23
53000
2000
01:10
We just put a blind person in it and we're done, right?
24
55000
2000
01:12
(Laughter)
25
57000
2000
01:14
We couldn't have been more wrong.
26
59000
2000
01:16
What NFB wanted
27
61000
2000
01:18
was not a vehicle that can drive a blind person around,
28
63000
3000
01:21
but a vehicle where a blind person can make active decisions and drive.
29
66000
3000
01:24
So we had to throw everything out the window
30
69000
2000
01:26
and start from scratch.
31
71000
2000
01:28
So to test this crazy idea,
32
73000
2000
01:30
we developed a small dune buggy prototype vehicle
33
75000
2000
01:32
to test the feasibility.
34
77000
2000
01:34
And in the summer of 2009,
35
79000
2000
01:36
we invited dozens of blind youth from all over the country
36
81000
3000
01:39
and gave them a chance to take it for a spin.
37
84000
2000
01:41
It was an absolutely amazing experience.
38
86000
2000
01:43
But the problem with this car was
39
88000
2000
01:45
it was designed to only be driven in a very controlled environment,
40
90000
3000
01:48
in a flat, closed-off parking lot --
41
93000
2000
01:50
even the lanes defined by red traffic cones.
42
95000
2000
01:52
So with this success,
43
97000
2000
01:54
we decided to take the next big step,
44
99000
2000
01:56
to develop a real car that can be driven on real roads.
45
101000
3000
01:59
So how does it work?
46
104000
2000
02:01
Well, it's a rather complex system,
47
106000
2000
02:03
but let me try to explain it, maybe simplify it.
48
108000
3000
02:06
So we have three steps.
49
111000
2000
02:08
We have perception, computation
50
113000
2000
02:10
and non-visual interfaces.
51
115000
2000
02:12
Now obviously the driver cannot see,
52
117000
2000
02:14
so the system needs to perceive the environment
53
119000
2000
02:16
and gather information for the driver.
54
121000
2000
02:18
For that, we use an initial measurement unit.
55
123000
3000
02:21
So it measures acceleration, angular acceleration --
56
126000
2000
02:23
like a human ear, inner ear.
57
128000
2000
02:25
We fuse that information with a GPS unit
58
130000
2000
02:27
to get an estimate of the location of the car.
59
132000
3000
02:30
We also use two cameras to detect the lanes of the road.
60
135000
3000
02:33
And we also use three laser range finders.
61
138000
2000
02:35
The lasers scan the environment to detect obstacles --
62
140000
3000
02:38
a car approaching from the front, the back
63
143000
2000
02:40
and also any obstacles that run into the roads,
64
145000
3000
02:43
any obstacles around the vehicle.
65
148000
2000
02:45
So all this vast amount of information is then fed into the computer,
66
150000
3000
02:48
and the computer can do two things.
67
153000
2000
02:50
One is, first of all, process this information
68
155000
3000
02:53
to have an understanding of the environment --
69
158000
2000
02:55
these are the lanes of the road, there's the obstacles --
70
160000
3000
02:58
and convey this information to the driver.
71
163000
2000
03:00
The system is also smart enough
72
165000
2000
03:02
to figure out the safest way to operate the car.
73
167000
2000
03:04
So we can also generate instructions
74
169000
2000
03:06
on how to operate the controls of the vehicle.
75
171000
2000
03:08
But the problem is this: How do we convey
76
173000
2000
03:10
this information and instructions
77
175000
2000
03:12
to a person who cannot see
78
177000
2000
03:14
fast enough and accurate enough so he can drive?
79
179000
3000
03:17
So for this, we developed many different types
80
182000
2000
03:19
of non-visual user interface technology.
81
184000
3000
03:22
So starting from a three-dimensional ping sound system,
82
187000
2000
03:24
a vibrating vest,
83
189000
2000
03:26
a click wheel with voice commands, a leg strip,
84
191000
3000
03:29
even a shoe that applies pressure to the foot.
85
194000
2000
03:31
But today we're going to talk about
86
196000
2000
03:33
three of these non-visual user interfaces.
87
198000
2000
03:35
Now the first interface is called a DriveGrip.
88
200000
3000
03:38
So these are a pair of gloves,
89
203000
2000
03:40
and it has vibrating elements on the knuckle part
90
205000
2000
03:42
so you can convey instructions about how to steer --
91
207000
3000
03:45
the direction and the intensity.
92
210000
2000
03:47
Another device is called SpeedStrip.
93
212000
2000
03:49
So this is a chair -- as a matter of fact, it's actually a massage chair.
94
214000
3000
03:52
We gut it out, and we rearrange the vibrating elements in different patterns,
95
217000
4000
03:56
and we actuate them to convey information about the speed,
96
221000
3000
03:59
and also instructions how to use the gas and the brake pedal.
97
224000
3000
04:02
So over here, you can see
98
227000
2000
04:04
how the computer understands the environment,
99
229000
2000
04:06
and because you cannot see the vibration,
100
231000
2000
04:08
we actually put red LED's on the driver so that you can see what's happening.
101
233000
3000
04:11
This is the sensory data,
102
236000
2000
04:13
and that data is transferred to the devices through the computer.
103
238000
3000
04:16
So these two devices, DriveGrip and SpeedStrip,
104
241000
2000
04:18
are very effective.
105
243000
2000
04:20
But the problem is
106
245000
2000
04:22
these are instructional cue devices.
107
247000
2000
04:24
So this is not really freedom, right?
108
249000
2000
04:26
The computer tells you how to drive --
109
251000
2000
04:28
turn left, turn right, speed up, stop.
110
253000
2000
04:30
We call this the "backseat-driver problem."
111
255000
2000
04:32
So we're moving away from the instructional cue devices,
112
257000
3000
04:35
and we're now focusing more
113
260000
2000
04:37
on the informational devices.
114
262000
2000
04:39
A good example for this informational non-visual user interface
115
264000
2000
04:41
is called AirPix.
116
266000
2000
04:43
So think of it as a monitor for the blind.
117
268000
2000
04:45
So it's a small tablet, has many holes in it,
118
270000
2000
04:47
and compressed air comes out,
119
272000
2000
04:49
so it can actually draw images.
120
274000
2000
04:51
So even though you are blind, you can put your hand over it,
121
276000
2000
04:53
you can see the lanes of the road and obstacles.
122
278000
2000
04:55
Actually, you can also change the frequency of the air coming out
123
280000
3000
04:58
and possibly the temperature.
124
283000
2000
05:00
So it's actually a multi-dimensional user interface.
125
285000
3000
05:03
So here you can see the left camera, the right camera from the vehicle
126
288000
3000
05:06
and how the computer interprets that and sends that information to the AirPix.
127
291000
3000
05:09
For this, we're showing a simulator,
128
294000
2000
05:11
a blind person driving using the AirPix.
129
296000
3000
05:14
This simulator was also very useful for training the blind drivers
130
299000
3000
05:17
and also quickly testing different types of ideas
131
302000
2000
05:19
for different types of non-visual user interfaces.
132
304000
2000
05:21
So basically that's how it works.
133
306000
2000
05:23
So just a month ago,
134
308000
2000
05:25
on January 29th,
135
310000
2000
05:27
we unveiled this vehicle for the very first time to the public
136
312000
2000
05:29
at the world-famous Daytona International Speedway
137
314000
3000
05:32
during the Rolex 24 racing event.
138
317000
2000
05:34
We also had some surprises. Let's take a look.
139
319000
3000
05:37
(Music)
140
322000
10000
05:47
(Video) Announcer: This is an historic day in January.
141
332000
4000
05:51
He's coming up to the grandstand, fellow Federationists.
142
336000
4000
05:55
(Cheering)
143
340000
6000
06:01
(Honking)
144
346000
3000
06:04
There's the grandstand now.
145
349000
2000
06:06
And he's [unclear] following that van that's out in front of him.
146
351000
4000
06:10
Well there comes the first box.
147
355000
2000
06:12
Now let's see if Mark avoids it.
148
357000
3000
06:15
He does. He passes it on the right.
149
360000
3000
06:20
Third box is out. The fourth box is out.
150
365000
3000
06:23
And he's perfectly making his way between the two.
151
368000
3000
06:26
He's closing in on the van
152
371000
2000
06:28
to make the moving pass.
153
373000
3000
06:32
Well this is what it's all about,
154
377000
2000
06:34
this kind of dynamic display of audacity and ingenuity.
155
379000
4000
06:39
He's approaching the end of the run,
156
384000
3000
06:42
makes his way between the barrels that are set up there.
157
387000
5000
06:47
(Honking)
158
392000
3000
06:50
(Applause)
159
395000
3000
06:56
Dennis Hong: I'm so happy for you.
160
401000
2000
06:58
Mark's going to give me a ride back to the hotel.
161
403000
2000
07:00
Mark Riccobono: Yes.
162
405000
2000
07:05
(Applause)
163
410000
9000
07:14
DH: So since we started this project,
164
419000
2000
07:16
we've been getting hundreds of letters, emails, phone calls
165
421000
3000
07:19
from people from all around the world.
166
424000
2000
07:21
Letters thanking us, but sometimes you also get funny letters like this one:
167
426000
3000
07:24
"Now I understand why there is Braille on a drive-up ATM machine."
168
429000
4000
07:28
(Laughter)
169
433000
2000
07:30
But sometimes --
170
435000
2000
07:32
(Laughter)
171
437000
2000
07:34
But sometimes I also do get --
172
439000
2000
07:36
I wouldn't call it hate mail --
173
441000
2000
07:38
but letters of really strong concern:
174
443000
2000
07:40
"Dr. Hong, are you insane,
175
445000
2000
07:42
trying to put blind people on the road?
176
447000
2000
07:44
You must be out of your mind."
177
449000
2000
07:46
But this vehicle is a prototype vehicle,
178
451000
2000
07:48
and it's not going to be on the road
179
453000
2000
07:50
until it's proven as safe as, or safer than, today's vehicle.
180
455000
2000
07:52
And I truly believe that this can happen.
181
457000
3000
07:55
But still, will the society,
182
460000
2000
07:57
would they accept such a radical idea?
183
462000
2000
07:59
How are we going to handle insurance?
184
464000
2000
08:01
How are we going to issue driver's licenses?
185
466000
2000
08:03
There's many of these different kinds of hurdles besides technology challenges
186
468000
3000
08:06
that we need to address before this becomes a reality.
187
471000
3000
08:09
Of course, the main goal of this project
188
474000
2000
08:11
is to develop a car for the blind.
189
476000
2000
08:13
But potentially more important than this
190
478000
2000
08:15
is the tremendous value of the spin-off technology
191
480000
3000
08:18
that can come from this project.
192
483000
2000
08:20
The sensors that are used can see through the dark,
193
485000
2000
08:22
the fog and rain.
194
487000
2000
08:24
And together with this new type of interfaces,
195
489000
2000
08:26
we can use these technologies
196
491000
2000
08:28
and apply them to safer cars for sighted people.
197
493000
2000
08:30
Or for the blind, everyday home appliances --
198
495000
3000
08:33
in the educational setting, in the office setting.
199
498000
2000
08:35
Just imagine, in a classroom a teacher writes on the blackboard
200
500000
3000
08:38
and a blind student can see what's written and read
201
503000
3000
08:41
using these non-visual interfaces.
202
506000
2000
08:43
This is priceless.
203
508000
3000
08:46
So today, the things I've showed you today, is just the beginning.
204
511000
3000
08:49
Thank you very much.
205
514000
2000
08:51
(Applause)
206
516000
11000
ABOUT THE SPEAKER
Dennis Hong - RoboticistDennis Hong is the founder and director of RoMeLa -- a Virginia Tech robotics lab that has pioneered several breakthroughs in robot design and engineering.
Why you should listen
As director of a groundbreaking robotics lab, Dennis Hong guides his team of students through projects on robot locomotion and mechanism design, creating award-winning humanoid robots like DARwIn (Dynamic Anthropomorphic Robot with Intelligence). His team is known as RoMeLa (Robotics & Mechanisms Laboratory) and operates at Virginia Tech.
Hong has also pioneered various innovations in soft-body robots, using a “whole-skin locomotion” as inspired by amoebae. Marrying robotics with biochemistry, he has been able to generate new types of motion with these ingenious forms. For his contributions to the field, Hong was selected as a NASA Summer Faculty Fellow in 2005, given the CAREER award by the National Science Foundation in 2007 and in 2009, named as one of Popular Science's Brilliant 10. He is also a gourmet chef and a magician, performing shows for charity and lecturing on the science of magic.
Dennis Hong | Speaker | TED.com