TED2010
Blaise Agüera y Arcas: Augmented-reality maps
Filmed:
Readability: 3.4
1,856,682 views
In a demo that drew gasps at TED2010, Blaise Aguera y Arcas demos new augmented-reality mapping technology from Microsoft.
Blaise Agüera y Arcas - Software architect
Blaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces. Full bio
Blaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces. Full bio
Double-click the English transcript below to play the video.
00:15
About a year and a half ago,
0
0
2000
00:17
Stephen Lawler, who also gave a talk
1
2000
3000
00:20
here at TED in 2007 on Virtual Earth,
2
5000
2000
00:22
brought me over to become the architect of Bing Maps,
3
7000
4000
00:26
which is Microsoft's online-mapping effort.
4
11000
3000
00:29
In the past two and a half, we've been very hard at work
5
14000
2000
00:31
on redefining the way maps work online.
6
16000
4000
00:35
And we really are seeing this in very different terms
7
20000
2000
00:37
from the kind of mapping and direction site
8
22000
2000
00:39
that one is used to.
9
24000
2000
00:41
So, the first thing that you might notice about the mapping site
10
26000
2000
00:43
is just the fluidity of the zooming and the panning,
11
28000
2000
00:45
which, if you're familiar at all with Seadragon,
12
30000
2000
00:47
that's where it comes from.
13
32000
2000
00:49
Mapping is, of course,
14
34000
2000
00:51
not just about cartography,
15
36000
2000
00:53
it's also about imagery.
16
38000
2000
00:55
So, as we zoom-in beyond a certain level
17
40000
3000
00:58
this resolves into a kind of Sim City-like
18
43000
3000
01:01
virtual view at 45 degrees.
19
46000
2000
01:03
This can be viewed from any of the cardinal directions
20
48000
2000
01:05
to show you the 3D structure of the city, all the facades.
21
50000
3000
01:08
Now, we see this space, this three-dimensional environment,
22
53000
6000
01:14
as being a canvas on which
23
59000
3000
01:17
all sorts of applications can play out,
24
62000
6000
01:23
and map's directions are really just one of them.
25
68000
3000
01:26
If you click on this, you'll see
26
71000
3000
01:29
some of the ones that we've put out, just in the past couple of months
27
74000
3000
01:32
since we've launched.
28
77000
2000
01:34
So, for example, a couple of days after the disaster in Haiti,
29
79000
2000
01:36
we had an earthquake map that showed
30
81000
2000
01:38
before and after pictures from the sky.
31
83000
2000
01:40
This wonderful one which I don't have time to show you
32
85000
2000
01:42
is taking hyper-local blogs in real time
33
87000
3000
01:45
and mapping those stories, those entries
34
90000
2000
01:47
to the places that are referred to on the blogs.
35
92000
2000
01:49
It's wonderful.
36
94000
3000
01:52
But I'm going to show you some more candy sort of stuff.
37
97000
3000
01:55
So, we see the imagery, of course,
38
100000
4000
01:59
not stopping at the sky.
39
104000
3000
02:02
These little green bubbles represent
40
107000
3000
02:05
photosynths that users have made.
41
110000
2000
02:07
I'm not going to dive into them either, but photosynths are integrated into the map.
42
112000
3000
02:10
Everything that's cased in blue
43
115000
2000
02:12
is an area where we've taken imagery on the ground as well.
44
117000
5000
02:17
And so, when you fly down --
45
122000
2000
02:19
(Applause)
46
124000
2000
02:21
Thank you. When you fly down to the ground,
47
126000
2000
02:23
and you see this kind of panoramic imagery,
48
128000
3000
02:26
the first thing that you might notice is that it's not just a picture,
49
131000
4000
02:30
there's just as much three-dimensional understanding of this environment
50
135000
3000
02:33
as there is of the three-dimensional city from above,
51
138000
3000
02:36
so if I click on something to get a closer view of it,
52
141000
4000
02:40
then, the fact that that transition looks as it does,
53
145000
3000
02:43
is a function of all of that geometry,
54
148000
2000
02:45
all of that 3D understanding behind this model.
55
150000
4000
02:49
Now, I'll show you a fun app
56
154000
3000
02:52
that -- we've been working on a collaboration with our friends at Flickr.
57
157000
7000
02:59
This takes Flickr, georegistered imagery
58
164000
3000
03:02
and uses photosynth-like processes
59
167000
3000
03:05
to connect that imagery to our imagery, so --
60
170000
4000
03:09
I'm not sure if that's the one I actually meant to pull up, but --
61
174000
3000
03:12
(Laughter)
62
177000
3000
03:15
But notice -- this is, of course, a popular tourist site,
63
180000
4000
03:19
and there are lots of photos around here,
64
184000
2000
03:21
and these photos are all taken at different times.
65
186000
3000
03:24
So this one was taken around five.
66
189000
2000
03:26
So that's the Flickr photo,
67
191000
2000
03:28
that's our imagery.
68
193000
2000
03:30
So you really see how this kind of crowd-sourced imagery
69
195000
3000
03:33
is integrating, in a very deep way, into the map itself.
70
198000
3000
03:36
(Applause)
71
201000
2000
03:38
Thank you.
72
203000
2000
03:40
(Applause)
73
205000
2000
03:42
There are several reasons why this is interesting
74
207000
2000
03:44
and one of them, of course, is time travel.
75
209000
2000
03:46
And I'm not going to show you some of the wonderful historic imagery in here,
76
211000
2000
03:48
but there are some with horses and carriages and so on as well.
77
213000
3000
03:51
But what's cool about this is that, not only is it augmenting
78
216000
3000
03:54
this visual representation of the world
79
219000
2000
03:56
with things that are coming in from users,
80
221000
3000
03:59
but it also is the foundation for augmented reality,
81
224000
4000
04:03
and that's something that I'll be showing you more of in just a moment.
82
228000
3000
04:06
Now I just made a transition indoors. That's also interesting.
83
231000
4000
04:10
OK, notice there's now a roof above us.
84
235000
2000
04:12
We're inside the Pike Place Market.
85
237000
3000
04:15
And this is something that we're able to do with a backpack camera,
86
240000
2000
04:17
so, we're now not only imaging in the street
87
242000
4000
04:21
with this camera on tops of cars,
88
246000
2000
04:23
but we're also imaging inside.
89
248000
4000
04:27
And from here, we're able to do the same sorts of registration,
90
252000
5000
04:32
not only of still images, but also of video.
91
257000
4000
04:36
So this is something that we're now going to try
92
261000
2000
04:38
for the first time, live,
93
263000
2000
04:40
and this is really, truly, very frightening.
94
265000
3000
04:43
(Laughter)
95
268000
2000
04:46
OK.
96
271000
2000
04:48
(Ringing)
97
273000
2000
04:50
All right, guys, are you there?
98
275000
2000
04:52
(Noise)
99
277000
2000
04:54
All right. I'm hitting it. I'm punching play.
100
279000
2000
04:56
I'm live. All right. There we go.
101
281000
2000
04:58
So, these are our friends in Pike Place Market, the lab.
102
283000
5000
05:03
(Applause)
103
288000
8000
05:12
So they're broadcasting this live.
104
297000
2000
05:14
OK, George, can you pan back over to the corner market?
105
299000
4000
05:18
Because I want to show points of interest.
106
303000
4000
05:22
No, no. The other way.
107
307000
3000
05:25
Yeah, yeah, back to the corner, back to the corner.
108
310000
2000
05:27
I don't want to see you guys yet.
109
312000
3000
05:31
OK, OK, back to the corner, back to the corner, back to the corner.
110
316000
6000
05:37
OK, never mind.
111
322000
2000
05:40
What I wanted to show you was these points of interest
112
325000
2000
05:42
over here on top of the image
113
327000
2000
05:44
because what that gives you a sense of is the way,
114
329000
2000
05:46
if you're actually on the spot,
115
331000
2000
05:48
you can think about this --
116
333000
2000
05:50
this is taking a step in addition to augmented reality.
117
335000
3000
05:53
What the hell are you guys -- oh, sorry.
118
338000
2000
05:55
(Laughter)
119
340000
3000
05:58
We're doing two different --
120
343000
3000
06:01
OK, I'm hanging up now.
121
346000
3000
06:04
We're doing two different things here.
122
349000
2000
06:06
One of them is to take that real ...
123
351000
2000
06:08
(Laughter)
124
353000
7000
06:15
All right, let me just take a moment and thank the team.
125
360000
3000
06:18
They've done a fantastic job of pulling this together.
126
363000
2000
06:20
(Applause)
127
365000
2000
06:22
I'm going to abandon them now and walk back outside.
128
367000
3000
06:25
And while I walk outside, I'll just mention that
129
370000
3000
06:28
here we're using this for telepresence,
130
373000
2000
06:30
but you can equally well use this
131
375000
2000
06:32
on the spot, for augmented reality.
132
377000
2000
06:34
When you use it on the spot, it means that
133
379000
2000
06:36
you're able to bring all of that metadata
134
381000
2000
06:38
and information about the world to you.
135
383000
2000
06:40
So here, we're taking the extra step of also broadcasting it.
136
385000
2000
06:42
That was being broadcast, by the way, on a 4G network
137
387000
3000
06:45
from the market.
138
390000
3000
06:48
All right, and now there's one last TED talk
139
393000
3000
06:51
that Microsoft has given in the past several years.
140
396000
2000
06:53
And that's Curtis Wong, WorldWide Telescope.
141
398000
3000
06:56
So, we're going to head over to the dumpsters,
142
401000
2000
06:58
where it's traditional, after a long day at the market,
143
403000
3000
07:01
to go out for a break, but also stare up at the sky.
144
406000
4000
07:05
This is the integration
145
410000
2000
07:07
of WorldWide Telescope into our maps.
146
412000
3000
07:10
(Applause)
147
415000
2000
07:12
This is the current -- thank you --
148
417000
3000
07:15
this is the current time. If we scrub the time,
149
420000
2000
07:17
then we can see how the sky will look at different times,
150
422000
3000
07:20
and we can get all of this very detailed
151
425000
2000
07:22
information about different times, different dates:
152
427000
4000
07:27
Let's move the moon a little higher in the sky,
153
432000
2000
07:29
maybe change the date.
154
434000
4000
07:33
I would like to kind of zoom in on the moon.
155
438000
3000
07:39
So, this is an astronomically complete
156
444000
2000
07:41
representation of the sky
157
446000
3000
07:44
integrated right into the Earth.
158
449000
2000
07:46
All right now, I've overrun my time,
159
451000
2000
07:48
so I've got to stop.
160
453000
2000
07:50
Thank you all very much.
161
455000
2000
07:52
(Applause)
162
457000
12000
ABOUT THE SPEAKER
Blaise Agüera y Arcas - Software architectBlaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces.
Why you should listen
Blaise Agüera y Arcas is principal scientist at Google, where he leads a team working on machine intelligence for mobile devices. His group works extensively with deep neural nets for machine perception and distributed learning, and it also investigates so-called "connectomics" research, assessing maps of connections within the brain.
Agüera y Arcas' background is as multidimensional as the visions he helps create. In the 1990s, he authored patents on both video compression and 3D visualization techniques, and in 2001, he made an influential computational discovery that cast doubt on Gutenberg's role as the father of movable type.
He also created Seadragon (acquired by Microsoft in 2006), the visualization technology that gives Photosynth its amazingly smooth digital rendering and zoom capabilities. Photosynth itself is a vastly powerful piece of software capable of taking a wide variety of images, analyzing them for similarities, and grafting them together into an interactive three-dimensional space. This seamless patchwork of images can be viewed via multiple angles and magnifications, allowing us to look around corners or “fly” in for a (much) closer look. Simply put, it could utterly transform the way we experience digital images.
He joined Microsoft when Seadragon was acquired by Live Labs in 2006. Shortly after the acquisition of Seadragon, Agüera y Arcas directed his team in a collaboration with Microsoft Research and the University of Washington, leading to the first public previews of Photosynth several months later. His TED Talk on Seadragon and Photosynth in 2007 is rated one of TED's "most jaw-dropping." He returned to TED in 2010 to demo Bing’s augmented reality maps.
Fun fact: According to the author, Agüera y Arcas is the inspiration for the character Elgin in the 2012 best-selling novel Where'd You Go, Bernadette?
Blaise Agüera y Arcas | Speaker | TED.com