sponsored links
TED2010

Blaise Agüera y Arcas: Augmented-reality maps

February 11, 2010

In a demo that drew gasps at TED2010, Blaise Aguera y Arcas demos new augmented-reality mapping technology from Microsoft.

Blaise Agüera y Arcas - Software architect
Blaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces. Full bio

sponsored links
Double-click the English subtitles below to play the video.
About a year and a half ago,
00:15
Stephen Lawler, who also gave a talk
00:17
here at TED in 2007 on Virtual Earth,
00:20
brought me over to become the architect of Bing Maps,
00:22
which is Microsoft's online-mapping effort.
00:26
In the past two and a half, we've been very hard at work
00:29
on redefining the way maps work online.
00:31
And we really are seeing this in very different terms
00:35
from the kind of mapping and direction site
00:37
that one is used to.
00:39
So, the first thing that you might notice about the mapping site
00:41
is just the fluidity of the zooming and the panning,
00:43
which, if you're familiar at all with Seadragon,
00:45
that's where it comes from.
00:47
Mapping is, of course,
00:49
not just about cartography,
00:51
it's also about imagery.
00:53
So, as we zoom-in beyond a certain level
00:55
this resolves into a kind of Sim City-like
00:58
virtual view at 45 degrees.
01:01
This can be viewed from any of the cardinal directions
01:03
to show you the 3D structure of the city, all the facades.
01:05
Now, we see this space, this three-dimensional environment,
01:08
as being a canvas on which
01:14
all sorts of applications can play out,
01:17
and map's directions are really just one of them.
01:23
If you click on this, you'll see
01:26
some of the ones that we've put out, just in the past couple of months
01:29
since we've launched.
01:32
So, for example, a couple of days after the disaster in Haiti,
01:34
we had an earthquake map that showed
01:36
before and after pictures from the sky.
01:38
This wonderful one which I don't have time to show you
01:40
is taking hyper-local blogs in real time
01:42
and mapping those stories, those entries
01:45
to the places that are referred to on the blogs.
01:47
It's wonderful.
01:49
But I'm going to show you some more candy sort of stuff.
01:52
So, we see the imagery, of course,
01:55
not stopping at the sky.
01:59
These little green bubbles represent
02:02
photosynths that users have made.
02:05
I'm not going to dive into them either, but photosynths are integrated into the map.
02:07
Everything that's cased in blue
02:10
is an area where we've taken imagery on the ground as well.
02:12
And so, when you fly down --
02:17
(Applause)
02:19
Thank you. When you fly down to the ground,
02:21
and you see this kind of panoramic imagery,
02:23
the first thing that you might notice is that it's not just a picture,
02:26
there's just as much three-dimensional understanding of this environment
02:30
as there is of the three-dimensional city from above,
02:33
so if I click on something to get a closer view of it,
02:36
then, the fact that that transition looks as it does,
02:40
is a function of all of that geometry,
02:43
all of that 3D understanding behind this model.
02:45
Now, I'll show you a fun app
02:49
that -- we've been working on a collaboration with our friends at Flickr.
02:52
This takes Flickr, georegistered imagery
02:59
and uses photosynth-like processes
03:02
to connect that imagery to our imagery, so --
03:05
I'm not sure if that's the one I actually meant to pull up, but --
03:09
(Laughter)
03:12
But notice -- this is, of course, a popular tourist site,
03:15
and there are lots of photos around here,
03:19
and these photos are all taken at different times.
03:21
So this one was taken around five.
03:24
So that's the Flickr photo,
03:26
that's our imagery.
03:28
So you really see how this kind of crowd-sourced imagery
03:30
is integrating, in a very deep way, into the map itself.
03:33
(Applause)
03:36
Thank you.
03:38
(Applause)
03:40
There are several reasons why this is interesting
03:42
and one of them, of course, is time travel.
03:44
And I'm not going to show you some of the wonderful historic imagery in here,
03:46
but there are some with horses and carriages and so on as well.
03:48
But what's cool about this is that, not only is it augmenting
03:51
this visual representation of the world
03:54
with things that are coming in from users,
03:56
but it also is the foundation for augmented reality,
03:59
and that's something that I'll be showing you more of in just a moment.
04:03
Now I just made a transition indoors. That's also interesting.
04:06
OK, notice there's now a roof above us.
04:10
We're inside the Pike Place Market.
04:12
And this is something that we're able to do with a backpack camera,
04:15
so, we're now not only imaging in the street
04:17
with this camera on tops of cars,
04:21
but we're also imaging inside.
04:23
And from here, we're able to do the same sorts of registration,
04:27
not only of still images, but also of video.
04:32
So this is something that we're now going to try
04:36
for the first time, live,
04:38
and this is really, truly, very frightening.
04:40
(Laughter)
04:43
OK.
04:46
(Ringing)
04:48
All right, guys, are you there?
04:50
(Noise)
04:52
All right. I'm hitting it. I'm punching play.
04:54
I'm live. All right. There we go.
04:56
So, these are our friends in Pike Place Market, the lab.
04:58
(Applause)
05:03
So they're broadcasting this live.
05:12
OK, George, can you pan back over to the corner market?
05:14
Because I want to show points of interest.
05:18
No, no. The other way.
05:22
Yeah, yeah, back to the corner, back to the corner.
05:25
I don't want to see you guys yet.
05:27
OK, OK, back to the corner, back to the corner, back to the corner.
05:31
OK, never mind.
05:37
What I wanted to show you was these points of interest
05:40
over here on top of the image
05:42
because what that gives you a sense of is the way,
05:44
if you're actually on the spot,
05:46
you can think about this --
05:48
this is taking a step in addition to augmented reality.
05:50
What the hell are you guys -- oh, sorry.
05:53
(Laughter)
05:55
We're doing two different --
05:58
OK, I'm hanging up now.
06:01
We're doing two different things here.
06:04
One of them is to take that real ...
06:06
(Laughter)
06:08
All right, let me just take a moment and thank the team.
06:15
They've done a fantastic job of pulling this together.
06:18
(Applause)
06:20
I'm going to abandon them now and walk back outside.
06:22
And while I walk outside, I'll just mention that
06:25
here we're using this for telepresence,
06:28
but you can equally well use this
06:30
on the spot, for augmented reality.
06:32
When you use it on the spot, it means that
06:34
you're able to bring all of that metadata
06:36
and information about the world to you.
06:38
So here, we're taking the extra step of also broadcasting it.
06:40
That was being broadcast, by the way, on a 4G network
06:42
from the market.
06:45
All right, and now there's one last TED talk
06:48
that Microsoft has given in the past several years.
06:51
And that's Curtis Wong, WorldWide Telescope.
06:53
So, we're going to head over to the dumpsters,
06:56
where it's traditional, after a long day at the market,
06:58
to go out for a break, but also stare up at the sky.
07:01
This is the integration
07:05
of WorldWide Telescope into our maps.
07:07
(Applause)
07:10
This is the current -- thank you --
07:12
this is the current time. If we scrub the time,
07:15
then we can see how the sky will look at different times,
07:17
and we can get all of this very detailed
07:20
information about different times, different dates:
07:22
Let's move the moon a little higher in the sky,
07:27
maybe change the date.
07:29
I would like to kind of zoom in on the moon.
07:33
So, this is an astronomically complete
07:39
representation of the sky
07:41
integrated right into the Earth.
07:44
All right now, I've overrun my time,
07:46
so I've got to stop.
07:48
Thank you all very much.
07:50
(Applause)
07:52

sponsored links

Blaise Agüera y Arcas - Software architect
Blaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces.

Why you should listen

Blaise Agüera y Arcas is principal scientist at Google, where he leads a team working on machine intelligence for mobile devices. His group works extensively with deep neural nets for machine perception and distributed learning, and it also investigates so-called "connectomics" research, assessing maps of connections within the brain.

Agüera y Arcas' background is as multidimensional as the visions he helps create. In the 1990s, he authored patents on both video compression and 3D visualization techniques, and in 2001, he made an influential computational discovery that cast doubt on Gutenberg's role as the father of movable type.

He also created Seadragon (acquired by Microsoft in 2006), the visualization technology that gives Photosynth its amazingly smooth digital rendering and zoom capabilities. Photosynth itself is a vastly powerful piece of software capable of taking a wide variety of images, analyzing them for similarities, and grafting them together into an interactive three-dimensional space. This seamless patchwork of images can be viewed via multiple angles and magnifications, allowing us to look around corners or “fly” in for a (much) closer look. Simply put, it could utterly transform the way we experience digital images.

He joined Microsoft when Seadragon was acquired by Live Labs in 2006. Shortly after the acquisition of Seadragon, Agüera y Arcas directed his team in a collaboration with Microsoft Research and the University of Washington, leading to the first public previews of Photosynth several months later. His TED Talk on Seadragon and Photosynth in 2007 is rated one of TED's "most jaw-dropping." He returned to TED in 2010 to demo Bing’s augmented reality maps.

Fun fact: According to the author, Agüera y Arcas is the inspiration for the character Elgin in the 2012 best-selling novel Where'd You Go, Bernadette?

The original video is available on TED.com
sponsored links

If you need translations, you can install "Google Translate" extension into your Chrome Browser.
Furthermore, you can change playback rate by installing "Video Speed Controller" extension.

Data provided by TED.

This website is owned and operated by Tokyo English Network.
The developer's blog is here.