TEDGlobal 2012

Matt Mills: Image recognition that triggers augmented reality

Filmed:

Matt Mills and Tamara Roukaerts demonstrate Aurasma, a new augmented reality tool that can seamlessly animate the world as seen through a smartphone. Going beyond previous augmented reality, their "auras" can do everything from making a painting talk to overlaying live news onto a printed newspaper.

- Technologist
Matt Mills comes from Aurasma, a startup that makes augmented-reality technology for mobile phones. Full bio

-
Tamara Roukaerts is head of marketing at Aurasma, working to build an augmented-reality lens via smartphones. Full bio

So wouldn't it be amazing if our phones
00:16
could see the world in the same way that we do,
00:19
as we're walking around being able
00:21
to point a phone at anything,
00:23
and then have it actually recognize images and objects
00:25
like the human brain, and then be able to pull in information
00:27
from an almost infinite library of knowledge
00:30
and experiences and ideas.
00:33
Well, traditionally that was seen as science fiction,
00:35
but now we've moved to a world
00:37
where actually this has become possible.
00:38
So the best way of explaining it is to just show it.
00:41
What you can see over here is Tamara,
00:43
who is holding my phone that's now plugged in.
00:45
So let me start with this.
00:47
What we have here is a painting
00:49
of the great poet Rabbie Burns,
00:50
and it's just a normal image,
00:52
but if we now switch inputs over to the phone,
00:54
running our technology, you can see
00:57
effectively what Tamara's seeing on the screen,
00:59
and when she points at this image,
01:01
something magical happens.
01:03
(Laughter) (Bagpipes)
01:05
(Bagpipes) (Applause)
01:09
(Bagpipes)
01:12
Voice: Now simmer blinks on flowery braes ...
01:19
Matt Mills: Now, what's great about this is,
01:21
there's no trickery here.
01:22
There's nothing done to this image.
01:24
And what's great about this is the technology's
01:27
actually allowing the phone to start to see and understand
01:30
much like how the human brain does.
01:32
Not only that, but as I move the object around,
01:34
it's going to track it and overlay that content seamlessly.
01:37
Again, the thing that's incredible about this is
01:43
this is how advanced these devices have become.
01:45
All the processing to do that was actually done
01:47
on the device itself.
01:49
Now, this has applications everywhere,
01:52
whether in things like art in museums, like you just saw,
01:54
or in the world of, say, advertising, or print journalism.
01:58
So a newspaper becomes out of date as soon as it's printed.
02:01
And here is this morning's newspaper,
02:04
and we have some Wimbledon news, which is great.
02:06
Now what we can do is point at the front of the newspaper
02:08
and immediately get the bulletin.
02:11
Voice: ... To the grass, and it's very important that you adapt
02:13
and you, you have to be flexible, you have to be willing
02:17
to change direction at a split second,
02:19
and she does all that. She's won this title.
02:22
MM: And that linking of the digital content
02:25
to something that's physical is what we call an aura, and
02:28
I'll be using that term a little bit as we go through the talk.
02:30
So, what's great about this is it isn't just a faster,
02:33
more convenient way to get information in the real world,
02:36
but there are times when actually using this medium
02:39
allows you to be able to display information in a way
02:41
that was never before possible.
02:44
So what I have here is a wireless router.
02:46
My American colleagues have told me I've got to call it a router,
02:48
so that everyone here understands — (Laughter) —
02:52
but nonetheless, here is the device.
02:54
So now what I can do is, rather than getting the instructions
02:57
for the device online, I can simply point at it,
03:00
the device is recognized, and then --
03:03
Voice: Begin by plugging in the grey ADSL cable.
03:07
Then connect the power. Finally, the yellow ethernet cable.
03:10
Congratulations. You have now completed setup.
03:16
(Laughter) MM: Awesome. Thank you.
03:19
(Applause)
03:21
The incredible work that made that possible was done
03:24
here in the U.K. by scientists at Cambridge,
03:27
and they work in our offices,
03:30
and I've got a lovely picture of them here.
03:32
They couldn't all be on stage, but we're going to
03:34
bring their aura to the stage, so here they are.
03:35
They're not very animated. (Laughter)
03:43
This was the fourth take, I'm told. (Laughter)
03:45
Okay. So, as we're talking about Cambridge,
03:49
let's now move on to technical advancements,
03:52
because since we started putting this technology
03:55
on mobile phones less than 12 months ago,
03:57
the speed and the processing in these devices
04:01
has grown at a really phenomenal rate,
04:03
and that means that I can now take cinema-quality
04:06
3D models and place them in the world around me,
04:08
so I have one over here.
04:11
Tamara, would you like to jump in?
04:13
(Music)
04:19
(Dinosaur roaring) (Laughter)
04:21
MM: I should leap in.
04:26
(Music) (Dinosaur roaring)
04:29
(Applause)
04:32
So then, after the fun, comes the more emotional side
04:38
of what we do, because effectively, this technology
04:40
allows you to see the world through someone's eyes,
04:42
and for that person to be able to take a moment in time
04:45
and effectively store it and tag it to something physical
04:48
that exists in the real world.
04:51
What's great about this is, the tools to do this are free.
04:53
They're open, they're available to everyone within our application,
04:55
and educators have really got on board with the classrooms.
04:58
So we have teachers who've tagged up textbooks,
05:02
teachers who've tagged up school classrooms,
05:05
and a great example of this is a school in the U.K.
05:07
I have a picture here from a video, and we're now going to play it.
05:09
Teacher: See what happens. (Children talking) Keep going.
05:13
Child: TV. (Children react)
05:21
Child: Oh my God.
05:25
Teacher: Now move it either side. See what happens.
05:26
Move away from it and come back to it.
05:29
Child: Oh, that is so cool.
05:32
Teacher: And then, have you got it again?
05:35
Child: Oh my God! How did you do that?
05:38
Second child: It's magic.
05:43
(Laughter) MM: (Laughs) So, it's not magic.
05:45
It's available for everyone to do,
05:49
and actually I'm going to show you how easy it is to do
05:51
by doing one right now.
05:53
So, as sort of — I'm told it's called a stadium wave,
05:54
so we're going to start from this side of the room
05:57
on the count of three, and go over to here.
05:58
Tamara, are you recording?
06:00
Okay, so are you all ready?
06:01
One, two, three. Go!
06:03
Audience: Whooooooo!
06:05
MM: Fellows are really good at that. (Laughs) (Laughter)
06:09
Okay. Now we're going to switch back
06:12
into the Aurasma application,
06:14
and what Tamara's going to do is tag that video that we just
06:16
took onto my badge, so that I can remember it forever.
06:19
Now, we have lots of people who are doing this already,
06:25
and we've talked a little bit about the educational side.
06:29
On the emotional side, we have people who've
06:31
done things like send postcards and Christmas cards
06:33
back to their family with little messages on them.
06:36
We have people who have, for example,
06:39
taken the inside of the engine bay of an old car
06:42
and tagged up different components within an engine,
06:44
so that if you're stuck and you want to find out more,
06:46
you can point and discover the information.
06:48
We're all very, very familiar with the Internet.
06:51
In the last 20 years, it's really
06:54
changed the way that we live and work,
06:57
and the way that we see the world, and what's great is,
06:59
we sort of think this is the next paradigm shift,
07:02
because now we can literally take the content
07:04
that we share, we discover, and that we enjoy
07:08
and make it a part of the world around us.
07:11
It's completely free to download this application.
07:13
If you have a good Wi-Fi connection or 3G,
07:15
this process is very, very quick.
07:18
Oh, there we are. We can save it now.
07:19
It's just going to do a tiny bit of processing
07:21
to convert that image that we just took
07:23
into a sort of digital fingerprint,
07:25
and the great thing is, if you're a professional user,
07:27
-- so, a newspaper -- the tools are pretty much identical
07:29
to what we've just used to create this demonstration.
07:32
The only difference is that you've got the ability
07:34
to add in links and slightly more content. Are you now ready?
07:36
Tamara Roukaerts: We're ready to go.
07:39
MM: Okay. So, I'm told we're ready, which means
07:40
we can now point at the image, and there you all are.
07:42
MM on video: One, two, three. Go!
07:44
MM: Well done. We've been Aurasma. Thank you.
07:51
(Applause)
07:53
Translated by Joseph Geni
Reviewed by Morton Bast

▲Back to top

About the Speakers:

Matt Mills - Technologist
Matt Mills comes from Aurasma, a startup that makes augmented-reality technology for mobile phones.

Why you should listen

Augmented reality is the process of adding layers of information onto the world we see, viewable only through technology. In this space, the Aurasma Lite app uses a smartphone and camera to lay "auras"--3D images, games, animations--onto real-world places. Point your phone at a movie poster on the street and launch a trailer; point at a building to pull up an interactive map... or create your own "aura" and upload it to the virtual space for anyone to see. Matt Mills and Tamara Roukaerts both joined Aurasma when it launched in June 2011.

They say: "This is the next step on from simply browsing the internet because now the digital content we discover, create and share can be woven seamlessly in to the world around us."


Tamara Roukaerts -
Tamara Roukaerts is head of marketing at Aurasma, working to build an augmented-reality lens via smartphones.

Why you should listen

Tamara joined Aurasma at its launch in June 2011 as Head of Marketing, having previously led marketing at venture-backed start-up Light Blue Optics for four years. Prior to that, she earned her high-tech stripes working for the Cambridge-MIT Institute - a $100 million joint venture between the University of Cambridge and the Massachusetts Institute of Technology. Tamara started her career working in advertising for Saatchi & Saatchi and has a Masters degree from Oxford University.