11:03
TEDGlobal 2012

Ramesh Raskar: Imaging at a trillion frames per second

Filmed:

Ramesh Raskar presents femto-photography, a new type of imaging so fast it visualizes the world one trillion frames per second, so detailed it shows light itself in motion. This technology may someday be used to build cameras that can look “around” corners or see inside the body without X-rays.

- Femto-photographer
Photography is about creating images by recording light. At the MIT media lab, professor Ramesh Raskar and his team members have invented a camera that can photograph light itself as it moves at, well, the speed of light. Full bio

Doc Edgerton inspired us with awe and curiosity
00:16
with this photo of a bullet piercing through an apple,
00:22
and exposure just a millionth of a second.
00:28
But now, 50 years later, we can go a million times faster
00:32
and see the world not at a million,
00:40
or a billion,
00:43
but one trillion frames per second.
00:45
I present you a new type of photography,
00:49
femto-photography,
00:53
a new imaging technique so fast
00:55
that it can create slow motion videos of light in motion.
01:00
And with that, we can create cameras
01:05
that can look around corners,
01:08
beyond line of sight
01:10
or see inside our body without an X-ray,
01:12
and really challenge what we mean by a camera.
01:16
Now if I take a laser pointer and turn it on and off
01:21
in one trillionth of a second --
01:25
which is several femtoseconds --
01:28
I'll create a packet of photons
01:31
barely a millimeter wide,
01:33
and that packet of photons, that bullet,
01:35
will travel at the speed of light,
01:38
and, again, a million times faster than an ordinary bullet.
01:40
Now, if you take that bullet and take this packet of photons
01:45
and fire into this bottle,
01:50
how will those photons shatter into this bottle?
01:53
How does light look in slow motion?
01:57
Now, the whole event -- (Applause)
02:21
(Applause)
02:25
Now, remember, the whole event
02:29
is effectively taking place in less than a nanosecond
02:32
— that's how much time it takes for light to travel —
02:35
but I'm slowing down in this video by a factor of 10 billion
02:38
so you can see the light in motion.
02:42
But, Coca-Cola did not sponsor this research. (Laughter)
02:46
Now, there's a lot going on in this movie,
02:50
so let me break this down and show you what's going on.
02:52
So, the pulse enters the bottle, our bullet,
02:55
with a packet of photons that start traveling through
02:58
and that start scattering inside.
03:01
Some of the light leaks, goes on the table,
03:02
and you start seeing these ripples of waves.
03:05
Many of the photons eventually reach the cap
03:08
and then they explode in various directions.
03:10
As you can see, there's a bubble of air,
03:13
and it's bouncing around inside.
03:15
Meanwhile, the ripples are traveling on the table,
03:17
and because of the reflections at the top,
03:19
you see at the back of the bottle, after several frames,
03:21
the reflections are focused.
03:25
Now, if you take an ordinary bullet
03:28
and let it go the same distance and slow down the video
03:34
again by a factor of 10 billion, do you know
03:37
how long you'll have to sit here to watch that movie?
03:40
A day, a week? Actually, a whole year.
03:45
It'll be a very boring movie — (Laughter) —
03:50
of a slow, ordinary bullet in motion.
03:54
And what about some still-life photography?
03:58
You can watch the ripples again washing over the table,
04:08
the tomato and the wall in the back.
04:14
It's like throwing a stone in a pond of water.
04:16
I thought, this is how nature paints a photo,
04:23
one femto frame at a time,
04:26
but of course our eye sees an integral composite.
04:29
But if you look at this tomato one more time,
04:34
you will notice, as the light washes over the tomato,
04:38
it continues to glow. It doesn't become dark.
04:40
Why is that? Because the tomato is actually ripe,
04:43
and the light is bouncing around inside the tomato,
04:47
and it comes out after several trillionths of a second.
04:49
So, in the future, when this femto-camera
04:53
is in your camera phone,
04:56
you might be able to go to a supermarket
04:58
and check if the fruit is ripe without actually touching it.
05:00
So how did my team at MIT create this camera?
05:04
Now, as photographers, you know,
05:09
if you take a short exposure photo, you get very little light,
05:11
but we're going to go a billion times faster
05:15
than your shortest exposure,
05:17
so you're going to get hardly any light.
05:19
So, what we do is we send that bullet,
05:21
those packet of photons, millions of times,
05:23
and record again and again with very clever synchronization,
05:25
and from the gigabytes of data,
05:28
we computationally weave together
05:30
to create those femto-videos I showed you.
05:33
And we can take all that raw data
05:36
and treat it in very interesting ways.
05:39
So, Superman can fly.
05:41
Some other heroes can become invisible,
05:43
but what about a new power for a future superhero:
05:46
to see around corners?
05:51
The idea is that we could shine some light on the door.
05:53
It's going to bounce, go inside the room,
05:58
some of that is going to reflect back on the door,
06:01
and then back to the camera,
06:03
and we could exploit these multiple bounces of light.
06:05
And it's not science fiction. We have actually built it.
06:08
On the left, you see our femto-camera.
06:10
There's a mannequin hidden behind a wall,
06:13
and we're going to bounce light off the door.
06:15
So after our paper was published
06:18
in Nature Communications,
06:20
it was highlighted by Nature.com,
06:22
and they created this animation.
06:24
(Music)
06:27
We're going to fire those bullets of light,
06:33
and they're going to hit this wall,
06:36
and because the packet of the photons,
06:40
they will scatter in all the directions,
06:42
and some of them will reach our hidden mannequin,
06:45
which in turn will again scatter that light,
06:47
and again in turn the door will reflect
06:50
some of that scattered light,
06:53
and a tiny fraction of the photons will actually
06:56
come back to the camera, but most interestingly,
06:58
they will all arrive at a slightly different time slot.
07:01
(Music)
07:04
And because we have a camera that can run so fast,
07:09
our femto-camera, it has some unique abilities.
07:12
It has very good time resolution,
07:15
and it can look at the world at the speed of light.
07:18
And this way, we know the distances, of course to the door,
07:21
but also to the hidden objects,
07:25
but we don't know which point corresponds
07:27
to which distance.
07:28
(Music)
07:31
By shining one laser, we can record one raw photo, which,
07:34
you look on the screen, doesn't really make any sense,
07:38
but then we will take a lot of such pictures,
07:40
dozens of such pictures, put them together,
07:42
and try to analyze the multiple bounces of light,
07:45
and from that, can we see the hidden object?
07:47
Can we see it in full 3D?
07:51
So this is our reconstruction. (Music)
07:54
(Music)
07:56
(Music) (Applause)
08:00
Now we have some ways to go before we take this
08:08
outside the lab on the road, but in the future,
08:11
we could create cars that avoid collisions
08:14
with what's around the bend,
08:17
or we can look for survivors in hazardous conditions
08:19
by looking at light reflected through open windows,
08:23
or we can build endoscopes that can see
08:27
deep inside the body around occluders,
08:30
and also for cardioscopes.
08:33
But of course, because of tissue and blood,
08:35
this is quite challenging, so this is really a call
08:37
for scientists to start thinking about femto-photography
08:39
as really a new imaging modality to solve
08:42
the next generation of health imaging problems.
08:45
Now, like Doc Edgerton, a scientist himself,
08:48
science became art, an art of ultra-fast photography,
08:52
and I realized that all the gigabytes of data
08:58
that we're collecting every time
09:01
is not just for scientific imaging, but we can also do
09:03
a new form of computational photography
09:06
with time-lapse and color-coding,
09:10
and we look at those ripples. Remember,
09:14
the time between each of those ripples is only
09:17
a few trillionths of a second.
09:20
But there's also something funny going on here.
09:24
When you look at the ripples under the cap,
09:26
the ripples are moving away from us.
09:28
The ripples should be moving towards us.
09:32
What's going on here?
09:34
It turns out, because we're recording
09:36
nearly at the speed of light,
09:38
we have strange effects,
09:43
and Einstein would have loved to see this picture.
09:45
The order at which events take place in the world
09:49
appear in the camera with sometimes reversed order,
09:52
so by applying the corresponding space and time warp,
09:56
we can correct for this distortion.
10:00
So whether it's for photography around corners,
10:04
or creating the next generation of health imaging,
10:08
or creating new visualizations,
10:12
since our invention, we have open-sourced
10:15
all the data and details on our website, and our hope
10:19
is that the DIY, the creative and the research community
10:22
will show us that we should stop obsessing about
10:29
the megapixels in cameras — (Laughter) —
10:33
and start focusing on the next dimension in imaging.
10:36
It's about time. Thank you. (Applause)
10:41
(Applause)
10:46
Translated by Joseph Geni
Reviewed by Thu-Huong Ha

▲Back to top

About the Speaker:

Ramesh Raskar - Femto-photographer
Photography is about creating images by recording light. At the MIT media lab, professor Ramesh Raskar and his team members have invented a camera that can photograph light itself as it moves at, well, the speed of light.

Why you should listen

In 1964 MIT professor Harold Edgerton, pioneer of stop-action photography, famously took a photo of a bullet piercing an apple using exposures as short as a few nanoseconds. Inspired by his work, Ramesh Raskar and his team set out to create a camera that could capture not just a bullet (traveling at 850 meters per second) but light itself (nearly 300 million meters per second).

Stop a moment to take that in: photographing light as it moves. For that, they built a camera and software that can visualize pictures as if they are recorded at 1 trillion frames per second. The same photon-imaging technology can also be used to create a camera that can peer "around" corners , by exploiting specific properties of the photons when they bounce off surfaces and objects.

Among the other projects that Raskar is leading, with the MIT Media Lab's Camera Culture research group, are low-cost eye care devices, a next generation CAT-Scan machine and human-computer interaction systems.

Papers: 

Andreas Velten, Thomas Willwacher, Otkrist Gupta, Ashok Veeraraghavan, Moungi G. Bawendi and Ramesh Raskar, “Recovering ThreeDimensional Shape around a Corner using Ultra-Fast Time-of-Flight Imaging.” Nature Communications, March 2012

Andreas Velten, Adrian Jarabo, Belen Masia, Di Wu, Christopher Barsi, Everett Lawson, Chinmaya Joshi, Diego Gutierrez, Moungi G. Bawendi and Ramesh Raskar, "Ultra-fast Imaging for Light in Motion" (in progress). http://femtocamera.info

More profile about the speaker
Ramesh Raskar | Speaker | TED.com