TED2015
Abe Davis: New video technology that reveals an object's hidden properties
亞伯 戴維斯: 揭示物體隱藏特質的新攝影科技
Filmed:
Readability: 4.3
1,482,525 views
細微的動作無時不刻在我們身邊所發生，包括由聲音產生的細小振動。 一個新的技術展現出通過搜集振動，我們可以從一個看起來靜止的物體中重現聲音和對話。 但是現在亞伯戴維斯更進一步：讓他在一段簡單影片中教你如何和這些隱而不顯的特質互動。
Abe Davis - Computer scientist
Computer vision expert Abe Davis pioneers methods to extract audio from silent digital videos, even footage shot on ordinary consumer cameras. Full bio
Computer vision expert Abe Davis pioneers methods to extract audio from silent digital videos, even footage shot on ordinary consumer cameras. Full bio
Double-click the English transcript below to play the video.
00:13
Most of us think of motion
as a very visual thing.
as a very visual thing.
0
1373
3349
絕大部分的人認為
「動作」是非常視覺的。
「動作」是非常視覺的。
00:17
If I walk across this stage
or gesture with my hands while I speak,
or gesture with my hands while I speak,
1
5889
5088
如果我走過這個舞台，
或在說話的時候使用手勢，
或在說話的時候使用手勢，
00:22
that motion is something that you can see.
2
10977
2261
這些動作都是你可以看見的。
00:26
But there's a world of important motion
that's too subtle for the human eye,
that's too subtle for the human eye,
3
14255
5482
但有一些細微的重要動作，
是人類的眼睛無法看見的。
是人類的眼睛無法看見的。
00:31
and over the past few years,
4
19737
2041
而在過去的幾年中，
00:33
we've started to find that cameras
5
21778
1997
我們發現鏡頭，
能夠看到這些人類所看不見的動作。
00:35
can often see this motion
even when humans can't.
even when humans can't.
6
23775
3410
00:40
So let me show you what I mean.
7
28305
1551
所以讓我來帶大家了解一下，
00:42
On the left here, you see video
of a person's wrist,
of a person's wrist,
8
30717
3622
在左邊，你可以看到一個人手腕的影片；
00:46
and on the right, you see video
of a sleeping infant,
of a sleeping infant,
9
34339
3147
在右邊，你可以看到一個睡著嬰兒的影片。
00:49
but if I didn't tell you
that these were videos,
that these were videos,
10
37486
3146
但是如果我沒有告訴你這是影片的話，
00:52
you might assume that you were looking
at two regular images,
at two regular images,
11
40632
3761
你有可能會認為
你只是在看兩張非常普通的圖片。
你只是在看兩張非常普通的圖片。
因為在這兩個情況之下，
00:56
because in both cases,
12
44393
1672
這些影片看起來只是完全靜止的。
00:58
these videos appear to be
almost completely still.
almost completely still.
13
46065
3047
01:02
But there's actually a lot
of subtle motion going on here,
of subtle motion going on here,
14
50175
3885
事實上這裏發生著許多細微的動作。
01:06
and if you were to touch
the wrist on the left,
the wrist on the left,
15
54060
2392
如果你觸碰左邊手腕的話，
01:08
you would feel a pulse,
16
56452
1996
你會感覺到脈搏。
01:10
and if you were to hold
the infant on the right,
the infant on the right,
17
58448
2485
如果你抱起這個右邊嬰兒的話，
01:12
you would feel the rise
and fall of her chest
and fall of her chest
18
60933
2391
你會感覺到她的胸部上下起伏，
01:15
as she took each breath.
19
63324
1390
當她正在呼吸的時候。
01:17
And these motions carry
a lot of significance,
a lot of significance,
20
65762
3576
這些動作十分重要，
01:21
but they're usually
too subtle for us to see,
too subtle for us to see,
21
69338
3343
但對我們來說太細微
以至於我們看不到。
以至於我們看不到。
01:24
so instead, we have to observe them
22
72681
2276
所以我們要用
01:26
through direct contact, through touch.
23
74957
2900
直接的觸碰去感知它們。
01:30
But a few years ago,
24
78997
1265
但是，幾年前，
01:32
my colleagues at MIT developed
what they call a motion microscope,
what they call a motion microscope,
25
80262
4405
我麻省理工的同事
研究出了動作顯微鏡。
研究出了動作顯微鏡。
01:36
which is software that finds
these subtle motions in video
these subtle motions in video
26
84667
4384
這軟體可用來尋找影片中的細微動作，
01:41
and amplifies them so that they
become large enough for us to see.
become large enough for us to see.
27
89051
3562
然後將它們放大使得我們可以看到。
01:45
And so, if we use their software
on the left video,
on the left video,
28
93416
3483
所以說，如果我們在左邊的影片上
使用那個軟體，
使用那個軟體，
01:48
it lets us see the pulse in this wrist,
29
96899
3250
它能讓我們看到腕部的脈搏。
01:52
and if we were to count that pulse,
30
100149
1695
而且如果我們數一數脈搏的話，
01:53
we could even figure out
this person's heart rate.
this person's heart rate.
31
101844
2355
我們甚至可以得出這個人的心率。
01:57
And if we used the same software
on the right video,
on the right video,
32
105095
3065
如果我們在右邊的影片上用同一軟體的話，
02:00
it lets us see each breath
that this infant takes,
that this infant takes,
33
108160
3227
它可以讓我們看到嬰兒的每一個呼吸。
02:03
and we can use this as a contact-free way
to monitor her breathing.
to monitor her breathing.
34
111387
4137
我們可以將這個軟體視為
不用接觸就能觀察嬰兒呼吸的探測器。
不用接觸就能觀察嬰兒呼吸的探測器。
02:08
And so this technology is really powerful
because it takes these phenomena
because it takes these phenomena
35
116884
5348
所以這種科技非常強大，
它記錄的現象
它記錄的現象
是原本我們得觸摸才能感受到的現象，
02:14
that we normally have
to experience through touch
to experience through touch
36
122232
2367
02:16
and it lets us capture them visually
and non-invasively.
and non-invasively.
37
124599
2957
而且它可以讓我們可視地、無創地觀察他們。
02:21
So a couple years ago, I started working
with the folks that created that software,
with the folks that created that software,
38
129104
4411
所以數年以前，我開始
和這些開發軟體的人一起工作，
和這些開發軟體的人一起工作，
02:25
and we decided to pursue a crazy idea.
39
133515
3367
然後我們決定去追尋一個瘋狂的主意。
02:28
We thought, it's cool
that we can use software
that we can use software
40
136882
2693
我們認為用這個軟體發現細小的動作
02:31
to visualize tiny motions like this,
41
139575
3135
是很酷炫的，
而且可以將它視為我們觸覺的延伸。
02:34
and you can almost think of it
as a way to extend our sense of touch.
as a way to extend our sense of touch.
42
142710
4458
02:39
But what if we could do the same thing
with our ability to hear?
with our ability to hear?
43
147168
4059
但是如果我們可以
做出擴展我們聽覺的軟體呢？
做出擴展我們聽覺的軟體呢？
02:44
What if we could use video
to capture the vibrations of sound,
to capture the vibrations of sound,
44
152508
4665
如果我們可以通過影片
從而獲得聲音的振動，
從而獲得聲音的振動，
02:49
which are just another kind of motion,
45
157173
2827
振動是另外一種動作，
02:52
and turn everything that we see
into a microphone?
into a microphone?
46
160000
3346
然後將我們看到的所有東西
轉化為聲音進入麥克風呢？
轉化為聲音進入麥克風呢？
02:56
Now, this is a bit of a strange idea,
47
164236
1971
這是一個有點奇怪的主意，
02:58
so let me try to put it
in perspective for you.
in perspective for you.
48
166207
2586
所以讓我將它變得更加易懂一些。
03:01
Traditional microphones
work by converting the motion
work by converting the motion
49
169523
3488
傳統的麥克風通過將內部隔膜的振動
03:05
of an internal diaphragm
into an electrical signal,
into an electrical signal,
50
173011
3599
轉換為電信號，
03:08
and that diaphragm is designed
to move readily with sound
to move readily with sound
51
176610
4318
設計讓隔膜隨著聲音方便移動。
03:12
so that its motion can be recorded
and interpreted as audio.
and interpreted as audio.
52
180928
4807
它的振動可以被記錄和轉換成聲音。
03:17
But sound causes all objects to vibrate.
53
185735
3668
但是聲音可以使任何物體產生振動。
03:21
Those vibrations are just usually
too subtle and too fast for us to see.
too subtle and too fast for us to see.
54
189403
5480
那些振動與我們來說太小太快，
以至於我們不能看不見。
以至於我們不能看不見。
03:26
So what if we record them
with a high-speed camera
with a high-speed camera
55
194883
3738
要是我們用高速錄影機記錄下振動，
03:30
and then use software
to extract tiny motions
to extract tiny motions
56
198621
3576
然後用軟體從高速錄影機的影片中
分離出細微的動作，
03:34
from our high-speed video,
57
202197
2090
03:36
and analyze those motions to figure out
what sounds created them?
what sounds created them?
58
204287
4274
然後分析那些動作
並且搞清楚是什麼聲音創造了振動呢？
並且搞清楚是什麼聲音創造了振動呢？
03:41
This would let us turn visible objects
into visual microphones from a distance.
into visual microphones from a distance.
59
209859
5449
這樣我們可以在一定距離內
將可視物體轉換到可視話筒中。
將可視物體轉換到可視話筒中。
03:49
And so we tried this out,
60
217080
2183
然後我們進行了實驗。
03:51
and here's one of our experiments,
61
219263
1927
這裏可以看到我們的實驗。
03:53
where we took this potted plant
that you see on the right
that you see on the right
62
221190
2949
在右邊我們放置了一盆盆栽，
03:56
and we filmed it with a high-speed camera
63
224139
2438
然後我們用高速錄影機記錄了下來。
03:58
while a nearby loudspeaker
played this sound.
played this sound.
64
226577
3529
同事在邊上用擴音器發出這個聲音。
04:02
(Music: "Mary Had a Little Lamb")
65
230275
8190
（音樂：“瑪麗有隻小綿羊”）
04:11
And so here's the video that we recorded,
66
239820
2824
然後下面是我們記錄下的聲音。
04:14
and we recorded it at thousands
of frames per second,
of frames per second,
67
242644
3924
我們每秒鐘記錄下上千次畫面，
04:18
but even if you look very closely,
68
246568
2322
但是就算你再仔細地看，
04:20
all you'll see are some leaves
69
248890
1951
你只會看到一些好看的樹葉，
04:22
that are pretty much
just sitting there doing nothing,
just sitting there doing nothing,
70
250841
3065
就只是靜止在那什麼也不做。
04:25
because our sound only moved those leaves
by about a micrometer.
by about a micrometer.
71
253906
4806
那是因為我們的聲音
只移動了樹葉大約一微米的距離，
只移動了樹葉大約一微米的距離，
04:31
That's one ten-thousandth of a centimeter,
72
259103
4276
那個距離大概是萬分之一釐米，
04:35
which spans somewhere between
a hundredth and a thousandth
a hundredth and a thousandth
73
263379
4156
在千分之一和百分之一之間，
04:39
of a pixel in this image.
74
267535
2299
只是這一個圖像的像素點。
04:41
So you can squint all you want,
75
269881
2887
所以你可以盡量瞇著眼睛看，
04:44
but motion that small is pretty much
perceptually invisible.
perceptually invisible.
76
272768
3335
但是細小的動作是不容易被感知到的。
04:49
But it turns out that something
can be perceptually invisible
can be perceptually invisible
77
277667
4157
但結果卻是一些不容易被看到的物體，
04:53
and still be numerically significant,
78
281824
2809
在數字上仍然非常重要。
04:56
because with the right algorithms,
79
284633
2002
因為當使用了正確的演算法之後，
04:58
we can take this silent,
seemingly still video
seemingly still video
80
286635
3687
我們可以獲取這段看起來靜止無聲的影片，
05:02
and we can recover this sound.
81
290322
1527
然後還原出聲音。
05:04
(Music: "Mary Had a Little Lamb")
82
292690
7384
（音樂：“瑪麗有隻小綿羊”）
05:12
(Applause)
83
300074
5828
（掌聲）
05:22
So how is this possible?
84
310058
1939
這是為什麼呢？
05:23
How can we get so much information
out of so little motion?
out of so little motion?
85
311997
4344
我們是如何在這細小的動作中
得到如此多訊息的呢？
得到如此多訊息的呢？
05:28
Well, let's say that those leaves
move by just a single micrometer,
move by just a single micrometer,
86
316341
5361
那麼讓我們假設這些樹葉
只是移動了一點點距離，
只是移動了一點點距離，
05:33
and let's say that that shifts our image
by just a thousandth of a pixel.
by just a thousandth of a pixel.
87
321702
4308
再者樹葉只是移動了
千分之一個像素的距離。
千分之一個像素的距離。
05:39
That may not seem like much,
88
327269
2572
那看起來並不多，
05:41
but a single frame of video
89
329841
1996
但是一個單一幀率的影片，
05:43
may have hundreds of thousands
of pixels in it,
of pixels in it,
90
331837
3257
可能有不計其數的像素。
05:47
and so if we combine all
of the tiny motions that we see
of the tiny motions that we see
91
335094
3454
所以如果我們將這些細小的動作
05:50
from across that entire image,
92
338548
2298
從整個畫面中截取出來的話，可以看到
05:52
then suddenly a thousandth of a pixel
93
340846
2623
一個像素的千分之一
05:55
can start to add up
to something pretty significant.
to something pretty significant.
94
343469
2775
可以累計變得十分重要。
05:58
On a personal note, we were pretty psyched
when we figured this out.
when we figured this out.
95
346870
3635
就我個人來說，
我們研究出來時高興得都要瘋了。
我們研究出來時高興得都要瘋了。
06:02
(Laughter)
96
350505
2320
（笑聲）
06:04
But even with the right algorithm,
97
352825
3253
但是就算擁有正確的算法，
06:08
we were still missing
a pretty important piece of the puzzle.
a pretty important piece of the puzzle.
98
356078
3617
我們仍然會丟失整個拼圖中最重要的部分。
06:11
You see, there are a lot of factors
that affect when and how well
that affect when and how well
99
359695
3604
你們知道有許多的因素
會對這個技術正常工作造成影響。
06:15
this technique will work.
100
363299
1997
06:17
There's the object and how far away it is;
101
365296
3204
這些因素包括，物體離得有多少遠、
06:20
there's the camera
and the lens that you use;
and the lens that you use;
102
368500
2394
拍攝的時候使用的鏡頭、
06:22
how much light is shining on the object
and how loud your sound is.
and how loud your sound is.
103
370894
4091
有多少光照在物體上，還有放出的聲音多響。
06:27
And even with the right algorithm,
104
375945
3375
而且就算擁有正確的算法，
06:31
we had to be very careful
with our early experiments,
with our early experiments,
105
379320
3390
我們在早期的試驗中必須十分小心。
06:34
because if we got
any of these factors wrong,
any of these factors wrong,
106
382710
2392
如果說我們弄錯了其中任何一個細節，
06:37
there was no way to tell
what the problem was.
what the problem was.
107
385102
2368
我們沒有辦法找出問題所在，
06:39
We would just get noise back.
108
387470
2647
只會得到一段噪音。
06:42
And so a lot of our early
experiments looked like this.
experiments looked like this.
109
390117
3320
所以我們早期的研究是像這樣的。
06:45
And so here I am,
110
393437
2206
這就是我。
06:47
and on the bottom left, you can kind of
see our high-speed camera,
see our high-speed camera,
111
395643
4040
在畫面的左下角可以看到我們的高速錄影機，
06:51
which is pointed at a bag of chips,
112
399683
2183
它正對著一包洋芋片，
06:53
and the whole thing is lit
by these bright lamps.
by these bright lamps.
113
401866
2949
所有的事物被一盞燈所照亮。
06:56
And like I said, we had to be
very careful in these early experiments,
very careful in these early experiments,
114
404815
4365
就像我說的，我們需要格外小心。
07:01
so this is how it went down.
115
409180
2508
這就是這個試驗如何進行的。
07:03
(Video) Abe Davis: Three, two, one, go.
116
411688
3761
（影片）亞伯戴維斯：三二一開始。
07:07
Mary had a little lamb!
Little lamb! Little lamb!
Little lamb! Little lamb!
117
415449
5387
瑪麗有隻小綿羊
小綿羊！小綿羊！
小綿羊！小綿羊！
07:12
(Laughter)
118
420836
4500
（笑聲）
07:17
AD: So this experiment
looks completely ridiculous.
looks completely ridiculous.
119
425336
2814
亞伯戴維斯：所以這個實驗看起來十分可笑。
07:20
(Laughter)
120
428150
1788
（笑聲）
07:21
I mean, I'm screaming at a bag of chips --
121
429938
2345
我對著一袋洋芋片尖叫。
07:24
(Laughter) --
122
432283
1551
（笑聲）
07:25
and we're blasting it with so much light,
123
433834
2117
我們在充足的光照下對著它大叫，
07:27
we literally melted the first bag
we tried this on. (Laughter)
we tried this on. (Laughter)
124
435951
4479
我們確實將第一個實驗的洋芋融化了。（笑聲）
07:32
But ridiculous as this experiment looks,
125
440525
3274
儘管看上去很可笑，
07:35
it was actually really important,
126
443799
1788
這確實是十分重要，
07:37
because we were able
to recover this sound.
to recover this sound.
127
445587
2926
因為我們可以復原這個聲音。
07:40
(Audio) Mary had a little lamb!
Little lamb! Little lamb!
Little lamb! Little lamb!
128
448513
4712
（聲音）瑪麗有隻小綿羊，小綿羊！小綿羊！
07:45
(Applause)
129
453225
4088
（掌聲）
07:49
AD: And this was really significant,
130
457313
1881
這是十分重要的。
07:51
because it was the first time
we recovered intelligible human speech
we recovered intelligible human speech
131
459194
4119
因為這是第一次
我們從一個物體靜止的影像中，
我們從一個物體靜止的影像中，
07:55
from silent video of an object.
132
463424
2341
復原出了清楚的人聲。
07:57
And so it gave us this point of reference,
133
465765
2391
所以這給了我們一個參考，
08:00
and gradually we could start
to modify the experiment,
to modify the experiment,
134
468156
3871
並且可以逐漸去修改這個實驗。
08:04
using different objects
or moving the object further away,
or moving the object further away,
135
472106
3805
用不同的物體或者把物體移到更遠的地方，
08:07
using less light or quieter sounds.
136
475911
2770
或者使用少量的光和更加輕的聲音。
08:11
And we analyzed all of these experiments
137
479887
2874
我們分析了實驗，
08:14
until we really understood
the limits of our technique,
the limits of our technique,
138
482761
3622
直到我們弄清楚我們技術的侷限性在哪，
08:18
because once we understood those limits,
139
486383
1950
因為只要我們明白它們的極限，
08:20
we could figure out how to push them.
140
488333
2346
就可以知道如何去推動它們。
08:22
And that led to experiments like this one,
141
490679
3181
我們的實驗就可能變成這一個，
08:25
where again, I'm going to speak
to a bag of chips,
to a bag of chips,
142
493860
2739
我在同一個地方
再一次向一包洋芋片說話，
再一次向一包洋芋片說話，
08:28
but this time we've moved our camera
about 15 feet away,
about 15 feet away,
143
496599
4830
但是這一次我們把攝影機往後移動了15英尺，
08:33
outside, behind a soundproof window,
144
501429
2833
放置在隔音玻璃後面，
08:36
and the whole thing is lit
by only natural sunlight.
by only natural sunlight.
145
504262
2803
所有的東西僅僅是被太陽光所照亮。
08:40
And so here's the video that we captured.
146
508529
2155
這是我們拍攝到的影片。
08:44
And this is what things sounded like
from inside, next to the bag of chips.
from inside, next to the bag of chips.
147
512450
4559
聽起來這個聲音是從洋芋片內部發出來的。
08:49
(Audio) Mary had a little lamb
whose fleece was white as snow,
whose fleece was white as snow,
148
517009
5038
（聲音）瑪麗有隻小綿羊，
牠的毛白得像雪一樣。
牠的毛白得像雪一樣。
08:54
and everywhere that Mary went,
that lamb was sure to go.
that lamb was sure to go.
149
522047
5619
並且瑪麗走到哪裏，
小綿羊就跟到哪裏。
小綿羊就跟到哪裏。
08:59
AD: And here's what we were able
to recover from our silent video
to recover from our silent video
150
527666
4017
這是我們能夠通過在玻璃外面捕捉的
09:03
captured outside behind that window.
151
531683
2345
靜止影像中還原出來的。
09:06
(Audio) Mary had a little lamb
whose fleece was white as snow,
whose fleece was white as snow,
152
534028
4435
（聲音）瑪麗有隻小綿羊，
牠的毛白得像雪一樣。
牠的毛白得像雪一樣。
09:10
and everywhere that Mary went,
that lamb was sure to go.
that lamb was sure to go.
153
538463
5457
並且瑪麗走到哪裏，
小綿羊就跟到哪裏。
小綿羊就跟到哪裏。
09:15
(Applause)
154
543920
6501
（掌聲）
09:22
AD: And there are other ways
that we can push these limits as well.
that we can push these limits as well.
155
550421
3542
還有其他辦法去推動這些限制，
09:25
So here's a quieter experiment
156
553963
1798
所以下面是一個更安靜的實驗。
09:27
where we filmed some earphones
plugged into a laptop computer,
plugged into a laptop computer,
157
555761
4110
我們拍攝了一些插在電腦上的耳機。
09:31
and in this case, our goal was to recover
the music that was playing on that laptop
the music that was playing on that laptop
158
559871
4110
我們的目標是還原出
在手提電腦上所放出的聲音，
在手提電腦上所放出的聲音，
09:35
from just silent video
159
563981
2299
從這兩個小耳機中的
09:38
of these two little plastic earphones,
160
566280
2507
從靜止影片中得到。
09:40
and we were able to do this so well
161
568787
2183
並且我們可以做得很好，
09:42
that I could even Shazam our results.
162
570970
2461
甚至能夠用聽歌識曲軟體鑑別我們的結果。
09:45
(Laughter)
163
573431
2411
（笑聲）
09:49
(Music: "Under Pressure" by Queen)
164
577191
10034
（音樂：皇后樂隊 “壓力之下”）
10:01
(Applause)
165
589615
4969
（掌聲）
10:06
And we can also push things
by changing the hardware that we use.
by changing the hardware that we use.
166
594584
4551
我們也可以通過改變硬體來推動事物。
10:11
Because the experiments
I've shown you so far
I've shown you so far
167
599135
2461
我給你們看的這些實驗
10:13
were done with a camera,
a high-speed camera,
a high-speed camera,
168
601596
2322
都使用了攝影機，高速攝影機，
10:15
that can record video
about a 100 times faster
about a 100 times faster
169
603918
2879
我們可以比大多數手機
10:18
than most cell phones,
170
606797
1927
快一百倍地記錄影片。
10:20
but we've also found a way
to use this technique
to use this technique
171
608724
2809
但是我們也找到了用普通攝影機
10:23
with more regular cameras,
172
611533
2230
使用這一項技術的方法。
10:25
and we do that by taking advantage
of what's called a rolling shutter.
of what's called a rolling shutter.
173
613763
4069
我們採用普通照相機
像百葉窗一樣記錄東西優點來記錄。
像百葉窗一樣記錄東西優點來記錄。
10:29
You see, most cameras
record images one row at a time,
record images one row at a time,
174
617832
4798
你們知道，大多數照相機
一段時間記錄一排的圖像，
一段時間記錄一排的圖像，
10:34
and so if an object moves
during the recording of a single image,
during the recording of a single image,
175
622630
5702
所以如果一個物體
只在被記錄的圖像中移動，
只在被記錄的圖像中移動，
10:40
there's a slight time delay
between each row,
between each row,
176
628344
2717
在記錄的每一排之間幾乎沒有延遲。
10:43
and this causes slight artifacts
177
631061
3157
這樣就可以使物體
10:46
that get coded into each frame of a video.
178
634218
3483
被記錄到影片的每一個部分之中。
10:49
And so what we found
is that by analyzing these artifacts,
is that by analyzing these artifacts,
179
637701
3806
我們發現透過分析這些東西，
10:53
we can actually recover sound
using a modified version of our algorithm.
using a modified version of our algorithm.
180
641507
4615
實際上只是用了更改過的算法來還原出聲音。
10:58
So here's an experiment we did
181
646122
1912
下面就是我們所做的實驗。
11:00
where we filmed a bag of candy
182
648034
1695
我們拍攝了一袋糖，
11:01
while a nearby loudspeaker played
183
649729
1741
同時邊上有一個擴音器正在播放著
11:03
the same "Mary Had a Little Lamb"
music from before,
music from before,
184
651470
2972
與之前相同的“瑪麗有隻小綿羊”。
11:06
but this time, we used just a regular
store-bought camera,
store-bought camera,
185
654442
4203
但是這一次我們僅使用從商店買來的攝影機。
11:10
and so in a second, I'll play for you
the sound that we recovered,
the sound that we recovered,
186
658645
3174
馬上我就向你們播放我們還原出的聲音。
11:13
and it's going to sound
distorted this time,
distorted this time,
187
661819
2050
這一次聲音聽起來有一些扭曲，
11:15
but listen and see if you can still
recognize the music.
recognize the music.
188
663869
2836
但是請聽聽看能否分辨出這音樂。
11:19
(Audio: "Mary Had a Little Lamb")
189
667723
6223
（聲音：“瑪麗有隻小綿羊”）
11:37
And so, again, that sounds distorted,
190
685527
3465
聲音確實被扭曲了，
11:40
but what's really amazing here
is that we were able to do this
is that we were able to do this
191
688992
4386
但是神奇的是，我們能夠做這個事情，
11:45
with something
that you could literally run out
that you could literally run out
192
693378
2626
運用一些用完以後
11:48
and pick up at a Best Buy.
193
696004
1444
就可以在 Best Buy 買到的東西。
11:51
So at this point,
194
699122
1363
所以在這時
11:52
a lot of people see this work,
195
700485
1974
很多人可以看到我們的研究結果，
11:54
and they immediately think
about surveillance.
about surveillance.
196
702459
3413
然後他們立刻會想到監視。
11:57
And to be fair,
197
705872
2415
公平的說，
12:00
it's not hard to imagine how you might use
this technology to spy on someone.
this technology to spy on someone.
198
708287
4133
不難想到你們可以用這項技術去監視其他人。
12:04
But keep in mind that there's already
a lot of very mature technology
a lot of very mature technology
199
712420
3947
但是要記住早就有許多成熟的技術
12:08
out there for surveillance.
200
716367
1579
為監視所準備。
12:09
In fact, people have been using lasers
201
717946
2090
事實上，人們數十年來使用雷射
12:12
to eavesdrop on objects
from a distance for decades.
from a distance for decades.
202
720036
2799
去竊聽別的事物。
12:15
But what's really new here,
203
723978
2025
但是，這個技術新穎的地方、
12:18
what's really different,
204
726003
1440
完全不同的地方，
12:19
is that now we have a way
to picture the vibrations of an object,
to picture the vibrations of an object,
205
727443
4295
是我們現在有辦法拍攝出物體的振動。
12:23
which gives us a new lens
through which to look at the world,
through which to look at the world,
206
731738
3413
讓我們獲得了觀察這個世界的新鏡頭，
12:27
and we can use that lens
207
735151
1510
並且可以使用這鏡頭，
12:28
to learn not just about forces like sound
that cause an object to vibrate,
that cause an object to vibrate,
208
736661
4899
不僅僅是為了去瞭解導致物體振動的聲音，
12:33
but also about the object itself.
209
741560
2288
還瞭解了物體本身。
12:36
And so I want to take a step back
210
744975
1693
所以我想往回退一步
12:38
and think about how that might change
the ways that we use video,
the ways that we use video,
211
746668
4249
去思考這個技術會如何改變
我們應用影片的方法。
我們應用影片的方法。
12:42
because we usually use video
to look at things,
to look at things,
212
750917
3553
因為我們用影片通常來看一些東西，
12:46
and I've just shown you how we can use it
213
754470
2322
並且我剛才已經展示如何使用它
12:48
to listen to things.
214
756792
1857
去聽一些東西。
12:50
But there's another important way
that we learn about the world:
that we learn about the world:
215
758649
3971
但是另外有一個我們瞭解世界的重要方法，
12:54
that's by interacting with it.
216
762620
2275
那就是和它互動。
12:56
We push and pull and poke and prod things.
217
764895
3111
我們推、拉、戳、刺一些事物，
13:00
We shake things and see what happens.
218
768006
3181
我們搖動物體來明白發生了什麼事。
13:03
And that's something that video
still won't let us do,
still won't let us do,
219
771187
4273
那是影片無法做到的。
13:07
at least not traditionally.
220
775460
2136
至少過去不行。
13:09
So I want to show you some new work,
221
777596
1950
所以我想向你們展示一些新的成品，
13:11
and this is based on an idea I had
just a few months ago,
just a few months ago,
222
779546
2667
這源自我幾個月之前的想法，
13:14
so this is actually the first time
I've shown it to a public audience.
I've shown it to a public audience.
223
782213
3301
所以這是我第一次公眾展示。
13:17
And the basic idea is that we're going
to use the vibrations in a video
to use the vibrations in a video
224
785514
5363
而且基本的想法就是
我們會用影片之中的振動，
我們會用影片之中的振動，
13:22
to capture objects in a way
that will let us interact with them
that will let us interact with them
225
790877
4481
來捕捉物體
在某種程度上這樣可以使我們與物體互動，
在某種程度上這樣可以使我們與物體互動，
13:27
and see how they react to us.
226
795358
1974
並且可以知道它們如何對我們進行反應。
13:31
So here's an object,
227
799120
1764
這是一個物體。
13:32
and in this case, it's a wire figure
in the shape of a human,
in the shape of a human,
228
800884
3832
這是一個用線做成的人。
13:36
and we're going to film that object
with just a regular camera.
with just a regular camera.
229
804716
3088
我們要用普通的相機去拍攝它，
所以這個相機沒有什麼特別之處。
13:39
So there's nothing special
about this camera.
about this camera.
230
807804
2124
事實上我曾經用我的手機完成過這件事
13:41
In fact, I've actually done this
with my cell phone before.
with my cell phone before.
231
809928
2961
13:44
But we do want to see the object vibrate,
232
812889
2252
但是我們確實希望這個物體振動。
13:47
so to make that happen,
233
815141
1133
所以為了做到這點，
13:48
we're just going to bang a little bit
on the surface where it's resting
on the surface where it's resting
234
816274
3346
我們在拍攝的時候
13:51
while we record this video.
235
819620
2138
在它放置的地方敲擊。
13:59
So that's it: just five seconds
of regular video,
of regular video,
236
827398
3671
這就是全部了：一個僅僅五秒鐘的普通影片，
14:03
while we bang on this surface,
237
831069
2136
拍攝我們敲擊表面的時候。
14:05
and we're going to use
the vibrations in that video
the vibrations in that video
238
833205
3513
我們將要用這個影片的震動
14:08
to learn about the structural
and material properties of our object,
and material properties of our object,
239
836718
4544
去瞭解這個物體的結構和組織組成，
14:13
and we're going to use that information
to create something new and interactive.
to create something new and interactive.
240
841262
4834
然後使用這個訊息
去創造新穎和互動性的東西。
去創造新穎和互動性的東西。
14:24
And so here's what we've created.
241
852866
2653
這就是我們所創造的。
14:27
And it looks like a regular image,
242
855519
2229
這看起來像一個普通的圖片，
14:29
but this isn't an image,
and it's not a video,
and it's not a video,
243
857748
3111
但是這不是圖片也不是影片。
14:32
because now I can take my mouse
244
860859
2368
因為我可以用我的游標，
14:35
and I can start interacting
with the object.
with the object.
245
863227
2859
也可以和我這個物體互動。
14:44
And so what you see here
246
872936
2357
所以你們看到的
14:47
is a simulation of how this object
247
875389
2226
是一個我們從來沒看到過的，
14:49
would respond to new forces
that we've never seen before,
that we've never seen before,
248
877615
4458
關於物體如何對新的力量進行反應。
14:54
and we created it from just
five seconds of regular video.
five seconds of regular video.
249
882073
3633
我們只是使用了五秒鐘的影片。
14:59
(Applause)
250
887249
4715
（掌聲）
15:09
And so this is a really powerful
way to look at the world,
way to look at the world,
251
897421
3227
所以這是個十分有力的看世界的方法，
15:12
because it lets us predict
how objects will respond
how objects will respond
252
900648
2972
讓我們能推測物體是如何在新環境
15:15
to new situations,
253
903620
1823
做出反應的。
15:17
and you could imagine, for instance,
looking at an old bridge
looking at an old bridge
254
905443
3473
並且可以想像，例如看一個古老的橋樑，
15:20
and wondering what would happen,
how would that bridge hold up
how would that bridge hold up
255
908916
3527
並思考開車經過那座橋時
15:24
if I were to drive my car across it.
256
912443
2833
它會如何支撐住。
15:27
And that's a question
that you probably want to answer
that you probably want to answer
257
915276
2774
那是一個在你開車穿過之前，
15:30
before you start driving
across that bridge.
across that bridge.
258
918050
2560
你會想先知道答案的問題。
15:33
And of course, there are going to be
limitations to this technique,
limitations to this technique,
259
921988
3272
的確，這項技術還是有侷限性的，
15:37
just like there were
with the visual microphone,
with the visual microphone,
260
925260
2462
就像視覺麥克風也有缺陷一樣，
15:39
but we found that it works
in a lot of situations
in a lot of situations
261
927722
3181
但是我們發現它適用於許多情況，
15:42
that you might not expect,
262
930903
1875
你可能沒有想到的，
15:44
especially if you give it longer videos.
263
932778
2768
尤其是拍攝更長影片的時候。
15:47
So for example,
here's a video that I captured
here's a video that I captured
264
935546
2508
例如，這是我拍的一段影片，
15:50
of a bush outside of my apartment,
265
938054
2299
是我公寓外的灌木叢。
15:52
and I didn't do anything to this bush,
266
940353
3088
我沒有對這灌木叢做什麼事，
15:55
but by capturing a minute-long video,
267
943441
2705
但是透過拍攝一段一分鐘的影片，
15:58
a gentle breeze caused enough vibrations
268
946146
3378
一陣輕風可以產生足夠的振動，
16:01
that we could learn enough about this bush
to create this simulation.
to create this simulation.
269
949524
3587
我們可以足夠地瞭解這個灌木叢
從而創造出這樣的模擬情況。
從而創造出這樣的模擬情況。
16:07
(Applause)
270
955270
6142
（掌聲）
16:13
And so you could imagine giving this
to a film director,
to a film director,
271
961412
2972
所以你可以想像將這個技術給一個電影導演，
16:16
and letting him control, say,
272
964384
1719
讓他來控制
16:18
the strength and direction of wind
in a shot after it's been recorded.
in a shot after it's been recorded.
273
966103
4922
影片拍攝完後的風力強度和方向。
16:24
Or, in this case, we pointed our camera
at a hanging curtain,
at a hanging curtain,
274
972810
4535
我們也將相機指向了一個掛著的窗簾，
16:29
and you can't even see
any motion in this video,
any motion in this video,
275
977345
4129
你幾乎看不到影片中有任何動作，
16:33
but by recording a two-minute-long video,
276
981474
2925
但是拍攝兩分鐘的影片後，
16:36
natural air currents in this room
277
984399
2438
在這個房間中的天然氣流
16:38
created enough subtle,
imperceptible motions and vibrations
imperceptible motions and vibrations
278
986837
4412
創造了足夠細微、不可被察覺的動作和振動，
16:43
that we could learn enough
to create this simulation.
to create this simulation.
279
991249
2565
這樣我們也可以透過振動製造出模擬。
16:48
And ironically,
280
996243
2366
可笑的是，
16:50
we're kind of used to having
this kind of interactivity
this kind of interactivity
281
998609
3088
我們只是在
16:53
when it comes to virtual objects,
282
1001697
2647
虛擬的物體上，
16:56
when it comes to video games
and 3D models,
and 3D models,
283
1004344
3297
電視遊戲和3D模型中使用這種互動。
16:59
but to be able to capture this information
from real objects in the real world
from real objects in the real world
284
1007641
4404
但是僅僅使用簡單普通的影片
17:04
using just simple, regular video,
285
1012045
2817
去捕捉現實世界中的真實物體，
17:06
is something new that has
a lot of potential.
a lot of potential.
286
1014862
2183
仍然有很大的潛力。
17:10
So here are the amazing people
who worked with me on these projects.
who worked with me on these projects.
287
1018410
4904
這裏有許多傑出的人
與我共同研究這些計劃。
與我共同研究這些計劃。
17:16
(Applause)
288
1024057
5596
（掌聲）
17:24
And what I've shown you today
is only the beginning.
is only the beginning.
289
1032819
3057
我今天展示給你們看的只是個開始。
17:27
We've just started to scratch the surface
290
1035876
2113
我們僅僅開始挖出表面的一部分，
17:29
of what you can do
with this kind of imaging,
with this kind of imaging,
291
1037989
2972
看看這樣的成像技術能做到什麼事。
17:32
because it gives us a new way
292
1040961
2286
因為它給了我們一個新的方法
17:35
to capture our surroundings
with common, accessible technology.
with common, accessible technology.
293
1043342
4724
透過平常可得到的技術去捕捉周圍的東西。
17:40
And so looking to the future,
294
1048066
1929
所以展望未來，
17:41
it's going to be
really exciting to explore
really exciting to explore
295
1049995
2037
探索這個技術
可以告訴我們關於這個世界
可以告訴我們關於這個世界
17:44
what this can tell us about the world.
296
1052032
1856
會變得格外激動人心。
17:46
Thank you.
297
1054381
1204
謝謝。
17:47
(Applause)
298
1055610
6107
（掌聲）
ABOUT THE SPEAKER
Abe Davis - Computer scientistComputer vision expert Abe Davis pioneers methods to extract audio from silent digital videos, even footage shot on ordinary consumer cameras.
Why you should listen
MIT PhD student, computer vision wizard and rap artist Abe Davis has co-created the world’s most improbable audio instrument. In 2014, Davis and his collaborators debuted the “visual microphone,” an algorithm that samples the sympathetic vibrations of ordinary objects (such as a potato chip bag) from ordinary high-speed video footage and transduces them into intelligible audio tracks.
Davis is also the author of Caperture, a 3D-imaging app designed to create and share 3D images on any compatible smartphone.
Abe Davis | Speaker | TED.com