sponsored links
TEDxBeaconStreet

Michael Rubinstein: See invisible motion, hear silent sounds

マイケル・ルービンスタイン: 見えない動きを見、聞こえない音を聞く。すごい? それとも気味が悪い?

November 13, 2014

肉眼では見られない微細な動きや色の変化を拡大する「顕微鏡」をご紹介します。画像技術者マイケル・ルービンスタインは、驚くばかりの映像を次から次へと見せ、この技術によって普通の映像から人の心拍を読み取ったり、音波により生じるポテトチップ袋の振動を拡大して会話を再現できることを示します。恐ろしい応用の可能性も秘めたこの驚きの技術は、実際に目にするまで信じられないかもしれません。

Michael Rubinstein - Research scientist, Google
Computer scientist Michael Rubinstein and his team have developed a "motion microscope" that can show video footage of barely perceivable movements, like breaths and heartbeats. Full bio

sponsored links
Double-click the English subtitles below to play the video.
So over the past few centuries,
microscopes have revolutionized our world.
過去数世紀に渡って
顕微鏡は世界を変えてきました
00:12
They revealed to us a tiny world
of objects, life and structures
小さすぎて肉眼では見えない
物や生物や構造の世界を
00:20
that are too small for us
to see with our naked eyes.
顕微鏡が明らかにし
00:26
They are a tremendous contribution
to science and technology.
科学や技術に対して
大いなる貢献をしました
00:28
Today I'd like to introduce you
to a new type of microscope,
今日ご紹介したいのは
新しいタイプの顕微鏡
00:31
a microscope for changes.
「変化を見る顕微鏡」です
00:35
It doesn't use optics
like a regular microscope
普通の顕微鏡のように
光学的に小さなものを大きく
00:37
to make small objects bigger,
見せるのではなく
00:40
but instead it uses a video camera
and image processing
ビデオと画像処理を使って
00:42
to reveal to us the tiniest motions
and color changes in objects and people,
肉眼では見えないような
人や物の微細な動きや
00:47
changes that are impossible
for us to see with our naked eyes.
色の変化を
見えるようにします
00:52
And it lets us look at our world
in a completely new way.
これは世界に対する
まったく別の見方を与えてくれます
00:56
So what do I mean by color changes?
色の変化とは
どんなものかですが
01:00
Our skin, for example,
changes its color very slightly
例えば人の肌というのは
血の流れに応じて
01:02
when the blood flows under it.
色がかすかに変化します
01:05
That change is incredibly subtle,
これはとても
微妙な変化であるため
01:07
which is why, when you
look at other people,
隣に座っている人を
見たところで
01:09
when you look at the person
sitting next to you,
肌や顔の色が
01:11
you don't see their skin
or their face changing color.
変わっているようには
見えません
01:13
When we look at this video of Steve here,
it appears to us like a static picture,
このスティーブの映像を見ても
静止画のように見えます
01:17
but once we look at this video
through our new, special microscope,
しかし私達の新しい顕微鏡を
通して見ると
01:21
suddenly we see
a completely different image.
まったく異なる
イメージが現れます
01:25
What you see here are small changes
in the color of Steve's skin,
肌の色の小さな変化を
100倍増幅することで
01:28
magnified 100 times
so that they become visible.
目で見て分かる
ようにしています
01:32
We can actually see a human pulse.
脈拍を見て取る
こともできます
01:36
We can see how fast
Steve's heart is beating,
心拍の早さだけでなく
01:39
but we can also see the actual way
that the blood flows in his face.
顔を血がどう流れているかも
分かります
01:43
And we can do that not just
to visualize the pulse,
脈拍を可視化
できるだけでなく
01:48
but also to actually
recover our heart rates,
心拍数を正確に
01:50
and measure our heart rates.
計測することもできます
01:54
And we can do it with regular cameras
and without touching the patients.
普通のカメラでできて
患者に触れる必要もありません
01:56
So here you see the pulse and heart rate
we extracted from a neonatal baby
ここでは普通のDSLRカメラで撮った
新生児の映像から
02:00
from a video we took
with a regular DSLR camera,
脈拍と心拍数を
取り出しています
02:06
and the heart rate measurement we get
これで計測した心拍数は
02:09
is as accurate as the one you'd get
with a standard monitor in a hospital.
病院にある通常の計器によるのと
同様の正確さがあります
02:11
And it doesn't even have to be
a video we recorded.
映像も自分で撮ったもの
である必要はなく
02:15
We can do it essentially
with other videos as well.
既存のビデオを使うこともできます
02:18
So I just took a short clip
from "Batman Begins" here
これは『バットマン ビギンズ』の一場面から
02:21
just to show Christian Bale's pulse.
クリスチャン・ベールの心拍が
見えるようにしたものです(笑)
02:25
(Laughter)
クリスチャン・ベールの心拍が
見えるようにしたものです(笑)
02:27
And you know, presumably
he's wearing makeup,
映画なのでメークも
しているだろうし
02:29
the lighting here is kind of challenging,
光の条件にも
難しい面がありますが
02:31
but still, just from the video,
we're able to extract his pulse
それでも映像から
彼の心拍を
02:33
and show it quite well.
非常にうまく
取り出せています
02:36
So how do we do all that?
どうやっているのかですが
02:38
We basically analyze the changes
in the light that are recorded
ビデオのそれぞれの
ピクセルに記録された
02:40
at every pixel in the video over time,
光の時間的変化を分析し
02:44
and then we crank up those changes.
その変化を拡大しています
02:46
We make them bigger
so that we can see them.
変化が見て分かるくらいに
大きくするわけです
02:48
The tricky part is that those signals,
難しいのは
捉えたい変化が
02:50
those changes that we're after,
are extremely subtle,
非常に小さなものだ
ということで
02:52
so we have to be very careful
when you try to separate them
その変化を
録画につきもののノイズから
02:55
from noise that always exists in videos.
注意深く分離する
必要があります
02:58
So we use some clever
image processing techniques
それぞれのピクセルの
ごく正確な色を得るために
03:02
to get a very accurate measurement
of the color at each pixel in the video,
巧妙な画像処理を
行っています
03:05
and then the way the color
changes over time,
それから色の
時間変化の仕方を捉え
03:09
and then we amplify those changes.
それを拡大して
03:11
We make them bigger to create those types
of enhanced videos, or magnified videos,
変化が目で見て分かるよう
変化の強調された
03:14
that actually show us those changes.
映像を作ります
03:18
But it turns out we can do that
not just to show tiny changes in color,
このようにして見えるようにできるものには
微細な色の変化だけでなく
03:20
but also tiny motions,
微細な動きもあります
03:25
and that's because the light
that gets recorded in our cameras
カメラに記録される光は
色の変化によってだけでなく
03:27
will change not only if the color
of the object changes,
物の動きによっても
03:30
but also if the object moves.
変化するからです
03:33
So this is my daughter
when she was about two months old.
これは生後2ヶ月の頃の
私の娘です
03:36
It's a video I recorded
about three years ago.
3年ほど前に
録画したものです
03:39
And as new parents, we all want
to make sure our babies are healthy,
親になったばかりの人は
赤ちゃんが健康か 息をしているか
03:42
that they're breathing,
that they're alive, of course.
生きているか
いつも気にかけています
03:45
So I too got one of those baby monitors
私も娘の眠っている姿を
見られるよう
03:48
so that I could see my daughter
when she was asleep.
ベビーモニターを
買いました
03:50
And this is pretty much what you'll see
with a standard baby monitor.
普通のベビーモニターで見られるのは
このような映像です
03:53
You can see the baby's sleeping, but
there's not too much information there.
眠っている様子は分かりますが
情報は大して得られません
03:56
There's not too much we can see.
見て分かる事は
殆どありません
04:00
Wouldn't it be better,
or more informative, or more useful,
もしこんな風に
見えたとしたら
04:01
if instead we could look
at the view like this.
もっと情報が得られて
有用ではないでしょうか?
04:04
So here I took the motions
and I magnified them 30 times,
動きを30倍拡大することで
娘の動きがはっきり見て取れるようになりました
04:07
and then I could clearly see that my
daughter was indeed alive and breathing.
これで娘が確かに生きて
呼吸しているのが分かります
04:14
(Laughter)
(笑)
04:17
Here is a side-by-side comparison.
並べて比較したところですが
04:20
So again, in the source video,
in the original video,
元々のビデオでは
04:22
there's not too much we can see,
動きが分かりません
04:24
but once we magnify the motions,
the breathing becomes much more visible.
しかし動きを拡大した映像では
呼吸の様子がよく分かります
04:26
And it turns out, there's
a lot of phenomena
この「変化を見る顕微鏡」によって
04:29
we can reveal and magnify
with our new motion microscope.
明らかにできる身の回りの現象は
たくさんあります
04:31
We can see how our veins and arteries
are pulsing in our bodies.
体の中で静脈や動脈が
どう脈打っているか分かります
04:35
We can see that our eyes
are constantly moving
目が絶えずユラユラ
動いていることも
04:40
in this wobbly motion.
よく分かります
04:42
And that's actually my eye,
これは私の目で
04:44
and again this video was taken
right after my daughter was born,
娘が生まれた頃に
撮ったので
04:46
so you can see I wasn't getting
too much sleep. (Laughter)
あまり寝ていないのが
分かるかと思います(笑)
04:49
Even when a person is sitting still,
じっと座っている人からでさえ
04:53
there's a lot of information
we can extract
多くの情報が得られます
04:56
about their breathing patterns,
small facial expressions.
呼吸のパターンとか
小さな顔の表情とか
04:58
Maybe we could use those motions
このような動きから
05:01
to tell us something about
our thoughts or our emotions.
その人の思っていることや
感情も分かるかもしれません
05:03
We can also magnify small
mechanical movements,
エンジンの振動のような
小さな機械の動きも
05:06
like vibrations in engines,
拡大して見えるようにできます
05:09
that can help engineers detect
and diagnose machinery problems,
機械の問題の検出や診断を技術者がするのに
役立つかもしれません
05:11
or see how our buildings and structures
sway in the wind and react to forces.
建物や構造物が風に揺れたり
反発したりする様子も見て取れます
05:15
Those are all things that our society
knows how to measure in various ways,
こういった動きを計測する方法なら
以前からありましたが
05:19
but measuring those motions is one thing,
その動いている様子を
05:24
and actually seeing those
motions as they happen
実際に目で見えるようにする
というのは
05:26
is a whole different thing.
また別の話になります
05:29
And ever since we discovered
this new technology,
私達はこの技術を開発して以来
05:31
we made our code available online so that
others could use and experiment with it.
ネット上でプログラムを公開して
誰でも実験できるようにしています
05:34
It's very simple to use.
とても簡単に使え
05:38
It can work on your own videos.
自分のビデオで
試すことができます
05:40
Our collaborators at Quanta Research
even created this nice website
私達の協力者のQuanta Researchは
ご覧のようなサイトも用意していて
05:42
where you can upload your videos
and process them online,
ビデオをアップするだけで
結果を見られます
05:45
so even if you don't have any experience
in computer science or programming,
だからコンピュータサイエンスや
プログラミングの知識がまったくなくても
05:48
you can still very easily experiment
with this new microscope.
簡単にこの顕微鏡で
実験ができます
05:52
And I'd like to show you
just a couple of examples
これを使ってみんなが
どんなことをしているのか
05:55
of what others have done with it.
いくつかご覧に入れましょう
05:57
So this video was made by
a YouTube user called Tamez85.
このビデオはTomez85という
YouTubeユーザーが作ったもので
06:00
I don't know who that user is,
どういう人なのか知りませんが
06:05
but he, or she, used our code
私達のプログラムを使って
06:07
to magnify small belly
movements during pregnancy.
妊婦のお腹の動きを
拡大しています
06:09
It's kind of creepy.
ちょっと不気味ですね
06:13
(Laughter)
(笑)
06:14
People have used it to magnify
pulsing veins in their hands.
ここでは手の静脈の拍動を
拡大しています
06:16
And you know it's not real science
unless you use guinea pigs,
しかしモルモットを使わなくちゃ
科学っぽくなりませんよね
06:21
and apparently this guinea pig
is called Tiffany,
このモルモットは
ティファニーという名前で
06:25
and this YouTube user claims
it is the first rodent on Earth
作者はこれが
微細な動きを拡大された
06:28
that was motion-magnified.
最初の齧歯類だと
主張しています
06:31
You can also do some art with it.
美術作品を作ることもできます
06:34
So this video was sent to me
by a design student at Yale.
イェール大のデザイン科の学生が
送ってきたもので
06:36
She wanted to see
if there's any difference
友人の身動きの仕方に
違いがあるか
06:39
in the way her classmates move.
知りたかったのだそうです
06:41
She made them all stand still,
and then magnified their motions.
じっとしているように頼んで
それから動きを拡大したものです
06:42
It's like seeing
still pictures come to life.
写真が動き始めたみたいな
感じがします
06:47
And the nice thing with
all those examples
これらの例の良いところは
06:50
is that we had nothing to do with them.
我々自身何もする必要が
なかったことです
06:53
We just provided this new tool,
a new way to look at the world,
ただ新しい道具と
世界を見る新しい方法を提供するだけで
06:55
and then people find other interesting,
new and creative ways of using it.
いろんな人が新しくて面白い
創造的な使い方を見つけてくれます
06:59
But we didn't stop there.
しかしそれで終わりではありません
07:04
This tool not only allows us
to look at the world in a new way,
このツールは世界に対し
新しい見方ができるようにするだけでなく
07:06
it also redefines what we can do
カメラで出来ることを再定義し
07:09
and pushes the limits of what
we can do with our cameras.
可能性の限界を
押し広げもします
07:11
So as scientists, we started wondering,
科学者として
私達は考え始めました
07:14
what other types of physical phenomena
produce tiny motions
カメラで計測できる
微細な動きを生み出す物理現象として
07:17
that we could now use
our cameras to measure?
他にどんなものがあるだろう?
07:20
And one such phenomenon
that we focused on recently is sound.
そのような現象の1つとして
我々が最近取り組んでいるのが「音」です
07:23
Sound, as we all know,
is basically changes
音というのは基本的に
07:27
in air pressure that
travel through the air.
空気中を伝わる空気圧の変化です
07:29
Those pressure waves hit objects
and they create small vibrations in them,
圧力の波が物にぶつかる時
小さな振動を生じ
07:32
which is how we hear
and how we record sound.
それを使って私達は音を聞いたり
録音したりしています
07:35
But it turns out that sound
also produces visual motions.
しかし音は視覚的な
動きも作り出します
07:38
Those are motions
that are not visible to us
肉眼では見えなくとも
07:41
but are visible to a camera
with the right processing.
カメラを使って適切に処理すれば
見えるようになります
07:44
So here are two examples.
例を2つお見せします
07:47
This is me demonstrating
my great singing skills.
これは私が素晴らしい歌唱力を
披露しているところです
07:49
(Singing)
アー
07:52
(Laughter)
(笑)
07:54
And I took a high-speed video
of my throat while I was humming.
声を出している時の喉を
高速度カメラで撮影しました
07:55
Again, if you stare at that video,
元の映像を見ても
07:58
there's not too much
you'll be able to see,
ほとんど動きは見られませんが
08:00
but once we magnify the motions 100 times,
we can see all the motions and ripples
動きを100倍拡大してやると
発声に関わる首の部分に
08:02
in the neck that are involved
in producing the sound.
波のような動きが広がっているのが
分かります
08:07
That signal is there in that video.
音の痕跡が映像に
残されているわけです
08:10
We also know that singers
can break a wine glass
歌手は特定の音程の声を出して
グラスを割れる
08:13
if they hit the correct note.
という話は
良く知られています
08:15
So here, we're going to play a note
ここではグラスの
08:17
that's in the resonance
frequency of that glass
共鳴周波数の音を
08:19
through a loudspeaker that's next to it.
横のスピーカーから
出しています
08:21
Once we play that note
and magnify the motions 250 times,
その時の動きを
250倍拡大すると
08:23
we can very clearly see
how the glass vibrates
グラスが音に共鳴して
振動しているのが
08:28
and resonates in response to the sound.
はっきり分かります
08:30
It's not something you're used
to seeing every day.
あまり日常で目にする光景では
ありませんね
08:33
But this made us think.
It gave us this crazy idea.
しかしここから
突飛なアイデアを思いつきました
08:36
Can we actually invert this process
and recover sound from video
この過程を逆にして 映像から音を
復元できないでしょうか?
08:39
by analyzing the tiny vibrations
that sound waves create in objects,
音波が物の表面に作り出す
微細な振動を解析して
08:45
and essentially convert those
back into the sounds that produced them.
元になった音を
生成するのです
08:49
In this way, we can turn
everyday objects into microphones.
そのようにすれば 身の回りにある物を
マイクに変えることができます
08:54
So that's exactly what we did.
私達はまさにそれを
やってみました
08:58
So here's an empty bag of chips
that was lying on a table,
テーブルの上にポテトチップの
空き袋があります
09:00
and we're going to turn that
bag of chips into a microphone
これをビデオ撮影して
音により生じた —
09:03
by filming it with a video camera
微細な動きを
解析することで
09:06
and analyzing the tiny motions
that sound waves create in it.
ポテトチップの袋をマイクロフォンに
変えようというわけです
09:08
So here's the sound
that we played in the room.
この部屋では
こんな音楽を流しています
09:11
(Music: "Mary Had a Little Lamb")
(曲『メリーさんのひつじ』)
09:14
And this is a high-speed video
we recorded of that bag of chips.
そしてポテトチップの袋を
高速度カメラで撮影しました
09:21
Again it's playing.
これを見ても
09:24
There's no chance you'll be able
to see anything going on in that video
何かが起きているようには
09:26
just by looking at it,
見えませんが
09:29
but here's the sound we were able
to recover just by analyzing
映像の中の微細な動きを
解析することで
09:30
the tiny motions in that video.
このような音を
再現できました
09:33
(Music: "Mary Had a Little Lamb")
(曲『メリーさんのひつじ』)
09:35
I call it -- Thank you.
私はこれを—
09:52
(Applause)
(拍手)
09:54
I call it the visual microphone.
「ビジュアル・マイクロフォン」
と呼んでいます
10:01
We actually extract audio signals
from video signals.
ビデオ信号からオーディオ信号を
取り出しているのです
10:04
And just to give you a sense
of the scale of the motions here,
動きの大きさが
どれくらいかというと
10:07
a pretty loud sound will cause that bag
of chips to move less than a micrometer.
かなり大きな音でも
ポテトチップの袋の動きは1ミクロン未満です
10:10
That's one thousandth of a millimeter.
1ミリの千分の1です
10:15
That's how tiny the motions are
that we are now able to pull out
そのような小さな動きでも
10:18
just by observing how light
bounces off objects
映像の中で物に反射する光を
観察することによって
10:22
and gets recorded by our cameras.
検出できるのです
10:25
We can recover sounds
from other objects, like plants.
他の物を使うこともできます
たとえば植物とか
10:27
(Music: "Mary Had a Little Lamb")
(曲『メリーさんのひつじ』)
10:30
And we can recover speech as well.
声を復元することもできます
10:39
So here's a person speaking in a room.
こちらでは部屋の中で
人が話しています
10:41
Voice: Mary had a little lamb
whose fleece was white as snow,
Mary had a little lamb whose fleece was white as snow,
(メリーさんは小さな羊を飼っていた 雪のように白い毛をして)
10:43
and everywhere that Mary went,
that lamb was sure to go.
and everywhere that Mary went, that lamb was sure to go.
(メリーさんの行くところは どこにでも付いてきた)
10:47
Michael Rubinstein: And here's
that speech again recovered
前と同じ
ポテトチップの袋の映像から
10:52
just from this video
of that same bag of chips.
復元した声です
10:54
Voice: Mary had a little lamb
whose fleece was white as snow,
Mary had a little lamb
whose fleece was white as snow,
10:58
and everywhere that Mary went,
that lamb was sure to go.
and everywhere that Mary went,
that lamb was sure to go.
11:02
MR: We used "Mary Had a Little Lamb"
『メリーさんのひつじ』を使ったのは
11:07
because those are said to be
the first words
エジソンが1877年に
蓄音機で
11:10
that Thomas Edison spoke
into his phonograph in 1877.
最初に録音したのが
この歌だったからです
11:12
It was one of the first sound
recording devices in history.
それは音を記録する
最初の装置の1つでした
11:16
It basically directed the sounds
onto a diaphragm
音を振動板で受け
11:19
that vibrated a needle that essentially
engraved the sound on tinfoil
その振動を針に伝え
それが筒に巻いたアルミ箔に
11:22
that was wrapped around the cylinder.
記録される仕掛けでした
11:27
Here's a demonstration of recording and
replaying sound with Edison's phonograph.
これはエジソンの蓄音機で
録音し再生するデモです
11:29
(Video) Voice: Testing,
testing, one two three.
(録音)Testing, testing, one two three.
11:35
Mary had a little lamb
whose fleece was white as snow,
Mary had a little lamb
whose fleece was white as snow,
11:38
and everywhere that Mary went,
the lamb was sure to go.
and everywhere that Mary went,
the lamb was sure to go.
11:41
Testing, testing, one two three.
(再生)Testing, testing, one two three.
11:45
Mary had a little lamb
whose fleece was white as snow,
Mary had a little lamb
whose fleece was white as snow,
11:48
and everywhere that Mary went,
the lamb was sure to go.
and everywhere that Mary went,
the lamb was sure to go.
11:52
MR: And now, 137 years later,
137年後の今日
11:57
we're able to get sound
in pretty much similar quality
音に振動する物の映像だけから
同程度のクオリティの音を
12:01
but by just watching objects
vibrate to sound with cameras,
再現できるようになりました
12:05
and we can even do that when the camera
しかも防音ガラスの
向こう側にある
12:09
is 15 feet away from the object,
behind soundproof glass.
5メートル離れた物を使って
それができるのです
12:11
So this is the sound that we were
able to recover in that case.
そうやって復元した音がこれです
12:15
Voice: Mary had a little lamb
whose fleece was white as snow,
Mary had a little lamb
whose fleece was white as snow,
12:19
and everywhere that Mary went,
the lamb was sure to go.
and everywhere that Mary went,
the lamb was sure to go.
12:24
MR: And of course, surveillance is
the first application that comes to mind.
すぐ思いつく応用は
スパイ活動でしょう
12:29
(Laughter)
(笑)
12:32
But it might actually be useful
for other things as well.
しかし他のことにも使えます
12:35
Maybe in the future, we'll be able
to use it, for example,
もしかしたら
将来宇宙の向こうの音を
12:39
to recover sound across space,
再現できるように
なるかもしれません
12:42
because sound can't travel
in space, but light can.
音は宇宙を伝わりませんが
光なら伝わるからです
12:44
We've only just begun exploring
私達は
この新しい技術の可能性を
12:48
other possible uses
for this new technology.
探り始めたばかりです
12:50
It lets us see physical processes
that we know are there
これまで存在するのは
知っていたけれど
12:53
but that we've never been able
to see with our own eyes until now.
自分の目で見られなかった物理現象を
見えるようにしてくれるのです
12:56
This is our team.
これが私の仲間です
13:00
Everything I showed you today
is a result of a collaboration
今日お見せしたものは
みんなここに出ている人達の
13:01
with this great group
of people you see here,
協力の結果
得られたものです
13:04
and I encourage you and welcome you
to check out our website,
みなさんにもぜひ
私達のウェブサイトを訪れ
13:06
try it out yourself,
自分で試してみて
13:09
and join us in exploring
this world of tiny motions.
一緒に微細な動きの世界を
探索していただきたいと思います
13:11
Thank you.
ありがとうございました
13:14
(Applause)
(拍手)
13:15
Translator:Yasushi Aoki
Reviewer:Eriko T.

sponsored links

Michael Rubinstein - Research scientist, Google
Computer scientist Michael Rubinstein and his team have developed a "motion microscope" that can show video footage of barely perceivable movements, like breaths and heartbeats.

Why you should listen

Michael Rubinstein zooms in on what we can't see and mangnifies it by thirty or a hundred times. His "motion microscope," developed at MIT with Microsoft and Quanta Research, picks up on subtle motion and color changes in videos and blows them up for the naked eye to see. The result: fun, cool, creepy videos.

Rubinstein is a research scientist at a new Cambridge-based Google lab for computer vision research. He has a PhD in computer science and electrical engineering from MIT.

sponsored links

If you need translations, you can install "Google Translate" extension into your Chrome Browser.
Furthermore, you can change playback rate by installing "Video Speed Controller" extension.

Data provided by TED.

This website is owned and operated by Tokyo English Network.
The developer's blog is here.