TED2007
Blaise Agüera y Arcas: How PhotoSynth can connect the world's images
布雷斯•A•雅克斯的「相片合成」技術展示
Filmed:
Readability: 4.3
5,831,957 views
布雷斯•雅克斯進行了炫目的「相片合成」技術示範，這個軟體可以改變我們觀看數位影像的方式。使用從網路上所挑出的靜態照片，「相片合成」創建了令人歎為觀止的夢幻景象，並且讓我們自由穿梭在其中。
Blaise Agüera y Arcas - Software architect
Blaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces. Full bio
Blaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces. Full bio
Double-click the English transcript below to play the video.
00:25
What I'm going to show you first, as quickly as I can,
0
0
2000
首先，我要用最快的速度為大家示範
00:27
is some foundational work, some new technology
1
2000
4000
一些新技術的基礎研究成果。
00:31
that we brought to Microsoft as part of an acquisition
2
6000
3000
正好是一年前， 微軟收購了我們公司，
00:34
almost exactly a year ago. This is Seadragon,
3
9000
3000
而我們為微軟帶來了這項技術，它就是「海龍」(Seadragon)。
00:37
and it's an environment in which you can either locally or remotely
4
12000
3000
「海龍」是一個軟體環境，你可以通過它以近景或遠景的方式
00:40
interact with vast amounts of visual data.
5
15000
3000
流覽浩瀚的視覺化資料。
00:43
We're looking at many, many gigabytes of digital photos here
6
18000
3000
我們在這裏看到的是，非常多千兆位元組的數位像片，
00:46
and kind of seamlessly and continuously zooming in,
7
21000
3000
我們可以對它們可以進行持續並且平滑的放大，
00:50
panning through the thing, rearranging it in any way we want.
8
25000
2000
可以通過全景的方式流覽它們，還可以對它們進行重新排列。
00:52
And it doesn't matter how much information we're looking at,
9
27000
4000
不管所見到的資料有多少、
00:56
how big these collections are or how big the images are.
10
31000
3000
圖像集有多大或是圖像本身有多大。
00:59
Most of them are ordinary digital camera photos,
11
34000
2000
以上展示的圖片，大部分是來自一般數位相機的照片，
01:01
but this one, for example, is a scan from the Library of Congress,
12
36000
3000
但這個例子不同，它是一張來自國會圖書館的掃描圖片，
01:05
and it's in the 300 megapixel range.
13
40000
2000
擁有3億個像素。
01:08
It doesn't make any difference
14
43000
1000
然而，這沒有造成任何不同，
01:09
because the only thing that ought to limit the performance
15
44000
3000
因為能夠限制像這樣的系統效能的唯一因素，
01:12
of a system like this one is the number of pixels on your screen
16
47000
3000
是你所使用的螢幕所正在顯示的像素數量。
01:15
at any given moment. It's also very flexible architecture.
17
50000
3000
「海龍」同時也是一個非常靈活的架構。
01:18
This is an entire book, so this is an example of non-image data.
18
53000
3000
這是一本完整的書，是非圖形式資料的一個例子。
01:22
This is "Bleak House" by Dickens. Every column is a chapter.
19
57000
5000
這是狄更斯所著的《荒涼山莊》，每一欄就是一個章節。
01:27
To prove to you that it's really text, and not an image,
20
62000
4000
為了向你們證明這真的是文字而非圖片，
01:31
we can do something like so, to really show
21
66000
2000
我們可以這樣操作，
01:33
that this is a real representation of the text; it's not a picture.
22
68000
3000
大家可以看得出來這真的是文字，而不是一張圖片。
01:37
Maybe this is a kind of an artificial way to read an e-book.
23
72000
2000
也許這會是一種閱讀電子書的方式，
01:39
I wouldn't recommend it.
24
74000
1000
但是我個人不會推薦這麼做。
01:40
This is a more realistic case. This is an issue of The Guardian.
25
75000
3000
接下來是一個更加實際的例子，這是一期《衛報》。
01:43
Every large image is the beginning of a section.
26
78000
2000
你所看到的每一張大圖片，就是各版頭條，
01:45
And this really gives you the joy and the good experience
27
80000
3000
而報紙或雜誌的紙本，本身就包含了各種比例的媒材，
01:48
of reading the real paper version of a magazine or a newspaper,
28
83000
5000
因此這樣閱讀的時候，讀者會得到更好的閱讀體驗，
01:54
which is an inherently multi-scale kind of medium.
29
89000
1000
從而享受閱讀的樂趣。
01:56
We've also done a little something
30
91000
1000
我們在這裏做了點小小的更動，
01:57
with the corner of this particular issue of The Guardian.
31
92000
3000
在這一期《衛報》的一角。
02:00
We've made up a fake ad that's very high resolution --
32
95000
3000
我們刊登了一個非常高解析度的虛構廣告 —
02:03
much higher than you'd be able to get in an ordinary ad --
33
98000
2000
比你平常看到的普通廣告的解析度要高很多，
02:05
and we've embedded extra content.
34
100000
2000
我們並在圖片中嵌入了額外的內容。
02:07
If you want to see the features of this car, you can see it here.
35
102000
2000
如果你希望看到這輛車的特性，你可以看這裏，
02:10
Or other models, or even technical specifications.
36
105000
4000
你還能看到其他的型號，甚至技術規格。
02:15
And this really gets at some of these ideas
37
110000
2000
這種方式在一定程度上，
02:18
about really doing away with those limits on screen real estate.
38
113000
4000
避開了螢幕面積的限制。
02:22
We hope that this means no more pop-ups
39
117000
2000
我們希望這個技術能夠減少不必要的彈出視窗，
02:24
and other kind of rubbish like that -- shouldn't be necessary.
40
119000
2000
及其他類似的，不必要的垃圾。
02:27
Of course, mapping is one of those really obvious applications
41
122000
2000
當然，對於像這樣的技術，
02:29
for a technology like this.
42
124000
2000
數位地圖也是顯而易見的應用之一。
02:31
And this one I really won't spend any time on,
43
126000
2000
對此，我真的不想花費太多的時間進行介紹，
02:33
except to say that we have things to contribute to this field as well.
44
128000
2000
我只想告訴大家，我們對這個領域也貢獻了一己之力。
02:37
But those are all the roads in the U.S.
45
132000
2000
這些是將美國的所有道路，
02:39
superimposed on top of a NASA geospatial image.
46
134000
4000
疊加在太空總署的地理空間影像上。
02:44
So let's pull up, now, something else.
47
139000
2000
現在，我們先放下這些，看看其他的。
02:46
This is actually live on the Web now; you can go check it out.
48
141000
3000
實際上，這項技術已經放到網路上了，大家可以自己去體驗一下。
02:49
This is a project called Photosynth,
49
144000
1000
這個計畫名叫「相片合成」 (Photosynth)，
02:51
which really marries two different technologies.
50
146000
1000
它實際上融合了兩個不同的技術：
02:52
One of them is Seadragon
51
147000
1000
一個是「海龍」，
02:54
and the other is some very beautiful computer vision research
52
149000
2000
而另一個則是源自華盛頓大學的研究生 Noah Snavely，
02:57
done by Noah Snavely, a graduate student at the University of Washington,
53
152000
2000
所進行的電腦視覺化研究的美麗成果。
03:00
co-advised by Steve Seitz at U.W.
54
155000
2000
這項研究還得到了華盛頓大學 Steve Seitz
03:02
and Rick Szeliski at Microsoft Research. A very nice collaboration.
55
157000
4000
和微軟研究中心 Rick Szeliski 的共同指導。這是一個非常漂亮的合作成果。
03:07
And so this is live on the Web. It's powered by Seadragon.
56
162000
2000
現在各位看到的是我們連上網路的即時示範，它是根基於「海龍」技術。
03:09
You can see that when we kind of do these sorts of views,
57
164000
2000
你可以看到，我們輕鬆地對圖片進行多種方式的查看，
03:12
where we can dive through images
58
167000
1000
就好像潛入這些影像一般，
03:14
and have this kind of multi-resolution experience.
59
169000
1000
擁有了這種多解析度的瀏覽體驗。
03:16
But the spatial arrangement of the images here is actually meaningful.
60
171000
4000
不過，在這邊，這些圖片空間上的關係事實上是有意義的。
03:20
The computer vision algorithms have registered these images together
61
175000
3000
電腦視覺演算法將這些圖片聯繫到一起，
03:23
so that they correspond to the real space in which these shots --
62
178000
4000
那麼這些圖片就能將真實空間給呈現出來了，
03:27
all taken near Grassi Lakes in the Canadian Rockies --
63
182000
2000
而我們正是在這個地方拍了上述的照片 — 這些照片都是在
03:31
all these shots were taken. So you see elements here
64
186000
2000
加拿大洛磯山脈的格拉西湖附近拍下的 — 所有照片都是在這裏拍下的。
03:33
of stabilized slide-show or panoramic imaging,
65
188000
4000
因此，這邊你可以看到穩定幻燈片播放的元素或者環景影像，
03:40
and these things have all been related spatially.
66
195000
2000
而這些內容在空間上都是互相關聯的。
03:42
I'm not sure if I have time to show you any other environments.
67
197000
3000
我不確定我是否有時間為你們示範其他環境的例子。
03:45
There are some that are much more spatial.
68
200000
1000
有些其他例子比這個的空間感還要強。
03:47
I would like to jump straight to one of Noah's original data-sets --
69
202000
3000
下面讓我們來看一下去年夏天，
03:50
and this is from an early prototype of Photosynth
70
205000
2000
Noah 早期所建立的資料集之一，
03:52
that we first got working in the summer --
71
207000
2000
這是來自於「相片合成」技術早期的原型階段。
03:54
to show you what I think
72
209000
1000
我認為，
03:55
is really the punch line behind this technology,
73
210000
3000
這是我們這項技術最搶眼之處。
03:59
the Photosynth technology. And it's not necessarily so apparent
74
214000
2000
「相片合成」技術不單單像我們剛剛在
04:01
from looking at the environments that we've put up on the website.
75
216000
3000
網站上所示範的環境般，那麼的簡單明瞭。
04:04
We had to worry about the lawyers and so on.
76
219000
2000
主要因為我們製作網站時，要顧慮很多法律問題。
04:07
This is a reconstruction of Notre Dame Cathedral
77
222000
1000
這裡是利用 Flickr 網站上
04:09
that was done entirely computationally
78
224000
2000
的照片，並完全以電腦重建的巴黎聖母院。
04:11
from images scraped from Flickr. You just type Notre Dame into Flickr,
79
226000
3000
你所要做的只是在 Flickr 網站上輸入「巴黎聖母院」，
04:14
and you get some pictures of guys in t-shirts, and of the campus
80
229000
3000
然後便能看到很多照片，包括在那邊留影的遊客等等。
04:17
and so on. And each of these orange cones represents an image
81
232000
4000
每一個橘色的錐形都代表了一張
04:22
that was discovered to belong to this model.
82
237000
2000
用來建立模型的照片。
04:26
And so these are all Flickr images,
83
241000
2000
這些全部是來自 Flickr 的圖片，
04:28
and they've all been related spatially in this way.
84
243000
3000
被這樣在空間裡被串聯起來。
04:31
And we can just navigate in this very simple way.
85
246000
2000
接著，我們便可如此自然的進行瀏覽。
04:35
(Applause)
86
250000
9000
（掌聲）
04:44
You know, I never thought that I'd end up working at Microsoft.
87
259000
2000
說實話，我從來沒想過我會為微軟工作，
04:46
It's very gratifying to have this kind of reception here.
88
261000
4000
這樣受到歡迎，真挺令人高興的。
04:50
(Laughter)
89
265000
3000
（笑聲）
04:53
I guess you can see
90
268000
3000
我想你們可以看出，
04:56
this is lots of different types of cameras:
91
271000
2000
這些照片來自很多不同的相機：
04:58
it's everything from cell phone cameras to professional SLRs,
92
273000
3000
從手機鏡頭到專業的單眼相機。
05:02
quite a large number of them, stitched
93
277000
1000
如此大量的不同品質的照片，全被在這個環境下
05:03
together in this environment.
94
278000
1000
拼湊在一 起。
05:04
And if I can, I'll find some of the sort of weird ones.
95
279000
2000
讓我來找些比較詭異的照片。
05:08
So many of them are occluded by faces, and so on.
96
283000
3000
看，不少照片包含了遊客的大頭照等等。
05:13
Somewhere in here there are actually
97
288000
1000
我記得這裡應該有
05:15
a series of photographs -- here we go.
98
290000
1000
一系列的照片 — 啊，在這兒。
05:17
This is actually a poster of Notre Dame that registered correctly.
99
292000
3000
這實際上是一張有巴黎聖母院照片的海報，
05:21
We can dive in from the poster
100
296000
2000
我們可以鑽到海報裡，
05:24
to a physical view of this environment.
101
299000
3000
去看整個重建的環境。
05:31
What the point here really is is that we can do things
102
306000
3000
這裏的重點呢？便是我們可以
05:34
with the social environment. This is now taking data from everybody --
103
309000
5000
有效地利用網路社群。我們可以從每個人那裡得到資料，
05:39
from the entire collective memory
104
314000
1000
將每個人對不同環境
05:40
of, visually, of what the Earth looks like --
105
315000
2000
的視覺記憶蒐集在一起，
05:43
and link all of that together.
106
318000
1000
並將它們連結起來。
05:44
All of those photos become linked together,
107
319000
2000
當所有這些圖片交織在一起時，
05:46
and they make something emergent
108
321000
1000
所衍生出的東西，
05:47
that's greater than the sum of the parts.
109
322000
2000
要遠遠超過各部件的總和，
05:49
You have a model that emerges of the entire Earth.
110
324000
2000
這個模型所衍生出的，是整個地球。
05:51
Think of this as the long tail to Stephen Lawler's Virtual Earth work.
111
326000
5000
將之想像是 Stephen Lawler《虛擬地球》的長尾市場。（Stephen Lawler 是微軟「虛擬地球」專案主管）
05:56
And this is something that grows in complexity
112
331000
2000
這類模型， 會隨著人們的
05:58
as people use it, and whose benefits become greater
113
333000
3000
使用而不斷變得更複雜，
06:01
to the users as they use it.
114
336000
2000
變得更加有價值。
06:03
Their own photos are getting tagged with meta-data
115
338000
2000
用戶的照片，會被其他人
06:05
that somebody else entered.
116
340000
1000
輸入標注資料。
06:07
If somebody bothered to tag all of these saints
117
342000
3000
如果有人願意，為聖母院裡的所有聖賢輸入標注，
06:10
and say who they all are, then my photo of Notre Dame Cathedral
118
345000
3000
表明他們是誰，那我們聖母院的照片便會
06:13
suddenly gets enriched with all of that data,
119
348000
2000
一下子增加了許多資訊，
06:15
and I can use it as an entry point to dive into that space,
120
350000
3000
然後呢，我們便能以這張照片為起點，進入這個空間，
06:18
into that meta-verse, using everybody else's photos,
121
353000
2000
這個由很多人的照片所搭建的虛擬世界，
06:21
and do a kind of a cross-modal
122
356000
2000
從而得到一種跨越模型，
06:25
and cross-user social experience that way.
123
360000
3000
跨越用戶的社交體驗。
06:28
And of course, a by-product of all of that
124
363000
1000
當然了，這一切所帶來另外一個寶貴產物便是，
06:30
is immensely rich virtual models
125
365000
2000
我們擁有地球上每一個有趣的地方，
06:32
of every interesting part of the Earth, collected
126
367000
2000
非常豐富的模型。
06:35
not just from overhead flights and from satellite images
127
370000
3000
這些模型的資料來源，不再僅限於空拍或衛星照片等等，
06:38
and so on, but from the collective memory.
128
373000
2000
而是來自全人類的集合記憶。
06:40
Thank you so much.
129
375000
2000
非常感謝！
06:42
(Applause)
130
377000
11000
（掌聲）
06:53
Chris Anderson: Do I understand this right? That what your software is going to allow,
131
388000
4000
Chris Anderson: 如果我理解正確的話，你們的這個軟體將能夠
06:58
is that at some point, really within the next few years,
132
393000
2000
在未來的幾年內，
07:01
all the pictures that are shared by anyone across the world
133
396000
4000
將來自全球網路使用者所共享的照片
07:05
are going to basically link together?
134
400000
2000
結合在一起？
07:07
BAA: Yes. What this is really doing is discovering.
135
402000
2000
BAA:是的。這個軟體的真正意義便是去探索，
07:09
It's creating hyperlinks, if you will, between images.
136
404000
3000
它在照片間建立超鏈結。
07:12
And it's doing that
137
407000
1000
這個結合的過程，
07:13
based on the content inside the images.
138
408000
1000
完全是基於照片的內容。
07:14
And that gets really exciting when you think about the richness
139
409000
3000
更令人興奮的
07:17
of the semantic information that a lot of those images have.
140
412000
2000
在於照片所包含的大量文字語義資訊。
07:19
Like when you do a web search for images,
141
414000
2000
譬如，你在網路上搜尋一張照片，
07:22
you type in phrases, and the text on the web page
142
417000
2000
鍵入關鍵字後，網頁上的文字內容
07:24
is carrying a lot of information about what that picture is of.
143
419000
3000
將包含大量與這個照片相關的資訊。
07:27
Now, what if that picture links to all of your pictures?
144
422000
2000
現在，假設這些照片，全部都與你的照片互相連結，那將會怎樣？
07:29
Then the amount of semantic interconnection
145
424000
2000
那時，所有這些語義資訊相互連結的
07:31
and the amount of richness that comes out of that
146
426000
1000
資訊量將是
07:32
is really huge. It's a classic network effect.
147
427000
3000
非常巨大的。這是非常典型的網路效應。
07:35
CA: Blaise, that is truly incredible. Congratulations.
148
430000
2000
CA:Blaise， 太難以置信了。祝賀你們！
07:37
BAA: Thanks so much.
149
432000
1000
BAA: 非常感謝各位！
ABOUT THE SPEAKER
Blaise Agüera y Arcas - Software architectBlaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces.
Why you should listen
Blaise Agüera y Arcas is principal scientist at Google, where he leads a team working on machine intelligence for mobile devices. His group works extensively with deep neural nets for machine perception and distributed learning, and it also investigates so-called "connectomics" research, assessing maps of connections within the brain.
Agüera y Arcas' background is as multidimensional as the visions he helps create. In the 1990s, he authored patents on both video compression and 3D visualization techniques, and in 2001, he made an influential computational discovery that cast doubt on Gutenberg's role as the father of movable type.
He also created Seadragon (acquired by Microsoft in 2006), the visualization technology that gives Photosynth its amazingly smooth digital rendering and zoom capabilities. Photosynth itself is a vastly powerful piece of software capable of taking a wide variety of images, analyzing them for similarities, and grafting them together into an interactive three-dimensional space. This seamless patchwork of images can be viewed via multiple angles and magnifications, allowing us to look around corners or “fly” in for a (much) closer look. Simply put, it could utterly transform the way we experience digital images.
He joined Microsoft when Seadragon was acquired by Live Labs in 2006. Shortly after the acquisition of Seadragon, Agüera y Arcas directed his team in a collaboration with Microsoft Research and the University of Washington, leading to the first public previews of Photosynth several months later. His TED Talk on Seadragon and Photosynth in 2007 is rated one of TED's "most jaw-dropping." He returned to TED in 2010 to demo Bing’s augmented reality maps.
Fun fact: According to the author, Agüera y Arcas is the inspiration for the character Elgin in the 2012 best-selling novel Where'd You Go, Bernadette?
Blaise Agüera y Arcas | Speaker | TED.com