TEDxUSC
Paul Debevec: Animating a photo-real digital face
Paul Debevec 使相片般真實的數位臉孔活了起來
Filmed:
Readability: 4.8
881,584 views
在 TEDxUSC ，電腦繪圖先驅 Paul Debevec 解釋數位 Emily 背後搶眼的科技，一種數位化構建的人物面孔，真實得可以應付多種需要。
Paul Debevec - Computer graphics pioneer
Paul Debevec's digital inventions have powered the breathtaking visual effects in films like The Matrix, Superman Returns, King Kong and The Curious Case of Benjamin Button. Full bio
Paul Debevec's digital inventions have powered the breathtaking visual effects in films like The Matrix, Superman Returns, King Kong and The Curious Case of Benjamin Button. Full bio
Double-click the English transcript below to play the video.
00:18
One of the biggest challenges in computer graphics
0
0
3000
電腦繪圖的巨大挑戰之一，
00:21
has been being able to create a photo-real,
1
3000
2000
是如何製作一個如相片般真實的
00:23
digital human face.
2
5000
2000
數位人物面孔。
00:25
And one of the reasons it is so difficult is that, unlike aliens and dinosaurs,
3
7000
3000
不像外星人和恐龍，它之所以如此困難的原因是，
00:28
we look at human faces every day.
4
10000
3000
我們每天都會看到人的面孔，
00:31
They are very important to how we communicate with each other.
5
13000
2000
它們對我們非常重要，因為我們藉此來彼此溝通，
00:33
As a result, we're tuned in to the subtlest things
6
15000
3000
因此，我們會注意到那些通過電腦渲染出來，
00:36
that could possibly be wrong with a computer rendering,
7
18000
2000
不正確的、非常細微的東西，
00:38
in order to believe whether these things are realistic.
8
20000
3000
並用它們來分辨這些事物是否真實的。
00:41
And what I'm going to do in the next five minutes
9
23000
2000
在接下來的五分鐘裡面，
00:43
is take you through a process
10
25000
2000
我將帶領大家一起看一個過程，
00:45
where we tried to create a reasonably photo-realistic computer-generated face,
11
27000
3000
從我們嘗試去創造一個以電腦制作的幾近相片般真實的面孔，
00:48
using some computer graphics technology we've developed,
12
30000
3000
通過我們
00:51
and also some collaborators at a company called Image Metrics.
13
33000
2000
和一些來自 Image Metrics 的伙伴開發的一系列電腦繪圖技術。
00:53
And we're going to try to do a photo-real face
14
35000
2000
我們將嘗試製作
00:55
of an actress named Emily O'Brien, who is right there.
15
37000
3000
一位名叫 Emily O'Brien 的女演員的面孔，她就在那兒。
00:58
And that's actually a completely computer-generated rendering of her face.
16
40000
3000
那實際上是一個完全依據她的面孔，由電腦渲染產生的結果。
01:01
By the end of the talk, we're going to see it move.
17
43000
2000
在演講的最後，我們將會看到她動起來。
01:03
The way that we did this is we tried to start with Emily herself,
18
45000
3000
我們嘗試的方式是由 Emily 她本身開始，
01:06
who was gracious enough to come to our laboratory
19
48000
3000
她非常親切地來到我們的實驗室。
01:09
in Marina Del Rey, and sit for a session in Light Stage 5.
20
51000
3000
在瑪麗安德爾灣，並坐到了我們的設備「光台五號」(Light Stage 5) 當中。
01:12
This is a face-scanning sphere, with 156 white LEDs all around
21
54000
4000
這是一個臉部掃描球體，有 156 個白色的發光二極管，
01:16
that allow us to photograph her
22
58000
2000
能讓我們在一系列嚴格控制的照明條件下，
01:18
in a series of very controlled illumination conditions.
23
60000
3000
對她進行拍照。
01:21
And the lighting that we use these days looks something like this.
24
63000
3000
我們當時使用的照明方式看起來是這樣的。
01:24
We shoot all of these photographs in about three seconds.
25
66000
2000
我們在三秒鐘之內拍攝出這裡全部的照片。
01:26
And we basically capture enough information
26
68000
3000
我們基本上採集到了足夠的資訊，
01:29
with video projector patterns that drape over the contours of her face,
27
71000
4000
能夠通過圖像投影模擬出她的面部輪廓，
01:33
and different principle directions of light from the light stage,
28
75000
2000
以及在燈光舞台不同角度的照明條件下，
01:35
to figure out both the coarse-scale
29
77000
2000
來計算出她面孔粗略的
01:37
and the fine-scale detail of her face.
30
79000
2000
以及精細的面部細節模型。
01:39
If we zoom in on this photograph right here,
31
81000
2000
如果我們放大這張照片的這個地方，
01:41
we can see it's a really nice photograph to have of her,
32
83000
2000
我們可以很清晰地看到她臉部的細節，
01:43
because she is lit from absolutely everywhere at the same time
33
85000
4000
因為她的臉部同時從各個角度被完全地照亮了，
01:47
to get a nice image of her facial texture.
34
89000
2000
因此得到一份非常優質的她的臉部紋理的相片。
01:49
And in addition, we've actually used polarizers on all the lights --
35
91000
3000
再加上，我們實際上在所有燈光上應用了偏光鏡
01:52
just like polarized sunglasses can block
36
94000
2000
就如同偏光的太陽眼鏡可以阻擋
01:54
the glare off of the road,
37
96000
2000
馬路上的強烈反強光，
01:56
polarizers can block the shine off of the skin,
38
98000
2000
偏光鏡可以阻擋皮膚上出現的光澤，
01:58
so we don't get all those specular reflections to take this map.
39
100000
3000
所以在這張圖中，我們沒有採集到那些球面反射。
02:01
Now, if we turn the polarizers around just a little bit,
40
103000
2000
現在，如果我們稍微調整一下偏光鏡，
02:03
we can actually bring that specular reflection
41
105000
2000
我們就可以將球面反射
02:05
of the skin back in,
42
107000
2000
帶回到皮膚上面，
02:07
and you can see she looks kind of shiny and oily at this point.
43
109000
3000
並且，你會看到她的面部看起來有些閃耀和油光。
02:10
If you take the difference between these two images here,
44
112000
3000
如果你取得這兩張圖像的差別，
02:13
you can get an image lit from the entire sphere of light
45
115000
2000
你可以得到的一個被球形光線照亮的
02:15
of just the shine off of Emily's skin.
46
117000
3000
Emily 的皮膚的相片。光照效果
02:18
I don't think any photograph like this had ever been taken
47
120000
2000
我認為沒有任何人拍過像這樣的照片，
02:20
before we had done this.
48
122000
2000
我們是第一個這麼做的。
02:22
And this is very important light to capture,
49
124000
3000
採集這種光線是非常重要的，
02:25
because this is the light that reflects off the first surface of the skin.
50
127000
2000
因為這是皮膚最表層所反射出來的光線，
02:27
It doesn't get underneath the translucent
51
129000
2000
它不會跑到皮膚的透明層下面去，
02:29
layers of the skin and blur out.
52
131000
2000
然後造成模糊不清。
02:31
And, as a result, it's a very good cue
53
133000
2000
因此，它是一個
02:33
to the detailed shape of the skin-pore structure
54
135000
3000
用來獲得皮膚細緻毛孔結構
02:36
and all of the fine wrinkles that all of us have,
55
138000
2000
以及我們都有的皮膚全部的褶皺細節的很好線索，
02:38
the things that actually make us look like real humans.
56
140000
2000
實際上，是這些東西讓我們看起來像真實人類。
02:40
So, if we use information that comes off of this specular reflection,
57
142000
4000
如果，我們使用來自高光反射的資訊，
02:44
we can go from a traditional face scan
58
146000
2000
我們可以從傳統的臉部掃描開始，
02:46
that might have the gross contours of the face and the basic shape,
59
148000
4000
來得到臉部的粗略輪廓和基本形狀，
02:50
and augment it with information
60
152000
2000
再加上大量
02:52
that puts in all of that skin pore structure and fine wrinkles.
61
154000
3000
皮膚的毛孔結構和紋理特徵的資訊。
02:55
And, even more importantly,
62
157000
2000
更重要的是，
02:57
since this is a photometric process that only takes three seconds to capture,
63
159000
3000
因為這是基於照片建模的流程而我們只採集了三秒鐘的照片資料，
03:00
we can shoot Emily
64
162000
2000
我們可以用一下午的時間
03:02
in just part of an afternoon,
65
164000
2000
來拍愛蜜麗 (Emily)，
03:04
in many different facial poses and facial expressions.
66
166000
3000
拍攝不同的面部姿態和表情變化。
03:07
So, here you can see her moving her eyes around, moving her mouth around.
67
169000
3000
這裡，你可以看到她轉動眼睛，嘴巴的開合。
03:10
And these we're actually going to use to create a photo-real digital character.
68
172000
3000
這些，實際上都會被我們用來創造數位虛擬的寫實人物。
03:13
If you take a look at these scans that we have of Emily,
69
175000
4000
來看一下這些我們對愛蜜麗 (Emily) 進行的掃描，
03:17
you can see that the human face does an enormous amount of amazing things
70
179000
3000
你會看到我們人類臉部表情是千變萬化的，
03:20
as it goes into different facial expressions.
71
182000
3000
在不同的臉部表情變換之中，
03:23
You can see things. Not only the face shape changes,
72
185000
3000
你會看到，不僅僅是臉部形態發生變化，
03:26
but all sorts of different skin buckling and skin wrinkling occurs.
73
188000
4000
各種各樣的皮膚的弧線和皮膚上的褶皺也會出現。
03:30
You can see that the skin pore structure changes enormously
74
192000
4000
你可以看到皮膚毛孔結構也有著巨大的變化，
03:34
from stretched skin pores
75
196000
2000
從被拉伸的皮膚毛孔，
03:36
to the regular skin texture.
76
198000
2000
到普通的皮膚質地。
03:38
You can see the furrows in the brow and how the microstructure changes there.
77
200000
3000
你可以看到前額眉骨那裡細微的變化，
03:41
You can see muscles pulling down at flesh to bring her eyebrows down.
78
203000
4000
你可以看到在她皺眉時候肌肉的收縮。
03:45
Her muscles bulging in her forehead when she winces like that.
79
207000
3000
在她畏懼的時候，她前額的肌肉會這樣地膨脹。
03:48
In addition to this kind of high-resolution geometry,
80
210000
3000
對於這樣高分辨率的幾何體，
03:51
since it's all captured with cameras, we've got a great texture map to use for the face.
81
213000
4000
因為這些都通過照相機採集，我們有她面部表情足夠的貼圖。
03:55
And by looking at how the different color channels of the illumination,
82
217000
2000
通過顏色和亮度不同的頻道，
03:57
the red and the green and the blue,
83
219000
2000
紅綠藍色彩頻道，
03:59
diffuse the light differently,
84
221000
2000
受光照不同的散射，
04:01
we can come up with a way of shading the skin on the computer.
85
223000
3000
我們可以通過電腦總結出為皮膚添加陰影的方式。
04:04
Then, instead of looking like a plaster mannequin,
86
226000
3000
然後，不同於看起來如同石膏像的假人，
04:07
it actually looks like it's made out of living human flesh.
87
229000
3000
她看起來如同人類一樣鮮活。
04:10
And this is what we used
88
232000
2000
這就是我們
04:12
to give to the company Image Metrics
89
234000
2000
為 Image Metrics 公司
04:14
to create a rigged, digital version of Emily.
90
236000
3000
所製作的寫實的數位虛擬人物 Emily。
04:17
We're just seeing the coarse-scale geometry here.
91
239000
2000
這裡，我們看一下粗糙的幾何模型，
04:19
But they basically created a digital puppet of her,
92
241000
3000
但他們已經對她創建了一些基本的數據程式，
04:22
where you can pull on these various strings,
93
244000
2000
你可以拖曳這些不同的座標繩，
04:24
and it actually moves her face in ways that are
94
246000
2000
它實際上運動的方式就會依照
04:26
completely consistent with the scans that we took.
95
248000
2000
我們掃描出的數據那樣。
04:28
And, in addition to the coarse-scale geometry,
96
250000
2000
並且，對於這個基本模型，
04:30
they also used all of that detail
97
252000
2000
他們也使用了全部的細節
04:32
to create a set of what are called "displacement maps"
98
254000
3000
來創建一系列的「置換貼圖」，
04:35
that animate as well.
99
257000
3000
同樣可以對其進行動畫設置。
04:38
These are the displacement maps here.
100
260000
2000
這裡是一些置換貼圖，
04:40
And you can see those different wrinkles actually show up as she animates.
101
262000
3000
你可以看到不同的褶皺出現在她的動畫之中。
04:43
So the next process was then to animate her.
102
265000
2000
所以，接下來的步驟是對她進行動畫設置，
04:45
We actually used one of her own performances to provide the source data.
103
267000
4000
我們實際上應用她自身的表演作為源數據。
04:49
So, by analyzing this video with computer vision techniques,
104
271000
3000
通過計算機視覺技術來分析這些影片，
04:52
they were able to drive the facial rig
105
274000
2000
他們可以使用電腦合成的表演，
04:54
with the computer-generated performance.
106
276000
2000
來驅動表情的變化。
04:56
So what you're going to see now, after this,
107
278000
2000
所以，我們將看到，通過這一切處理，
04:58
is a completely photo-real digital face.
108
280000
2000
是一個完全寫實的數位化臉部。
05:00
We can turn the volume up a little bit if that's available.
109
282000
2000
我們可以把聲音開大一點，如果可以的話。
05:02
Emily: Image Metrics is a markerless, performance-driven animation company.
110
284000
4000
Emily: Image Metrics 是一家追求高品質的動畫公司。
05:06
We specialize in high-quality facial animation
111
288000
3000
我們致力於為電動和電影
05:09
for video games and films.
112
291000
2000
製作高品質的臉部表情動畫。
05:11
Image Metrics is a markerless, performance-driven animation company.
113
293000
4000
Image Metrics 是一家追求高品質動畫的公司。
05:15
We specialize in high quality facial animation
114
297000
3000
我們致力於為電動和電影
05:18
for video games and films.
115
300000
3000
製作高品質的臉部表情動畫。
05:21
Paul Debevec: So, if we break that down into layers, here's that diffuse component
116
303000
2000
保羅•德貝維奇：因此，如果我們把它分解到各個層，這是散射部分，
05:23
we saw in the first slide.
117
305000
2000
我們在第一張投影片裡面看到的。
05:25
Here is the specular component animating.
118
307000
2000
這是高光部分的動畫。
05:27
You can see all the wrinkles happening there.
119
309000
2000
你們可以看到各處皮膚褶皺的出現。
05:29
And there is the underlying wireframe mesh.
120
311000
5000
這是原始的線框結構層。
05:34
And that is Emily herself.
121
316000
2000
那是 Emily 她自己。
05:36
Now, where are we going with this here?
122
318000
2000
現在，我們接下來會做些什麼？
05:38
We've gone a little bit beyond Light Stage 5. This is Light Stage 6,
123
320000
4000
我們已經有了高於「光台五」的裝置設備。這是「光台六」。
05:42
and we're looking at taking this technology
124
324000
2000
我們期望將這一技術
05:44
and applying it to whole human bodies.
125
326000
2000
應用到人的全身。
05:46
This is Bruce Lawmen, one of our researchers in the group,
126
328000
2000
這是布魯斯•勞曼，我們研發團隊中的一位，
05:48
who graciously agreed to get captured running in the Light Stage.
127
330000
3000
非常樂意配合我們，採集他在「光台」中跑步的數據。
05:51
And let's take a look at a computer-generated version
128
333000
2000
接著讓我們看一下電腦合成的，
05:53
of Bruce, running in a new environment.
129
335000
2000
在一個全新的環境中跑步的，布魯斯的影像。
06:01
And thank you very much.
130
343000
2000
感謝大家。
06:03
(Applause)
131
345000
1000
（掌聲）
ABOUT THE SPEAKER
Paul Debevec - Computer graphics pioneerPaul Debevec's digital inventions have powered the breathtaking visual effects in films like The Matrix, Superman Returns, King Kong and The Curious Case of Benjamin Button.
Why you should listen
Called "Hollywood's Master of Light" by MIT's Technology Review, Paul Debevec leads the Graphics Laboratory at USC's Institute for Creative Technologies, where he directs the development of the Light Stage systems, which capture and simulate how people and objects appear in real-world illumination.
His work on lighting digital objects has been used in films like X-Men, The Golden Compass and Terminator: Salvation. He leads the design of HDR Shop, the first high-dynamic-range image editing program. He has collaborated with TEDster Golan Levin on an interactive art installation called Rouen Revisited.
Paul Debevec | Speaker | TED.com