TEDxUSC
Paul Debevec: Animating a photo-real digital face
保罗·德贝维奇：动态化人物面部表情捕捉
Filmed:
Readability: 4.8
881,584 views
计算机图形界的先驱保罗·德贝维奇（Paul Debevec）为我们阐述了备受关注的数字虚拟人物艾米丽（Emily）的幕后制作技术——动态化写实面部表情捕捉。
Paul Debevec - Computer graphics pioneer
Paul Debevec's digital inventions have powered the breathtaking visual effects in films like The Matrix, Superman Returns, King Kong and The Curious Case of Benjamin Button. Full bio
Paul Debevec's digital inventions have powered the breathtaking visual effects in films like The Matrix, Superman Returns, King Kong and The Curious Case of Benjamin Button. Full bio
Double-click the English transcript below to play the video.
00:18
One of the biggest challenges in computer graphics
0
0
3000
计算机图形制作面临的巨大挑战之一
00:21
has been being able to create a photo-real,
1
3000
2000
是如何制作一个真实的
00:23
digital human face.
2
5000
2000
人物面部表情。
00:25
And one of the reasons it is so difficult is that, unlike aliens and dinosaurs,
3
7000
3000
不像外星人和恐龙，它之所以如此困难的原因是
00:28
we look at human faces every day.
4
10000
3000
我们每天都会看到人的面部表情
00:31
They are very important to how we communicate with each other.
5
13000
2000
面部表情对我们非常重要，因为我们通过表情来彼此沟通
00:33
As a result, we're tuned in to the subtlest things
6
15000
3000
由此可见我们会注意那些非常微妙的东西
00:36
that could possibly be wrong with a computer rendering,
7
18000
2000
通过计算机直接渲染出来的很可能是不正确的
00:38
in order to believe whether these things are realistic.
8
20000
3000
更无法使人相信这些是真实的
00:41
And what I'm going to do in the next five minutes
9
23000
2000
在接下来的5分钟里面
00:43
is take you through a process
10
25000
2000
我将带领大家一起浏览一个流程
00:45
where we tried to create a reasonably photo-realistic computer-generated face,
11
27000
3000
来通过计算机创造一个写实的人物表情
00:48
using some computer graphics technology we've developed,
12
30000
3000
通过我们开发的一系列计算机图形技术
00:51
and also some collaborators at a company called Image Metrics.
13
33000
2000
同时和一些来自Image Metrics公司的伙伴
00:53
And we're going to try to do a photo-real face
14
35000
2000
我们将尝试制作一个写实的
00:55
of an actress named Emily O'Brien, who is right there.
15
37000
3000
名叫艾米丽(Emily O'Brien)的女演员的表情，她就在那儿。
00:58
And that's actually a completely computer-generated rendering of her face.
16
40000
3000
那实际上是一个完全由计算机渲染生成的她的面部头像。
01:01
By the end of the talk, we're going to see it move.
17
43000
2000
在讲座的结尾，我们将看到她动起来
01:03
The way that we did this is we tried to start with Emily herself,
18
45000
3000
我们尝试的方式是通过Emily她自己
01:06
who was gracious enough to come to our laboratory
19
48000
3000
她非常优雅高尚地进入我们的实验室
01:09
in Marina Del Rey, and sit for a session in Light Stage 5.
20
51000
3000
在玛丽安德尔湾，并坐到了我们的设备灯光舞台5当中
01:12
This is a face-scanning sphere, with 156 white LEDs all around
21
54000
4000
这是一个面部扫描球体，有156个白色的发光二极管分部在其中，
01:16
that allow us to photograph her
22
58000
2000
配合我们对她进行拍照
01:18
in a series of very controlled illumination conditions.
23
60000
3000
通过一系列严格控制的光照条件
01:21
And the lighting that we use these days looks something like this.
24
63000
3000
我们近期使用的光照方式看起来是这样的。
01:24
We shoot all of these photographs in about three seconds.
25
66000
2000
我们在三秒钟之内拍摄出全部这些照片。
01:26
And we basically capture enough information
26
68000
3000
我们主要地会采集足量的信息
01:29
with video projector patterns that drape over the contours of her face,
27
71000
4000
通过图像投影模拟出她的面部轮廓
01:33
and different principle directions of light from the light stage,
28
75000
2000
以及通过灯光舞台得到的不同角度光照条件下的数据
01:35
to figure out both the coarse-scale
29
77000
2000
来计算出粗略的骨骼模型
01:37
and the fine-scale detail of her face.
30
79000
2000
以及她的精细的面部细节模型数据
01:39
If we zoom in on this photograph right here,
31
81000
2000
如果我们看放大的这张照片
01:41
we can see it's a really nice photograph to have of her,
32
83000
2000
我们可以很清晰地看到她的面部细节
01:43
because she is lit from absolutely everywhere at the same time
33
85000
4000
因为她的面部被同时从每个角度完全地照亮了
01:47
to get a nice image of her facial texture.
34
89000
2000
来得到一份非常高质量的她的面部表情贴图
01:49
And in addition, we've actually used polarizers on all the lights --
35
91000
3000
并且，我们实际上在所有灯上面应用了偏振镜
01:52
just like polarized sunglasses can block
36
94000
2000
就如同偏振的太阳镜可以阻挡
01:54
the glare off of the road,
37
96000
2000
马路上的耀眼光斑
01:56
polarizers can block the shine off of the skin,
38
98000
2000
偏振镜可以组织皮肤上出现光芒
01:58
so we don't get all those specular reflections to take this map.
39
100000
3000
所以我们没有采集到这些贴图的高光反射部分
02:01
Now, if we turn the polarizers around just a little bit,
40
103000
2000
现在，如果我们稍微调整一下偏振镜
02:03
we can actually bring that specular reflection
41
105000
2000
我们就可以得到高光的反射
02:05
of the skin back in,
42
107000
2000
在皮肤上面
02:07
and you can see she looks kind of shiny and oily at this point.
43
109000
3000
并且，你会看到她的面部看起来有些闪耀和油光。
02:10
If you take the difference between these two images here,
44
112000
3000
如果你取一下两张图像的差
02:13
you can get an image lit from the entire sphere of light
45
115000
2000
你得到的合成图像就是360度的光照条件下
02:15
of just the shine off of Emily's skin.
46
117000
3000
艾米丽(Emily)皮肤的光照效果
02:18
I don't think any photograph like this had ever been taken
47
120000
2000
我认为没有任何图像会这样拍摄
02:20
before we had done this.
48
122000
2000
在我们这样做之前。
02:22
And this is very important light to capture,
49
124000
3000
采集光照信息是非常重要的，
02:25
because this is the light that reflects off the first surface of the skin.
50
127000
2000
因为光照会反射出皮肤表面的最上一层
02:27
It doesn't get underneath the translucent
51
129000
2000
它不会出现在下层半透明层，
02:29
layers of the skin and blur out.
52
131000
2000
它会模糊掉。
02:31
And, as a result, it's a very good cue
53
133000
2000
由此可见，它是一个很好的线索
02:33
to the detailed shape of the skin-pore structure
54
135000
3000
来获得皮肤细致的毛孔结构
02:36
and all of the fine wrinkles that all of us have,
55
138000
2000
以及我们都有的皮肤褶皱细节的全部，
02:38
the things that actually make us look like real humans.
56
140000
2000
实际上，这些都使得它看起来更接近人类的真实效果。
02:40
So, if we use information that comes off of this specular reflection,
57
142000
4000
如果，我们使用来自高光反射的信息
02:44
we can go from a traditional face scan
58
146000
2000
我们可以通过传统的面部扫描而
02:46
that might have the gross contours of the face and the basic shape,
59
148000
4000
得到面部的粗略的轮廓和基本的形状，
02:50
and augment it with information
60
152000
2000
同时输入大量的信息
02:52
that puts in all of that skin pore structure and fine wrinkles.
61
154000
3000
皮肤的毛孔结构和纹理特征
02:55
And, even more importantly,
62
157000
2000
更重要的是，
02:57
since this is a photometric process that only takes three seconds to capture,
63
159000
3000
因为这是基于照片建模的流程而我们只采集了3秒钟的照片数据
03:00
we can shoot Emily
64
162000
2000
我们可以用一下午的时间
03:02
in just part of an afternoon,
65
164000
2000
拍艾米丽(Emily),
03:04
in many different facial poses and facial expressions.
66
166000
3000
拍摄不同的面部姿态和表情变化。
03:07
So, here you can see her moving her eyes around, moving her mouth around.
67
169000
3000
这里，你可以看到她转动眼睛，嘴巴的开合。
03:10
And these we're actually going to use to create a photo-real digital character.
68
172000
3000
这些，实际上都会被我们用来创造数字虚拟的写实人物。
03:13
If you take a look at these scans that we have of Emily,
69
175000
4000
来看一下这些我们对艾米丽(Emily)进行的扫描
03:17
you can see that the human face does an enormous amount of amazing things
70
179000
3000
你会看到我们人类面部表情是千变万化的
03:20
as it goes into different facial expressions.
71
182000
3000
在不同的面部表情变换之中
03:23
You can see things. Not only the face shape changes,
72
185000
3000
你会看到，不仅仅是面部形态发生变化
03:26
but all sorts of different skin buckling and skin wrinkling occurs.
73
188000
4000
以及各种各样的皮肤的弧线和皮肤上的褶皱出现。
03:30
You can see that the skin pore structure changes enormously
74
192000
4000
你可以看到皮肤毛孔结构也有着巨大的变化
03:34
from stretched skin pores
75
196000
2000
从被拉伸的皮肤毛孔，
03:36
to the regular skin texture.
76
198000
2000
到普通的皮肤质地
03:38
You can see the furrows in the brow and how the microstructure changes there.
77
200000
3000
你可以看到前额眉骨那里细微的变化。
03:41
You can see muscles pulling down at flesh to bring her eyebrows down.
78
203000
4000
你可以看到肌肉的收缩在她皱眉的时候。
03:45
Her muscles bulging in her forehead when she winces like that.
79
207000
3000
她前额的肌肉会这样的膨胀，在她畏惧的时候。
03:48
In addition to this kind of high-resolution geometry,
80
210000
3000
对于这样高分辨率的几何体，
03:51
since it's all captured with cameras, we've got a great texture map to use for the face.
81
213000
4000
因为这些都通过照相机采集，我们有足够的她的面部表情贴图。
03:55
And by looking at how the different color channels of the illumination,
82
217000
2000
通过颜色和亮度不同的通道，
03:57
the red and the green and the blue,
83
219000
2000
红绿蓝色彩通道
03:59
diffuse the light differently,
84
221000
2000
受光照不同的漫反射
04:01
we can come up with a way of shading the skin on the computer.
85
223000
3000
我们可以通过计算机总结出为皮肤添加阴影的方式。
04:04
Then, instead of looking like a plaster mannequin,
86
226000
3000
那么，不同于看起来如同石膏像的假人，
04:07
it actually looks like it's made out of living human flesh.
87
229000
3000
她看起来如同人类一样鲜活。
04:10
And this is what we used
88
232000
2000
这就是我们使用的
04:12
to give to the company Image Metrics
89
234000
2000
为Image Metrics公司
04:14
to create a rigged, digital version of Emily.
90
236000
3000
制作的写实的数字虚拟人物Emily。
04:17
We're just seeing the coarse-scale geometry here.
91
239000
2000
这里，我们看看一下粗糙的几何模型
04:19
But they basically created a digital puppet of her,
92
241000
3000
但他们已经对她创建了一些基本的数据程式，
04:22
where you can pull on these various strings,
93
244000
2000
你可以拖拽这些不同的坐标，
04:24
and it actually moves her face in ways that are
94
246000
2000
它实际上运动的方式就会依照
04:26
completely consistent with the scans that we took.
95
248000
2000
我们扫描出的数据那样。
04:28
And, in addition to the coarse-scale geometry,
96
250000
2000
并且，对于这个基本模型，
04:30
they also used all of that detail
97
252000
2000
他们也使用了全部的细节
04:32
to create a set of what are called "displacement maps"
98
254000
3000
来创建一些列的置换贴图
04:35
that animate as well.
99
257000
3000
同样可以对其进行动画设置。
04:38
These are the displacement maps here.
100
260000
2000
这里是一些置换贴图
04:40
And you can see those different wrinkles actually show up as she animates.
101
262000
3000
同时你可以看到不同的褶皱出现在她的动画之中。
04:43
So the next process was then to animate her.
102
265000
2000
所以，接下来的步骤是对她进行动画设置。
04:45
We actually used one of her own performances to provide the source data.
103
267000
4000
我们实际上应用她自身的表演作为源数据。
04:49
So, by analyzing this video with computer vision techniques,
104
271000
3000
通过计算机视觉技术来分析这些视频，
04:52
they were able to drive the facial rig
105
274000
2000
他们可以驱动表情的变化
04:54
with the computer-generated performance.
106
276000
2000
通过计算机的预置效果生成。
04:56
So what you're going to see now, after this,
107
278000
2000
所以，我们将看到，通过这一切处理，
04:58
is a completely photo-real digital face.
108
280000
2000
是一个完全写实的数字话面部。
05:00
We can turn the volume up a little bit if that's available.
109
282000
2000
我们可以开大一点声音，如果可以的话。
05:02
Emily: Image Metrics is a markerless, performance-driven animation company.
110
284000
4000
Emily:Image Metrics是一家追求高效的动画公司
05:06
We specialize in high-quality facial animation
111
288000
3000
我们致力于制作高质量的面部表情动画
05:09
for video games and films.
112
291000
2000
为视频游戏和电影。
05:11
Image Metrics is a markerless, performance-driven animation company.
113
293000
4000
Image Metrics是一家追求高效动画的公司。
05:15
We specialize in high quality facial animation
114
297000
3000
我们致力于制作高质量的面部表情动画
05:18
for video games and films.
115
300000
3000
为视频游戏和电影。
05:21
Paul Debevec: So, if we break that down into layers, here's that diffuse component
116
303000
2000
保罗·德贝维奇：如果我们把它分解到各个层，这是漫反射部分
05:23
we saw in the first slide.
117
305000
2000
我们在第一张幻灯片里面看到的。
05:25
Here is the specular component animating.
118
307000
2000
这是高光部分的动画。
05:27
You can see all the wrinkles happening there.
119
309000
2000
你们可以看到各处皮肤褶皱的出现。
05:29
And there is the underlying wireframe mesh.
120
311000
5000
这是原始的线框结构层。
05:34
And that is Emily herself.
121
316000
2000
那是Emily她自己。
05:36
Now, where are we going with this here?
122
318000
2000
现在，我们接下来会做些什么？
05:38
We've gone a little bit beyond Light Stage 5. This is Light Stage 6,
123
320000
4000
我们已经有了高于灯光舞台5的装置设备。这是灯光舞台6.
05:42
and we're looking at taking this technology
124
324000
2000
我们期望将这一技术
05:44
and applying it to whole human bodies.
125
326000
2000
应用到人的全身。
05:46
This is Bruce Lawmen, one of our researchers in the group,
126
328000
2000
这是布鲁斯 劳曼，我们研发团队中的一位，
05:48
who graciously agreed to get captured running in the Light Stage.
127
330000
3000
非常乐意配合我们采集他在灯光舞台中跑步的数据。
05:51
And let's take a look at a computer-generated version
128
333000
2000
下面让我们看一下计算机生成的
05:53
of Bruce, running in a new environment.
129
335000
2000
布鲁斯的影像，在一个全新的环境中跑步。
06:01
And thank you very much.
130
343000
2000
非常感谢。
06:03
(Applause)
131
345000
1000
（掌声）
ABOUT THE SPEAKER
Paul Debevec - Computer graphics pioneerPaul Debevec's digital inventions have powered the breathtaking visual effects in films like The Matrix, Superman Returns, King Kong and The Curious Case of Benjamin Button.
Why you should listen
Called "Hollywood's Master of Light" by MIT's Technology Review, Paul Debevec leads the Graphics Laboratory at USC's Institute for Creative Technologies, where he directs the development of the Light Stage systems, which capture and simulate how people and objects appear in real-world illumination.
His work on lighting digital objects has been used in films like X-Men, The Golden Compass and Terminator: Salvation. He leads the design of HDR Shop, the first high-dynamic-range image editing program. He has collaborated with TEDster Golan Levin on an interactive art installation called Rouen Revisited.
Paul Debevec | Speaker | TED.com