ABOUT THE SPEAKER
Paul Debevec - Computer graphics pioneer
Paul Debevec's digital inventions have powered the breathtaking visual effects in films like The Matrix, Superman Returns, King Kong and The Curious Case of Benjamin Button.

Why you should listen

Called "Hollywood's Master of Light" by MIT's Technology Review, Paul Debevec leads the Graphics Laboratory at USC's Institute for Creative Technologies, where he directs the development of the Light Stage systems, which capture and simulate how people and objects appear in real-world illumination.

His work on lighting digital objects has been used in films like X-Men, The Golden Compass and Terminator: Salvation. He leads the design of HDR Shop, the first high-dynamic-range image editing program. He has collaborated with TEDster Golan Levin on an interactive art installation called Rouen Revisited.

More profile about the speaker
Paul Debevec | Speaker | TED.com
TEDxUSC

Paul Debevec: Animating a photo-real digital face

保罗·德贝维奇:动态化人物面部表情捕捉

Filmed:
881,584 views

计算机图形界的先驱保罗·德贝维奇(Paul Debevec)为我们阐述了备受关注的数字虚拟人物艾米丽(Emily)的幕后制作技术——动态化写实面部表情捕捉。
- Computer graphics pioneer
Paul Debevec's digital inventions have powered the breathtaking visual effects in films like The Matrix, Superman Returns, King Kong and The Curious Case of Benjamin Button. Full bio

Double-click the English transcript below to play the video.

00:18
One of the biggest最大 challenges挑战 in computer电脑 graphics图像
0
0
3000
计算机图形制作面临的巨大挑战之一
00:21
has been being存在 able能够 to create创建 a photo-real照片真实,
1
3000
2000
是如何制作一个真实的
00:23
digital数字 human人的 face面对.
2
5000
2000
人物面部表情。
00:25
And one of the reasons原因 it is so difficult is that, unlike不像 aliens外星人 and dinosaurs恐龙,
3
7000
3000
不像外星人和恐龙,它之所以如此困难的原因是
00:28
we look at human人的 faces面孔 every一切 day.
4
10000
3000
我们每天都会看到人的面部表情
00:31
They are very important重要 to how we communicate通信 with each other.
5
13000
2000
面部表情对我们非常重要,因为我们通过表情来彼此沟通
00:33
As a result结果, we're tuned调整 in to the subtlest微妙 things
6
15000
3000
由此可见我们会注意那些非常微妙的东西
00:36
that could possibly或者 be wrong错误 with a computer电脑 rendering翻译,
7
18000
2000
通过计算机直接渲染出来的很可能是不正确的
00:38
in order订购 to believe whether是否 these things are realistic实际.
8
20000
3000
更无法使人相信这些是真实的
00:41
And what I'm going to do in the next下一个 five minutes分钟
9
23000
2000
在接下来的5分钟里面
00:43
is take you through通过 a process处理
10
25000
2000
我将带领大家一起浏览一个流程
00:45
where we tried试着 to create创建 a reasonably合理 photo-realistic照片般逼真的 computer-generated计算机生成的 face面对,
11
27000
3000
来通过计算机创造一个写实的人物表情
00:48
using运用 some computer电脑 graphics图像 technology技术 we've我们已经 developed发达,
12
30000
3000
通过我们开发的一系列计算机图形技术
00:51
and also some collaborators合作者 at a company公司 called Image图片 Metrics度量.
13
33000
2000
同时和一些来自Image Metrics公司的伙伴
00:53
And we're going to try to do a photo-real照片真实 face面对
14
35000
2000
我们将尝试制作一个写实的
00:55
of an actress演员 named命名 Emily艾米莉 O'Brien奥布莱恩, who is right there.
15
37000
3000
名叫艾米丽(Emily O'Brien)的女演员的表情,她就在那儿。
00:58
And that's actually其实 a completely全然 computer-generated计算机生成的 rendering翻译 of her face面对.
16
40000
3000
那实际上是一个完全由计算机渲染生成的她的面部头像。
01:01
By the end结束 of the talk, we're going to see it move移动.
17
43000
2000
在讲座的结尾,我们将看到她动起来
01:03
The way that we did this is we tried试着 to start开始 with Emily艾米莉 herself她自己,
18
45000
3000
我们尝试的方式是通过Emily她自己
01:06
who was gracious亲切 enough足够 to come to our laboratory实验室
19
48000
3000
她非常优雅高尚地进入我们的实验室
01:09
in Marina码头 Del德尔 Rey雷伊, and sit for a session会议 in Light Stage阶段 5.
20
51000
3000
在玛丽安德尔湾,并坐到了我们的设备灯光舞台5当中
01:12
This is a face-scanning面扫描 sphere领域, with 156 white白色 LEDs发光二极管 all around
21
54000
4000
这是一个面部扫描球体,有156个白色的发光二极管分部在其中,
01:16
that allow允许 us to photograph照片 her
22
58000
2000
配合我们对她进行拍照
01:18
in a series系列 of very controlled受控 illumination照明 conditions条件.
23
60000
3000
通过一系列严格控制的光照条件
01:21
And the lighting灯光 that we use these days looks容貌 something like this.
24
63000
3000
我们近期使用的光照方式看起来是这样的。
01:24
We shoot射击 all of these photographs照片 in about three seconds.
25
66000
2000
我们在三秒钟之内拍摄出全部这些照片。
01:26
And we basically基本上 capture捕获 enough足够 information信息
26
68000
3000
我们主要地会采集足量的信息
01:29
with video视频 projector放映机 patterns模式 that drape悬垂 over the contours轮廓 of her face面对,
27
71000
4000
通过图像投影模拟出她的面部轮廓
01:33
and different不同 principle原理 directions方向 of light from the light stage阶段,
28
75000
2000
以及通过灯光舞台得到的不同角度光照条件下的数据
01:35
to figure数字 out both the coarse-scale粗尺度
29
77000
2000
来计算出粗略的骨骼模型
01:37
and the fine-scale精细尺度 detail详情 of her face面对.
30
79000
2000
以及她的精细的面部细节模型数据
01:39
If we zoom放大 in on this photograph照片 right here,
31
81000
2000
如果我们看放大的这张照片
01:41
we can see it's a really nice不错 photograph照片 to have of her,
32
83000
2000
我们可以很清晰地看到她的面部细节
01:43
because she is lit发光的 from absolutely绝对 everywhere到处 at the same相同 time
33
85000
4000
因为她的面部被同时从每个角度完全地照亮了
01:47
to get a nice不错 image图片 of her facial面部 texture质地.
34
89000
2000
来得到一份非常高质量的她的面部表情贴图
01:49
And in addition加成, we've我们已经 actually其实 used polarizers偏光片 on all the lights灯火 --
35
91000
3000
并且,我们实际上在所有灯上面应用了偏振镜
01:52
just like polarized偏振 sunglasses墨镜 can block
36
94000
2000
就如同偏振的太阳镜可以阻挡
01:54
the glare强光 off of the road,
37
96000
2000
马路上的耀眼光斑
01:56
polarizers偏光片 can block the shine闪耀 off of the skin皮肤,
38
98000
2000
偏振镜可以组织皮肤上出现光芒
01:58
so we don't get all those specular镜面 reflections思考 to take this map地图.
39
100000
3000
所以我们没有采集到这些贴图的高光反射部分
02:01
Now, if we turn the polarizers偏光片 around just a little bit,
40
103000
2000
现在,如果我们稍微调整一下偏振镜
02:03
we can actually其实 bring带来 that specular镜面 reflection反射
41
105000
2000
我们就可以得到高光的反射
02:05
of the skin皮肤 back in,
42
107000
2000
在皮肤上面
02:07
and you can see she looks容貌 kind of shiny闪亮 and oily油腻 at this point.
43
109000
3000
并且,你会看到她的面部看起来有些闪耀和油光。
02:10
If you take the difference区别 between之间 these two images图片 here,
44
112000
3000
如果你取一下两张图像的差
02:13
you can get an image图片 lit发光的 from the entire整个 sphere领域 of light
45
115000
2000
你得到的合成图像就是360度的光照条件下
02:15
of just the shine闪耀 off of Emily's艾米莉的 skin皮肤.
46
117000
3000
艾米丽(Emily)皮肤的光照效果
02:18
I don't think any photograph照片 like this had ever been taken采取
47
120000
2000
我认为没有任何图像会这样拍摄
02:20
before we had doneDONE this.
48
122000
2000
在我们这样做之前。
02:22
And this is very important重要 light to capture捕获,
49
124000
3000
采集光照信息是非常重要的,
02:25
because this is the light that reflects反映 off the first surface表面 of the skin皮肤.
50
127000
2000
因为光照会反射出皮肤表面的最上一层
02:27
It doesn't get underneath the translucent半透明
51
129000
2000
它不会出现在下层半透明层,
02:29
layers of the skin皮肤 and blur模糊 out.
52
131000
2000
它会模糊掉。
02:31
And, as a result结果, it's a very good cue球杆
53
133000
2000
由此可见,它是一个很好的线索
02:33
to the detailed详细 shape形状 of the skin-pore皮肤毛孔 structure结构体
54
135000
3000
来获得皮肤细致的毛孔结构
02:36
and all of the fine wrinkles皱纹 that all of us have,
55
138000
2000
以及我们都有的皮肤褶皱细节的全部,
02:38
the things that actually其实 make us look like real真实 humans人类.
56
140000
2000
实际上,这些都使得它看起来更接近人类的真实效果。
02:40
So, if we use information信息 that comes off of this specular镜面 reflection反射,
57
142000
4000
如果,我们使用来自高光反射的信息
02:44
we can go from a traditional传统 face面对 scan扫描
58
146000
2000
我们可以通过传统的面部扫描而
02:46
that might威力 have the gross contours轮廓 of the face面对 and the basic基本 shape形状,
59
148000
4000
得到面部的粗略的轮廓和基本的形状,
02:50
and augment增加 it with information信息
60
152000
2000
同时输入大量的信息
02:52
that puts看跌期权 in all of that skin皮肤 pore structure结构体 and fine wrinkles皱纹.
61
154000
3000
皮肤的毛孔结构和纹理特征
02:55
And, even more importantly重要的,
62
157000
2000
更重要的是,
02:57
since以来 this is a photometric光度 process处理 that only takes three seconds to capture捕获,
63
159000
3000
因为这是基于照片建模的流程而我们只采集了3秒钟的照片数据
03:00
we can shoot射击 Emily艾米莉
64
162000
2000
我们可以用一下午的时间
03:02
in just part部分 of an afternoon下午,
65
164000
2000
拍艾米丽(Emily),
03:04
in many许多 different不同 facial面部 poses姿势 and facial面部 expressions表达式.
66
166000
3000
拍摄不同的面部姿态和表情变化。
03:07
So, here you can see her moving移动 her eyes眼睛 around, moving移动 her mouth around.
67
169000
3000
这里,你可以看到她转动眼睛,嘴巴的开合。
03:10
And these we're actually其实 going to use to create创建 a photo-real照片真实 digital数字 character字符.
68
172000
3000
这些,实际上都会被我们用来创造数字虚拟的写实人物。
03:13
If you take a look at these scans扫描 that we have of Emily艾米莉,
69
175000
4000
来看一下这些我们对艾米丽(Emily)进行的扫描
03:17
you can see that the human人的 face面对 does an enormous巨大 amount of amazing惊人 things
70
179000
3000
你会看到我们人类面部表情是千变万化的
03:20
as it goes into different不同 facial面部 expressions表达式.
71
182000
3000
在不同的面部表情变换之中
03:23
You can see things. Not only the face面对 shape形状 changes变化,
72
185000
3000
你会看到,不仅仅是面部形态发生变化
03:26
but all sorts排序 of different不同 skin皮肤 buckling屈曲 and skin皮肤 wrinkling起皱 occurs发生.
73
188000
4000
以及各种各样的皮肤的弧线和皮肤上的褶皱出现。
03:30
You can see that the skin皮肤 pore structure结构体 changes变化 enormously巨大
74
192000
4000
你可以看到皮肤毛孔结构也有着巨大的变化
03:34
from stretched拉伸 skin皮肤 pores毛孔
75
196000
2000
从被拉伸的皮肤毛孔,
03:36
to the regular定期 skin皮肤 texture质地.
76
198000
2000
到普通的皮肤质地
03:38
You can see the furrows in the brow眉头 and how the microstructure微观结构 changes变化 there.
77
200000
3000
你可以看到前额眉骨那里细微的变化。
03:41
You can see muscles肌肉 pulling down at flesh to bring带来 her eyebrows眉毛 down.
78
203000
4000
你可以看到肌肉的收缩在她皱眉的时候。
03:45
Her muscles肌肉 bulging挺着 in her forehead前额 when she winces畏缩 like that.
79
207000
3000
她前额的肌肉会这样的膨胀,在她畏惧的时候。
03:48
In addition加成 to this kind of high-resolution高分辨率 geometry几何,
80
210000
3000
对于这样高分辨率的几何体,
03:51
since以来 it's all captured捕获 with cameras相机, we've我们已经 got a great texture质地 map地图 to use for the face面对.
81
213000
4000
因为这些都通过照相机采集,我们有足够的她的面部表情贴图。
03:55
And by looking at how the different不同 color颜色 channels渠道 of the illumination照明,
82
217000
2000
通过颜色和亮度不同的通道,
03:57
the red and the green绿色 and the blue蓝色,
83
219000
2000
红绿蓝色彩通道
03:59
diffuse扩散 the light differently不同,
84
221000
2000
受光照不同的漫反射
04:01
we can come up with a way of shading阴影 the skin皮肤 on the computer电脑.
85
223000
3000
我们可以通过计算机总结出为皮肤添加阴影的方式。
04:04
Then, instead代替 of looking like a plaster石膏 mannequin模特,
86
226000
3000
那么,不同于看起来如同石膏像的假人,
04:07
it actually其实 looks容貌 like it's made制作 out of living活的 human人的 flesh.
87
229000
3000
她看起来如同人类一样鲜活。
04:10
And this is what we used
88
232000
2000
这就是我们使用的
04:12
to give to the company公司 Image图片 Metrics度量
89
234000
2000
为Image Metrics公司
04:14
to create创建 a rigged非法操纵的, digital数字 version of Emily艾米莉.
90
236000
3000
制作的写实的数字虚拟人物Emily。
04:17
We're just seeing眼看 the coarse-scale粗尺度 geometry几何 here.
91
239000
2000
这里,我们看看一下粗糙的几何模型
04:19
But they basically基本上 created创建 a digital数字 puppet木偶 of her,
92
241000
3000
但他们已经对她创建了一些基本的数据程式,
04:22
where you can pull on these various各个 strings字符串,
93
244000
2000
你可以拖拽这些不同的坐标,
04:24
and it actually其实 moves移动 her face面对 in ways方法 that are
94
246000
2000
它实际上运动的方式就会依照
04:26
completely全然 consistent一贯 with the scans扫描 that we took.
95
248000
2000
我们扫描出的数据那样。
04:28
And, in addition加成 to the coarse-scale粗尺度 geometry几何,
96
250000
2000
并且,对于这个基本模型,
04:30
they also used all of that detail详情
97
252000
2000
他们也使用了全部的细节
04:32
to create创建 a set of what are called "displacement移位 maps地图"
98
254000
3000
来创建一些列的置换贴图
04:35
that animate活跃 as well.
99
257000
3000
同样可以对其进行动画设置。
04:38
These are the displacement移位 maps地图 here.
100
260000
2000
这里是一些置换贴图
04:40
And you can see those different不同 wrinkles皱纹 actually其实 show显示 up as she animates动画.
101
262000
3000
同时你可以看到不同的褶皱出现在她的动画之中。
04:43
So the next下一个 process处理 was then to animate活跃 her.
102
265000
2000
所以,接下来的步骤是对她进行动画设置。
04:45
We actually其实 used one of her own拥有 performances演出 to provide提供 the source资源 data数据.
103
267000
4000
我们实际上应用她自身的表演作为源数据。
04:49
So, by analyzing分析 this video视频 with computer电脑 vision视力 techniques技术,
104
271000
3000
通过计算机视觉技术来分析这些视频,
04:52
they were able能够 to drive驾驶 the facial面部 rig操纵
105
274000
2000
他们可以驱动表情的变化
04:54
with the computer-generated计算机生成的 performance性能.
106
276000
2000
通过计算机的预置效果生成。
04:56
So what you're going to see now, after this,
107
278000
2000
所以,我们将看到,通过这一切处理,
04:58
is a completely全然 photo-real照片真实 digital数字 face面对.
108
280000
2000
是一个完全写实的数字话面部。
05:00
We can turn the volume up a little bit if that's available可得到.
109
282000
2000
我们可以开大一点声音,如果可以的话。
05:02
Emily艾米莉: Image图片 Metrics度量 is a markerless无标记, performance-driven业绩驱动 animation动画 company公司.
110
284000
4000
Emily:Image Metrics是一家追求高效的动画公司
05:06
We specialize专攻 in high-quality高质量 facial面部 animation动画
111
288000
3000
我们致力于制作高质量的面部表情动画
05:09
for video视频 games游戏 and films影片.
112
291000
2000
为视频游戏和电影。
05:11
Image图片 Metrics度量 is a markerless无标记, performance-driven业绩驱动 animation动画 company公司.
113
293000
4000
Image Metrics是一家追求高效动画的公司。
05:15
We specialize专攻 in high quality质量 facial面部 animation动画
114
297000
3000
我们致力于制作高质量的面部表情动画
05:18
for video视频 games游戏 and films影片.
115
300000
3000
为视频游戏和电影。
05:21
Paul保罗 Debevec中找到适合: So, if we break打破 that down into layers, here's这里的 that diffuse扩散 component零件
116
303000
2000
保罗·德贝维奇:如果我们把它分解到各个层,这是漫反射部分
05:23
we saw in the first slide滑动.
117
305000
2000
我们在第一张幻灯片里面看到的。
05:25
Here is the specular镜面 component零件 animating动画.
118
307000
2000
这是高光部分的动画。
05:27
You can see all the wrinkles皱纹 happening事件 there.
119
309000
2000
你们可以看到各处皮肤褶皱的出现。
05:29
And there is the underlying底层 wireframe线框 mesh网孔.
120
311000
5000
这是原始的线框结构层。
05:34
And that is Emily艾米莉 herself她自己.
121
316000
2000
那是Emily她自己。
05:36
Now, where are we going with this here?
122
318000
2000
现在,我们接下来会做些什么?
05:38
We've我们已经 gone走了 a little bit beyond Light Stage阶段 5. This is Light Stage阶段 6,
123
320000
4000
我们已经有了高于灯光舞台5的装置设备。这是灯光舞台6.
05:42
and we're looking at taking服用 this technology技术
124
324000
2000
我们期望将这一技术
05:44
and applying应用 it to whole整个 human人的 bodies身体.
125
326000
2000
应用到人的全身。
05:46
This is Bruce布鲁斯 Lawmen执法官, one of our researchers研究人员 in the group,
126
328000
2000
这是布鲁斯 劳曼,我们研发团队中的一位,
05:48
who graciously和蔼地 agreed同意 to get captured捕获 running赛跑 in the Light Stage阶段.
127
330000
3000
非常乐意配合我们采集他在灯光舞台中跑步的数据。
05:51
And let's take a look at a computer-generated计算机生成的 version
128
333000
2000
下面让我们看一下计算机生成的
05:53
of Bruce布鲁斯, running赛跑 in a new environment环境.
129
335000
2000
布鲁斯的影像,在一个全新的环境中跑步。
06:01
And thank you very much.
130
343000
2000
非常感谢。
06:03
(Applause掌声)
131
345000
1000
(掌声)
Translated by Yin`ai Sun
Reviewed by Halei Liu

▲Back to top

ABOUT THE SPEAKER
Paul Debevec - Computer graphics pioneer
Paul Debevec's digital inventions have powered the breathtaking visual effects in films like The Matrix, Superman Returns, King Kong and The Curious Case of Benjamin Button.

Why you should listen

Called "Hollywood's Master of Light" by MIT's Technology Review, Paul Debevec leads the Graphics Laboratory at USC's Institute for Creative Technologies, where he directs the development of the Light Stage systems, which capture and simulate how people and objects appear in real-world illumination.

His work on lighting digital objects has been used in films like X-Men, The Golden Compass and Terminator: Salvation. He leads the design of HDR Shop, the first high-dynamic-range image editing program. He has collaborated with TEDster Golan Levin on an interactive art installation called Rouen Revisited.

More profile about the speaker
Paul Debevec | Speaker | TED.com