ABOUT THE SPEAKER
Stuart Russell - AI expert
Stuart Russell wrote the standard text on AI; now he thinks deeply on AI's future -- and the future of us humans, too.

Why you should listen

Stuart Russell is a professor (and formerly chair) of Electrical Engineering and Computer Sciences at University of California at Berkeley. His book Artificial Intelligence: A Modern Approach (with Peter Norvig) is the standard text in AI; it has been translated into 13 languages and is used in more than 1,300 universities in 118 countries. His research covers a wide range of topics in artificial intelligence including machine learning, probabilistic reasoning, knowledge representation, planning, real-time decision making, multitarget tracking, computer vision, computational physiology, global seismic monitoring and philosophical foundations.

He also works for the United Nations, developing a new global seismic monitoring system for the nuclear-test-ban treaty. His current concerns include the threat of autonomous weapons and the long-term future of artificial intelligence and its relation to humanity.

More profile about the speaker
Stuart Russell | Speaker | TED.com
TED2017

Stuart Russell: 3 principles for creating safer AI

斯图尔特·罗素: 人工智能是如何让我们变得更好的

Filmed:
1,465,832 views

我们应该如何在发挥人工智能最大用途的同时,预防那些机器人可能带来的威胁呢?随着人工智能的日益完善和发展,人工智能先驱斯图尔特·罗素正在创造一些不同的东西:那就是具有无限可能的机器人。让我们听听他对人类该如何兼容人工智能的看法,如何才能真正利用人工智能使其利用常识、利他主义以及人类的价值观来解决问题。
- AI expert
Stuart Russell wrote the standard text on AI; now he thinks deeply on AI's future -- and the future of us humans, too. Full bio

Double-click the English transcript below to play the video.

这是李世石。
00:12
This is Lee背风处 SedolSEDOL.
0
712
1552
李世石是全世界
最顶尖的围棋高手之一,
00:14
Lee背风处 SedolSEDOL is one of the world's世界
greatest最大 Go players玩家,
1
2288
3997
在这一刻,他所经历的
足以让我硅谷的朋友们
00:18
and he's having what my friends朋友
in Silicon Valley call
2
6309
2885
00:21
a "Holy Cow" moment时刻 --
3
9218
1510
喊一句”我的天啊“——
00:22
(Laughter笑声)
4
10752
1073
(笑声)
00:23
a moment时刻 where we realize实现
5
11849
2188
在这一刻,我们意识到
原来人工智能发展的进程
比我们预想的要快得多。
00:26
that AIAI is actually其实 progressing进展
a lot faster更快 than we expected预期.
6
14061
3296
00:30
So humans人类 have lost丢失 on the Go board.
What about the real真实 world世界?
7
18154
3047
人们在围棋棋盘上已经输了,
那在现实世界中又如何呢?
当然了,现实世界要
比围棋棋盘要大得多,
00:33
Well, the real真实 world世界 is much bigger,
8
21225
2100
复杂得多。
00:35
much more complicated复杂 than the Go board.
9
23349
2249
相比之下每一步也没那么明确,
00:37
It's a lot less visible可见,
10
25622
1819
00:39
but it's still a decision决定 problem问题.
11
27465
2038
但现实世界仍然是一个选择性问题。
00:42
And if we think about some
of the technologies技术
12
30948
2321
如果我们想想那一些在不久的未来,
即将来临的新科技……
00:45
that are coming未来 down the pike梭子鱼 ...
13
33293
1749
00:47
Noriko纪子 [Arai新井] mentioned提到 that reading
is not yet然而 happening事件 in machines,
14
35738
4335
Noriko提到机器还不能进行阅读,
至少达不到理解的程度,
00:52
at least最小 with understanding理解.
15
40097
1500
但这迟早会发生,
00:53
But that will happen发生,
16
41621
1536
而当它发生时,
00:55
and when that happens发生,
17
43181
1771
不久之后,
00:56
very soon不久 afterwards之后,
18
44976
1187
机器就将读遍人类写下的所有东西。
00:58
machines will have read everything
that the human人的 race种族 has ever written书面.
19
46187
4572
01:03
And that will enable启用 machines,
20
51850
2030
这将使机器除了拥有
比人类看得更远的能力,
01:05
along沿 with the ability能力 to look
further进一步 ahead than humans人类 can,
21
53904
2920
就像我们在围棋中看到的那样,
01:08
as we've我们已经 already已经 seen看到 in Go,
22
56848
1680
如果机器能接触到比人类更多的信息,
01:10
if they also have access访问
to more information信息,
23
58552
2164
则将能够在现实世界中
做出比人类更好的选择。
01:12
they'll他们会 be able能够 to make better decisions决定
in the real真实 world世界 than we can.
24
60740
4268
那这是一件好事吗?
01:18
So is that a good thing?
25
66792
1606
01:21
Well, I hope希望 so.
26
69898
2232
我当然希望如此。
01:26
Our entire整个 civilization文明,
everything that we value,
27
74694
3255
人类的全部文明,
我们所珍视的一切,
都是基于我们的智慧之上。
01:29
is based基于 on our intelligence情报.
28
77973
2068
如果我们能掌控更强大的智能,
01:32
And if we had access访问
to a lot more intelligence情报,
29
80065
3694
那我们人类的 创造力
就真的没有极限了。
01:35
then there's really no limit限制
to what the human人的 race种族 can do.
30
83783
3302
01:40
And I think this could be,
as some people have described描述 it,
31
88665
3325
我认为这可能就像很多人描述的那样
会成为人类历史上最重要的事件。
01:44
the biggest最大 event事件 in human人的 history历史.
32
92014
2016
01:48
So why are people saying things like this,
33
96665
2829
那为什么有的人会说出以下的言论,
说人工智能将是人类的末日呢?
01:51
that AIAI might威力 spell拼写 the end结束
of the human人的 race种族?
34
99518
2876
01:55
Is this a new thing?
35
103438
1659
这是一个新事物吗?
这只关乎伊隆马斯克、
比尔盖茨,和斯提芬霍金吗?
01:57
Is it just Elon伊隆 Musk and Bill法案 Gates盖茨
and Stephen斯蒂芬 Hawking霍金?
36
105121
4110
02:01
Actually其实, no. This idea理念
has been around for a while.
37
109953
3262
其实不是的,人工智能
这个概念已经存在很长时间了。
请看这段话:
02:05
Here's这里的 a quotation行情:
38
113239
1962
“即便我们能够将机器
维持在一个屈服于我们的地位,
02:07
"Even if we could keep the machines
in a subservient奴颜婢膝 position位置,
39
115225
4350
比如说,在战略性时刻将电源关闭。”——
02:11
for instance, by turning车削 off the power功率
at strategic战略 moments瞬间" --
40
119599
2984
我等会儿再来讨论
”关闭电源“这一话题,
02:14
and I'll come back to that
"turning车削 off the power功率" idea理念 later后来 on --
41
122607
3237
”我们,作为一个物种,
仍然应该自感惭愧。“
02:17
"we should, as a species种类,
feel greatly非常 humbled自愧不如."
42
125868
2804
02:22
So who said this?
This is Alan艾伦 Turing图灵 in 1951.
43
130177
3448
这段话是谁说的呢?
是阿兰图灵,他在1951年说的。
02:26
Alan艾伦 Turing图灵, as you know,
is the father父亲 of computer电脑 science科学
44
134300
2763
阿兰图灵,众所皆知,
是计算机科学之父。
从很多意义上说,
他也是人工智能之父。
02:29
and in many许多 ways方法,
the father父亲 of AIAI as well.
45
137087
3048
02:33
So if we think about this problem问题,
46
141239
1882
当我们考虑这个问题,
创造一个比自己更智能的
物种的问题时,
02:35
the problem问题 of creating创建 something
more intelligent智能 than your own拥有 species种类,
47
143145
3787
我们不妨将它称为”大猩猩问题“,
02:38
we might威力 call this "the gorilla大猩猩 problem问题,"
48
146956
2622
02:42
because gorillas'大猩猩 ancestors祖先 did this
a few少数 million百万 years年份 ago,
49
150345
3750
因为这正是大猩猩的
祖先们几百万年前所经历的。
我们今天可以去问大猩猩们:
02:46
and now we can ask the gorillas大猩猩:
50
154119
1745
02:48
Was this a good idea理念?
51
156752
1160
那么做是不是一个好主意?
在这幅图里,大猩猩们正在
开会讨论那么做是不是一个好主意,
02:49
So here they are having a meeting会议
to discuss讨论 whether是否 it was a good idea理念,
52
157936
3530
片刻后他们下定结论,不是的。
02:53
and after a little while,
they conclude得出结论, no,
53
161490
3346
那是一个很糟糕的主意。
02:56
this was a terrible可怕 idea理念.
54
164860
1345
我们的物种已经奄奄一息了,
02:58
Our species种类 is in dire可怕的 straits海峡.
55
166229
1782
03:00
In fact事实, you can see the existential存在
sadness in their eyes眼睛.
56
168538
4263
你都可以从它们的眼神中看到这种忧伤,
(笑声)
03:04
(Laughter笑声)
57
172825
1640
所以创造比你自己更聪明的物种,
03:06
So this queasy动荡 feeling感觉 that making制造
something smarter聪明 than your own拥有 species种类
58
174489
4840
也许不是一个好主意——
03:11
is maybe not a good idea理念 --
59
179353
2365
03:14
what can we do about that?
60
182488
1491
那我们能做些什么呢?
其实没什么能做的,
除了停止研究人工智能,
03:16
Well, really nothing,
except stop doing AIAI,
61
184003
4767
03:20
and because of all
the benefits好处 that I mentioned提到
62
188794
2510
但因为人工智能能带来
我之前所说的诸多益处,
也因为我是
人工智能的研究者之一,
03:23
and because I'm an AIAI researcher研究员,
63
191328
1716
我可不同意就这么止步。
03:25
I'm not having that.
64
193068
1791
03:27
I actually其实 want to be able能够
to keep doing AIAI.
65
195283
2468
实际上,我想继续做人工智能。
03:30
So we actually其实 need to nail down
the problem问题 a bit more.
66
198615
2678
所以我们需要把这个问题更细化一点,
它到底是什么呢?
03:33
What exactly究竟 is the problem问题?
67
201317
1371
那就是为什么更强大的
人工智能可能会是灾难呢?
03:34
Why is better AIAI possibly或者 a catastrophe灾难?
68
202712
3246
03:39
So here's这里的 another另一个 quotation行情:
69
207398
1498
再来看这段话:
03:41
"We had better be quite相当 sure
that the purpose目的 put into the machine
70
209935
3335
”我们一定得确保我们
给机器输入的目的和价值
是我们确实想要的目的和价值。“
03:45
is the purpose目的 which哪一个 we really desire欲望."
71
213294
2298
03:48
This was said by Norbert诺伯特 Wiener维纳 in 1960,
72
216282
3498
这是诺博特维纳在1960年说的,
他说这话时是刚看到
一个早期的学习系统,
03:51
shortly不久 after he watched看着
one of the very early learning学习 systems系统
73
219804
4002
这个系统在学习如何能把
西洋棋下得比它的创造者更好。
03:55
learn学习 to play checkers跳棋
better than its creator创造者.
74
223830
2583
04:00
But this could equally一样 have been said
75
228602
2683
与此如出一辙的一句话,
迈达斯国王也说过。
04:03
by King国王 Midas迈达斯.
76
231309
1167
04:05
King国王 Midas迈达斯 said, "I want everything
I touch触摸 to turn to gold,"
77
233083
3134
迈达斯国王说:”我希望
我触碰的所有东西都变成金子。“
结果他真的获得了点石成金的能力。
04:08
and he got exactly究竟 what he asked for.
78
236241
2473
那就是他所输入的目的,
04:10
That was the purpose目的
that he put into the machine,
79
238738
2751
从一定程度上说,
04:13
so to speak说话,
80
241513
1450
后来他的食物、
他的家人都变成了金子,
04:14
and then his food餐饮 and his drink
and his relatives亲戚们 turned转身 to gold
81
242987
3444
他死在痛苦与饥饿之中。
04:18
and he died死亡 in misery苦难 and starvation饥饿.
82
246455
2281
04:22
So we'll call this
"the King国王 Midas迈达斯 problem问题"
83
250444
2341
我们可以把这个问题
叫做”迈达斯问题“,
这个问题是我们阐述的目标,但实际上
04:24
of stating说明 an objective目的
which哪一个 is not, in fact事实,
84
252809
3305
与我们真正想要的不一致,
04:28
truly aligned对齐 with what we want.
85
256138
2413
用现代的术语来说,
我们把它称为”价值一致性问题“。
04:30
In modern现代 terms条款, we call this
"the value alignment对准 problem问题."
86
258575
3253
04:37
Putting in the wrong错误 objective目的
is not the only part部分 of the problem问题.
87
265047
3485
而输入错误的目标
仅仅是问题的一部分。
它还有另一部分。
04:40
There's another另一个 part部分.
88
268556
1152
04:42
If you put an objective目的 into a machine,
89
270160
1943
如果你为机器输入一个目标,
即便是一个很简单的目标,
比如说”去把咖啡端来“,
04:44
even something as simple简单 as,
"Fetch the coffee咖啡,"
90
272127
2448
04:47
the machine says to itself本身,
91
275908
1841
机器会对自己说:
04:50
"Well, how might威力 I fail失败
to fetch the coffee咖啡?
92
278733
2623
”好吧,那我要怎么去拿咖啡呢?
说不定有人会把我的电源关掉。
04:53
Someone有人 might威力 switch开关 me off.
93
281380
1580
04:55
OK, I have to take steps脚步 to prevent避免 that.
94
283645
2387
好吧,那我要想办法
阻止别人把我关掉。
我得让我的‘关闭’开关失效。
04:58
I will disable禁用 my 'off'“关” switch开关.
95
286056
1906
05:00
I will do anything to defend保卫 myself
against反对 interference干扰
96
288534
2959
我得尽一切可能自我防御,
不让别人干涉我,
这都是因为我被赋予的目标。”
05:03
with this objective目的
that I have been given特定."
97
291517
2629
这种一根筋的思维,
05:06
So this single-minded专一 pursuit追求
98
294170
2012
05:09
in a very defensive防御性 mode模式
of an objective目的 that is, in fact事实,
99
297213
2945
以一种十分防御型的
模式去实现某一目标,
实际上与我们人类最初
想实现的目标并不一致——
05:12
not aligned对齐 with the true真正 objectives目标
of the human人的 race种族 --
100
300182
2814
这就是我们面临的问题。
05:16
that's the problem问题 that we face面对.
101
304122
1862
05:19
And in fact事实, that's the high-value高价值
takeaway带走 from this talk.
102
307007
4767
实际上,这就是今天这个演讲的核心。
如果你在我的演讲中只记住一件事,
05:23
If you want to remember记得 one thing,
103
311798
2055
那就是:如果你死了,
你就不能去端咖啡了。
05:25
it's that you can't fetch
the coffee咖啡 if you're dead.
104
313877
2675
(笑声)
05:28
(Laughter笑声)
105
316576
1061
这很简单。记住它就行了。
每天对自己重复三遍。
05:29
It's very simple简单. Just remember记得 that.
Repeat重复 it to yourself你自己 three times a day.
106
317661
3829
(笑声)
05:33
(Laughter笑声)
107
321514
1821
实际上,这正是电影
05:35
And in fact事实, this is exactly究竟 the plot情节
108
323359
2754
《2001太空漫步》的剧情。
05:38
of "2001: [A Space空间 Odyssey奥德赛]"
109
326137
2648
05:41
HALHAL has an objective目的, a mission任务,
110
329226
2090
HAL有一个目标,一个任务,
但这个目标和人类的目标不一致,
05:43
which哪一个 is not aligned对齐
with the objectives目标 of the humans人类,
111
331340
3732
这就导致了矛盾的产生。
05:47
and that leads引线 to this conflict冲突.
112
335096
1810
05:49
Now fortunately幸好, HALHAL
is not superintelligent超智.
113
337494
2969
幸运的是,HAL并不具备超级智能,
他挺聪明的,但还是
比不过人类主角戴夫,
05:52
He's pretty漂亮 smart聪明,
but eventually终于 Dave戴夫 outwitsoutwits him
114
340487
3587
戴夫成功地把HAL关掉了。
05:56
and manages管理 to switch开关 him off.
115
344098
1849
但我们可能就没有这么幸运了。
06:01
But we might威力 not be so lucky幸运.
116
349828
1619
06:08
So what are we going to do?
117
356193
1592
那我们应该怎么办呢?
06:12
I'm trying to redefine重新定义 AIAI
118
360371
2601
我想要重新定义人工智能,
远离传统的定义,
06:14
to get away from this classical古典 notion概念
119
362996
2061
将其仅限定为
机器通过智能去达成目标。
06:17
of machines that intelligently智能
pursue追求 objectives目标.
120
365081
4567
06:22
There are three principles原则 involved参与.
121
370712
1798
新的定义涉及到三个原则:
第一个原则是利他主义原则,
06:24
The first one is a principle原理
of altruism利他主义, if you like,
122
372534
3289
也就是说,机器的唯一目标
06:27
that the robot's机器人 only objective目的
123
375847
3262
就是去最大化地实现人类的目标,
06:31
is to maximize最大化 the realization实现
of human人的 objectives目标,
124
379133
4246
人类的价值。
06:35
of human人的 values.
125
383403
1390
至于价值,我指的不是感情化的价值,
06:36
And by values here I don't mean
touchy-feely煽情, goody-goody伪善 values.
126
384817
3330
而是指人类对生活所向往的,
06:40
I just mean whatever随你 it is
that the human人的 would prefer比较喜欢
127
388171
3787
无论是什么。
06:43
their life to be like.
128
391982
1343
06:47
And so this actually其实 violates违反 Asimov's阿西莫夫 law
129
395364
2309
这实际上违背了阿西莫夫定律,
他指出机器人一定要维护自己的生存。
06:49
that the robot机器人 has to protect保护
its own拥有 existence存在.
130
397697
2329
但我定义的机器
对维护自身生存毫无兴趣。
06:52
It has no interest利益 in preserving
its existence存在 whatsoever任何.
131
400050
3723
06:57
The second第二 law is a law
of humility谦逊, if you like.
132
405420
3768
第二个原则不妨称之为谦逊原则。
07:01
And this turns out to be really
important重要 to make robots机器人 safe安全.
133
409974
3743
这一条对于制造安全的机器十分重要。
它说的是机器不知道
07:05
It says that the robot机器人 does not know
134
413741
3142
人类的价值是什么,
07:08
what those human人的 values are,
135
416907
2028
机器知道它需要将人类的价值最大化,
却不知道这价值究竟是什么。
07:10
so it has to maximize最大化 them,
but it doesn't know what they are.
136
418959
3178
07:15
And that avoids避免 this problem问题
of single-minded专一 pursuit追求
137
423254
2626
为了避免一根筋地追求
某一目标,
07:17
of an objective目的.
138
425904
1212
这种不确定性是至关重要的。
07:19
This uncertainty不确定 turns out to be crucial关键.
139
427140
2172
07:21
Now, in order订购 to be useful有用 to us,
140
429726
1639
那机器为了对我们有用,
它就得掌握一些
关于我们想要什么的信息。
07:23
it has to have some idea理念 of what we want.
141
431389
2731
07:27
It obtains取得 that information信息 primarily主要
by observation意见 of human人的 choices选择,
142
435223
5427
它主要通过观察人类
做的选择来获取这样的信息,
我们自己做出的选择会包含着
07:32
so our own拥有 choices选择 reveal揭示 information信息
143
440674
2801
关于我们希望我们的生活
是什么样的信息,
07:35
about what it is that we prefer比较喜欢
our lives生活 to be like.
144
443499
3300
07:40
So those are the three principles原则.
145
448632
1683
这就是三条原则。
让我们来看看它们是如何应用到
07:42
Let's see how that applies适用
to this question of:
146
450339
2318
像图灵说的那样,
“将机器关掉”这个问题上来。
07:44
"Can you switch开关 the machine off?"
as Turing图灵 suggested建议.
147
452681
2789
07:49
So here's这里的 a PRPR2 robot机器人.
148
457073
2120
这是一个PR2机器人。
我们实验室里有一个。
07:51
This is one that we have in our lab实验室,
149
459217
1821
它的背面有一个大大的红色的开关。
07:53
and it has a big red "off" switch开关
right on the back.
150
461062
2903
07:56
The question is: Is it
going to let you switch开关 it off?
151
464541
2615
那问题来了:它会让你把它关掉吗?
如果我们按传统的方法,
07:59
If we do it the classical古典 way,
152
467180
1465
给它一个目标,让它拿咖啡,
它会想:”我必须去拿咖啡,
08:00
we give it the objective目的 of, "Fetch
the coffee咖啡, I must必须 fetch the coffee咖啡,
153
468669
3482
但我死了就不能拿咖啡了。“
08:04
I can't fetch the coffee咖啡 if I'm dead,"
154
472175
2580
显然PR2听过我的演讲了,
08:06
so obviously明显 the PRPR2
has been listening to my talk,
155
474779
3341
所以它说:”我必须让我的开关失灵,
08:10
and so it says, therefore因此,
"I must必须 disable禁用 my 'off'“关” switch开关,
156
478144
3753
08:14
and probably大概 taser泰瑟枪 all the other
people in Starbucks星巴克
157
482976
2694
可能还要把那些在星巴克里,
可能干扰我的人都电击一下。“
08:17
who might威力 interfere干扰 with me."
158
485694
1560
(笑声)
08:19
(Laughter笑声)
159
487278
2062
这看起来必然会发生,对吗?
08:21
So this seems似乎 to be inevitable必然, right?
160
489364
2153
这种失败看起来是必然的,
08:23
This kind of failure失败 mode模式
seems似乎 to be inevitable必然,
161
491541
2398
因为机器人在遵循
一个十分确定的目标。
08:25
and it follows如下 from having
a concrete具体, definite objective目的.
162
493963
3543
08:30
So what happens发生 if the machine
is uncertain不确定 about the objective目的?
163
498812
3144
那如果机器对目标
不那么确定会发生什么呢?
那它的思路就不一样了。
08:33
Well, it reasons原因 in a different不同 way.
164
501980
2127
它会说:”好的,人类可能会把我关掉,
08:36
It says, "OK, the human人的
might威力 switch开关 me off,
165
504131
2424
08:39
but only if I'm doing something wrong错误.
166
507144
1866
但只在我做错事的时候。
08:41
Well, I don't really know what wrong错误 is,
167
509747
2475
我不知道什么是错事,
但我知道我不该做那些事。”
08:44
but I know that I don't want to do it."
168
512246
2044
这就是第一和第二原则。
08:46
So that's the first and second第二
principles原则 right there.
169
514314
3010
“那我就应该让人类把我关掉。”
08:49
"So I should let the human人的 switch开关 me off."
170
517348
3359
08:53
And in fact事实 you can calculate计算
the incentive激励 that the robot机器人 has
171
521721
3956
事实上你可以计算出机器人
让人类把它关掉的动机,
08:57
to allow允许 the human人的 to switch开关 it off,
172
525701
2493
而且这个动机是
09:00
and it's directly tied to the degree
173
528218
1914
与对目标的不确定程度直接相关的。
09:02
of uncertainty不确定 about
the underlying底层 objective目的.
174
530156
2746
09:05
And then when the machine is switched交换的 off,
175
533977
2949
当机器被关闭后,
第三条原则就起作用了。
09:08
that third第三 principle原理 comes into play.
176
536950
1805
机器开始学习它所追求的目标,
09:10
It learns获悉 something about the objectives目标
it should be pursuing追求,
177
538779
3062
因为它知道它刚做的事是不对的。
09:13
because it learns获悉 that
what it did wasn't right.
178
541865
2533
实际上,我们可以用希腊字母
09:16
In fact事实, we can, with suitable适当 use
of Greek希腊语 symbols符号,
179
544422
3570
就像数学家们经常做的那样,
09:20
as mathematicians数学家 usually平时 do,
180
548016
2131
直接证明这一定理,
09:22
we can actually其实 prove证明 a theorem定理
181
550171
1984
那就是这样的一个机器人
对人们是绝对有利的。
09:24
that says that such这样 a robot机器人
is provably可证明 beneficial有利 to the human人的.
182
552179
3553
可以证明我们的生活
有如此设计的机器人会变得
09:27
You are provably可证明 better off
with a machine that's designed设计 in this way
183
555756
3803
比没有这样的机器人更好。
09:31
than without it.
184
559583
1246
09:33
So this is a very simple简单 example,
but this is the first step
185
561237
2906
这是一个很简单的例子,但这只是
我们尝试实现与人类
兼容的人工智能的第一步。
09:36
in what we're trying to do
with human-compatible与人相容 AIAI.
186
564167
3903
09:42
Now, this third第三 principle原理,
187
570657
3257
现在来看第三个原则。
我知道你们可能正在
为这一个原则而大伤脑筋。
09:45
I think is the one that you're probably大概
scratching搔抓 your head over.
188
573938
3112
你可能会想:“你知道,
我有时不按规矩办事。
09:49
You're probably大概 thinking思维, "Well,
you know, I behave表现 badly.
189
577074
3239
我可不希望我的机器人
像我一样行事。
09:52
I don't want my robot机器人 to behave表现 like me.
190
580337
2929
我有时大半夜偷偷摸摸地
从冰箱里找东西吃,
09:55
I sneak潜行 down in the middle中间 of the night
and take stuff东东 from the fridge冰箱.
191
583290
3434
诸如此类的事。”
09:58
I do this and that."
192
586748
1168
有各种各样的事你是
不希望机器人去做的。
09:59
There's all kinds of things
you don't want the robot机器人 doing.
193
587940
2797
但实际上并不一定会这样。
10:02
But in fact事实, it doesn't
quite相当 work that way.
194
590761
2071
仅仅是因为你表现不好,
10:04
Just because you behave表现 badly
195
592856
2155
并不代表机器人就会复制你的行为。
10:07
doesn't mean the robot机器人
is going to copy复制 your behavior行为.
196
595035
2623
它会去尝试理解你做事的动机,
而且可能会在合适的情况下制止你去做
10:09
It's going to understand理解 your motivations动机
and maybe help you resist them,
197
597682
3910
那些不该做的事。
10:13
if appropriate适当.
198
601616
1320
10:16
But it's still difficult.
199
604206
1464
但这仍然十分困难。
10:18
What we're trying to do, in fact事实,
200
606302
2545
实际上,我们在做的是
让机器去预测任何一个人,
在他们的任何一种
10:20
is to allow允许 machines to predict预测
for any person and for any possible可能 life
201
608871
5796
可能的生活中
10:26
that they could live生活,
202
614691
1161
以及别人的生活中,
10:27
and the lives生活 of everybody每个人 else其他:
203
615876
1597
10:29
Which哪一个 would they prefer比较喜欢?
204
617497
2517
他们会更倾向于哪一种?
10:34
And there are many许多, many许多
difficulties困难 involved参与 in doing this;
205
622061
2954
这涉及到诸多困难;
我不认为这会很快地就被解决。
10:37
I don't expect期望 that this
is going to get solved解决了 very quickly很快.
206
625039
2932
实际上,真正的困难是我们自己。
10:39
The real真实 difficulties困难, in fact事实, are us.
207
627995
2643
10:44
As I have already已经 mentioned提到,
we behave表现 badly.
208
632149
3117
就像我刚说的那样,
我们做事不守规矩,
我们中有的人甚至行为肮脏。
10:47
In fact事实, some of us are downright彻头彻尾 nasty讨厌.
209
635290
2321
10:50
Now the robot机器人, as I said,
doesn't have to copy复制 the behavior行为.
210
638431
3052
就像我说的,
机器人并不会复制那些行为,
机器人没有自己的目标,
10:53
The robot机器人 does not have
any objective目的 of its own拥有.
211
641507
2791
它是完全无私的。
10:56
It's purely纯粹 altruistic利他.
212
644322
1737
10:59
And it's not designed设计 just to satisfy满足
the desires欲望 of one person, the user用户,
213
647293
5221
它的设计不是去满足
某一个人、一个用户的欲望,
而是去尊重所有人的意愿。
11:04
but in fact事实 it has to respect尊重
the preferences优先 of everybody每个人.
214
652538
3138
11:09
So it can deal合同 with a certain某些
amount of nastiness污秽,
215
657263
2570
所以它能对付一定程度的肮脏行为。
它甚至能理解你的不端行为,比如说
11:11
and it can even understand理解
that your nastiness污秽, for example,
216
659857
3701
假如你是一个边境护照官员,
很可能收取贿赂,
11:15
you may可能 take bribes行贿 as a passport护照 official官方
217
663582
2671
因为你得养家、
得供你的孩子们上学。
11:18
because you need to feed饲料 your family家庭
and send发送 your kids孩子 to school学校.
218
666277
3812
机器人能理解这一点,
它不会因此去偷,
11:22
It can understand理解 that;
it doesn't mean it's going to steal.
219
670113
2906
它反而会帮助你去供孩子们上学。
11:25
In fact事实, it'll它会 just help you
send发送 your kids孩子 to school学校.
220
673043
2679
11:28
We are also computationally计算 limited有限.
221
676976
3012
我们的计算能力也是有限的。
李世石是一个杰出的围棋大师,
11:32
Lee背风处 SedolSEDOL is a brilliant辉煌 Go player播放机,
222
680012
2505
但他还是输了。
11:34
but he still lost丢失.
223
682541
1325
如果我们看他的行动,
他最终输掉了棋局。
11:35
So if we look at his actions行动,
he took an action行动 that lost丢失 the game游戏.
224
683890
4239
但这不意味着他想要输。
11:40
That doesn't mean he wanted to lose失去.
225
688153
2161
所以要理解他的行为,
11:43
So to understand理解 his behavior行为,
226
691340
2040
我们得从人类认知模型来反过来想,
11:45
we actually其实 have to invert倒置
through通过 a model模型 of human人的 cognition认识
227
693404
3644
这包含了我们的计算能力限制,
是一个很复杂的模型,
11:49
that includes包括 our computational计算
limitations限制 -- a very complicated复杂 model模型.
228
697072
4977
但仍然是我们可以尝试去理解的。
11:54
But it's still something
that we can work on understanding理解.
229
702073
2993
11:57
Probably大概 the most difficult part部分,
from my point of view视图 as an AIAI researcher研究员,
230
705876
4320
可能对于我这样一个
人工智能研究人员来说最大的困难,
是我们彼此各不相同。
12:02
is the fact事实 that there are lots of us,
231
710220
2575
12:06
and so the machine has to somehow不知何故
trade贸易 off, weigh称重 up the preferences优先
232
714294
3581
所以机器必须想办法去判别衡量
不同人的不同需求,
12:09
of many许多 different不同 people,
233
717899
2225
而又有众多方法去做这样的判断。
12:12
and there are different不同 ways方法 to do that.
234
720148
1906
经济学家、社会学家、
哲学家都理解这一点,
12:14
Economists经济学家, sociologists社会学家,
moral道德 philosophers哲学家 have understood了解 that,
235
722078
3689
我们正在积极地去寻求合作。
12:17
and we are actively积极地
looking for collaboration合作.
236
725791
2455
让我们来看看如果我们
把这一步弄错了会怎么样。
12:20
Let's have a look and see what happens发生
when you get that wrong错误.
237
728270
3251
举例来说,你可能会
与你的人工智能助理,
12:23
So you can have
a conversation会话, for example,
238
731545
2133
有这样的对话:
12:25
with your intelligent智能 personal个人 assistant助理
239
733702
1944
这样的人工智能可能几年内就会出现,
12:27
that might威力 be available可得到
in a few少数 years'年份' time.
240
735670
2285
可以把它想做加强版的Siri。
12:29
Think of a SiriSiri的 on steroids类固醇.
241
737979
2524
12:33
So SiriSiri的 says, "Your wife妻子 called
to remind提醒 you about dinner晚餐 tonight今晚."
242
741627
4322
Siri对你说:“你的妻子打电话
提醒你今晚要跟她共进晚餐。”
12:38
And of course课程, you've forgotten忘记了.
"What? What dinner晚餐?
243
746616
2508
而你呢,自然忘了这回事:
“什么?什么晚饭?
你在说什么?”
12:41
What are you talking about?"
244
749148
1425
12:42
"Uh, your 20th anniversary周年 at 7pm下午."
245
750597
3746
“啊,你们晚上7点,
庆祝结婚20周年纪念日。”
12:48
"I can't do that. I'm meeting会议
with the secretary-general秘书长 at 7:30.
246
756915
3719
“我可去不了。
我约了晚上7点半见领导。
怎么会这样呢?”
12:52
How could this have happened发生?"
247
760658
1692
“呃,我可是提醒过你的,
但你不听我的建议。”
12:54
"Well, I did warn警告 you, but you overrode凌驾于
my recommendation建议."
248
762374
4660
13:00
"Well, what am I going to do?
I can't just tell him I'm too busy."
249
768146
3328
“我该怎么办呢?我可不能
跟领导说我有事,没空见他。”
13:04
"Don't worry担心. I arranged安排
for his plane平面 to be delayed延迟."
250
772490
3281
“别担心。我已经安排了,
让他的航班延误。
(笑声)
13:07
(Laughter笑声)
251
775795
1682
13:10
"Some kind of computer电脑 malfunction故障."
252
778249
2101
“像是因为某种计算机故障那样。”
(笑声)
13:12
(Laughter笑声)
253
780374
1212
“真的吗?这个你也能做到?”
13:13
"Really? You can do that?"
254
781610
1617
13:16
"He sends发送 his profound深刻 apologies道歉
255
784400
2179
“领导很不好意思,跟你道歉,
并且告诉你明天
中午午饭不见不散。”
13:18
and looks容貌 forward前锋 to meeting会议 you
for lunch午餐 tomorrow明天."
256
786603
2555
(笑声)
13:21
(Laughter笑声)
257
789182
1299
这里就有一个小小的问题。
13:22
So the values here --
there's a slight轻微 mistake错误 going on.
258
790505
4403
这显然是在遵循我妻子的价值论,
13:26
This is clearly明确地 following以下 my wife's妻子 values
259
794932
3009
那就是“老婆开心,生活舒心”。
13:29
which哪一个 is "Happy快乐 wife妻子, happy快乐 life."
260
797965
2069
(笑声)
13:32
(Laughter笑声)
261
800058
1583
它也有可能发展成另一种情况。
13:33
It could go the other way.
262
801665
1444
13:35
You could come home
after a hard day's work,
263
803821
2201
你忙碌一天,回到家里,
电脑对你说:“像是繁忙的一天啊?”
13:38
and the computer电脑 says, "Long day?"
264
806046
2195
“是啊,我连午饭都没来得及吃。”
13:40
"Yes, I didn't even have time for lunch午餐."
265
808265
2288
“那你一定很饿了吧。”
13:42
"You must必须 be very hungry饥饿."
266
810577
1282
“快饿晕了。你能做点晚饭吗?”
13:43
"Starving饥饿, yeah.
Could you make some dinner晚餐?"
267
811883
2646
13:48
"There's something I need to tell you."
268
816070
2090
“有一件事我得告诉你。
(笑声)
13:50
(Laughter笑声)
269
818184
1155
13:52
"There are humans人类 in South Sudan苏丹
who are in more urgent紧急 need than you."
270
820193
4905
”南苏丹的人们可比你更需要照顾。
(笑声)
13:57
(Laughter笑声)
271
825122
1104
“所以我要离开了。
你自己做饭去吧。”
13:58
"So I'm leaving离开. Make your own拥有 dinner晚餐."
272
826250
2075
(笑声)
14:00
(Laughter笑声)
273
828349
2000
我们得解决这些问题,
14:02
So we have to solve解决 these problems问题,
274
830823
1739
我也很期待去解决。
14:04
and I'm looking forward前锋
to working加工 on them.
275
832586
2515
我们有理由感到乐观。
14:07
There are reasons原因 for optimism乐观.
276
835125
1843
理由之一是
14:08
One reason原因 is,
277
836992
1159
我们有大量的数据,
14:10
there is a massive大规模的 amount of data数据.
278
838175
1868
记住,我说过机器将能够阅读一切
14:12
Because remember记得 -- I said
they're going to read everything
279
840067
2794
人类所写下来的东西,
14:14
the human人的 race种族 has ever written书面.
280
842885
1546
而我们写下的大多数是
我们做的什么事情,
14:16
Most of what we write about
is human人的 beings众生 doing things
281
844455
2724
以及其他人对此有什么意见。
14:19
and other people getting得到 upset烦乱 about it.
282
847203
1914
所以机器可以从大量的数据中去学习。
14:21
So there's a massive大规模的 amount
of data数据 to learn学习 from.
283
849141
2398
同时从经济的角度,
我们也有足够的动机
14:23
There's also a very
strong强大 economic经济 incentive激励
284
851563
2236
14:27
to get this right.
285
855331
1186
去把这件事做对。
想象一下,你家里有个居家机器人,
14:28
So imagine想像 your domestic国内 robot's机器人 at home.
286
856541
2001
而你又得加班,
机器人得给孩子们做饭,
14:30
You're late晚了 from work again
and the robot机器人 has to feed饲料 the kids孩子,
287
858566
3067
孩子们很饿,
但冰箱里什么都没有。
14:33
and the kids孩子 are hungry饥饿
and there's nothing in the fridge冰箱.
288
861657
2823
然后机器人看到了家里的猫,
14:36
And the robot机器人 sees看到 the cat.
289
864504
2605
(笑声)
14:39
(Laughter笑声)
290
867133
1692
机器人还没学透人类的价值论,
14:40
And the robot机器人 hasn't有没有 quite相当 learned学到了
the human人的 value function功能 properly正确,
291
868849
4190
所以它不知道
14:45
so it doesn't understand理解
292
873063
1251
猫的感情价值
大于猫的营养价值。
14:46
the sentimental感伤 value of the cat outweighs胜过
the nutritional营养 value of the cat.
293
874338
4844
(笑声)
14:51
(Laughter笑声)
294
879206
1095
接下来会发生什么?
14:52
So then what happens发生?
295
880325
1748
差不多是这样的:
14:54
Well, it happens发生 like this:
296
882097
3297
头版头条:“疯狂的机器人
把猫煮了给主人当晚饭!”
14:57
"Deranged发疯 robot机器人 cooks厨师 kitty猫咪
for family家庭 dinner晚餐."
297
885418
2964
这一个事故就足以结束
整个居家机器人产业。
15:00
That one incident事件 would be the end结束
of the domestic国内 robot机器人 industry行业.
298
888406
4523
所以我们有足够的动机在我们实现
15:04
So there's a huge巨大 incentive激励
to get this right
299
892953
3372
超级智能机器让它更加完善。
15:08
long before we reach达到
superintelligent超智 machines.
300
896349
2715
15:12
So to summarize总结:
301
900128
1535
总结来说:
我想要改变人工智能的定义,
15:13
I'm actually其实 trying to change更改
the definition定义 of AIAI
302
901687
2881
让我们可以证明机器对我们是有利的。
15:16
so that we have provably可证明
beneficial有利 machines.
303
904592
2993
这三个原则是:
15:19
And the principles原则 are:
304
907609
1222
机器是利他的,
15:20
machines that are altruistic利他,
305
908855
1398
只想着实现我们的目标,
15:22
that want to achieve实现 only our objectives目标,
306
910277
2804
但它不确定我们的目标是什么,
15:25
but that are uncertain不确定
about what those objectives目标 are,
307
913105
3116
所以它会观察我们,
15:28
and will watch all of us
308
916245
1998
从中学习我们想要的究竟是什么。
15:30
to learn学习 more about what it is
that we really want.
309
918267
3203
15:34
And hopefully希望 in the process处理,
we will learn学习 to be better people.
310
922373
3559
希望在这个过程中,
我们也能学会成为更好的人。
谢谢大家。
15:37
Thank you very much.
311
925956
1191
(掌声)
15:39
(Applause掌声)
312
927171
3709
克里斯安德森:
非常有意思,斯图尔特。
15:42
Chris克里斯 Anderson安德森: So interesting有趣, Stuart斯图尔特.
313
930904
1868
我们趁着工作人员
为下一位演讲者布置的时候
15:44
We're going to stand here a bit
because I think they're setting设置 up
314
932796
3170
来简单聊几句。
15:47
for our next下一个 speaker扬声器.
315
935990
1151
15:49
A couple一对 of questions问题.
316
937165
1538
我有几个问题。
从直觉上来看,将无知编入到程序中
似乎是一个很重要的理念,
15:50
So the idea理念 of programming程序设计 in ignorance无知
seems似乎 intuitively直观地 really powerful强大.
317
938727
5453
当你要实现超级智能时,
15:56
As you get to superintelligence超级智能,
318
944204
1594
什么能阻止机器人?
15:57
what's going to stop a robot机器人
319
945822
2258
当它在阅读和学习的过程中发现,
16:00
reading literature文学 and discovering发现
this idea理念 that knowledge知识
320
948104
2852
16:02
is actually其实 better than ignorance无知
321
950980
1572
知识比无知更强大,
16:04
and still just shifting its own拥有 goals目标
and rewriting重写 that programming程序设计?
322
952576
4218
然后就改变它的目标
去重新编写程序呢?
斯图尔特拉塞尔:是的,
我们想要它去学习,就像我说的,
16:09
Stuart斯图尔特 Russell罗素: Yes, so we want
it to learn学习 more, as I said,
323
957692
6356
学习我们的目标。
16:16
about our objectives目标.
324
964072
1287
16:17
It'll它会 only become成为 more certain某些
as it becomes more correct正确,
325
965383
5521
它只有在理解得越来越正确的时候,
才会变得更确定,
16:22
so the evidence证据 is there
326
970928
1945
我们有证据显示,
它的设计使它能按正确的方式理解。
16:24
and it's going to be designed设计
to interpret it correctly正确地.
327
972897
2724
16:27
It will understand理解, for example,
that books图书 are very biased
328
975645
3956
比如说,它能够理解书中的论证是
带有非常强的偏见的。
16:31
in the evidence证据 they contain包含.
329
979625
1483
书中只会讲述国王、王子
16:33
They only talk about kings国王 and princes王子
330
981132
2397
和那些精英白人男性做的事。
16:35
and elite原种 white白色 male people doing stuff东东.
331
983553
2800
这是一个复杂的问题,
16:38
So it's a complicated复杂 problem问题,
332
986377
2096
但当它更深入地学习我们的目标时,
16:40
but as it learns获悉 more about our objectives目标
333
988497
3872
16:44
it will become成为 more and more useful有用 to us.
334
992393
2063
它就变得对我们更有用。
CA:那你不能把这些
都集中在一条准则里吗?
16:46
CACA: And you couldn't不能
just boil it down to one law,
335
994480
2526
把这样的命令写在它的程序里:
16:49
you know, hardwired硬线 in:
336
997030
1650
“如果人类什么时候想把我关掉,
16:50
"if any human人的 ever tries尝试 to switch开关 me off,
337
998704
3293
我服从。我服从。”
16:54
I comply执行. I comply执行."
338
1002021
1935
SR:绝对不行,
16:55
SRSR: Absolutely绝对 not.
339
1003980
1182
那将是一个很糟糕的主意。
16:57
That would be a terrible可怕 idea理念.
340
1005186
1499
试想一下,你有一辆无人驾驶汽车,
16:58
So imagine想像 that you have
a self-driving自驾车 car汽车
341
1006709
2689
你想让它送你五岁的孩子
17:01
and you want to send发送 your five-year-old五十岁
342
1009422
2433
去上学。
17:03
off to preschool幼儿.
343
1011879
1174
你希望你五岁的孩子
能在汽车运行过程中
17:05
Do you want your five-year-old五十岁
to be able能够 to switch开关 off the car汽车
344
1013077
3101
将它关闭吗?
17:08
while it's driving主动 along沿?
345
1016202
1213
应该不会吧。
17:09
Probably大概 not.
346
1017439
1159
它得理解下指令的人有多理智,
是不是讲道理。
17:10
So it needs需求 to understand理解 how rational合理的
and sensible明智 the person is.
347
1018622
4703
这个人越理智,
17:15
The more rational合理的 the person,
348
1023349
1676
它就越愿意自己被关掉。
17:17
the more willing愿意 you are
to be switched交换的 off.
349
1025049
2103
如果这个人是完全思绪混乱
或者甚至是有恶意的,
17:19
If the person is completely全然
random随机 or even malicious恶毒,
350
1027176
2543
那你就不愿意它被关掉。
17:21
then you're less willing愿意
to be switched交换的 off.
351
1029743
2512
CA:好吧。斯图尔特,我得说
17:24
CACA: All right. Stuart斯图尔特, can I just say,
352
1032279
1866
我真的希望你为我们
能把这一切研究出来,
17:26
I really, really hope希望 you
figure数字 this out for us.
353
1034169
2314
很感谢你的演讲,太精彩了。
17:28
Thank you so much for that talk.
That was amazing惊人.
354
1036507
2375
SR:谢谢。
17:30
SRSR: Thank you.
355
1038906
1167
(掌声)
17:32
(Applause掌声)
356
1040097
1837
Translated by Yichen Zheng
Reviewed by Yanyan Hong

▲Back to top

ABOUT THE SPEAKER
Stuart Russell - AI expert
Stuart Russell wrote the standard text on AI; now he thinks deeply on AI's future -- and the future of us humans, too.

Why you should listen

Stuart Russell is a professor (and formerly chair) of Electrical Engineering and Computer Sciences at University of California at Berkeley. His book Artificial Intelligence: A Modern Approach (with Peter Norvig) is the standard text in AI; it has been translated into 13 languages and is used in more than 1,300 universities in 118 countries. His research covers a wide range of topics in artificial intelligence including machine learning, probabilistic reasoning, knowledge representation, planning, real-time decision making, multitarget tracking, computer vision, computational physiology, global seismic monitoring and philosophical foundations.

He also works for the United Nations, developing a new global seismic monitoring system for the nuclear-test-ban treaty. His current concerns include the threat of autonomous weapons and the long-term future of artificial intelligence and its relation to humanity.

More profile about the speaker
Stuart Russell | Speaker | TED.com