TEDxCambridge
Iyad Rahwan: What moral decisions should driverless cars make?
Lyad Rahwan: 无人车的道德评判标准是什么？
Filmed:
Readability: 4.4
1,147,381 views
你的无人车是否应该为了挽救5个行人的生命而牺牲你的生命？在这个关于无人车社会道德困境的前瞻性演讲中，Lyad Rahwan探索了技术会如何挑战我们的道德伦理，并解释了他的工作是如何从群众中搜集我们希望（或不希望）做出的道德权衡数据的。
Iyad Rahwan - Computational social scientist
Iyad Rahwan's work lies at the intersection of the computer and social sciences, with a focus on collective intelligence, large-scale cooperation and the social aspects of artificial intelligence. Full bio
Iyad Rahwan's work lies at the intersection of the computer and social sciences, with a focus on collective intelligence, large-scale cooperation and the social aspects of artificial intelligence. Full bio
Double-click the English transcript below to play the video.
00:13
Today I'm going to talk
about technology and society.
about technology and society.
0
1000
4080
今天我想谈谈技术和社会。
00:19
The Department of Transport
estimated that last year
estimated that last year
1
7040
3696
据交通部的估算，
在美国，仅去年就有
在美国，仅去年就有
3万5千人死于交通事故。
00:22
35,000 people died
from traffic crashes in the US alone.
from traffic crashes in the US alone.
2
10760
4080
00:28
Worldwide, 1.2 million people
die every year in traffic accidents.
die every year in traffic accidents.
3
16040
4800
而在全世界，每年则有
120万人死于交通事故。
120万人死于交通事故。
00:33
If there was a way we could eliminate
90 percent of those accidents,
90 percent of those accidents,
4
21760
4096
如果有一种方法能减少90%的交通事故，
你会支持它吗？
00:37
would you support it?
5
25880
1200
00:39
Of course you would.
6
27720
1296
答案绝对是肯定的。
这就是无人车技术所承诺实现的目标，
00:41
This is what driverless car technology
promises to achieve
promises to achieve
7
29040
3655
通过消除造成事故的主要原因——
00:44
by eliminating the main
source of accidents --
source of accidents --
8
32720
2816
人为过错。
00:47
human error.
9
35560
1200
00:49
Now picture yourself
in a driverless car in the year 2030,
in a driverless car in the year 2030,
10
37920
5416
现在想象一下你在2030年中的一天，
坐在一辆无人车里
坐在一辆无人车里
00:55
sitting back and watching
this vintage TEDxCambridge video.
this vintage TEDxCambridge video.
11
43360
3456
悠闲地观看我这个过时的
TEDxCambridge视频。
TEDxCambridge视频。
00:58
(Laughter)
12
46840
2000
（笑声）
01:01
All of a sudden,
13
49520
1216
突然间，
车子出现了机械故障，刹车失灵了。
01:02
the car experiences mechanical failure
and is unable to stop.
and is unable to stop.
14
50760
3280
01:07
If the car continues,
15
55360
1520
如果车继续行驶，
01:09
it will crash into a bunch
of pedestrians crossing the street,
of pedestrians crossing the street,
16
57720
4120
就会冲入正在穿越人行道的人群中，
01:15
but the car may swerve,
17
63080
2135
但是车还可能转向，
01:17
hitting one bystander,
18
65239
1857
撞到路边一个不相干的人，
01:19
killing them to save the pedestrians.
19
67120
2080
用他的生命来换那些行人的生命。
01:22
What should the car do,
and who should decide?
and who should decide?
20
70040
2600
这辆车该怎么做，
又该是谁来做这个决定呢？
又该是谁来做这个决定呢？
01:25
What if instead the car
could swerve into a wall,
could swerve into a wall,
21
73520
3536
再如果，这辆车会转向并撞墙，
01:29
crashing and killing you, the passenger,
22
77080
3296
连你在内人车俱毁，
从而挽救其他人的生命，
从而挽救其他人的生命，
01:32
in order to save those pedestrians?
23
80400
2320
这会是个更好的选择吗？
01:35
This scenario is inspired
by the trolley problem,
by the trolley problem,
24
83240
3080
这个场景假设是受到了
“电车问题”的启发，
“电车问题”的启发，
01:38
which was invented
by philosophers a few decades ago
by philosophers a few decades ago
25
86960
3776
这是几十年前由一群哲学家
01:42
to think about ethics.
26
90760
1240
发起的对道德的拷问。
01:46
Now, the way we think
about this problem matters.
about this problem matters.
27
94120
2496
我们如何思考这个问题非常关键。
01:48
We may for example
not think about it at all.
not think about it at all.
28
96640
2616
我们也许压根儿就
不应该去纠结这个问题。
不应该去纠结这个问题。
01:51
We may say this scenario is unrealistic,
29
99280
3376
我们可以辩称这个场景假设不现实，
01:54
incredibly unlikely, or just silly.
30
102680
2320
太不靠谱，简直无聊透顶。
01:57
But I think this criticism
misses the point
misses the point
31
105760
2736
不过我觉得这种批判没有切中要害，
02:00
because it takes
the scenario too literally.
the scenario too literally.
32
108520
2160
因为仅仅是停留在了问题表面。
02:03
Of course no accident
is going to look like this;
is going to look like this;
33
111920
2736
当然没有任何事故会出现这种情况；
02:06
no accident has two or three options
34
114680
3336
没有哪个事故会同时出现2-3种选择，
02:10
where everybody dies somehow.
35
118040
2000
而每种选择中都会有人失去生命。
02:13
Instead, the car is going
to calculate something
to calculate something
36
121480
2576
相反，车辆自身会做些计算，
02:16
like the probability of hitting
a certain group of people,
a certain group of people,
37
124080
4896
比如撞击一群人的可能性，
02:21
if you swerve one direction
versus another direction,
versus another direction,
38
129000
3336
如果转向另一个方向，
相对于行人来说，你可能略微
增加了乘客，或者其他驾驶员
增加了乘客，或者其他驾驶员
02:24
you might slightly increase the risk
to passengers or other drivers
to passengers or other drivers
39
132360
3456
受伤的可能性。
02:27
versus pedestrians.
40
135840
1536
02:29
It's going to be
a more complex calculation,
a more complex calculation,
41
137400
2160
这将会是一个更加复杂的计算，
02:32
but it's still going
to involve trade-offs,
to involve trade-offs,
42
140480
2520
不过仍然会涉及到某种权衡，
02:35
and trade-offs often require ethics.
43
143840
2880
而这种权衡经常需要做出道德考量。
02:39
We might say then,
"Well, let's not worry about this.
"Well, let's not worry about this.
44
147840
2736
我们可能会说，“还是别杞人忧天了。
02:42
Let's wait until technology
is fully ready and 100 percent safe."
is fully ready and 100 percent safe."
45
150600
4640
不如等到技术完全成熟，能达到
100%安全的时候再用吧。”
100%安全的时候再用吧。”
02:48
Suppose that we can indeed
eliminate 90 percent of those accidents,
eliminate 90 percent of those accidents,
46
156520
3680
假设我们的确可以在
未来的10年内消除90%，
未来的10年内消除90%，
02:53
or even 99 percent in the next 10 years.
47
161080
2840
甚至99%的事故。
02:56
What if eliminating
the last one percent of accidents
the last one percent of accidents
48
164920
3176
如果消除最后这1%的事故
却需要再研究50年才能实现呢？
03:00
requires 50 more years of research?
49
168120
3120
03:04
Should we not adopt the technology?
50
172400
1800
我们是不是应该放弃这项技术了？
03:06
That's 60 million people
dead in car accidents
dead in car accidents
51
174720
4776
按照目前的死亡率计算，
那可还要牺牲
那可还要牺牲
03:11
if we maintain the current rate.
52
179520
1760
6千万人的生命啊。
03:14
So the point is,
53
182760
1216
所以关键在于，
03:16
waiting for full safety is also a choice,
54
184000
3616
等待万无一失的技术也是一种选择，
03:19
and it also involves trade-offs.
55
187640
2160
这里也有权衡的考虑。
03:23
People online on social media
have been coming up with all sorts of ways
have been coming up with all sorts of ways
56
191560
4336
社交媒体上的人们想尽了办法
03:27
to not think about this problem.
57
195920
2016
去回避这个问题。
03:29
One person suggested
the car should just swerve somehow
the car should just swerve somehow
58
197960
3216
有人建议无人车应该把握好角度，
刚好从人群和路边的
刚好从人群和路边的
03:33
in between the passengers --
59
201200
2136
无辜者之间的缝隙——
（笑声）
03:35
(Laughter)
60
203360
1016
穿过去。
03:36
and the bystander.
61
204400
1256
03:37
Of course if that's what the car can do,
that's what the car should do.
that's what the car should do.
62
205680
3360
当然，如果车辆能做到这一点，
毫无疑问就应该这么做。
毫无疑问就应该这么做。
03:41
We're interested in scenarios
in which this is not possible.
in which this is not possible.
63
209920
2840
我们讨论的是无法实现这一点的情况。
03:45
And my personal favorite
was a suggestion by a blogger
was a suggestion by a blogger
64
213280
5416
我个人比较赞同一个博主的点子，
03:50
to have an eject button in the car
that you press --
that you press --
65
218720
3016
在车里加装一个弹射按钮——
03:53
(Laughter)
66
221760
1216
（笑声）
在车辆自毁前按一下就行了。
03:55
just before the car self-destructs.
67
223000
1667
（笑声）
03:56
(Laughter)
68
224691
1680
03:59
So if we acknowledge that cars
will have to make trade-offs on the road,
will have to make trade-offs on the road,
69
227840
5200
那么如果我们认同车辆
将不得不在行驶中做出权衡的话，
将不得不在行驶中做出权衡的话，
04:06
how do we think about those trade-offs,
70
234200
1880
我们要如何考量这种权衡
04:09
and how do we decide?
71
237320
1576
并做出决策呢？
04:10
Well, maybe we should run a survey
to find out what society wants,
to find out what society wants,
72
238920
3136
也许我们应该做些调查问卷
看看大众是什么想法，
看看大众是什么想法，
04:14
because ultimately,
73
242080
1456
毕竟最终，
04:15
regulations and the law
are a reflection of societal values.
are a reflection of societal values.
74
243560
3960
规则和法律应该反映社会价值。
04:20
So this is what we did.
75
248040
1240
所以我们做了这么件事儿。
04:21
With my collaborators,
76
249880
1616
跟我的合作者
朗·弗朗索瓦·伯尼夫和
阿米滋·谢里夫一起，
阿米滋·谢里夫一起，
04:23
Jean-François Bonnefon and Azim Shariff,
77
251520
2336
04:25
we ran a survey
78
253880
1616
我们做了一项调查问卷，
为人们列举了这些假设的场景。
04:27
in which we presented people
with these types of scenarios.
with these types of scenarios.
79
255520
2855
04:30
We gave them two options
inspired by two philosophers:
inspired by two philosophers:
80
258399
3777
受哲学家杰里米·边沁（英国）和
伊曼努尔·康德（德国）的启发，
伊曼努尔·康德（德国）的启发，
04:34
Jeremy Bentham and Immanuel Kant.
81
262200
2640
我们给出了两种选择。
04:37
Bentham says the car
should follow utilitarian ethics:
should follow utilitarian ethics:
82
265600
3096
边沁认为车辆应该
遵循功利主义道德：
遵循功利主义道德：
它应该采取最小伤害的行动——
04:40
it should take the action
that will minimize total harm --
that will minimize total harm --
83
268720
3416
04:44
even if that action will kill a bystander
84
272160
2816
即使是以牺牲一个无辜者为代价，
04:47
and even if that action
will kill the passenger.
will kill the passenger.
85
275000
2440
即使会令乘客身亡。
04:50
Immanuel Kant says the car
should follow duty-bound principles,
should follow duty-bound principles,
86
278120
4976
伊曼努尔·康德则认为
车辆应该遵循义不容辞的原则，
车辆应该遵循义不容辞的原则，
04:55
like "Thou shalt not kill."
87
283120
1560
比如“不可杀人。”
04:57
So you should not take an action
that explicitly harms a human being,
that explicitly harms a human being,
88
285480
4456
因此你不应该有意
去伤害一个人，
去伤害一个人，
应该让车顺其自然行驶，
05:01
and you should let the car take its course
89
289960
2456
即使这样会伤害到更多的人。
05:04
even if that's going to harm more people.
90
292440
1960
05:07
What do you think?
91
295640
1200
你会怎么选择？
05:09
Bentham or Kant?
92
297360
1520
支持边沁还是康德？
05:11
Here's what we found.
93
299760
1256
我们得到的结果是这样的。
大部分人赞同边沁的观点。
05:13
Most people sided with Bentham.
94
301040
1800
05:16
So it seems that people
want cars to be utilitarian,
want cars to be utilitarian,
95
304160
3776
所以人们似乎希望
车辆是功利主义的，
车辆是功利主义的，
将伤害降到最小，
05:19
minimize total harm,
96
307960
1416
我们都应该这么做。
05:21
and that's what we should all do.
97
309400
1576
问题解决了。
05:23
Problem solved.
98
311000
1200
05:25
But there is a little catch.
99
313240
1480
不过这里还有个小插曲。
05:27
When we asked people
whether they would purchase such cars,
whether they would purchase such cars,
100
315920
3736
当我们问大家他们
会不会买这样一辆车时，
会不会买这样一辆车时，
他们不约而同地回答，“绝对不会。”
05:31
they said, "Absolutely not."
101
319680
1616
05:33
(Laughter)
102
321320
2296
（笑声）
他们更希望买能够
不顾一切保障自己安全的车，
不顾一切保障自己安全的车，
05:35
They would like to buy cars
that protect them at all costs,
that protect them at all costs,
103
323640
3896
05:39
but they want everybody else
to buy cars that minimize harm.
to buy cars that minimize harm.
104
327560
3616
不过却指望其他人
都买能将伤害降到最低的车。
都买能将伤害降到最低的车。
05:43
(Laughter)
105
331200
2520
（笑声）
05:46
We've seen this problem before.
106
334720
1856
这个问题以前就出现过。
叫做社会道德困境。
05:48
It's called a social dilemma.
107
336600
1560
05:51
And to understand the social dilemma,
108
339160
1816
为了理解这个概念，
我们要先简单回顾一下历史。
05:53
we have to go a little bit
back in history.
back in history.
109
341000
2040
05:56
In the 1800s,
110
344000
2576
在19世纪，
05:58
English economist William Forster Lloyd
published a pamphlet
published a pamphlet
111
346600
3736
英国经济学家威廉·福斯特·劳埃德
出版了一个宣传册，
出版了一个宣传册，
里面描述了这样一个场景。
06:02
which describes the following scenario.
112
350360
2216
有一群农场主，
06:04
You have a group of farmers --
113
352600
1656
06:06
English farmers --
114
354280
1336
英国农场主，
共同在一片地里放羊。
06:07
who are sharing a common land
for their sheep to graze.
for their sheep to graze.
115
355640
2680
06:11
Now, if each farmer
brings a certain number of sheep --
brings a certain number of sheep --
116
359520
2576
如果每个农场主都
带了一定数量的羊，
带了一定数量的羊，
比如每家三只，
06:14
let's say three sheep --
117
362120
1496
06:15
the land will be rejuvenated,
118
363640
2096
这片土地上的植被还可以正常再生，
农场主们自然高兴，
06:17
the farmers are happy,
119
365760
1216
06:19
the sheep are happy,
120
367000
1616
羊群也自在逍遥，
一切都相安无事。
06:20
everything is good.
121
368640
1200
06:22
Now, if one farmer brings one extra sheep,
122
370440
2520
如果有一个农场主多放了一只羊，
06:25
that farmer will do slightly better,
and no one else will be harmed.
and no one else will be harmed.
123
373800
4720
他就会获益更多，
不过其他人也都没什么损失。
不过其他人也都没什么损失。
06:31
But if every farmer made
that individually rational decision,
that individually rational decision,
124
379160
3640
但是如果每个农场主
都擅自增加羊的数量，
都擅自增加羊的数量，
06:35
the land will be overrun,
and it will be depleted
and it will be depleted
125
383840
2720
土地容量就会饱和，变得不堪重负，
06:39
to the detriment of all the farmers,
126
387360
2176
所有农场主都会受损，
06:41
and of course,
to the detriment of the sheep.
to the detriment of the sheep.
127
389560
2120
当然，羊群也会开始挨饿。
06:44
We see this problem in many places:
128
392720
3680
我们在很多场合都见到过这个问题：
06:49
in the difficulty of managing overfishing,
129
397080
3176
比如过度捕捞的困境，
06:52
or in reducing carbon emissions
to mitigate climate change.
to mitigate climate change.
130
400280
4560
或者应对气候变化的碳减排。
06:59
When it comes to the regulation
of driverless cars,
of driverless cars,
131
407160
2920
而到了无人车的制度问题，
07:03
the common land now
is basically public safety --
is basically public safety --
132
411080
4336
公共土地在这里指的就是公共安全，
07:07
that's the common good --
133
415440
1240
也就是公共利益，
07:09
and the farmers are the passengers
134
417400
1976
而农场主就是乘客，
07:11
or the car owners who are choosing
to ride in those cars.
to ride in those cars.
135
419400
3600
或者车主，决定乘车出行的人。
07:16
And by making the individually
rational choice
rational choice
136
424960
2616
通过自作主张把自己的安全凌驾于
07:19
of prioritizing their own safety,
137
427600
2816
其他人的利益之上，
他们可能共同损害了
能将总损失降到最低的
能将总损失降到最低的
07:22
they may collectively be
diminishing the common good,
diminishing the common good,
138
430440
3136
公共利益。
07:25
which is minimizing total harm.
139
433600
2200
07:30
It's called the tragedy of the commons,
140
438320
2136
传统上把这称为
07:32
traditionally,
141
440480
1296
公地悲剧，
不过我认为对于无人车来说，
07:33
but I think in the case
of driverless cars,
of driverless cars,
142
441800
3096
07:36
the problem may be
a little bit more insidious
a little bit more insidious
143
444920
2856
问题可能是更深层次的，
07:39
because there is not necessarily
an individual human being
an individual human being
144
447800
3496
因为并没有一个人
07:43
making those decisions.
145
451320
1696
去做决策。
07:45
So car manufacturers
may simply program cars
may simply program cars
146
453040
3296
那么无人车制造商可能会
简单的把行车电脑程序
简单的把行车电脑程序
07:48
that will maximize safety
for their clients,
for their clients,
147
456360
2520
设定成最大程度保护车主的安全，
07:52
and those cars may learn
automatically on their own
automatically on their own
148
460080
2976
而那些车可能会自主学习，
07:55
that doing so requires slightly
increasing risk for pedestrians.
increasing risk for pedestrians.
149
463080
3520
而这一过程也就会略微增加
对行人的潜在危险。
对行人的潜在危险。
07:59
So to use the sheep metaphor,
150
467520
1416
跟羊群的比喻类似，
08:00
it's like we now have electric sheep
that have a mind of their own.
that have a mind of their own.
151
468960
3616
这就好像换成了一批
可以自己思考的机器羊。
可以自己思考的机器羊。
08:04
(Laughter)
152
472600
1456
（笑声）
08:06
And they may go and graze
even if the farmer doesn't know it.
even if the farmer doesn't know it.
153
474080
3080
它们可能会自己去吃草，
而农场主对此毫不知情。
而农场主对此毫不知情。
08:10
So this is what we may call
the tragedy of the algorithmic commons,
the tragedy of the algorithmic commons,
154
478640
3976
这就是我们所谓的算法共享悲剧，
08:14
and if offers new types of challenges.
155
482640
2360
这会带来新的类型的挑战。
08:22
Typically, traditionally,
156
490520
1896
通常在传统模式下，
08:24
we solve these types
of social dilemmas using regulation,
of social dilemmas using regulation,
157
492440
3336
我们可以通过制定规则
来解决这些社会道德困境，
来解决这些社会道德困境，
08:27
so either governments
or communities get together,
or communities get together,
158
495800
2736
政府或者社区共同商讨决定
08:30
and they decide collectively
what kind of outcome they want
what kind of outcome they want
159
498560
3736
他们能够接受什么样的后果，
08:34
and what sort of constraints
on individual behavior
on individual behavior
160
502320
2656
以及需要对个人行为施加
08:37
they need to implement.
161
505000
1200
什么形式的限制。
08:39
And then using monitoring and enforcement,
162
507600
2616
通过监管和强制执行，
08:42
they can make sure
that the public good is preserved.
that the public good is preserved.
163
510240
2559
就可以确定公共利益得到了保障。
08:45
So why don't we just,
164
513440
1575
那么我们为什么不像
08:47
as regulators,
165
515039
1496
立法者那样，
08:48
require that all cars minimize harm?
166
516559
2897
让所有无人车把危险降到最小？
08:51
After all, this is
what people say they want.
what people say they want.
167
519480
2240
毕竟这是所有人的共同意愿。
08:55
And more importantly,
168
523200
1416
更重要的是，
08:56
I can be sure that as an individual,
169
524640
3096
作为一个个体，我很确定
08:59
if I buy a car that may
sacrifice me in a very rare case,
sacrifice me in a very rare case,
170
527760
3856
如果我买了一辆会在极端情况下
牺牲我的利益的车，
牺牲我的利益的车，
09:03
I'm not the only sucker doing that
171
531640
1656
我不会是唯一一个自残，
让其他所有人都受到无条件保护的人。
09:05
while everybody else
enjoys unconditional protection.
enjoys unconditional protection.
172
533320
2680
09:09
In our survey, we did ask people
whether they would support regulation
whether they would support regulation
173
537120
3336
在我们的调查问卷中确实
也问了人们，是否会支持立法，
也问了人们，是否会支持立法，
调查结果如下。
09:12
and here's what we found.
174
540480
1200
09:14
First of all, people
said no to regulation;
said no to regulation;
175
542360
3760
首先，人们并不赞同立法，
09:19
and second, they said,
176
547280
1256
其次，他们认为，
“如果你们要制定规则保证
这些车造成的损失最小，
这些车造成的损失最小，
09:20
"Well if you regulate cars to do this
and to minimize total harm,
and to minimize total harm,
177
548560
3936
那我肯定不会买。”
09:24
I will not buy those cars."
178
552520
1480
09:27
So ironically,
179
555400
1376
讽刺的是，
让无人车遵循最小损失原则，
09:28
by regulating cars to minimize harm,
180
556800
3496
我们得到的反而可能是更大的损失，
09:32
we may actually end up with more harm
181
560320
1840
09:35
because people may not
opt into the safer technology
opt into the safer technology
182
563040
3656
因为人们可能放弃
使用这种更安全的技术，
使用这种更安全的技术，
即便其安全性远超过人类驾驶员。
09:38
even if it's much safer
than human drivers.
than human drivers.
183
566720
2080
09:42
I don't have the final
answer to this riddle,
answer to this riddle,
184
570360
3416
对于这场争论
我并没有得到最终的答案，
我并没有得到最终的答案，
不过我认为作为一个开始，
09:45
but I think as a starting point,
185
573800
1576
09:47
we need society to come together
186
575400
3296
我们需要团结整个社会
09:50
to decide what trade-offs
we are comfortable with
we are comfortable with
187
578720
2760
来决定哪种折中方案
是大家都可以接受的，
是大家都可以接受的，
09:54
and to come up with ways
in which we can enforce those trade-offs.
in which we can enforce those trade-offs.
188
582360
3480
更要商讨出可以有效推行
这种权衡决策的方法。
这种权衡决策的方法。
09:58
As a starting point,
my brilliant students,
my brilliant students,
189
586520
2536
以此为基础，我的两位出色的学生，
Edmond Awad和Sohan Dsouza,
10:01
Edmond Awad and Sohan Dsouza,
190
589080
2456
10:03
built the Moral Machine website,
191
591560
1800
创建了“道德衡量器”网站，
10:06
which generates random scenarios at you --
192
594200
2680
可以为你设计出随机的场景——
10:10
basically a bunch
of random dilemmas in a sequence
of random dilemmas in a sequence
193
598080
2456
简单来说就是一系列
按顺序发生的随机困境，
按顺序发生的随机困境，
10:12
where you have to choose what
the car should do in a given scenario.
the car should do in a given scenario.
194
600560
3920
你需要据此判断无人车应该如何抉择。
10:17
And we vary the ages and even
the species of the different victims.
the species of the different victims.
195
605040
4600
我们还对不同的（潜在）受害者
设置了年龄，甚至种族信息。
设置了年龄，甚至种族信息。
10:23
So far we've collected
over five million decisions
over five million decisions
196
611040
3696
目前我们已经搜集到了
超过5百万份决定，
超过5百万份决定，
10:26
by over one million people worldwide
197
614760
2200
来自于全世界超过1百万人
10:30
from the website.
198
618400
1200
在网上给出的答案。
10:32
And this is helping us
form an early picture
form an early picture
199
620360
2416
这帮助我们
形成了一个概念雏形，
形成了一个概念雏形，
10:34
of what trade-offs
people are comfortable with
people are comfortable with
200
622800
2616
告诉了我们对人们来说
哪些折中方案最适用，
哪些折中方案最适用，
他们最在意的是什么——
10:37
and what matters to them --
201
625440
1896
10:39
even across cultures.
202
627360
1440
甚至跨越了文化障碍。
10:42
But more importantly,
203
630240
1496
不过更重要的是，
10:43
doing this exercise
is helping people recognize
is helping people recognize
204
631760
3376
这项练习能帮助人们认识到
10:47
the difficulty of making those choices
205
635160
2816
做出这些选择有多难，
10:50
and that the regulators
are tasked with impossible choices.
are tasked with impossible choices.
206
638000
3800
而立法者更是被要求做出
不现实的选择。
不现实的选择。
10:55
And maybe this will help us as a society
understand the kinds of trade-offs
understand the kinds of trade-offs
207
643360
3576
这还可能帮助我们整个社会
去理解那些最终将会被
去理解那些最终将会被
10:58
that will be implemented
ultimately in regulation.
ultimately in regulation.
208
646960
3056
纳入法规的折中方案。
11:02
And indeed, I was very happy to hear
209
650040
1736
诚然，我很高兴听到
11:03
that the first set of regulations
210
651800
2016
第一套由交通部
11:05
that came from
the Department of Transport --
the Department of Transport --
211
653840
2136
批准的法规——
上周刚刚公布——
11:08
announced last week --
212
656000
1376
11:09
included a 15-point checklist
for all carmakers to provide,
for all carmakers to provide,
213
657400
6576
囊括了需要所有无人车厂商
提供的15点清单，
提供的15点清单，
11:16
and number 14 was ethical consideration --
214
664000
3256
而其中第14点就是道德考量——
11:19
how are you going to deal with that.
215
667280
1720
要如何处理道德困境。
11:23
We also have people
reflect on their own decisions
reflect on their own decisions
216
671800
2656
我们还通过为大家
提供自己选择的概要，
提供自己选择的概要，
11:26
by giving them summaries
of what they chose.
of what they chose.
217
674480
3000
让人们反思自己的决定。
11:30
I'll give you one example --
218
678440
1656
给大家举个例子——
我要提醒大家
这不是一个典型的例子，
这不是一个典型的例子，
11:32
I'm just going to warn you
that this is not your typical example,
that this is not your typical example,
219
680120
3536
也不是典型的车主。
11:35
your typical user.
220
683680
1376
11:37
This is the most sacrificed and the most
saved character for this person.
saved character for this person.
221
685080
3616
这个人有着最容易牺牲（儿童），
也最容易被保护的特征（宠物）。
也最容易被保护的特征（宠物）。
11:40
(Laughter)
222
688720
5200
（笑声）
11:46
Some of you may agree with him,
223
694680
1896
你们中有人可能会赞同他，
11:48
or her, we don't know.
224
696600
1640
或者她，我们并不知道其性别。
11:52
But this person also seems to slightly
prefer passengers over pedestrians
prefer passengers over pedestrians
225
700480
6136
不过这位调查对象
也似乎更愿意保护乘客，
也似乎更愿意保护乘客，
11:58
in their choices
226
706640
2096
而不是行人，
甚至相当支持严惩横穿马路的行人。
12:00
and is very happy to punish jaywalking.
227
708760
2816
12:03
(Laughter)
228
711600
3040
（笑声）
12:09
So let's wrap up.
229
717320
1216
那么我们来总结一下。
我们由一个问题开始——
就叫它道德困境问题——
就叫它道德困境问题——
12:10
We started with the question --
let's call it the ethical dilemma --
let's call it the ethical dilemma --
230
718559
3416
12:14
of what the car should do
in a specific scenario:
in a specific scenario:
231
722000
3056
关于在特定条件下
无人车应该如何抉择：
无人车应该如何抉择：
转向还是直行？
12:17
swerve or stay?
232
725080
1200
12:19
But then we realized
that the problem was a different one.
that the problem was a different one.
233
727240
2736
但之后我们意识到这并不是问题的核心。
12:22
It was the problem of how to get
society to agree on and enforce
society to agree on and enforce
234
730000
4536
关键的问题在于如何让
大众在他们能够接受的权衡方案中
大众在他们能够接受的权衡方案中
达成一致并付诸实施。
12:26
the trade-offs they're comfortable with.
235
734560
1936
这是个社会道德困境。
12:28
It's a social dilemma.
236
736520
1256
在20世纪40年代， 艾萨克·阿西莫夫
（俄科幻小说家）就写下了他那著名的
（俄科幻小说家）就写下了他那著名的
12:29
In the 1940s, Isaac Asimov
wrote his famous laws of robotics --
wrote his famous laws of robotics --
237
737800
5016
12:34
the three laws of robotics.
238
742840
1320
机器人三大法则。
12:37
A robot may not harm a human being,
239
745240
2456
机器人不能伤害人类，
12:39
a robot may not disobey a human being,
240
747720
2536
机器人不能违背人类的命令，
12:42
and a robot may not allow
itself to come to harm --
itself to come to harm --
241
750280
3256
机器人不能擅自伤害自己——
12:45
in this order of importance.
242
753560
1960
这是按重要性由高到低排序的。
12:48
But after 40 years or so
243
756360
2136
但是大约40年后，
12:50
and after so many stories
pushing these laws to the limit,
pushing these laws to the limit,
244
758520
3736
太多事件不断挑战这些法则的底线，
12:54
Asimov introduced the zeroth law
245
762280
3696
阿西莫夫又引入了第零号法则，
12:58
which takes precedence above all,
246
766000
2256
凌驾于之前所有法则之上，
13:00
and it's that a robot
may not harm humanity as a whole.
may not harm humanity as a whole.
247
768280
3280
说的是机器人不能伤害人类这个整体。
13:04
I don't know what this means
in the context of driverless cars
in the context of driverless cars
248
772480
4376
我不太明白在无人车和
其他特殊背景下
其他特殊背景下
13:08
or any specific situation,
249
776880
2736
这句话是什么意思，
13:11
and I don't know how we can implement it,
250
779640
2216
也不清楚我们要如何实践它，
13:13
but I think that by recognizing
251
781880
1536
但我认为通过认识到
13:15
that the regulation of driverless cars
is not only a technological problem
is not only a technological problem
252
783440
6136
针对无人车的立法不仅仅是个技术问题，
13:21
but also a societal cooperation problem,
253
789600
3280
还是一个社会合作问题，
13:25
I hope that we can at least begin
to ask the right questions.
to ask the right questions.
254
793800
2880
我希望我们至少可以
从提出正确的问题入手。
从提出正确的问题入手。
13:29
Thank you.
255
797200
1216
谢谢大家。
（掌声）
13:30
(Applause)
256
798440
2920
Reviewed by Bangyou Xiang
ABOUT THE SPEAKER
Iyad Rahwan - Computational social scientistIyad Rahwan's work lies at the intersection of the computer and social sciences, with a focus on collective intelligence, large-scale cooperation and the social aspects of artificial intelligence.
Why you should listen
Iyad Rahwan is the AT&T Career Development Professor and an associate professor of media arts & sciences at the MIT Media Lab, where he leads the Scalable Cooperation group. A native of Aleppo, Syria, Rahwan holds a PhD. from the University of Melbourne, Australia and is an affiliate faculty at the MIT Institute of Data, Systems and Society (IDSS). He led the winning team in the US State Department's Tag Challenge, using social media to locate individuals in remote cities within 12 hours using only their mug shots. Recently he crowdsourced 30 million decisions from people worldwide about the ethics of AI systems. Rahwan's work appeared in major academic journals, including Science and PNAS, and features regularly in major media outlets, including the New York Times, The Economist and the Wall Street Journal.
(Photo: Victoriano Izquierdo)
Iyad Rahwan | Speaker | TED.com