TEDSummit
Zeynep Tufekci: Machine intelligence makes human morals more important
日娜·土費琪: 在機器智慧時代，堅守人類道德更形重要
Filmed:
Readability: 4.9
1,648,711 views
機器智慧時代已來臨，且已被用於主觀判斷和做決定。但是人難以理解、控制越來越複雜的人工智慧。科技社會學家日娜·土費琪在演講中警示我們，人工智慧會在人類不會犯錯的地方出錯，可能會以我們意想不到、沒有準備的方式出錯。她說：「我們不能把責任外包給機器。我們必須更堅守人類的價值觀和人類的倫理。」
Zeynep Tufekci - Techno-sociologist
Techno-sociologist Zeynep Tufekci asks big questions about our societies and our lives, as both algorithms and digital connectivity spread. Full bio
Techno-sociologist Zeynep Tufekci asks big questions about our societies and our lives, as both algorithms and digital connectivity spread. Full bio
Double-click the English transcript below to play the video.
00:12
So, I started my first job
as a computer programmer
as a computer programmer
0
739
4122
大一時我開始了第一份工作：
程式設計師，
程式設計師，
00:16
in my very first year of college --
1
4885
1956
00:18
basically, as a teenager.
2
6865
1507
當時我還算是個青少女。
00:20
Soon after I started working,
3
8889
1732
開始為軟體公司寫程式後不久，
00:22
writing software in a company,
4
10645
1610
00:24
a manager who worked at the company
came down to where I was,
came down to where I was,
5
12799
3635
公司裡的一個經理走到我身邊，
00:28
and he whispered to me,
6
16458
1268
悄悄地問：
00:30
"Can he tell if I'm lying?"
7
18229
2861
「他能判斷我是否說謊嗎？」
00:33
There was nobody else in the room.
8
21806
2077
當時房裡沒別人。
「『誰』能不能判斷你說謊與否？
而且，我們為什麼耳語呢？」
而且，我們為什麼耳語呢？」
00:37
"Can who tell if you're lying?
And why are we whispering?"
And why are we whispering?"
9
25032
4389
00:42
The manager pointed
at the computer in the room.
at the computer in the room.
10
30266
3107
經理指著房裡的電腦，問：
00:45
"Can he tell if I'm lying?"
11
33397
3096
「『他』能判斷我是否說謊嗎？」
00:49
Well, that manager was having
an affair with the receptionist.
an affair with the receptionist.
12
37613
4362
當時那經理與接待員有曖昧關係。
00:53
(Laughter)
13
41999
1112
（笑聲）
00:55
And I was still a teenager.
14
43135
1766
那時我仍是個青少女。
00:57
So I whisper-shouted back to him,
15
45447
2019
所以，我用耳語大聲地回答他：
00:59
"Yes, the computer can tell
if you're lying."
if you're lying."
16
47490
3624
「能，電腦能判斷你撒謊與否。」
01:03
(Laughter)
17
51138
1806
（笑聲）
01:04
Well, I laughed, but actually,
the laugh's on me.
the laugh's on me.
18
52968
2923
沒錯，我笑了，
但可笑的人是我。
但可笑的人是我。
01:07
Nowadays, there are computational systems
19
55915
3268
如今，有些計算系統
01:11
that can suss out
emotional states and even lying
emotional states and even lying
20
59207
3548
靠分析、判讀面部表情，
就能判斷出情緒狀態，
就能判斷出情緒狀態，
01:14
from processing human faces.
21
62779
2044
甚至判斷是否說謊。
01:17
Advertisers and even governments
are very interested.
are very interested.
22
65248
4153
廣告商，甚至政府也對此很感興趣。
01:22
I had become a computer programmer
23
70319
1862
我之所以成為程式設計師
01:24
because I was one of those kids
crazy about math and science.
crazy about math and science.
24
72205
3113
乃因自幼便極為喜愛數學和科學。
01:27
But somewhere along the line
I'd learned about nuclear weapons,
I'd learned about nuclear weapons,
25
75942
3108
過程中我學到核子武器，
01:31
and I'd gotten really concerned
with the ethics of science.
with the ethics of science.
26
79074
2952
因而變得非常關心科學倫理。
01:34
I was troubled.
27
82050
1204
我很苦惱。
01:35
However, because of family circumstances,
28
83278
2641
但由於家庭狀況，
01:37
I also needed to start working
as soon as possible.
as soon as possible.
29
85943
3298
我必須儘早就業。
01:41
So I thought to myself, hey,
let me pick a technical field
let me pick a technical field
30
89265
3299
因此我告訴自己，
選擇一個在科技領域中
能簡單地找到頭路，
能簡單地找到頭路，
01:44
where I can get a job easily
31
92588
1796
01:46
and where I don't have to deal
with any troublesome questions of ethics.
with any troublesome questions of ethics.
32
94408
4018
又無需處理涉及倫理道德
這類麻煩問題的工作吧。
這類麻煩問題的工作吧。
01:51
So I picked computers.
33
99022
1529
所以我選擇了電腦。
01:52
(Laughter)
34
100575
1104
（笑聲）
01:53
Well, ha, ha, ha!
All the laughs are on me.
All the laughs are on me.
35
101703
3410
是啊，哈哈哈！大家都笑我。
01:57
Nowadays, computer scientists
are building platforms
are building platforms
36
105137
2754
如今，電腦科學家
正建構著可控制數十億人
每天接收訊息的平台。
每天接收訊息的平台。
01:59
that control what a billion
people see every day.
people see every day.
37
107915
4209
02:05
They're developing cars
that could decide who to run over.
that could decide who to run over.
38
113052
3822
他們設計的汽車
可以決定要輾過哪些人。
可以決定要輾過哪些人。
02:09
They're even building machines, weapons,
39
117707
3213
他們甚至建造能殺人的
戰爭機器和武器。
戰爭機器和武器。
02:12
that might kill human beings in war.
40
120944
2285
02:15
It's ethics all the way down.
41
123253
2771
從頭到尾都是倫理的問題。
02:19
Machine intelligence is here.
42
127183
2058
機器智慧已經在此。
02:21
We're now using computation
to make all sort of decisions,
to make all sort of decisions,
43
129823
3474
我們利用計算
來做各種決策，
來做各種決策，
02:25
but also new kinds of decisions.
44
133321
1886
同時也是種新形態的決策。
02:27
We're asking questions to computation
that have no single right answers,
that have no single right answers,
45
135231
5172
我們以計算來尋求解答，
但問題沒有單一的正解，
但問題沒有單一的正解，
02:32
that are subjective
46
140427
1202
而是主觀、開放、具價值觀的答案。
02:33
and open-ended and value-laden.
47
141653
2325
02:36
We're asking questions like,
48
144002
1758
問題像是，
02:37
"Who should the company hire?"
49
145784
1650
「公司應該聘誰？」
02:40
"Which update from which friend
should you be shown?"
should you be shown?"
50
148096
2759
「應該顯示哪個朋友的哪項更新？」
02:42
"Which convict is more
likely to reoffend?"
likely to reoffend?"
51
150879
2266
「哪個罪犯更可能再犯？」
02:45
"Which news item or movie
should be recommended to people?"
should be recommended to people?"
52
153514
3054
「應該推薦哪項新聞或哪部電影？」
02:48
Look, yes, we've been using
computers for a while,
computers for a while,
53
156592
3372
我們使用電腦雖有一段時間了，
02:51
but this is different.
54
159988
1517
但這是不同的。
02:53
This is a historical twist,
55
161529
2067
這是歷史性的轉折，
02:55
because we cannot anchor computation
for such subjective decisions
for such subjective decisions
56
163620
5337
因我們不能主導計算機
如何去做這樣的主觀決定，
如何去做這樣的主觀決定，
03:00
the way we can anchor computation
for flying airplanes, building bridges,
for flying airplanes, building bridges,
57
168981
5420
無法像主導計算機去開飛機、造橋樑
或登陸月球那樣。
03:06
going to the moon.
58
174425
1259
03:08
Are airplanes safer?
Did the bridge sway and fall?
Did the bridge sway and fall?
59
176449
3259
飛機會更安全嗎？
橋樑會搖擺或倒塌嗎？
橋樑會搖擺或倒塌嗎？
03:11
There, we have agreed-upon,
fairly clear benchmarks,
fairly clear benchmarks,
60
179732
4498
那兒已有相當明確的基準共識，
03:16
and we have laws of nature to guide us.
61
184254
2239
有自然的法則指引著我們。
03:18
We have no such anchors and benchmarks
62
186517
3394
但我們沒有
判斷凌亂人事的錨點或基準。
03:21
for decisions in messy human affairs.
63
189935
3963
03:25
To make things more complicated,
our software is getting more powerful,
our software is getting more powerful,
64
193922
4237
使事情變得更為複雜的是，
因軟體越來越強大，
因軟體越來越強大，
03:30
but it's also getting less
transparent and more complex.
transparent and more complex.
65
198183
3773
但也越來越不透明，越複雜難懂。
03:34
Recently, in the past decade,
66
202542
2040
過去十年
03:36
complex algorithms
have made great strides.
have made great strides.
67
204606
2729
複雜的演算法有長足的進步：
03:39
They can recognize human faces.
68
207359
1990
能辨識人臉，
03:41
They can decipher handwriting.
69
209985
2055
能解讀手寫的字，
03:44
They can detect credit card fraud
70
212436
2066
能檢測信用卡欺詐，
03:46
and block spam
71
214526
1189
阻擋垃圾郵件，
03:47
and they can translate between languages.
72
215739
2037
能翻譯不同的語言，
03:49
They can detect tumors in medical imaging.
73
217800
2574
能判讀醫學影像查出腫瘤，
03:52
They can beat humans in chess and Go.
74
220398
2205
能在西洋棋和圍棋賽中
擊敗人類棋手。
擊敗人類棋手。
03:55
Much of this progress comes
from a method called "machine learning."
from a method called "machine learning."
75
223264
4504
這些進步主要來自所謂的
「機器學習」法。
「機器學習」法。
04:00
Machine learning is different
than traditional programming,
than traditional programming,
76
228175
3187
機器學習不同於傳統的程式編寫。
04:03
where you give the computer
detailed, exact, painstaking instructions.
detailed, exact, painstaking instructions.
77
231386
3585
編寫程式是下詳細、精確、
齊全的計算機指令；
齊全的計算機指令；
04:07
It's more like you take the system
and you feed it lots of data,
and you feed it lots of data,
78
235378
4182
機器學習更像是
餵大量的數據給系統，
餵大量的數據給系統，
04:11
including unstructured data,
79
239584
1656
包括非結構化的數據，
04:13
like the kind we generate
in our digital lives.
in our digital lives.
80
241264
2278
像我們數位生活產生的數據；
04:15
And the system learns
by churning through this data.
by churning through this data.
81
243566
2730
系統翻撈這些數據來學習。
04:18
And also, crucially,
82
246669
1526
至關重要的是，
04:20
these systems don't operate
under a single-answer logic.
under a single-answer logic.
83
248219
4380
這些系統不在產生
單一答案的邏輯系統下運作；
單一答案的邏輯系統下運作；
04:24
They don't produce a simple answer;
it's more probabilistic:
it's more probabilistic:
84
252623
2959
它們不會給出一個簡單的答案，
而是以更接近機率的形式呈現：
04:27
"This one is probably more like
what you're looking for."
what you're looking for."
85
255606
3483
「這可能更接近你所要找的。」
04:32
Now, the upside is:
this method is really powerful.
this method is really powerful.
86
260023
3070
好處是：這方法強而有力。
04:35
The head of Google's AI systems called it,
87
263117
2076
谷歌的人工智慧系統負責人稱之為：
04:37
"the unreasonable effectiveness of data."
88
265217
2197
「不合理的數據有效性。」
04:39
The downside is,
89
267791
1353
缺點是，
04:41
we don't really understand
what the system learned.
what the system learned.
90
269738
3071
我們未能真正明白
系統學到了什麼。
系統學到了什麼。
04:44
In fact, that's its power.
91
272833
1587
事實上，這就是它的力量。
04:46
This is less like giving
instructions to a computer;
instructions to a computer;
92
274946
3798
這不像下指令給計算機；
04:51
it's more like training
a puppy-machine-creature
a puppy-machine-creature
93
279200
4064
而更像是訓練
我們未能真正了解
或無法控制的機器寵物狗。
或無法控制的機器寵物狗。
04:55
we don't really understand or control.
94
283288
2371
04:58
So this is our problem.
95
286362
1551
這是我們的問題。
05:00
It's a problem when this artificial
intelligence system gets things wrong.
intelligence system gets things wrong.
96
288427
4262
人工智慧系統出錯時會是個問題；
05:04
It's also a problem
when it gets things right,
when it gets things right,
97
292713
3540
即使它弄對了還是個問題，
05:08
because we don't even know which is which
when it's a subjective problem.
when it's a subjective problem.
98
296277
3628
因碰到主觀問題時，
我們不知哪個是哪個。
我們不知哪個是哪個。
05:11
We don't know what this thing is thinking.
99
299929
2339
我們不知道系統在想什麼。
05:15
So, consider a hiring algorithm --
100
303493
3683
就拿招募人員的演算法來說，
05:20
a system used to hire people,
using machine-learning systems.
using machine-learning systems.
101
308123
4311
亦即以機器學習來僱用人的系統，
05:25
Such a system would have been trained
on previous employees' data
on previous employees' data
102
313052
3579
這樣的系統用
已有的員工數據來訓練機器，
已有的員工數據來訓練機器，
05:28
and instructed to find and hire
103
316655
2591
指示它尋找和僱用那些
05:31
people like the existing
high performers in the company.
high performers in the company.
104
319270
3038
類似公司現有的高績效員工的人。
05:34
Sounds good.
105
322814
1153
聽起來不錯。
05:35
I once attended a conference
106
323991
1999
我曾參加某會議，
05:38
that brought together
human resources managers and executives,
human resources managers and executives,
107
326014
3125
聚集人資經理和高階主管，
05:41
high-level people,
108
329163
1206
高層人士，
05:42
using such systems in hiring.
109
330393
1559
使用這種系統招聘。
05:43
They were super excited.
110
331976
1646
他們超級興奮，
05:45
They thought that this would make hiring
more objective, less biased,
more objective, less biased,
111
333646
4653
認為這種系統會使招聘更為客觀，
較少偏見，
05:50
and give women
and minorities a better shot
and minorities a better shot
112
338323
3000
有利於婦女和少數民族
05:53
against biased human managers.
113
341347
2188
避開有偏見的管理人。
05:55
And look -- human hiring is biased.
114
343559
2843
看哪！靠人類僱用是有偏見的。
05:59
I know.
115
347099
1185
我知道。
06:00
I mean, in one of my early jobs
as a programmer,
as a programmer,
116
348308
3005
我的意思是，
在早期某個編寫程式的工作，
在早期某個編寫程式的工作，
06:03
my immediate manager would sometimes
come down to where I was
come down to where I was
117
351337
3868
有時候我的直屬主管會在
06:07
really early in the morning
or really late in the afternoon,
or really late in the afternoon,
118
355229
3753
大清早或下午很晚時來到我身旁，
06:11
and she'd say, "Zeynep,
let's go to lunch!"
let's go to lunch!"
119
359006
3062
說：「日娜，走，吃午飯！」
06:14
I'd be puzzled by the weird timing.
120
362724
2167
我被奇怪的時間點所困惑。
06:16
It's 4pm. Lunch?
121
364915
2129
下午4點。吃午餐？
06:19
I was broke, so free lunch. I always went.
122
367068
3094
我很窮，
因是免費的午餐，所以總是會去。
06:22
I later realized what was happening.
123
370618
2067
後來我明白到底是怎麼回事。
06:24
My immediate managers
had not confessed to their higher-ups
had not confessed to their higher-ups
124
372709
4546
我的直屬主管沒讓她的主管知道，
06:29
that the programmer they hired
for a serious job was a teen girl
for a serious job was a teen girl
125
377279
3113
他們僱來做重要職務的程式設計師，
是個穿牛仔褲和運動鞋
06:32
who wore jeans and sneakers to work.
126
380416
3930
來上班的十幾歲女孩。
06:37
I was doing a good job,
I just looked wrong
I just looked wrong
127
385174
2202
我工作做得很好，
只是外表形象看起來不符，
只是外表形象看起來不符，
06:39
and was the wrong age and gender.
128
387400
1699
年齡和性別不對。
06:41
So hiring in a gender- and race-blind way
129
389123
3346
因此，性別和種族
不列入考慮的僱用系統
不列入考慮的僱用系統
06:44
certainly sounds good to me.
130
392493
1865
對我而言當然不錯。
06:47
But with these systems,
it is more complicated, and here's why:
it is more complicated, and here's why:
131
395031
3341
但使用這些系統會更複雜，原因是：
06:50
Currently, computational systems
can infer all sorts of things about you
can infer all sorts of things about you
132
398968
5791
目前的計算系統
可從你零散的數位足跡
推斷出關於你的各種事物，
推斷出關於你的各種事物，
06:56
from your digital crumbs,
133
404783
1872
06:58
even if you have not
disclosed those things.
disclosed those things.
134
406679
2333
即使你未曾披露過。
07:01
They can infer your sexual orientation,
135
409506
2927
他們能推斷你的性取向，
07:04
your personality traits,
136
412994
1306
個性的特質，
07:06
your political leanings.
137
414859
1373
政治的傾向。
07:08
They have predictive power
with high levels of accuracy.
with high levels of accuracy.
138
416830
3685
他們的預測能力相當精準。
07:13
Remember -- for things
you haven't even disclosed.
you haven't even disclosed.
139
421362
2578
請記住：知道你未曾公開的事情
07:15
This is inference.
140
423964
1591
是推理。
07:17
I have a friend who developed
such computational systems
such computational systems
141
425579
3261
我有個朋友開發這樣的計算系統：
07:20
to predict the likelihood
of clinical or postpartum depression
of clinical or postpartum depression
142
428864
3641
從社交媒體數據來預測
臨床或產後抑鬱症的可能性。
臨床或產後抑鬱症的可能性。
07:24
from social media data.
143
432529
1416
07:26
The results are impressive.
144
434676
1427
結果非常優異。
07:28
Her system can predict
the likelihood of depression
the likelihood of depression
145
436492
3357
她的系統
能在出現任何症狀的幾個月前
預測出抑鬱的可能性，
預測出抑鬱的可能性，
07:31
months before the onset of any symptoms --
146
439873
3903
07:35
months before.
147
443800
1373
是好幾個月前。
07:37
No symptoms, there's prediction.
148
445197
2246
雖沒有症狀，已預測出來。
07:39
She hopes it will be used
for early intervention. Great!
for early intervention. Great!
149
447467
4812
她希望它被用來早期干預處理。
很好！
很好！
07:44
But now put this in the context of hiring.
150
452911
2040
但是，設想若把這系統
用在僱人的情況下。
用在僱人的情況下。
07:48
So at this human resources
managers conference,
managers conference,
151
456027
3046
在這人資經理會議中，
07:51
I approached a high-level manager
in a very large company,
in a very large company,
152
459097
4709
我走向一間大公司的高階經理，
對她說：
07:55
and I said to her, "Look,
what if, unbeknownst to you,
what if, unbeknownst to you,
153
463830
4578
「假設在你不知道的情形下，
08:00
your system is weeding out people
with high future likelihood of depression?
with high future likelihood of depression?
154
468432
6549
那個系統被用來排除
未來極有可能抑鬱的人呢？
未來極有可能抑鬱的人呢？
08:07
They're not depressed now,
just maybe in the future, more likely.
just maybe in the future, more likely.
155
475761
3376
他們現在不抑鬱，
只是未來『比較有可能』抑鬱。
只是未來『比較有可能』抑鬱。
08:11
What if it's weeding out women
more likely to be pregnant
more likely to be pregnant
156
479923
3406
如果它被用來排除
在未來一兩年比較有可能懷孕，
在未來一兩年比較有可能懷孕，
08:15
in the next year or two
but aren't pregnant now?
but aren't pregnant now?
157
483353
2586
但現在沒懷孕的婦女呢？
08:18
What if it's hiring aggressive people
because that's your workplace culture?"
because that's your workplace culture?"
158
486844
5636
如果它被用來招募激進性格者，
以符合你的職場文化呢？」
以符合你的職場文化呢？」
08:25
You can't tell this by looking
at gender breakdowns.
at gender breakdowns.
159
493173
2691
透過性別比例無法看到這些問題，
08:27
Those may be balanced.
160
495888
1502
因比例可能是均衡的。
08:29
And since this is machine learning,
not traditional coding,
not traditional coding,
161
497414
3557
而且由於這是機器學習，
不是傳統編碼，
不是傳統編碼，
08:32
there is no variable there
labeled "higher risk of depression,"
labeled "higher risk of depression,"
162
500995
4907
沒有標記為「更高抑鬱症風險」、
08:37
"higher risk of pregnancy,"
163
505926
1833
「更高懷孕風險」、
08:39
"aggressive guy scale."
164
507783
1734
「侵略性格者」的變數；
08:41
Not only do you not know
what your system is selecting on,
what your system is selecting on,
165
509995
3679
你不僅不知道系統在選什麼，
08:45
you don't even know
where to begin to look.
where to begin to look.
166
513698
2323
甚至不知道要從何找起。
08:48
It's a black box.
167
516045
1246
它就是個黑盒子，
08:49
It has predictive power,
but you don't understand it.
but you don't understand it.
168
517315
2807
具有預測能力，但你不了解它。
08:52
"What safeguards," I asked, "do you have
169
520486
2369
我問：「你有什麼能確保
08:54
to make sure that your black box
isn't doing something shady?"
isn't doing something shady?"
170
522879
3673
你的黑盒子沒在暗地裡
做了什麼不可告人之事？
做了什麼不可告人之事？
09:00
She looked at me as if I had
just stepped on 10 puppy tails.
just stepped on 10 puppy tails.
171
528863
3878
她看著我，彷彿我剛踩了
十隻小狗的尾巴。
十隻小狗的尾巴。
09:04
(Laughter)
172
532765
1248
（笑聲）
09:06
She stared at me and she said,
173
534037
2041
她盯著我，說：
09:08
"I don't want to hear
another word about this."
another word about this."
174
536556
4333
「關於這事，我不想
再聽妳多說一個字。」
再聽妳多說一個字。」
09:13
And she turned around and walked away.
175
541458
2034
然後她就轉身走開了。
09:16
Mind you -- she wasn't rude.
176
544064
1486
提醒你們，她不是粗魯。
09:17
It was clearly: what I don't know
isn't my problem, go away, death stare.
isn't my problem, go away, death stare.
177
545574
6308
她的意思很明顯：
我不知道的事不是我的問題。
走開。惡狠狠盯著。
09:23
(Laughter)
178
551906
1246
（笑聲）
09:25
Look, such a system
may even be less biased
may even be less biased
179
553862
3839
這樣的系統可能比人類經理
在某些方面更沒有偏見，
在某些方面更沒有偏見，
09:29
than human managers in some ways.
180
557725
2103
09:31
And it could make monetary sense.
181
559852
2146
可能也省錢；
09:34
But it could also lead
182
562573
1650
但也可能在不知不覺中逐步導致
09:36
to a steady but stealthy
shutting out of the job market
shutting out of the job market
183
564247
4748
抑鬱症風險較高的人
在就業市場裡吃到閉門羹。
在就業市場裡吃到閉門羹。
09:41
of people with higher risk of depression.
184
569019
2293
09:43
Is this the kind of society
we want to build,
we want to build,
185
571753
2596
我們要在不自覺的情形下
建立這種社會嗎？
建立這種社會嗎？
09:46
without even knowing we've done this,
186
574373
2285
09:48
because we turned decision-making
to machines we don't totally understand?
to machines we don't totally understand?
187
576682
3964
僅僅因我們讓給
我們不完全理解的機器做決策？
我們不完全理解的機器做決策？
09:53
Another problem is this:
188
581265
1458
另一個問題是：這些系統通常由
09:55
these systems are often trained
on data generated by our actions,
on data generated by our actions,
189
583314
4452
我們行動產生的數據，
即人類的印記所訓練。
即人類的印記所訓練。
09:59
human imprints.
190
587790
1816
10:02
Well, they could just be
reflecting our biases,
reflecting our biases,
191
590188
3808
它們可能只是反映我們的偏見，
10:06
and these systems
could be picking up on our biases
could be picking up on our biases
192
594020
3593
學習了我們的偏見
10:09
and amplifying them
193
597637
1313
並且放大，
10:10
and showing them back to us,
194
598974
1418
然後回饋給我們；
10:12
while we're telling ourselves,
195
600416
1462
而我們卻告訴自己：
10:13
"We're just doing objective,
neutral computation."
neutral computation."
196
601902
3117
「這樣做是客觀、不偏頗的計算。」
10:18
Researchers found that on Google,
197
606314
2677
研究人員在谷歌上發現，
10:22
women are less likely than men
to be shown job ads for high-paying jobs.
to be shown job ads for high-paying jobs.
198
610134
5313
女性看到高薪工作招聘的廣告
少於男性。
少於男性。
10:28
And searching for African-American names
199
616463
2530
蒐索非裔美國人的名字
10:31
is more likely to bring up ads
suggesting criminal history,
suggesting criminal history,
200
619017
4706
比較可能帶出暗示犯罪史的廣告，
10:35
even when there is none.
201
623747
1567
即使那人並無犯罪史。
10:38
Such hidden biases
and black-box algorithms
and black-box algorithms
202
626693
3549
這種隱藏偏見和黑箱的演算法，
10:42
that researchers uncover sometimes
but sometimes we don't know,
but sometimes we don't know,
203
630266
3973
有時被研究人員發現了，
但有時我們毫無所知，
但有時我們毫無所知，
10:46
can have life-altering consequences.
204
634263
2661
很可能產生改變生命的後果。
10:49
In Wisconsin, a defendant
was sentenced to six years in prison
was sentenced to six years in prison
205
637958
4159
在威斯康辛州，某個被告
因逃避警察而被判處六年監禁。
因逃避警察而被判處六年監禁。
10:54
for evading the police.
206
642141
1355
10:56
You may not know this,
207
644824
1186
你可能不知道
10:58
but algorithms are increasingly used
in parole and sentencing decisions.
in parole and sentencing decisions.
208
646034
3998
演算法越來越頻繁地被用在
假釋和量刑的決定上。
11:02
He wanted to know:
How is this score calculated?
How is this score calculated?
209
650056
2955
想知道分數如何計算出來的嗎？
11:05
It's a commercial black box.
210
653795
1665
這是個商業的黑盒子，
11:07
The company refused to have its algorithm
be challenged in open court.
be challenged in open court.
211
655484
4205
開發它的公司
拒絕讓演算法在公開法庭上受盤問。
11:12
But ProPublica, an investigative
nonprofit, audited that very algorithm
nonprofit, audited that very algorithm
212
660396
5532
但是 ProPublica 這家
非營利機構評估該演算法，
非營利機構評估該演算法，
11:17
with what public data they could find,
213
665952
2016
使用找得到的公共數據，
11:19
and found that its outcomes were biased
214
667992
2316
發現其結果偏頗，
11:22
and its predictive power
was dismal, barely better than chance,
was dismal, barely better than chance,
215
670332
3629
預測能力相當差，
僅比碰運氣稍強，
僅比碰運氣稍強，
11:25
and it was wrongly labeling
black defendants as future criminals
black defendants as future criminals
216
673985
4416
並錯誤地標記
黑人被告成為未來罪犯的機率，
黑人被告成為未來罪犯的機率，
11:30
at twice the rate of white defendants.
217
678425
3895
是白人被告的兩倍。
11:35
So, consider this case:
218
683891
1564
考慮這個情況：
11:38
This woman was late
picking up her godsister
picking up her godsister
219
686103
3852
這女人因來不及去佛州布勞沃德郡的
學校接她的乾妹妹，
學校接她的乾妹妹，
11:41
from a school in Broward County, Florida,
220
689979
2075
而與朋友狂奔趕赴學校。
11:44
running down the street
with a friend of hers.
with a friend of hers.
221
692757
2356
他們看到門廊上有一輛未上鎖的
兒童腳踏車和一台滑板車，
兒童腳踏車和一台滑板車，
11:47
They spotted an unlocked kid's bike
and a scooter on a porch
and a scooter on a porch
222
695137
4099
11:51
and foolishly jumped on it.
223
699260
1632
愚蠢地跳上去，
11:52
As they were speeding off,
a woman came out and said,
a woman came out and said,
224
700916
2599
當他們趕時間快速離去時，
一個女人出來說：
「嘿！那是我孩子的腳踏車！」
「嘿！那是我孩子的腳踏車！」
11:55
"Hey! That's my kid's bike!"
225
703539
2205
11:57
They dropped it, they walked away,
but they were arrested.
but they were arrested.
226
705768
3294
雖然他們留下車子走開，
但被逮捕了。
但被逮捕了。
12:01
She was wrong, she was foolish,
but she was also just 18.
but she was also just 18.
227
709086
3637
她錯了，她很蠢，
但她只有十八歲。
但她只有十八歲。
12:04
She had a couple of juvenile misdemeanors.
228
712747
2544
曾觸犯兩次少年輕罪。
12:07
Meanwhile, that man had been arrested
for shoplifting in Home Depot --
for shoplifting in Home Depot --
229
715808
5185
同時，
那個男人因在家得寶商店
偷竊八十五美元的東西而被捕，
偷竊八十五美元的東西而被捕，
12:13
85 dollars' worth of stuff,
a similar petty crime.
a similar petty crime.
230
721017
2924
類似的小罪，
12:16
But he had two prior
armed robbery convictions.
armed robbery convictions.
231
724766
4559
但他曾兩次因武裝搶劫而被定罪。
演算法認定她有再犯的高風險，
12:21
But the algorithm scored her
as high risk, and not him.
as high risk, and not him.
232
729955
3482
而他卻不然。
12:26
Two years later, ProPublica found
that she had not reoffended.
that she had not reoffended.
233
734746
3874
兩年後，ProPublica
發現她未曾再犯；
發現她未曾再犯；
12:30
It was just hard to get a job
for her with her record.
for her with her record.
234
738644
2550
但因有過犯罪紀錄而難以找到工作。
12:33
He, on the other hand, did reoffend
235
741218
2076
另一方面，他再犯了，
12:35
and is now serving an eight-year
prison term for a later crime.
prison term for a later crime.
236
743318
3836
現正因再犯之罪而入監服刑八年。
12:40
Clearly, we need to audit our black boxes
237
748088
3369
很顯然，我們必需審核黑盒子，
12:43
and not have them have
this kind of unchecked power.
this kind of unchecked power.
238
751481
2615
並且不賦予它們
這類未經檢查的權力。
這類未經檢查的權力。
12:46
(Applause)
239
754120
2879
（掌聲）
12:50
Audits are great and important,
but they don't solve all our problems.
but they don't solve all our problems.
240
758087
4242
審核極其重要，
但不足以解決所有的問題。
但不足以解決所有的問題。
12:54
Take Facebook's powerful
news feed algorithm --
news feed algorithm --
241
762353
2748
拿臉書強大的動態消息演算法來說，
12:57
you know, the one that ranks everything
and decides what to show you
and decides what to show you
242
765125
4843
就是通過你的朋友圈
和瀏覽過的頁面，
和瀏覽過的頁面，
排序並決定推薦
什麼給你看的演算法。
什麼給你看的演算法。
13:01
from all the friends and pages you follow.
243
769992
2284
13:04
Should you be shown another baby picture?
244
772898
2275
應該再讓你看一張嬰兒照片嗎？
13:07
(Laughter)
245
775197
1196
（笑聲）
13:08
A sullen note from an acquaintance?
246
776417
2596
或者一個熟人的哀傷筆記？
13:11
An important but difficult news item?
247
779449
1856
還是一則重要但艱澀的新聞？
13:13
There's no right answer.
248
781329
1482
沒有正確的答案。
13:14
Facebook optimizes
for engagement on the site:
for engagement on the site:
249
782835
2659
臉書根據在網站上的參與度來優化：
13:17
likes, shares, comments.
250
785518
1415
喜歡，分享，評論。
13:20
In August of 2014,
251
788168
2696
2014年八月，
13:22
protests broke out in Ferguson, Missouri,
252
790888
2662
在密蘇里州弗格森市
爆發了抗議遊行，
爆發了抗議遊行，
13:25
after the killing of an African-American
teenager by a white police officer,
teenager by a white police officer,
253
793574
4417
抗議一位白人警察在不明的狀況下
殺害一個非裔美國少年，
殺害一個非裔美國少年，
13:30
under murky circumstances.
254
798015
1570
13:31
The news of the protests was all over
255
799974
2007
抗議的消息充斥在
13:34
my algorithmically
unfiltered Twitter feed,
unfiltered Twitter feed,
256
802005
2685
我未經演算法篩選過的推特頁面上，
13:36
but nowhere on my Facebook.
257
804714
1950
但我的臉書上卻一則也沒有。
13:39
Was it my Facebook friends?
258
807182
1734
是我的臉書好友不關注這事嗎？
13:40
I disabled Facebook's algorithm,
259
808940
2032
我關閉了臉書的演算法，
13:43
which is hard because Facebook
keeps wanting to make you
keeps wanting to make you
260
811472
2848
但很麻煩惱人，
因為臉書不斷地
想讓你回到演算法的控制下，
想讓你回到演算法的控制下，
13:46
come under the algorithm's control,
261
814344
2036
13:48
and saw that my friends
were talking about it.
were talking about it.
262
816404
2238
臉書的朋友有在談論弗格森這事，
13:50
It's just that the algorithm
wasn't showing it to me.
wasn't showing it to me.
263
818666
2509
只是臉書的演算法沒有顯示給我看。
研究後，我發現這問題普遍存在。
13:53
I researched this and found
this was a widespread problem.
this was a widespread problem.
264
821199
3042
13:56
The story of Ferguson
wasn't algorithm-friendly.
wasn't algorithm-friendly.
265
824265
3813
弗格森一事和演算法不合，
14:00
It's not "likable."
266
828102
1171
它不討喜；
14:01
Who's going to click on "like?"
267
829297
1552
誰會點擊「讚」呢？
14:03
It's not even easy to comment on.
268
831500
2206
它甚至不易被評論。
14:05
Without likes and comments,
269
833730
1371
越是沒有讚、沒評論，
14:07
the algorithm was likely showing it
to even fewer people,
to even fewer people,
270
835125
3292
演算法就顯示給越少人看，
所以我們看不到這則新聞。
14:10
so we didn't get to see this.
271
838441
1542
14:12
Instead, that week,
272
840946
1228
相反地，
臉書的演算法在那星期特別突顯
為漸凍人募款的冰桶挑戰這事。
為漸凍人募款的冰桶挑戰這事。
14:14
Facebook's algorithm highlighted this,
273
842198
2298
14:16
which is the ALS Ice Bucket Challenge.
274
844520
2226
14:18
Worthy cause; dump ice water,
donate to charity, fine.
donate to charity, fine.
275
846770
3742
崇高的目標；傾倒冰水，捐贈慈善，
有意義，很好；
有意義，很好；
14:22
But it was super algorithm-friendly.
276
850536
1904
這事與演算法超級速配，
14:25
The machine made this decision for us.
277
853219
2613
機器已為我們決定了。
14:27
A very important
but difficult conversation
but difficult conversation
278
855856
3497
非常重要但艱澀的
新聞事件可能被埋沒掉，
新聞事件可能被埋沒掉，
14:31
might have been smothered,
279
859377
1555
14:32
had Facebook been the only channel.
280
860956
2696
倘若臉書是唯一的新聞渠道。
14:36
Now, finally, these systems
can also be wrong
can also be wrong
281
864117
3797
最後，這些系統
也可能以不像人類犯錯的方式出錯。
14:39
in ways that don't resemble human systems.
282
867938
2736
14:42
Do you guys remember Watson,
IBM's machine-intelligence system
IBM's machine-intelligence system
283
870698
2922
大家可還記得 IBM 的
機器智慧系統華生
機器智慧系統華生
14:45
that wiped the floor
with human contestants on Jeopardy?
with human contestants on Jeopardy?
284
873644
3128
在 Jeopardy 的
智力問答比賽中橫掃人類的對手？
智力問答比賽中橫掃人類的對手？
14:49
It was a great player.
285
877131
1428
它是個厲害的選手。
14:50
But then, for Final Jeopardy,
Watson was asked this question:
Watson was asked this question:
286
878583
3569
在 Final Jeopardy 中
華生被問到：
14:54
"Its largest airport is named
for a World War II hero,
for a World War II hero,
287
882659
2932
「它的最大機場以二戰英雄命名，
14:57
its second-largest
for a World War II battle."
for a World War II battle."
288
885615
2252
第二大機場以二戰戰場為名。」
14:59
(Hums Final Jeopardy music)
289
887891
1378
（哼 Jeopardy 的音樂）
15:01
Chicago.
290
889582
1182
「芝加哥」，
15:02
The two humans got it right.
291
890788
1370
兩個人類選手的答案正確；
15:04
Watson, on the other hand,
answered "Toronto" --
answered "Toronto" --
292
892697
4348
華生則回答「多倫多」。
15:09
for a US city category!
293
897069
1818
這是個猜「美國」城市的問題啊！
15:11
The impressive system also made an error
294
899596
2901
這個厲害的系統也犯了
15:14
that a human would never make,
a second-grader wouldn't make.
a second-grader wouldn't make.
295
902521
3651
人類永遠不會犯，
即使二年級學生也不會犯的錯誤。
15:18
Our machine intelligence can fail
296
906823
3109
我們的機器智慧可能敗在
15:21
in ways that don't fit
error patterns of humans,
error patterns of humans,
297
909956
3100
與人類犯錯模式迥異之處，
15:25
in ways we won't expect
and be prepared for.
and be prepared for.
298
913080
2950
在我們完全想不到、
沒準備的地方出錯。
沒準備的地方出錯。
15:28
It'd be lousy not to get a job
one is qualified for,
one is qualified for,
299
916054
3638
得不到一份可勝任的工作
確實很糟糕，
確實很糟糕，
15:31
but it would triple suck
if it was because of stack overflow
if it was because of stack overflow
300
919716
3727
但若起因是機器的子程式漫溢，
會是三倍的糟糕。
會是三倍的糟糕。
15:35
in some subroutine.
301
923467
1432
15:36
(Laughter)
302
924923
1579
（笑聲）
15:38
In May of 2010,
303
926526
2786
2010年五月，
15:41
a flash crash on Wall Street
fueled by a feedback loop
fueled by a feedback loop
304
929336
4044
華爾街「賣出」演算法的
回饋迴路觸發了股市的急速崩盤，
回饋迴路觸發了股市的急速崩盤，
15:45
in Wall Street's "sell" algorithm
305
933404
3028
15:48
wiped a trillion dollars
of value in 36 minutes.
of value in 36 minutes.
306
936456
4184
數萬億美元的市值
在36分鐘內蒸發掉了。
在36分鐘內蒸發掉了。
15:53
I don't even want to think
what "error" means
what "error" means
307
941722
2187
我甚至不敢想
若「錯誤」發生在致命的
自動武器上會是何種情況。
自動武器上會是何種情況。
15:55
in the context of lethal
autonomous weapons.
autonomous weapons.
308
943933
3589
16:01
So yes, humans have always made biases.
309
949894
3790
是啊，人類總是有偏見。
16:05
Decision makers and gatekeepers,
310
953708
2176
決策者和守門人
16:07
in courts, in news, in war ...
311
955908
3493
在法庭、新聞中、戰爭裡 …
都會犯錯；
都會犯錯；
16:11
they make mistakes;
but that's exactly my point.
but that's exactly my point.
312
959425
3038
但這正是我的觀點：
16:14
We cannot escape
these difficult questions.
these difficult questions.
313
962487
3521
我們不能逃避這些困難的問題。
16:18
We cannot outsource
our responsibilities to machines.
our responsibilities to machines.
314
966596
3516
我們不能把責任外包給機器。
16:22
(Applause)
315
970676
4208
（掌聲）
16:29
Artificial intelligence does not give us
a "Get out of ethics free" card.
a "Get out of ethics free" card.
316
977089
4447
人工智慧不會給我們
「倫理免責卡」。
「倫理免責卡」。
16:34
Data scientist Fred Benenson
calls this math-washing.
calls this math-washing.
317
982742
3381
數據科學家費德·本森
稱之為「數學粉飾」。
稱之為「數學粉飾」。
16:38
We need the opposite.
318
986147
1389
我們需要相反的東西。
16:39
We need to cultivate algorithm suspicion,
scrutiny and investigation.
scrutiny and investigation.
319
987560
5388
我們需要培養懷疑、審視
和調查演算法的能力。
和調查演算法的能力。
16:45
We need to make sure we have
algorithmic accountability,
algorithmic accountability,
320
993380
3198
我們需確保演算法有人負責，
16:48
auditing and meaningful transparency.
321
996602
2445
能被審查，並且確實公開透明。
16:51
We need to accept
that bringing math and computation
that bringing math and computation
322
999380
3234
我們必須體認，
把數學和演算法帶入凌亂、
具價值觀的人類事務
具價值觀的人類事務
16:54
to messy, value-laden human affairs
323
1002638
2970
16:57
does not bring objectivity;
324
1005632
2384
不能帶來客觀性；
17:00
rather, the complexity of human affairs
invades the algorithms.
invades the algorithms.
325
1008040
3633
相反地，人類事務的複雜性
侵入演算法。
侵入演算法。
17:04
Yes, we can and we should use computation
326
1012148
3487
是啊，我們可以、也應該用演算法
17:07
to help us make better decisions.
327
1015659
2014
來幫助我們做出更好的決定。
17:09
But we have to own up
to our moral responsibility to judgment,
to our moral responsibility to judgment,
328
1017697
5332
但我們也需要在判斷中
加入道德義務，
加入道德義務，
17:15
and use algorithms within that framework,
329
1023053
2818
並在該框架內使用演算法，
17:17
not as a means to abdicate
and outsource our responsibilities
and outsource our responsibilities
330
1025895
4935
而不是像人與人間相互推卸那樣，
17:22
to one another as human to human.
331
1030854
2454
就把責任轉移給機器。
17:25
Machine intelligence is here.
332
1033807
2609
機器智慧已經到來，
17:28
That means we must hold on ever tighter
333
1036440
3421
這意味著我們必須更堅守
人類價值觀和人類倫理。
17:31
to human values and human ethics.
334
1039885
2147
17:34
Thank you.
335
1042056
1154
謝謝。
17:35
(Applause)
336
1043234
5020
(掌聲）
ABOUT THE SPEAKER
Zeynep Tufekci - Techno-sociologistTechno-sociologist Zeynep Tufekci asks big questions about our societies and our lives, as both algorithms and digital connectivity spread.
Why you should listen
We've entered an era of digital connectivity and machine intelligence. Complex algorithms are increasingly used to make consequential decisions about us. Many of these decisions are subjective and have no right answer: who should be hired, fired or promoted; what news should be shown to whom; which of your friends do you see updates from; which convict should be paroled. With increasing use of machine learning in these systems, we often don't even understand how exactly they are making these decisions. Zeynep Tufekci studies what this historic transition means for culture, markets, politics and personal life.
Tufekci is a contributing opinion writer at the New York Times, an associate professor at the School of Information and Library Science at University of North Carolina, Chapel Hill, and a faculty associate at Harvard's Berkman Klein Center for Internet and Society.
Her book, Twitter and Tear Gas: The Power and Fragility of Networked Protest, was published in 2017 by Yale University Press. Her next book, from Penguin Random House, will be about algorithms that watch, judge and nudge us.
Zeynep Tufekci | Speaker | TED.com