TED2014
Del Harvey: Protecting Twitter users (sometimes from themselves)
Filmed:
Readability: 4.4
993,199 views
Del Harvey heads up Twitter’s Trust and Safety Team, and she thinks all day about how to prevent worst-case scenarios -- abuse, trolling, stalking -- while giving voice to people around the globe. With deadpan humor, she offers a window into how she works to keep 240 million users safe.
Del Harvey - Security maven
Del Harvey works to define policy and to ensure user safety and security in the challenging realm of modern social media. Full bio
Del Harvey works to define policy and to ensure user safety and security in the challenging realm of modern social media. Full bio
Double-click the English transcript below to play the video.
00:12
My job at Twitter
0
984
1291
00:14
is to ensure user trust,
1
2275
1978
00:16
protect user rights and keep users safe,
2
4253
2837
00:19
both from each other
3
7090
1260
00:20
and, at times, from themselves.
4
8350
3899
00:24
Let's talk about what scale looks like at Twitter.
5
12249
4275
00:28
Back in January 2009,
6
16524
2870
00:31
we saw more than two million new tweets each day
7
19394
3331
00:34
on the platform.
8
22725
1764
00:36
January 2014, more than 500 million.
9
24489
5908
00:42
We were seeing two million tweets
10
30397
2492
00:44
in less than six minutes.
11
32889
2176
00:47
That's a 24,900-percent increase.
12
35065
6984
00:54
Now, the vast majority of activity on Twitter
13
42049
3253
00:57
puts no one in harm's way.
14
45302
1503
00:58
There's no risk involved.
15
46805
1935
01:00
My job is to root out and prevent activity that might.
16
48740
5753
01:06
Sounds straightforward, right?
17
54493
1973
01:08
You might even think it'd be easy,
18
56466
1152
01:09
given that I just said the vast majority
19
57618
2170
01:11
of activity on Twitter puts no one in harm's way.
20
59788
3810
01:15
Why spend so much time
21
63598
2169
01:17
searching for potential calamities
22
65767
2743
01:20
in innocuous activities?
23
68510
2900
01:23
Given the scale that Twitter is at,
24
71410
2940
01:26
a one-in-a-million chance happens
25
74350
2357
01:28
500 times a day.
26
76707
4876
01:33
It's the same for other companies
27
81583
1445
01:35
dealing at this sort of scale.
28
83028
1471
01:36
For us, edge cases,
29
84499
1708
01:38
those rare situations that are unlikely to occur,
30
86207
3625
01:41
are more like norms.
31
89832
2622
01:44
Say 99.999 percent of tweets
32
92454
3942
01:48
pose no risk to anyone.
33
96396
1888
01:50
There's no threat involved.
34
98284
1066
01:51
Maybe people are documenting travel landmarks
35
99350
2954
01:54
like Australia's Heart Reef,
36
102304
1963
01:56
or tweeting about a concert they're attending,
37
104267
2921
01:59
or sharing pictures of cute baby animals.
38
107188
4747
02:03
After you take out that 99.999 percent,
39
111935
4509
02:08
that tiny percentage of tweets remaining
40
116444
3529
02:11
works out to roughly
41
119973
2389
02:14
150,000 per month.
42
122362
3475
02:17
The sheer scale of what we're dealing with
43
125837
2456
02:20
makes for a challenge.
44
128293
2312
02:22
You know what else makes my role
45
130605
1178
02:23
particularly challenging?
46
131783
3107
02:26
People do weird things.
47
134890
5123
02:32
(Laughter)
48
140013
1829
02:33
And I have to figure out what they're doing,
49
141842
2391
02:36
why, and whether or not there's risk involved,
50
144233
2249
02:38
often without much in terms of context
51
146482
2168
02:40
or background.
52
148650
1847
02:42
I'm going to show you some examples
53
150497
2077
02:44
that I've run into during my time at Twitter --
54
152574
2005
02:46
these are all real examples —
55
154579
1620
02:48
of situations that at first seemed cut and dried,
56
156199
2653
02:50
but the truth of the matter was something
57
158852
1643
02:52
altogether different.
58
160495
1550
02:54
The details have been changed
59
162045
1977
02:56
to protect the innocent
60
164022
1257
02:57
and sometimes the guilty.
61
165279
3233
03:00
We'll start off easy.
62
168512
3005
03:03
["Yo bitch"]
63
171517
1793
03:05
If you saw a Tweet that only said this,
64
173310
3228
03:08
you might think to yourself,
65
176538
1694
03:10
"That looks like abuse."
66
178232
1653
03:11
After all, why would you
want to receive the message,
want to receive the message,
67
179885
3107
03:14
"Yo, bitch."
68
182992
2218
03:17
Now, I try to stay relatively hip
69
185210
4663
03:21
to the latest trends and memes,
70
189873
2512
03:24
so I knew that "yo, bitch"
71
192385
2704
03:27
was also often a common greeting between friends,
72
195089
3154
03:30
as well as being a popular "Breaking Bad" reference.
73
198243
4262
03:34
I will admit that I did not expect
74
202505
2487
03:36
to encounter a fourth use case.
75
204992
2841
03:39
It turns out it is also used on Twitter
76
207833
3104
03:42
when people are role-playing as dogs.
77
210937
3062
03:45
(Laughter)
78
213999
5279
03:51
And in fact, in that case,
79
219278
1666
03:52
it's not only not abusive,
80
220944
1609
03:54
it's technically just an accurate greeting.
81
222553
3139
03:57
(Laughter)
82
225692
2889
04:00
So okay, determining whether or not
83
228581
2071
04:02
something is abusive without context,
84
230652
1848
04:04
definitely hard.
85
232500
1592
04:06
Let's look at spam.
86
234092
2717
04:08
Here's an example of an account engaged
87
236809
1960
04:10
in classic spammer behavior,
88
238769
1668
04:12
sending the exact same message
89
240437
1559
04:13
to thousands of people.
90
241996
1804
04:15
While this is a mockup I put
together using my account,
together using my account,
91
243800
2793
04:18
we see accounts doing this all the time.
92
246593
3001
04:21
Seems pretty straightforward.
93
249594
1979
04:23
We should just automatically suspend accounts
94
251573
2053
04:25
engaging in this kind of behavior.
95
253626
3307
04:28
Turns out there's some exceptions to that rule.
96
256933
3210
04:32
Turns out that that message
could also be a notification
could also be a notification
97
260143
2883
04:35
you signed up for that the International
Space Station is passing overhead
Space Station is passing overhead
98
263026
3889
04:38
because you wanted to go outside
99
266915
1846
04:40
and see if you could see it.
100
268761
1948
04:42
You're not going to get that chance
101
270709
1225
04:43
if we mistakenly suspend the account
102
271934
1847
04:45
thinking it's spam.
103
273781
2266
04:48
Okay. Let's make the stakes higher.
104
276047
3526
04:51
Back to my account,
105
279573
1916
04:53
again exhibiting classic behavior.
106
281489
3505
04:56
This time it's sending the same message and link.
107
284994
2643
04:59
This is often indicative of
something called phishing,
something called phishing,
108
287637
2774
05:02
somebody trying to steal another
person's account information
person's account information
109
290411
3178
05:05
by directing them to another website.
110
293589
2203
05:07
That's pretty clearly not a good thing.
111
295792
4194
05:11
We want to, and do, suspend accounts
112
299986
1930
05:13
engaging in that kind of behavior.
113
301916
2624
05:16
So why are the stakes higher for this?
114
304540
3247
05:19
Well, this could also be a bystander at a rally
115
307787
2999
05:22
who managed to record a video
116
310786
1910
05:24
of a police officer beating a non-violent protester
117
312696
3270
05:27
who's trying to let the world know what's happening.
118
315966
2975
05:30
We don't want to gamble
119
318941
1643
05:32
on potentially silencing that crucial speech
120
320584
2517
05:35
by classifying it as spam and suspending it.
121
323101
2929
05:38
That means we evaluate hundreds of parameters
122
326030
2879
05:40
when looking at account behaviors,
123
328909
1688
05:42
and even then, we can still get it wrong
124
330597
2016
05:44
and have to reevaluate.
125
332613
2236
05:46
Now, given the sorts of challenges I'm up against,
126
334849
3708
05:50
it's crucial that I not only predict
127
338557
2696
05:53
but also design protections for the unexpected.
128
341253
3784
05:57
And that's not just an issue for me,
129
345037
2342
05:59
or for Twitter, it's an issue for you.
130
347379
2087
06:01
It's an issue for anybody who's building or creating
131
349466
2406
06:03
something that you think is going to be amazing
132
351872
1925
06:05
and will let people do awesome things.
133
353797
2789
06:08
So what do I do?
134
356586
2866
06:11
I pause and I think,
135
359452
3318
06:14
how could all of this
136
362770
2095
06:16
go horribly wrong?
137
364865
3793
06:20
I visualize catastrophe.
138
368658
4453
06:25
And that's hard. There's a sort of
139
373111
2463
06:27
inherent cognitive dissonance in doing that,
140
375574
2848
06:30
like when you're writing your wedding vows
141
378422
1812
06:32
at the same time as your prenuptial agreement.
142
380234
2646
06:34
(Laughter)
143
382880
1696
06:36
But you still have to do it,
144
384576
2373
06:38
particularly if you're marrying
500 million tweets per day.
500 million tweets per day.
145
386949
4446
06:43
What do I mean by "visualize catastrophe?"
146
391395
3097
06:46
I try to think of how something as
147
394492
2762
06:49
benign and innocuous as a picture of a cat
148
397254
3228
06:52
could lead to death,
149
400482
1104
06:53
and what to do to prevent that.
150
401586
2326
06:55
Which happens to be my next example.
151
403912
2383
06:58
This is my cat, Eli.
152
406295
3110
07:01
We wanted to give users the ability
153
409405
1981
07:03
to add photos to their tweets.
154
411386
2073
07:05
A picture is worth a thousand words.
155
413459
1597
07:07
You only get 140 characters.
156
415056
2009
07:09
You add a photo to your tweet,
157
417065
1200
07:10
look at how much more content you've got now.
158
418265
3038
07:13
There's all sorts of great things you can do
159
421303
1677
07:14
by adding a photo to a tweet.
160
422980
2007
07:16
My job isn't to think of those.
161
424987
2280
07:19
It's to think of what could go wrong.
162
427267
2747
07:22
How could this picture
163
430014
1892
07:23
lead to my death?
164
431906
3539
07:27
Well, here's one possibility.
165
435445
3160
07:30
There's more in that picture than just a cat.
166
438605
3086
07:33
There's geodata.
167
441691
2092
07:35
When you take a picture with your smartphone
168
443783
2212
07:37
or digital camera,
169
445995
1299
07:39
there's a lot of additional information
170
447294
1654
07:40
saved along in that image.
171
448948
1616
07:42
In fact, this image also contains
172
450564
1932
07:44
the equivalent of this,
173
452496
1805
07:46
more specifically, this.
174
454301
3079
07:49
Sure, it's not likely that someone's going to try
175
457380
1956
07:51
to track me down and do me harm
176
459336
2285
07:53
based upon image data associated
177
461621
1784
07:55
with a picture I took of my cat,
178
463405
1948
07:57
but I start by assuming the worst will happen.
179
465353
3651
08:01
That's why, when we launched photos on Twitter,
180
469004
2338
08:03
we made the decision to strip that geodata out.
181
471342
3821
08:07
(Applause)
182
475163
5847
08:13
If I start by assuming the worst
183
481010
2613
08:15
and work backwards,
184
483623
947
08:16
I can make sure that the protections we build
185
484570
2553
08:19
work for both expected
186
487123
1768
08:20
and unexpected use cases.
187
488891
2078
08:22
Given that I spend my days and nights
188
490969
2945
08:25
imagining the worst that could happen,
189
493914
2541
08:28
it wouldn't be surprising if
my worldview was gloomy.
my worldview was gloomy.
190
496455
4257
08:32
(Laughter)
191
500712
1783
08:34
It's not.
192
502495
1417
08:35
The vast majority of interactions I see --
193
503912
3876
08:39
and I see a lot, believe me -- are positive,
194
507788
3901
08:43
people reaching out to help
195
511689
1924
08:45
or to connect or share information with each other.
196
513613
3448
08:49
It's just that for those of us dealing with scale,
197
517061
3323
08:52
for those of us tasked with keeping people safe,
198
520384
3800
08:56
we have to assume the worst will happen,
199
524184
2546
08:58
because for us, a one-in-a-million chance
200
526730
4227
09:02
is pretty good odds.
201
530957
2749
09:05
Thank you.
202
533706
1864
09:07
(Applause)
203
535570
4000
ABOUT THE SPEAKER
Del Harvey - Security mavenDel Harvey works to define policy and to ensure user safety and security in the challenging realm of modern social media.
Why you should listen
As Senior Director of Trust and Safety at Twitter, Del Harvey works to define policy and to ensure user safety and security in the challenging realm of modern social media. Prior to joining Twitter, she spent five years as the co-administrator and law enforcement liaison for a 501(c)3 non-profit charity, working with agencies ranging from local police departments to the FBI, U.S. Marshals and the Secret Service.
Del Harvey | Speaker | TED.com