sponsored links
TEDxPSU

Bruce Schneier: The security mirage

October 10, 2010

The feeling of security and the reality of security don't always match, says computer-security expert Bruce Schneier. In his talk, he explains why we spend billions addressing news story risks, like the "security theater" now playing at your local airport, while neglecting more probable risks -- and how we can break this pattern.

Bruce Schneier - Security expert
Bruce Schneier thinks hard about security -- as a computer security guru, and as a philosopher of the larger notion of making a safer world. Full bio

sponsored links
Double-click the English subtitles below to play the video.
So security is two different things:
00:15
it's a feeling, and it's a reality.
00:17
And they're different.
00:19
You could feel secure
00:21
even if you're not.
00:23
And you can be secure
00:25
even if you don't feel it.
00:27
Really, we have two separate concepts
00:29
mapped onto the same word.
00:31
And what I want to do in this talk
00:33
is to split them apart --
00:35
figuring out when they diverge
00:37
and how they converge.
00:39
And language is actually a problem here.
00:41
There aren't a lot of good words
00:43
for the concepts we're going to talk about.
00:45
So if you look at security
00:48
from economic terms,
00:50
it's a trade-off.
00:52
Every time you get some security,
00:54
you're always trading off something.
00:56
Whether this is a personal decision --
00:58
whether you're going to install a burglar alarm in your home --
01:00
or a national decision -- where you're going to invade some foreign country --
01:02
you're going to trade off something,
01:05
either money or time, convenience, capabilities,
01:07
maybe fundamental liberties.
01:10
And the question to ask when you look at a security anything
01:13
is not whether this makes us safer,
01:16
but whether it's worth the trade-off.
01:19
You've heard in the past several years,
01:22
the world is safer because Saddam Hussein is not in power.
01:24
That might be true, but it's not terribly relevant.
01:26
The question is, was it worth it?
01:29
And you can make your own decision,
01:32
and then you'll decide whether the invasion was worth it.
01:35
That's how you think about security --
01:37
in terms of the trade-off.
01:39
Now there's often no right or wrong here.
01:41
Some of us have a burglar alarm system at home,
01:44
and some of us don't.
01:46
And it'll depend on where we live,
01:48
whether we live alone or have a family,
01:50
how much cool stuff we have,
01:52
how much we're willing to accept
01:54
the risk of theft.
01:56
In politics also,
01:58
there are different opinions.
02:00
A lot of times, these trade-offs
02:02
are about more than just security,
02:04
and I think that's really important.
02:06
Now people have a natural intuition
02:08
about these trade-offs.
02:10
We make them every day --
02:12
last night in my hotel room,
02:14
when I decided to double-lock the door,
02:16
or you in your car when you drove here,
02:18
when we go eat lunch
02:20
and decide the food's not poison and we'll eat it.
02:22
We make these trade-offs again and again,
02:25
multiple times a day.
02:27
We often won't even notice them.
02:29
They're just part of being alive; we all do it.
02:31
Every species does it.
02:33
Imagine a rabbit in a field, eating grass,
02:36
and the rabbit's going to see a fox.
02:38
That rabbit will make a security trade-off:
02:41
"Should I stay, or should I flee?"
02:43
And if you think about it,
02:45
the rabbits that are good at making that trade-off
02:47
will tend to live and reproduce,
02:50
and the rabbits that are bad at it
02:52
will get eaten or starve.
02:54
So you'd think
02:56
that us, as a successful species on the planet --
02:58
you, me, everybody --
03:01
would be really good at making these trade-offs.
03:03
Yet it seems, again and again,
03:06
that we're hopelessly bad at it.
03:08
And I think that's a fundamentally interesting question.
03:11
I'll give you the short answer.
03:14
The answer is, we respond to the feeling of security
03:16
and not the reality.
03:18
Now most of the time, that works.
03:21
Most of the time,
03:25
feeling and reality are the same.
03:27
Certainly that's true
03:30
for most of human prehistory.
03:32
We've developed this ability
03:35
because it makes evolutionary sense.
03:38
One way to think of it
03:40
is that we're highly optimized
03:42
for risk decisions
03:44
that are endemic to living in small family groups
03:46
in the East African highlands in 100,000 B.C.
03:49
2010 New York, not so much.
03:52
Now there are several biases in risk perception.
03:56
A lot of good experiments in this.
03:59
And you can see certain biases that come up again and again.
04:01
So I'll give you four.
04:04
We tend to exaggerate spectacular and rare risks
04:06
and downplay common risks --
04:09
so flying versus driving.
04:11
The unknown is perceived
04:14
to be riskier than the familiar.
04:16
One example would be,
04:20
people fear kidnapping by strangers
04:22
when the data supports kidnapping by relatives is much more common.
04:25
This is for children.
04:28
Third, personified risks
04:30
are perceived to be greater than anonymous risks --
04:33
so Bin Laden is scarier because he has a name.
04:36
And the fourth
04:39
is people underestimate risks
04:41
in situations they do control
04:43
and overestimate them in situations they don't control.
04:45
So once you take up skydiving or smoking,
04:49
you downplay the risks.
04:52
If a risk is thrust upon you -- terrorism was a good example --
04:54
you'll overplay it because you don't feel like it's in your control.
04:57
There are a bunch of other of these biases, these cognitive biases,
05:02
that affect our risk decisions.
05:05
There's the availability heuristic,
05:08
which basically means
05:10
we estimate the probability of something
05:12
by how easy it is to bring instances of it to mind.
05:15
So you can imagine how that works.
05:19
If you hear a lot about tiger attacks, there must be a lot of tigers around.
05:21
You don't hear about lion attacks, there aren't a lot of lions around.
05:24
This works until you invent newspapers.
05:27
Because what newspapers do
05:30
is they repeat again and again
05:32
rare risks.
05:34
I tell people, if it's in the news, don't worry about it.
05:36
Because by definition,
05:38
news is something that almost never happens.
05:40
(Laughter)
05:43
When something is so common, it's no longer news --
05:45
car crashes, domestic violence --
05:48
those are the risks you worry about.
05:50
We're also a species of storytellers.
05:53
We respond to stories more than data.
05:55
And there's some basic innumeracy going on.
05:58
I mean, the joke "One, Two, Three, Many" is kind of right.
06:00
We're really good at small numbers.
06:03
One mango, two mangoes, three mangoes,
06:06
10,000 mangoes, 100,000 mangoes --
06:08
it's still more mangoes you can eat before they rot.
06:10
So one half, one quarter, one fifth -- we're good at that.
06:13
One in a million, one in a billion --
06:16
they're both almost never.
06:18
So we have trouble with the risks
06:21
that aren't very common.
06:23
And what these cognitive biases do
06:25
is they act as filters between us and reality.
06:27
And the result
06:30
is that feeling and reality get out of whack,
06:32
they get different.
06:34
Now you either have a feeling -- you feel more secure than you are.
06:37
There's a false sense of security.
06:40
Or the other way,
06:42
and that's a false sense of insecurity.
06:44
I write a lot about "security theater,"
06:46
which are products that make people feel secure,
06:49
but don't actually do anything.
06:52
There's no real word for stuff that makes us secure,
06:54
but doesn't make us feel secure.
06:56
Maybe it's what the CIA's supposed to do for us.
06:58
So back to economics.
07:03
If economics, if the market, drives security,
07:05
and if people make trade-offs
07:09
based on the feeling of security,
07:11
then the smart thing for companies to do
07:14
for the economic incentives
07:16
are to make people feel secure.
07:18
And there are two ways to do this.
07:21
One, you can make people actually secure
07:24
and hope they notice.
07:26
Or two, you can make people just feel secure
07:28
and hope they don't notice.
07:31
So what makes people notice?
07:35
Well a couple of things:
07:38
understanding of the security,
07:40
of the risks, the threats,
07:42
the countermeasures, how they work.
07:44
But if you know stuff,
07:47
you're more likely to have your feelings match reality.
07:49
Enough real world examples helps.
07:52
Now we all know the crime rate in our neighborhood,
07:55
because we live there, and we get a feeling about it
07:58
that basically matches reality.
08:01
Security theater's exposed
08:04
when it's obvious that it's not working properly.
08:07
Okay, so what makes people not notice?
08:10
Well, a poor understanding.
08:14
If you don't understand the risks, you don't understand the costs,
08:16
you're likely to get the trade-off wrong,
08:19
and your feeling doesn't match reality.
08:21
Not enough examples.
08:24
There's an inherent problem
08:26
with low probability events.
08:28
If, for example,
08:30
terrorism almost never happens,
08:32
it's really hard to judge
08:34
the efficacy of counter-terrorist measures.
08:36
This is why you keep sacrificing virgins,
08:40
and why your unicorn defenses are working just great.
08:43
There aren't enough examples of failures.
08:46
Also, feelings that are clouding the issues --
08:50
the cognitive biases I talked about earlier,
08:53
fears, folk beliefs,
08:55
basically an inadequate model of reality.
08:58
So let me complicate things.
09:02
I have feeling and reality.
09:05
I want to add a third element. I want to add model.
09:07
Feeling and model in our head,
09:10
reality is the outside world.
09:12
It doesn't change; it's real.
09:14
So feeling is based on our intuition.
09:17
Model is based on reason.
09:19
That's basically the difference.
09:21
In a primitive and simple world,
09:24
there's really no reason for a model
09:26
because feeling is close to reality.
09:29
You don't need a model.
09:32
But in a modern and complex world,
09:34
you need models
09:36
to understand a lot of the risks we face.
09:38
There's no feeling about germs.
09:42
You need a model to understand them.
09:44
So this model
09:47
is an intelligent representation of reality.
09:49
It's, of course, limited by science,
09:52
by technology.
09:55
We couldn't have a germ theory of disease
09:57
before we invented the microscope to see them.
10:00
It's limited by our cognitive biases.
10:04
But it has the ability
10:07
to override our feelings.
10:09
Where do we get these models? We get them from others.
10:11
We get them from religion, from culture,
10:14
teachers, elders.
10:17
A couple years ago,
10:19
I was in South Africa on safari.
10:21
The tracker I was with grew up in Kruger National Park.
10:23
He had some very complex models of how to survive.
10:26
And it depended on if you were attacked
10:29
by a lion or a leopard or a rhino or an elephant --
10:31
and when you had to run away, and when you couldn't run away, and when you had to climb a tree --
10:33
when you could never climb a tree.
10:36
I would have died in a day,
10:38
but he was born there,
10:41
and he understood how to survive.
10:43
I was born in New York City.
10:45
I could have taken him to New York, and he would have died in a day.
10:47
(Laughter)
10:50
Because we had different models
10:52
based on our different experiences.
10:54
Models can come from the media,
10:58
from our elected officials.
11:00
Think of models of terrorism,
11:03
child kidnapping,
11:06
airline safety, car safety.
11:09
Models can come from industry.
11:11
The two I'm following are surveillance cameras,
11:14
ID cards,
11:16
quite a lot of our computer security models come from there.
11:18
A lot of models come from science.
11:21
Health models are a great example.
11:24
Think of cancer, of bird flu, swine flu, SARS.
11:26
All of our feelings of security
11:29
about those diseases
11:32
come from models
11:34
given to us, really, by science filtered through the media.
11:36
So models can change.
11:40
Models are not static.
11:43
As we become more comfortable in our environments,
11:45
our model can move closer to our feelings.
11:48
So an example might be,
11:53
if you go back 100 years ago
11:55
when electricity was first becoming common,
11:57
there were a lot of fears about it.
12:00
I mean, there were people who were afraid to push doorbells,
12:02
because there was electricity in there, and that was dangerous.
12:04
For us, we're very facile around electricity.
12:07
We change light bulbs
12:10
without even thinking about it.
12:12
Our model of security around electricity
12:14
is something we were born into.
12:18
It hasn't changed as we were growing up.
12:21
And we're good at it.
12:24
Or think of the risks
12:27
on the Internet across generations --
12:29
how your parents approach Internet security,
12:31
versus how you do,
12:33
versus how our kids will.
12:35
Models eventually fade into the background.
12:38
Intuitive is just another word for familiar.
12:42
So as your model is close to reality,
12:45
and it converges with feelings,
12:47
you often don't know it's there.
12:49
So a nice example of this
12:52
came from last year and swine flu.
12:54
When swine flu first appeared,
12:57
the initial news caused a lot of overreaction.
12:59
Now it had a name,
13:03
which made it scarier than the regular flu,
13:05
even though it was more deadly.
13:07
And people thought doctors should be able to deal with it.
13:09
So there was that feeling of lack of control.
13:13
And those two things
13:15
made the risk more than it was.
13:17
As the novelty wore off, the months went by,
13:19
there was some amount of tolerance,
13:22
people got used to it.
13:24
There was no new data, but there was less fear.
13:26
By autumn,
13:29
people thought
13:31
the doctors should have solved this already.
13:33
And there's kind of a bifurcation --
13:35
people had to choose
13:37
between fear and acceptance --
13:39
actually fear and indifference --
13:43
they kind of chose suspicion.
13:45
And when the vaccine appeared last winter,
13:48
there were a lot of people -- a surprising number --
13:51
who refused to get it --
13:54
as a nice example
13:58
of how people's feelings of security change, how their model changes,
14:00
sort of wildly
14:03
with no new information,
14:05
with no new input.
14:07
This kind of thing happens a lot.
14:09
I'm going to give one more complication.
14:12
We have feeling, model, reality.
14:15
I have a very relativistic view of security.
14:18
I think it depends on the observer.
14:20
And most security decisions
14:23
have a variety of people involved.
14:25
And stakeholders
14:29
with specific trade-offs
14:31
will try to influence the decision.
14:34
And I call that their agenda.
14:36
And you see agenda --
14:38
this is marketing, this is politics --
14:40
trying to convince you to have one model versus another,
14:43
trying to convince you to ignore a model
14:46
and trust your feelings,
14:48
marginalizing people with models you don't like.
14:51
This is not uncommon.
14:54
An example, a great example, is the risk of smoking.
14:57
In the history of the past 50 years, the smoking risk
15:01
shows how a model changes,
15:04
and it also shows how an industry fights against
15:06
a model it doesn't like.
15:09
Compare that to the secondhand smoke debate --
15:11
probably about 20 years behind.
15:14
Think about seat belts.
15:17
When I was a kid, no one wore a seat belt.
15:19
Nowadays, no kid will let you drive
15:21
if you're not wearing a seat belt.
15:23
Compare that to the airbag debate --
15:26
probably about 30 years behind.
15:28
All examples of models changing.
15:31
What we learn is that changing models is hard.
15:36
Models are hard to dislodge.
15:39
If they equal your feelings,
15:41
you don't even know you have a model.
15:43
And there's another cognitive bias
15:46
I'll call confirmation bias,
15:48
where we tend to accept data
15:50
that confirms our beliefs
15:53
and reject data that contradicts our beliefs.
15:55
So evidence against our model,
15:59
we're likely to ignore, even if it's compelling.
16:01
It has to get very compelling before we'll pay attention.
16:04
New models that extend long periods of time are hard.
16:08
Global warming is a great example.
16:10
We're terrible
16:12
at models that span 80 years.
16:14
We can do to the next harvest.
16:16
We can often do until our kids grow up.
16:18
But 80 years, we're just not good at.
16:21
So it's a very hard model to accept.
16:24
We can have both models in our head simultaneously,
16:27
right, that kind of problem
16:31
where we're holding both beliefs together,
16:34
right, the cognitive dissonance.
16:37
Eventually,
16:39
the new model will replace the old model.
16:41
Strong feelings can create a model.
16:44
September 11th created a security model
16:47
in a lot of people's heads.
16:50
Also, personal experiences with crime can do it,
16:52
personal health scare,
16:55
a health scare in the news.
16:57
You'll see these called flashbulb events
16:59
by psychiatrists.
17:01
They can create a model instantaneously,
17:03
because they're very emotive.
17:06
So in the technological world,
17:09
we don't have experience
17:11
to judge models.
17:13
And we rely on others. We rely on proxies.
17:15
I mean, this works as long as it's to correct others.
17:17
We rely on government agencies
17:21
to tell us what pharmaceuticals are safe.
17:23
I flew here yesterday.
17:28
I didn't check the airplane.
17:30
I relied on some other group
17:32
to determine whether my plane was safe to fly.
17:34
We're here, none of us fear the roof is going to collapse on us,
17:37
not because we checked,
17:40
but because we're pretty sure
17:43
the building codes here are good.
17:45
It's a model we just accept
17:48
pretty much by faith.
17:50
And that's okay.
17:52
Now, what we want
17:57
is people to get familiar enough
17:59
with better models --
18:01
have it reflected in their feelings --
18:03
to allow them to make security trade-offs.
18:05
Now when these go out of whack,
18:09
you have two options.
18:11
One, you can fix people's feelings,
18:13
directly appeal to feelings.
18:15
It's manipulation, but it can work.
18:17
The second, more honest way
18:20
is to actually fix the model.
18:22
Change happens slowly.
18:26
The smoking debate took 40 years,
18:28
and that was an easy one.
18:31
Some of this stuff is hard.
18:34
I mean really though,
18:36
information seems like our best hope.
18:38
And I lied.
18:40
Remember I said feeling, model, reality;
18:42
I said reality doesn't change. It actually does.
18:44
We live in a technological world;
18:47
reality changes all the time.
18:49
So we might have -- for the first time in our species --
18:52
feeling chases model, model chases reality, reality's moving --
18:55
they might never catch up.
18:58
We don't know.
19:02
But in the long-term,
19:04
both feeling and reality are important.
19:06
And I want to close with two quick stories to illustrate this.
19:09
1982 -- I don't know if people will remember this --
19:12
there was a short epidemic
19:14
of Tylenol poisonings in the United States.
19:17
It's a horrific story. Someone took a bottle of Tylenol,
19:19
put poison in it, closed it up, put it back on the shelf.
19:22
Someone else bought it and died.
19:25
This terrified people.
19:27
There were a couple of copycat attacks.
19:29
There wasn't any real risk, but people were scared.
19:31
And this is how
19:34
the tamper-proof drug industry was invented.
19:36
Those tamper-proof caps, that came from this.
19:38
It's complete security theater.
19:40
As a homework assignment, think of 10 ways to get around it.
19:42
I'll give you one, a syringe.
19:44
But it made people feel better.
19:47
It made their feeling of security
19:50
more match the reality.
19:52
Last story, a few years ago, a friend of mine gave birth.
19:54
I visit her in the hospital.
19:57
It turns out when a baby's born now,
19:59
they put an RFID bracelet on the baby,
20:01
put a corresponding one on the mother,
20:03
so if anyone other than the mother takes the baby out of the maternity ward,
20:05
an alarm goes off.
20:07
I said, "Well, that's kind of neat.
20:09
I wonder how rampant baby snatching is
20:11
out of hospitals."
20:13
I go home, I look it up.
20:15
It basically never happens.
20:17
But if you think about it,
20:19
if you are a hospital,
20:21
and you need to take a baby away from its mother,
20:23
out of the room to run some tests,
20:25
you better have some good security theater,
20:27
or she's going to rip your arm off.
20:29
(Laughter)
20:31
So it's important for us,
20:33
those of us who design security,
20:35
who look at security policy,
20:37
or even look at public policy
20:40
in ways that affect security.
20:42
It's not just reality; it's feeling and reality.
20:44
What's important
20:47
is that they be about the same.
20:49
It's important that, if our feelings match reality,
20:51
we make better security trade-offs.
20:53
Thank you.
20:55
(Applause)
20:57

sponsored links

Bruce Schneier - Security expert
Bruce Schneier thinks hard about security -- as a computer security guru, and as a philosopher of the larger notion of making a safer world.

Why you should listen

Bruce Schneier is an internationally renowned security technologist and author. Described by the Economist as a "security guru," he is best known as a refreshingly candid and lucid security critic and commentator. When people want to know how security really works, they turn to Schneier.

His first bestseller, Applied Cryptography, explained how the arcane science of secret codes actually works, and was described by Wired as "the book the National Security Agency wanted never to be published." His book on computer and network security, Secrets and Lies, was called by Fortune "[a] jewel box of little surprises you can actually use." Beyond Fear tackles the problems of security from the small to the large: personal safety, crime, corporate security, national security. His current book, Schneier on Security, offers insight into everything from the risk of identity theft (vastly overrated) to the long-range security threat of unchecked presidential power and the surprisingly simple way to tamper-proof elections.

Schneier publishes a free monthly newsletter, Crypto-Gram, with over 150,000 readers. In its ten years of regular publication, Crypto-Gram has become one of the most widely read forums for free-wheeling discussions, pointed critiques and serious debate about security. As head curmudgeon at the table, Schneier explains, debunks and draws lessons from security stories that make the news.

The original video is available on TED.com
sponsored links

If you need translations, you can install "Google Translate" extension into your Chrome Browser.
Furthermore, you can change playback rate by installing "Video Speed Controller" extension.

Data provided by TED.

This website is owned and operated by Tokyo English Network.
The developer's blog is here.