19:00
TEDGlobal 2009

Beau Lotto: Optical illusions show how we see

Filmed:

Beau Lotto's color games puzzle your vision, but they also spotlight what you can't normally see: how your brain works. This fun, first-hand look at your own versatile sense of sight reveals how evolution tints your perception of what's really out there.

- Neuroscientist, Artist
Beau Lotto is founder of Lottolab, a hybrid art studio and science lab. With glowing, interactive sculpture -- and old-fashioned peer-reviewed research--he's illuminating the mysteries of the brain's visual system. Full bio

I want to start with a game.
00:13
And to win this game,
00:16
all you have to do is see the reality that's in front of you
00:18
as it really is. All right?
00:21
So, we have two panels here,
00:23
of colored dots.
00:25
And one of those dots is the same
00:27
in the two panels. Okay?
00:30
And you have to tell me which one.
00:33
Now, narrow it down to
00:35
the gray one, the green one and, say, the orange one.
00:38
So, by a show of hands -- we'll start with the easiest one --
00:41
Show of hands: how many people think it's the gray one?
00:44
Really? Okay.
00:48
How many people think it's the green one?
00:50
And how many people think it's the orange one?
00:55
Pretty even split.
00:59
Let's find out what the reality is.
01:02
Here is the orange one.
01:05
(Laughter)
01:08
Here is the green one.
01:10
And here is the gray one.
01:13
(Laughter)
01:17
So, for all of you who saw that, you're a complete realist. All right?
01:20
(Laughter)
01:24
So, this is pretty amazing, actually, isn't it?
01:26
Because nearly every living system
01:28
has evolved the ability to detect light in one way or another.
01:30
So, for us, seeing color is one of the simplest things the brain does.
01:33
And yet, even at this most fundamental level,
01:38
context is everything.
01:40
What I want to talk about is not that context is everything,
01:43
but why is context everything.
01:46
Because it's answering that question that tells us not only
01:48
why we see what we do,
01:52
but who we are as individuals,
01:54
and who we are as a society.
01:56
But first, we have to ask another question,
01:59
which is, "What is color for?"
02:01
And instead of telling you, I'll just show you.
02:03
What you see here is a jungle scene,
02:05
and you see the surfaces according to the amount
02:08
of light that those surfaces reflect.
02:10
Now, can any of you see the predator that's about to jump out at you?
02:12
And if you haven't seen it yet, you're dead. Right?
02:17
(Laughter)
02:19
Can anyone see it? Anyone? No?
02:21
Now, let's see the surfaces according to the quality of light that they reflect.
02:23
And now you see it.
02:27
So, color enables us to see
02:30
the similarities and differences between surfaces,
02:33
according to the full spectrum of light that they reflect.
02:35
But what you've just done is, in many respects, mathematically impossible.
02:38
Why? Because, as Berkeley tells us,
02:42
we have no direct access to our physical world,
02:45
other than through our senses.
02:48
And the light that falls onto our eyes
02:50
is determined by multiple things in the world --
02:52
not only the color of objects,
02:54
but also the color of their illumination,
02:56
and the color of the space between us and those objects.
02:58
You vary any one of those parameters,
03:01
and you'll change the color of the light that falls onto your eye.
03:03
This is a huge problem because it means that
03:08
the same image could have an infinite number
03:10
of possible real-world sources.
03:13
So let me show you what I mean. Imagine that this is the back of your eye.
03:16
And these are two projections from the world.
03:19
They are identical in every single way.
03:22
Identical in shape, size, spectral content.
03:25
They are the same, as far as your eye is concerned.
03:29
And yet they come from completely different sources.
03:33
The one on the right
03:38
comes from a yellow surface,
03:40
in shadow, oriented facing the left,
03:43
viewed through a pinkish medium.
03:45
The one on the left comes from an orange surface,
03:48
under direct light, facing to the right,
03:51
viewed through a sort of a bluish medium.
03:53
Completely different meanings,
03:55
giving rise to the exact same retinal information.
03:58
And yet it's only the retinal information
04:01
that we get.
04:03
So how on Earth do we even see?
04:05
So, if you remember anything in this next 18 minutes,
04:08
remember this: that the light that falls on to your eye,
04:12
sensory information, is meaningless,
04:15
because it could mean literally anything.
04:17
And what's true for sensory information is true for information generally.
04:20
There is no inherent meaning in information.
04:23
It's what we do with that information that matters.
04:25
So, how do we see? Well, we see by learning to see.
04:29
So, the brain evolved the mechanisms for finding patterns,
04:32
finding relationships in information
04:36
and associating those relationships
04:38
with a behavioral meaning,
04:40
a significance, by interacting with the world.
04:42
We're very aware of this
04:45
in the form of more cognitive attributes, like language.
04:47
So, I'm going to give you some letter strings. And I want you to read them out for me,
04:50
if you can.
04:52
Audience: "Can you read this?"
04:54
"You are not reading this."
04:57
"What are you reading?"
04:59
Beau Lotto: "What are you reading?" Half the letters are missing. Right?
05:01
There is no a priori reason why an "H" has to go
05:04
between that "W" and "A."
05:06
But you put one there. Why?
05:08
Because in the statistics of your past experience
05:10
it would have been useful to do so. So you do so again.
05:12
And yet you don't put a letter after that first "T."
05:15
Why? Because it wouldn't have been useful in the past.
05:18
So you don't do it again.
05:21
So let me show you how quickly our brains can redefine normality,
05:23
even at the simplest thing the brain does, which is color.
05:27
So, if I could have the lights down up here.
05:29
I want you to first notice that those two desert scenes are physically the same.
05:32
One is simply the flipping of the other. Okay?
05:35
Now I want you to look at that dot
05:40
between the green and the red. Okay?
05:42
And I want you to stare at that dot. Don't look anywhere else.
05:45
And we're going to look at that for about 30 seconds,
05:48
which is a bit of a killer in an 18-minute talk.
05:49
(Laughter)
05:52
But I really want you to learn.
05:53
And I'll tell you -- don't look anywhere else --
05:55
and I'll tell you what's happening inside your head.
05:58
Your brain is learning. And it's learning that the right side of its visual field
06:00
is under red illumination;
06:03
the left side of its visual field is under green illumination.
06:05
That's what it's learning. Okay?
06:08
Now, when I tell you, I want you to look at the dot between the two desert scenes.
06:11
So why don't you do that now?
06:16
(Laughter)
06:18
Can I have the lights up again?
06:21
I take it from your response they don't look the same anymore. Right?
06:23
(Applause)
06:27
Why? Because your brain is seeing that same information
06:28
as if the right one is still under red light,
06:31
and the left one is still under green light.
06:33
That's your new normal.
06:35
So, what does this mean for context?
06:37
It means that I can take these two identical squares,
06:39
and I can put them in light and dark surrounds.
06:41
And now the one on the dark surround looks lighter than the one on the light surround.
06:43
What's significant is not simply the light and dark surrounds that matter.
06:46
It's what those light and dark surrounds meant for your behavior in the past.
06:50
So I'll show you what I mean. Here we have
06:54
that exact same illusion.
06:56
We have two identical tiles, on the left,
06:58
one in a dark surround, one in a light surround.
07:00
And the same thing over on the right.
07:02
Now, what I'm going to do is I'm going to review those two scenes.
07:04
But I'm not going to change anything within those boxes,
07:07
except their meaning.
07:09
And see what happens to your perception.
07:11
Notice that on the left
07:13
the two tiles look nearly completely opposite:
07:15
one very white and one very dark.
07:18
All right? Whereas on the right,
07:20
the two tiles look nearly the same.
07:22
And yet there is still one on a dark surround and one on a light surround.
07:24
Why? Because if the tile in that shadow
07:28
were in fact in shadow,
07:31
and reflecting the same amount of light to your eye
07:33
as the one outside the shadow,
07:35
it would have to be more reflective -- just the laws of physics.
07:37
So you see it that way.
07:40
Whereas on the right, the information is consistent
07:42
with those two tiles being under the same light.
07:45
If they are under the same light, reflecting the same amount of light
07:47
to your eye,
07:49
then they must be equally reflective.
07:51
So you see it that way.
07:53
Which means we can bring all this information together
07:55
to create some incredibly strong illusions.
07:57
This is one I made a few years ago.
07:59
And you'll notice you see a dark brown tile at the top,
08:01
and a bright orange tile at the side.
08:04
That is your perceptual reality. The physical reality
08:07
is that those two tiles are the same.
08:09
Here you see four gray tiles on your left,
08:14
seven gray tiles on the right.
08:17
I'm not going to change those tiles at all,
08:19
but I'm going to reveal the rest of the scene
08:21
and see what happens to your perception.
08:23
The four blue tiles on the left are gray.
08:26
The seven yellow tiles on the right are also gray.
08:30
They are the same. Okay?
08:33
Don't believe me? Let's watch it again.
08:35
What's true for color is also true for complex perceptions of motion.
08:39
So here we have --
08:43
let's turn this around -- a diamond.
08:46
And what I'm going to do is, I'm going to hold it here,
08:51
and I'm going to spin it.
08:53
And for all of you, you'll see it probably spinning this direction.
08:57
Now I want you to keep looking at it.
09:00
Move your eyes around, blink, maybe close one eye.
09:03
And suddenly it will flip, and start spinning the opposite direction.
09:05
Yes? Raise your hand if you got that. Yes?
09:09
Keep blinking. Every time you blink it will switch. Alright?
09:12
So I can ask you, which direction is it rotating?
09:16
How do you know?
09:20
Your brain doesn't know. Because both are equally likely.
09:22
So depending on where it looks, it flips
09:25
between the two possibilities.
09:27
Are we the only ones that see illusions?
09:30
The answer to this question is no.
09:32
Even the beautiful bumblebee,
09:34
with its mere one million brain cells,
09:36
which is 250 times fewer cells than you have in one retina,
09:38
sees illusions, does the most complicated things
09:41
that even our most sophisticated computers can't do.
09:44
So in my lab, we of course work on bumblebees.
09:47
Because we can completely control their experience,
09:49
and see how that alters the architecture of their brain.
09:51
And we do this in what we call the Bee Matrix.
09:53
And here you have the hive. You can see the queen bee,
09:56
that large bee in the middle there. Those are all her daughters, the eggs.
09:58
And they go back and forth between this hive
10:01
and the arena, via this tube.
10:04
And you'll see one of the bees come out here.
10:09
You see how she has a little number on her?
10:11
Yeah there is another one coming out. She has another number on her.
10:14
Now, they are not born that way. Right?
10:17
We pull them out, put them in the fridge, and they fall asleep.
10:20
And then you can superglue little numbers on them.
10:22
(Laughter)
10:24
And now, in this experiment they get rewarded if they go to the blue flowers.
10:26
And they land on the flower. They stick their tongue in there,
10:30
called a proboscis, and they drink sugar water.
10:33
Now she is drinking a glass of water that's about that big to you and I,
10:35
will do that about three times, and then fly.
10:38
And sometimes they learn not to go to the blue,
10:44
but to go to where the other bees go.
10:46
So they copy each other. They can count to five. They can recognize faces.
10:48
And here she comes down the ladder.
10:51
And she'll come into the hive, find an empty honey pot
10:54
and throw up, and that's honey.
10:56
(Laughter)
10:58
Now remember -- (Laughter)
10:59
-- she's supposed to be going to the blue flowers.
11:02
But what are these bees doing in the upper right corner?
11:04
It looks like they're going to green flowers.
11:07
Now, are they getting it wrong?
11:09
And the answer to the question is no. Those are actually blue flowers.
11:12
But those are blue flowers under green light.
11:15
So they are using the relationships between the colors to solve the puzzle,
11:19
which is exactly what we do.
11:23
So, illusions are often used,
11:25
especially in art, in the words of a more contemporary artist,
11:27
"to demonstrate the fragility of our senses."
11:31
Okay, this is complete rubbish.
11:33
The senses aren't fragile. And if they were, we wouldn't be here.
11:36
Instead, color tells us something completely different,
11:39
that the brain didn't actually evolve to see the world the way it is.
11:43
We can't. Instead, the brain evolved to see the world
11:46
the way it was useful to see in the past.
11:50
And how we see is by continually redefining normality.
11:53
So how can we take this
11:59
incredible capacity of plasticity of the brain
12:03
and get people to experience their world differently?
12:06
Well, one of the ways we do in my lab and studio
12:09
is we translate the light into sound
12:12
and we enable people to hear their visual world.
12:15
And they can navigate the world using their ears.
12:19
Here is David, in the right. And he is holding a camera.
12:22
On the left is what his camera sees.
12:25
And you'll see there is a line, a faint line going across that image.
12:27
That line is broken up into 32 squares.
12:30
In each square we calculate the average color.
12:33
And then we just simply translate that into sound.
12:35
And now he's going to
12:37
turn around, close his eyes,
12:40
and find a plate on the ground with his eyes closed.
12:44
He finds it. Amazing. Right?
13:06
So not only can we create a prosthetic for the visually impaired,
13:08
but we can also investigate how people
13:10
literally make sense of the world.
13:13
But we can also do something else. We can also make music with color.
13:16
So, working with kids,
13:20
they created images,
13:22
thinking about what might the images you see
13:24
sound like if we could listen to them.
13:26
And then we translated these images.
13:28
And this is one of those images.
13:30
And this is a six-year-old child composing a piece of music
13:32
for a 32-piece orchestra.
13:35
And this is what it sounds like.
13:38
So, a six-year-old child. Okay?
14:06
Now, what does all this mean?
14:09
What this suggests is that no one is an outside observer
14:12
of nature. Okay?
14:15
We are not defined by our central properties,
14:17
by the bits that make us up.
14:19
We're defined by our environment and our interaction with that environment --
14:21
by our ecology.
14:24
And that ecology is necessarily relative,
14:26
historical and empirical.
14:30
So what I'd like to finish with is this over here.
14:32
Because what I've been trying to do is really celebrate uncertainty.
14:38
Because I think only through uncertainty is there potential for understanding.
14:41
So, if some of you are still feeling a bit too certain,
14:45
I'd like to do this one.
14:48
So, if we have the lights down.
14:50
And what we have here --
14:52
Can everyone see 25 purple surfaces
14:58
on your left,
15:01
and 25, call it yellowish, surfaces on your right?
15:03
So, now, what I want to do:
15:07
I'm going to put the middle nine surfaces here
15:09
under yellow illumination
15:11
by simply putting a filter behind them.
15:13
All right. Now you can see that changes the light
15:17
that's coming through there. Right?
15:20
Because now the light is going through a yellowish filter
15:22
and then a purplish filter.
15:24
I'm going to do this opposite on the left here.
15:26
I'm going to put the middle nine under a purplish light.
15:31
Now, some of you will notice that the consequence is that
15:38
the light coming through those middle nine on the right,
15:42
or your left,
15:45
is exactly the same as the light coming through
15:47
the middle nine on your right.
15:49
Agreed? Yes?
15:51
Okay. So they are physically the same.
15:54
Let's pull the covers off.
15:56
Now remember,
16:02
you know the middle nine are exactly the same.
16:06
Do they look the same? No.
16:09
The question is, "Is that an illusion?"
16:13
And I'll leave you with that.
16:15
So, thank you very much.
16:17
(Applause)
16:19

▲Back to top

About the Speaker:

Beau Lotto - Neuroscientist, Artist
Beau Lotto is founder of Lottolab, a hybrid art studio and science lab. With glowing, interactive sculpture -- and old-fashioned peer-reviewed research--he's illuminating the mysteries of the brain's visual system.

Why you should listen

"Let there be perception," was evolution's proclamation, and so it was that all creatures, from honeybees to humans, came to see the world not as it is, but as was most useful. This uncomfortable place--where what an organism's brain sees diverges from what is actually out there--is what Beau Lotto and his team at Lottolab are exploring through their dazzling art-sci experiments and public illusions. Their Bee Matrix installation, for example, places a live bee in a transparent enclosure where gallerygoers may watch it seek nectar in a virtual meadow of luminous Plexiglas flowers. (Bees, Lotto will tell you, see colors much like we humans do.) The data captured isn't just discarded, either: it's put to good use in probing scientific papers, and sometimes in more exhibits.

At their home in London’s Science Museum, the lab holds "synesthetic workshops" where kids and adults make abstract paintings that computers interpret into music, and they host regular Lates--evenings of science, music and "mass experiments." Lotto is passionate about involving people from all walks of life in research on perception--both as subjects and as fellow researchers. One such program, called "i,scientist," in fact led to the publication of the first ever peer-reviewed scientific paper written by schoolchildren ("Blackawton Bees," December 2010). It starts, "Once upon a time ..."

These and Lotto's other conjurings are slowly, charmingly bending the science of perception--and our perceptions of what science can be.

More profile about the speaker
Beau Lotto | Speaker | TED.com