English-Video.net comment policy

The comment field is common to all languages

Let's write in your language and use "Google Translate" together

Please refer to informative community guidelines on TED.com

TEDIndia 2009

Pawan Sinha: How brains learn to see

Filmed
Views 844,166

Pawan Sinha details his groundbreaking research into how the brain's visual system develops. Sinha and his team provide free vision-restoring treatment to children born blind, and then study how their brains learn to interpret visual data. The work offers insights into neuroscience, engineering and even autism.

- Visual neuroscientist
Pawan Sinha researches how our brains interpret what our eyes see -- and uses that research to give blind children the gift of sight. Full bio

If you are a blind child in India,
00:15
you will very likely have to contend with
00:19
at least two big pieces of bad news.
00:22
The first bad news
00:25
is that the chances of getting treatment
00:27
are extremely slim to none,
00:30
and that's because most of the blindness
00:33
alleviation programs in the country
00:35
are focused on adults,
00:37
and there are very, very few hospitals
00:39
that are actually equipped to treat children.
00:42
In fact, if you were to be treated,
00:46
you might well end up being treated
00:51
by a person who has no medical credentials
00:54
as this case from Rajasthan illustrates.
00:57
This is a three-year-old orphan girl
01:00
who had cataracts.
01:02
So, her caretakers took her
01:04
to the village medicine man,
01:06
and instead of suggesting to the caretakers
01:08
that the girl be taken to a hospital,
01:11
the person decided to burn her abdomen
01:14
with red-hot iron bars
01:16
to drive out the demons.
01:18
The second piece of bad news
01:20
will be delivered to you
01:23
by neuroscientists, who will tell you
01:25
that if you are older than four or five years of age,
01:28
that even if you have your eye corrected,
01:31
the chances of your brain learning how to see
01:34
are very, very slim --
01:37
again, slim or none.
01:39
So when I heard these two things,
01:42
it troubled me deeply,
01:44
both because of personal reasons
01:46
and scientific reasons.
01:48
So let me first start with the personal reason.
01:50
It'll sound corny, but it's sincere.
01:53
That's my son, Darius.
01:56
As a new father,
01:58
I have a qualitatively different sense
02:00
of just how delicate babies are,
02:04
what our obligations are towards them
02:07
and how much love
02:10
we can feel towards a child.
02:12
I would move heaven and earth
02:15
in order to get treatment for Darius,
02:17
and for me to be told
02:20
that there might be other Dariuses
02:22
who are not getting treatment,
02:24
that's just viscerally wrong.
02:26
So that's the personal reason.
02:29
Scientific reason is that this notion
02:31
from neuroscience of critical periods --
02:34
that if the brain is older
02:36
than four or five years of age,
02:39
it loses its ability to learn --
02:41
that doesn't sit well with me,
02:43
because I don't think that idea
02:45
has been tested adequately.
02:47
The birth of the idea is from
02:50
David Hubel and Torsten Wiesel's work,
02:52
two researchers who were at Harvard,
02:54
and they got the Nobel Prize in 1981
02:56
for their studies of visual physiology,
02:59
which are remarkably beautiful studies,
03:01
but I believe some of their work
03:03
has been extrapolated
03:05
into the human domain prematurely.
03:07
So, they did their work with kittens,
03:09
with different kinds of deprivation regiments,
03:11
and those studies,
03:13
which date back to the '60s,
03:15
are now being applied to human children.
03:17
So I felt that I needed to do two things.
03:20
One: provide care
03:23
to children who are currently
03:26
being deprived of treatment.
03:28
That's the humanitarian mission.
03:30
And the scientific mission would be
03:32
to test the limits
03:34
of visual plasticity.
03:36
And these two missions, as you can tell,
03:38
thread together perfectly. One adds to the other;
03:41
in fact, one would be impossible without the other.
03:44
So, to implement
03:49
these twin missions,
03:51
a few years ago, I launched Project Prakash.
03:53
Prakash, as many of you know,
03:56
is the Sanskrit word for light,
03:58
and the idea is that
04:00
in bringing light into the lives of children,
04:02
we also have a chance
04:05
of shedding light on some of the
04:07
deepest mysteries of neuroscience.
04:09
And the logo -- even though it looks extremely Irish,
04:12
it's actually derived from
04:15
the Indian symbol of Diya, an earthen lamp.
04:17
The Prakash, the overall effort
04:21
has three components:
04:24
outreach, to identify children in need of care;
04:26
medical treatment; and in subsequent study.
04:30
And I want to show you a short video clip
04:33
that illustrates the first two components of this work.
04:36
This is an outreach station
04:41
conducted at a school for the blind.
04:43
(Text: Most of the children are profoundly and permanently blind ...)
04:46
Pawan Sinha: So, because this is a school for the blind,
04:51
many children have permanent conditions.
04:56
That's a case of microphthalmos,
04:58
which is malformed eyes,
05:01
and that's a permanent condition;
05:03
it cannot be treated.
05:05
That's an extreme of micropthalmos
05:07
called enophthalmos.
05:09
But, every so often, we come across children
05:11
who show some residual vision,
05:13
and that is a very good sign
05:16
that the condition might actually be treatable.
05:19
So, after that screening, we bring the children to the hospital.
05:21
That's the hospital we're working with in Delhi,
05:24
the Schroff Charity Eye Hospital.
05:26
It has a very well-equipped
05:29
pediatric ophthalmic center,
05:31
which was made possible in part
05:35
by a gift from the Ronald McDonald charity.
05:37
So, eating burgers actually helps.
05:41
(Text: Such examinations allow us to improve
05:45
eye-health in many children, and ...
05:47
... help us find children who can participate in Project Prakash.)
05:54
PS: So, as I zoom in to the eyes of this child,
05:57
you will see the cause of his blindness.
05:59
The whites that you see in the middle of his pupils
06:03
are congenital cataracts,
06:06
so opacities of the lens.
06:09
In our eyes, the lens is clear,
06:11
but in this child, the lens has become opaque,
06:14
and therefore he can't see the world.
06:16
So, the child is given treatment. You'll see shots of the eye.
06:19
Here's the eye with the opaque lens,
06:22
the opaque lens extracted
06:24
and an acrylic lens inserted.
06:26
And here's the same child
06:29
three weeks post-operation,
06:31
with the right eye open.
06:34
(Applause)
06:40
Thank you.
06:46
So, even from that little clip, you can begin to get the sense
06:48
that recovery is possible,
06:51
and we have now
06:53
provided treatment to over 200 children,
06:55
and the story repeats itself.
06:58
After treatment, the child
07:00
gains significant functionality.
07:02
In fact, the story holds true
07:05
even if you have a person who got sight
07:08
after several years of deprivation.
07:10
We did a paper a few years ago
07:12
about this woman that you see on the right, SRD,
07:14
and she got her sight late in life,
07:18
and her vision is remarkable at this age.
07:20
I should add a tragic postscript to this --
07:24
she died two years ago
07:27
in a bus accident.
07:29
So, hers is just a truly inspiring story --
07:31
unknown, but inspiring story.
07:35
So when we started finding these results,
07:38
as you might imagine, it created quite a bit of stir
07:40
in the scientific and the popular press.
07:43
Here's an article in Nature
07:46
that profiled this work,
07:48
and another one in Time.
07:50
So, we were fairly convinced -- we are convinced --
07:52
that recovery is feasible,
07:54
despite extended visual deprivation.
07:56
The next obvious question to ask:
07:59
What is the process of recovery?
08:01
So, the way we study that is,
08:04
let's say we find a child who has light sensitivity.
08:07
The child is provided treatment,
08:09
and I want to stress that the treatment
08:11
is completely unconditional;
08:13
there is no quid pro quo.
08:15
We treat many more children then we actually work with.
08:17
Every child who needs treatment is treated.
08:20
After treatment, about every week,
08:23
we run the child
08:25
on a battery of simple visual tests
08:27
in order to see how their visual skills
08:30
are coming on line.
08:32
And we try to do this for as long as possible.
08:34
This arc of development
08:37
gives us unprecedented
08:39
and extremely valuable information
08:41
about how the scaffolding of vision
08:43
gets set up.
08:45
What might be the causal connections
08:47
between the early developing skills
08:49
and the later developing ones?
08:51
And we've used this general approach to study
08:53
many different visual proficiencies,
08:55
but I want to highlight one particular one,
08:58
and that is image parsing into objects.
09:02
So, any image of the kind that you see on the left,
09:05
be it a real image or a synthetic image,
09:07
it's made up of little regions
09:10
that you see in the middle column,
09:12
regions of different colors, different luminances.
09:14
The brain has this complex task
09:17
of putting together, integrating,
09:20
subsets of these regions
09:23
into something that's more meaningful,
09:25
into what we would consider to be objects,
09:27
as you see on the right.
09:29
And nobody knows how this integration happens,
09:31
and that's the question we asked with Project Prakash.
09:33
So, here's what happens
09:37
very soon after the onset of sight.
09:39
Here's a person who had gained sight just a couple of weeks ago,
09:42
and you see Ethan Myers, a graduate student from MIT,
09:45
running the experiment with him.
09:48
His visual-motor coordination is quite poor,
09:51
but you get a general sense
09:55
of what are the regions that he's trying to trace out.
09:57
If you show him real world images,
10:00
if you show others like him real world images,
10:02
they are unable to recognize most of the objects
10:05
because the world to them is over-fragmented;
10:07
it's made up of a collage, a patchwork,
10:10
of regions of different colors and luminances.
10:13
And that's what's indicated in the green outlines.
10:15
When you ask them,
10:17
"Even if you can't name the objects, just point to where the objects are,"
10:19
these are the regions that they point to.
10:22
So the world is this complex
10:24
patchwork of regions.
10:26
Even the shadow on the ball
10:28
becomes its own object.
10:30
Interestingly enough,
10:33
you give them a few months,
10:35
and this is what happens.
10:37
Doctor: How many are these?
10:43
Patient: These are two things.
10:45
Doctor: What are their shapes?
10:47
Patient: Their shapes ...
10:49
This one is a circle,
10:51
and this
10:54
is a square.
10:56
PS: A very dramatic transformation has come about.
10:58
And the question is:
11:01
What underlies this transformation?
11:03
It's a profound question,
11:05
and what's even more amazing is how simple
11:07
the answer is.
11:09
The answer lies in motion
11:11
and that's what I want to show you in the next clip.
11:13
Doctor: What shape do you see here?
11:18
Patient: I can't make it out.
11:20
Doctor: Now?
11:28
Patient: Triangle.
11:31
Doctor: How many things are these?
11:35
Now, how many things are these?
11:48
Patient: Two.
11:51
Doctor: What are these things?
11:53
Patient: A square and a circle.
11:56
PS: And we see this pattern over and over again.
11:58
The one thing the visual system needs
12:01
in order to begin parsing the world
12:04
is dynamic information.
12:06
So the inference we are deriving from this,
12:08
and several such experiments,
12:10
is that dynamic information processing,
12:12
or motion processing,
12:14
serves as the bedrock for building
12:16
the rest of the complexity of visual processing;
12:18
it leads to visual integration
12:22
and eventually to recognition.
12:24
This simple idea has far reaching implications.
12:27
And let me just quickly mention two,
12:30
one, drawing from the domain of engineering,
12:33
and one from the clinic.
12:35
So, from the perspective of engineering,
12:37
we can ask: Goven that we know
12:39
that motion is so important for the human visual system,
12:42
can we use this as a recipe
12:44
for constructing machine-based vision systems
12:47
that can learn on their own, that don't need to be programmed
12:50
by a human programmer?
12:53
And that's what we're trying to do.
12:55
I'm at MIT, at MIT you need to apply
12:57
whatever basic knowledge you gain.
13:00
So we are creating Dylan,
13:02
which is a computational system
13:04
with an ambitious goal
13:06
of taking in visual inputs
13:08
of the same kind that a human child would receive,
13:10
and autonomously discovering:
13:13
What are the objects in this visual input?
13:15
So, don't worry about the internals of Dylan.
13:18
Here, I'm just going to talk about
13:21
how we test Dylan.
13:24
The way we test Dylan is by giving it
13:26
inputs, as I said, of the same kind
13:28
that a baby, or a child in Project Prakash would get.
13:31
But for a long time we couldn't quite figure out:
13:34
Wow can we get these kinds of video inputs?
13:37
So, I thought,
13:41
could we have Darius
13:43
serve as our babycam carrier,
13:45
and that way get the inputs that we feed into Dylan?
13:48
So that's what we did.
13:51
(Laughter)
13:53
I had to have long conversations with my wife.
14:00
(Laughter)
14:03
In fact, Pam, if you're watching this,
14:08
please forgive me.
14:10
So, we modified the optics of the camera
14:13
in order to mimic the baby's visual acuity.
14:17
As some of you might know,
14:20
babyies are born pretty much legally blind.
14:22
Their acuity -- our acuity is 20/20;
14:26
babies' acuity is like 20/800,
14:29
so they are looking at the world
14:32
in a very, very blurry fashion.
14:34
Here's what a baby-cam video looks like.
14:37
(Laughter)
14:41
(Applause)
14:50
Thankfully, there isn't any audio
14:53
to go with this.
14:55
What's amazing is that working with such
14:58
highly degraded input,
15:00
the baby, very quickly, is able
15:02
to discover meaning in such input.
15:04
But then two or three days afterward,
15:07
babies begin to pay attention
15:09
to their mother's or their father's face.
15:11
How does that happen? We want Dylan to be able to do that,
15:13
and using this mantra of motion,
15:16
Dylan actually can do that.
15:19
So, given that kind of video input,
15:21
with just about six or seven minutes worth of video,
15:24
Dylan can begin to extract patterns
15:27
that include faces.
15:30
So, it's an important demonstration
15:33
of the power of motion.
15:35
The clinical implication, it comes from the domain of autism.
15:37
Visual integration has been associated with autism
15:40
by several researchers.
15:42
When we saw that, we asked:
15:44
Could the impairment in visual integration
15:46
be the manifestation of something underneath,
15:49
of dynamic information processing deficiencies in autism?
15:52
Because, if that hypothesis were to be true,
15:55
it would have massive repercussions in our understanding
15:58
of what's causing the many different aspects
16:01
of the autism phenotype.
16:03
What you're going to see are
16:06
video clips of two children -- one neurotypical,
16:08
one with autism, playing Pong.
16:11
So, while the child is playing Pong, we are tracking where they're looking.
16:13
In red are the eye movement traces.
16:16
This is the neurotypical child, and what you see
16:19
is that the child is able to make cues
16:22
of the dynamic information
16:24
to predict where the ball is going to go.
16:26
Even before the ball gets to a place,
16:28
the child is already looking there.
16:31
Contrast this with a child
16:34
with autism playing the same game.
16:36
Instead of anticipating,
16:38
the child always follows where the ball has been.
16:40
The efficiency of the use
16:43
of dynamic information
16:45
seems to be significantly compromised in autism.
16:47
So we are pursuing this line of work
16:51
and hopefully we'll have
16:54
more results to report soon.
16:56
Looking ahead, if you think of this disk
16:58
as representing all of the children
17:01
we've treated so far,
17:03
this is the magnitude of the problem.
17:05
The red dots are the children we have not treated.
17:07
So, there are many, many more children who need to be treated,
17:10
and in order to expand the scope of the project,
17:12
we are planning on launching
17:15
The Prakash Center for Children,
17:17
which will have a dedicated pediatric hospital,
17:19
a school for the children we are treating
17:22
and also a cutting-edge research facility.
17:24
The Prakash Center will integrate health care,
17:26
education and research in a way
17:29
that truly creates the whole
17:31
to be greater than the sum of the parts.
17:33
So, to summarize: Prakash, in its five years of existence,
17:36
it's had an impact in multiple areas,
17:39
ranging from basic neuroscience
17:42
plasticity and learning in the brain,
17:44
to clinically relevant hypotheses like in autism,
17:46
the development of autonomous machine vision systems,
17:50
education of the undergraduate and graduate students,
17:53
and most importantly in the alleviation
17:56
of childhood blindness.
17:58
And for my students and I, it's been
18:00
just a phenomenal experience
18:02
because we have gotten to do interesting research,
18:04
while at the same time
18:08
helping the many children that we have worked with.
18:10
Thank you very much.
18:12
(Applause)
18:14

▲Back to top

About the speaker:

Pawan Sinha - Visual neuroscientist
Pawan Sinha researches how our brains interpret what our eyes see -- and uses that research to give blind children the gift of sight.

Why you should listen

At Pawan Sinha's MIT lab, he and his team spend their days trying to understand how the brain learns to recognize and use the patterns and scenes we see around us. To do this, they often use computers to model the processes of the human brain, but they also study human subjects, some of whom are seeing the world for the very first time and can tell them about the experience as it happens. They find these unusual subjects through the humanitarian branch of their research, Project Prakash.

Project Prakash sets up eye-care camps in some of the most habitually underserved regions of India, and gives free eye-health screenings to, since 2003, more than 700 functionally blind children. The children are then treated without charge, even if they do not fit the profile that would make them eligible for Sinha's research.

Sinha's eventual goal is to help 500 children each year; plans are under way for a center for visual rehabilitation in new Delhi. The special relationship that Sinha has created between research and humanitarianism promises to deliver on both fronts.

More profile about the speaker
Pawan Sinha | Speaker | TED.com