sponsored links
TED2013

Mary Lou Jepsen: Could future devices read images from our brains?

March 12, 2013

As an expert on cutting-edge digital displays, Mary Lou Jepsen studies how to show our most creative ideas on screens. And as a brain surgery patient herself, she is driven to know more about the neural activity that underlies invention, creativity, thought. She meshes these two passions in a rather mind-blowing talk on two cutting-edge brain studies that might point to a new frontier in understanding how (and what) we think.

Mary Lou Jepsen - Screen engineer
Mary Lou Jepsen discovers astonishing ways to integrate digital screens into daily life. Full bio

sponsored links
Double-click the English subtitles below to play the video.
I had brain surgery 18 years ago,
00:12
and since that time, brain science has become
00:14
a personal passion of mine.
00:17
I'm actually an engineer.
00:19
And first let me say, I recently joined
00:21
Google's Moonshot group,
00:24
where I had a division,
00:25
the display division in Google X,
00:27
and the brain science work I'm speaking about today
00:29
is work I did before I joined Google
00:31
and on the side outside of Google.
00:34
So that said, there's a stigma
00:37
when you have brain surgery.
00:40
Are you still smart or not?
00:42
And if not, can you make yourself smart again?
00:45
After my neurosurgery,
00:49
part of my brain was missing,
00:50
and I had to deal with that.
00:52
It wasn't the grey matter, but it
was the gooey part dead center
00:55
that makes key hormones and neurotransmitters.
00:58
Immediately after my surgery,
01:02
I had to decide what amounts of each of over
01:04
a dozen powerful chemicals to take each day,
01:06
because if I just took nothing,
01:10
I would die within hours.
01:11
Every day now for 18 years -- every single day --
01:14
I've had to try to decide the combinations
01:18
and mixtures of chemicals,
01:21
and try to get them, to stay alive.
01:22
There have been several close calls.
01:26
But luckily, I'm an experimentalist at heart,
01:29
so I decided I would experiment
01:32
to try to find more optimal dosages
01:36
because there really isn't a clear road map
01:38
on this that's detailed.
01:40
I began to try different mixtures,
01:42
and I was blown away by how
01:44
tiny changes in dosages
01:47
dramatically changed my sense of self,
01:49
my sense of who I was, my thinking,
01:52
my behavior towards people.
01:54
One particularly dramatic case:
01:56
for a couple months I actually tried dosages
01:58
and chemicals typical of a man in his early 20s,
02:00
and I was blown away by how my thoughts changed.
02:04
(Laughter)
02:07
I was angry all the time,
02:10
I thought about sex constantly,
02:13
and I thought I was the smartest person
02:15
in the entire world, and
02:18
—(Laughter)—
02:20
of course over the years I'd
met guys kind of like that,
02:22
or maybe kind of toned-down versions of that.
02:25
I was kind of extreme.
02:28
But to me, the surprise was,
02:30
I wasn't trying to be arrogant.
02:32
I was actually trying,
02:35
with a little bit of insecurity,
02:38
to actually fix a problem in front of me,
02:40
and it just didn't come out that way.
02:43
So I couldn't handle it.
02:45
I changed my dosages.
02:46
But that experience, I think, gave me
02:48
a new appreciation for men
02:50
and what they might walk through,
02:52
and I've gotten along with men
02:54
a lot better since then.
02:56
What I was trying to do
02:58
with tuning these hormones
02:59
and neurotransmitters and so forth
03:01
was to try to get my intelligence back
03:03
after my illness and surgery,
03:07
my creative thought, my idea flow.
03:10
And I think mostly in images,
03:12
and so for me that became a key metric --
03:15
how to get these mental images
03:18
that I use as a way of rapid prototyping,
03:20
if you will, my ideas,
03:23
trying on different new ideas for size,
03:24
playing out scenarios.
03:27
This kind of thinking isn't new.
03:28
Philiosophers like Hume and Descartes and Hobbes
03:30
saw things similarly.
03:34
They thought that mental images and ideas
03:35
were actually the same thing.
03:38
There are those today that dispute that,
03:40
and lots of debates about how the mind works,
03:43
but for me it's simple:
03:46
Mental images, for most of us,
03:48
are central in inventive and creative thinking.
03:50
So after several years,
03:54
I tuned myself up and I have lots of great,
03:56
really vivid mental images with a lot of sophistication
03:59
and the analytical backbone behind them.
04:02
And so now I'm working on,
04:04
how can I get these mental images in my mind
04:06
out to my computer screen faster?
04:10
Can you imagine, if you will,
04:13
a movie director being able to use
04:15
her imagination alone to
direct the world in front of her?
04:17
Or a musician to get the music out of his head?
04:21
There are incredible possibilities with this
04:25
as a way for creative people
04:27
to share at light speed.
04:29
And the truth is, the remaining bottleneck
04:31
in being able to do this
04:33
is just upping the resolution of brain scan systems.
04:35
So let me show you why I think
we're pretty close to getting there
04:38
by sharing with you two recent experiments
04:41
from two top neuroscience groups.
04:44
Both used fMRI technology --
04:46
functional magnetic resonance imaging technology --
04:49
to image the brain,
04:51
and here is a brain scan set from Giorgio Ganis
04:53
and his colleagues at Harvard.
04:56
And the left-hand column shows a brain scan
04:58
of a person looking at an image.
05:01
The middle column shows the brainscan
05:04
of that same individual
05:06
imagining, seeing that same image.
05:08
And the right column was created
05:11
by subtracting the middle
column from the left column,
05:13
showing the difference to be nearly zero.
05:16
This was repeated on lots of different individuals
05:19
with lots of different images,
05:22
always with a similar result.
05:25
The difference between seeing an image
05:27
and imagining seeing that same image
05:29
is next to nothing.
05:31
Next let me share with you one other experiment,
05:33
this from Jack Gallant's lab at Cal Berkeley.
05:36
They've been able to decode brainwaves
05:41
into recognizable visual fields.
05:43
So let me set this up for you.
05:45
In this experiment, individuals were shown
05:46
hundreds of hours of YouTube videos
05:49
while scans were made of their brains
05:51
to create a large library of their brain reacting
05:53
to video sequences.
05:56
Then a new movie was shown with new images,
05:59
new people, new animals in it,
06:02
and a new scan set was recorded.
06:04
The computer, using brain scan data alone,
06:06
decoded that new brain scan
06:09
to show what it thought the
individual was actually seeing.
06:11
On the right-hand side, you
see the computer's guess,
06:15
and on the left-hand side, the presented clip.
06:19
This is the jaw-dropper.
06:23
We are so close to being able to do this.
06:25
We just need to up the resolution.
06:28
And now remember that when you see an image
06:31
versus when you imagine that same image,
06:34
it creates the same brain scan.
06:36
So this was done with the highest-resolution
06:39
brain scan systems available today,
06:42
and their resolution has increased really
06:44
about a thousandfold in the last several years.
06:46
Next we need to increase the resolution
06:50
another thousandfold
06:52
to get a deeper glimpse.
06:54
How do we do that?
06:56
There's a lot of techniques in this approach.
06:57
One way is to crack open your
skull and put in electrodes.
07:00
I'm not for that.
07:03
There's a lot of new imaging techniques
07:04
being proposed, some even by me,
07:07
but given the recent success of MRI,
07:09
first we need to ask the question,
07:12
is it the end of the road with this technology?
07:14
Conventional wisdom says the only way
07:17
to get higher resolution is with bigger magnets,
07:20
but at this point bigger magnets
07:22
only offer incremental resolution improvements,
07:24
not the thousandfold we need.
07:28
I'm putting forward an idea:
07:30
instead of bigger magnets,
07:32
let's make better magnets.
07:34
There's some new technology breakthroughs
07:36
in nanoscience
07:38
when applied to magnetic structures
07:40
that have created a whole new class of magnets,
07:41
and with these magnets, we can lay down
07:44
very fine detailed magnetic field patterns
07:47
throughout the brain,
07:49
and using those, we can actually create
07:51
holographic-like interference structures
07:54
to get precision control over many patterns,
07:57
as is shown here by shifting things.
08:00
We can create much more complicated structures
08:02
with slightly different arrangements,
08:06
kind of like making Spirograph.
08:08
So why does that matter?
08:11
A lot of effort in MRI over the years
08:13
has gone into making really big,
08:16
really huge magnets, right?
08:18
But yet most of the recent advances
08:21
in resolution have actually come from
08:23
ingeniously clever encoding and decoding solutions
08:26
in the F.M. radio frequency transmitters and receivers
08:30
in the MRI systems.
08:33
Let's also, instead of a uniform magnetic field,
08:36
put down structured magnetic patterns
08:39
in addition to the F.M. radio frequencies.
08:42
So by combining the magnetics patterns
08:45
with the patterns in the F.M. radio frequencies
08:47
processing which can massively increase
08:50
the information that we can extract
08:52
in a single scan.
08:54
And on top of that, we can then layer
08:56
our ever-growing knowledge
of brain structure and memory
08:59
to create a thousandfold increase that we need.
09:03
And using fMRI, we should be able to measure
09:07
not just oxygenated blood flow,
09:10
but the hormones and neurotransmitters
I've talked about
09:12
and maybe even the direct neural activity,
09:15
which is the dream.
09:17
We're going to be able to dump our ideas
09:19
directly to digital media.
09:21
Could you imagine if we could leapfrog language
09:24
and communicate directly with human thought?
09:26
What would we be capable of then?
09:30
And how will we learn to deal
09:34
with the truths of unfiltered human thought?
09:36
You think the Internet was big.
09:41
These are huge questions.
09:43
It might be irresistible as a tool
09:46
to amplify our thinking and communication skills.
09:48
And indeed, this very same tool
09:52
may prove to lead to the cure
09:54
for Alzheimer's and similar diseases.
09:56
We have little option but to open this door.
09:59
Regardless, pick a year --
10:02
will it happen in five years or 15 years?
10:04
It's hard to imagine it taking much longer.
10:06
We need to learn how to take this step together.
10:11
Thank you.
10:15
(Applause)
10:17

sponsored links

Mary Lou Jepsen - Screen engineer
Mary Lou Jepsen discovers astonishing ways to integrate digital screens into daily life.

Why you should listen

Mary Lou Jepsen is the head of the Display Division at Google [x].  Previously she has founded or co-founded 4 different startups and served as the CTO or CEO at all of them. In 2005, with Nicholas Negroponte, she co-founded One Laptop per Child (OLPC) to build affordable computers for the world’s poorest children. As CTO she invented, architected and delivered to high-volume production a machine that the titans of technology believed was impossible to make.  Dr. Jepsen then founded Pixel Qi Corp. in 2008 in an attempt to transform a broken display component industry into an innovation engine. In the past she has been a professor at MIT, the CTO of Intel's Display Division and a globe-trotting high-tech media artist. She has been ranked in the top 50 female computer scientists of all time, and Time Magazine inducted her into its "Time 100" as one of the 100 most influential people in the world.

sponsored links

If you need translations, you can install "Google Translate" extension into your Chrome Browser.
Furthermore, you can change playback rate by installing "Video Speed Controller" extension.

Data provided by TED.

This website is owned and operated by Tokyo English Network.
The developer's blog is here.