TED2013
Mary Lou Jepsen: Could future devices read images from our brains?
Filmed:
Readability: 4.6
1,035,856 views
As an expert on cutting-edge digital displays, Mary Lou Jepsen studies how to show our most creative ideas on screens. And as a brain surgery patient herself, she is driven to know more about the neural activity that underlies invention, creativity, thought. She meshes these two passions in a rather mind-blowing talk on two cutting-edge brain studies that might point to a new frontier in understanding how (and what) we think.
Mary Lou Jepsen - Inventor, entrepreneur, optical physicist
Mary Lou Jepsen pushes the edges of what's possible in optics and physics, to make new types of devices, leading teams and working with huge factories that can ship vast volumes of these strange, new things. Full bio
Mary Lou Jepsen pushes the edges of what's possible in optics and physics, to make new types of devices, leading teams and working with huge factories that can ship vast volumes of these strange, new things. Full bio
Double-click the English transcript below to play the video.
00:12
I had brain surgery 18 years ago,
0
607
2508
00:15
and since that time, brain science has become
1
3115
2567
00:17
a personal passion of mine.
2
5682
1999
00:19
I'm actually an engineer.
3
7681
2235
00:21
And first let me say, I recently joined
4
9916
2516
00:24
Google's Moonshot group,
5
12432
1549
00:25
where I had a division,
6
13981
1212
00:27
the display division in Google X,
7
15193
2181
00:29
and the brain science work I'm speaking about today
8
17374
2622
00:31
is work I did before I joined Google
9
19996
2921
00:34
and on the side outside of Google.
10
22917
2332
00:37
So that said, there's a stigma
11
25249
3183
00:40
when you have brain surgery.
12
28432
2285
00:42
Are you still smart or not?
13
30717
2823
00:45
And if not, can you make yourself smart again?
14
33540
3848
00:49
After my neurosurgery,
15
37388
1766
00:51
part of my brain was missing,
16
39154
1997
00:53
and I had to deal with that.
17
41151
2773
00:55
It wasn't the grey matter, but it
was the gooey part dead center
was the gooey part dead center
18
43924
2944
00:58
that makes key hormones and neurotransmitters.
19
46868
3402
01:02
Immediately after my surgery,
20
50270
2231
01:04
I had to decide what amounts of each of over
21
52501
2143
01:06
a dozen powerful chemicals to take each day,
22
54644
3702
01:10
because if I just took nothing,
23
58346
1809
01:12
I would die within hours.
24
60155
2732
01:14
Every day now for 18 years -- every single day --
25
62887
3920
01:18
I've had to try to decide the combinations
26
66807
2710
01:21
and mixtures of chemicals,
27
69517
1328
01:22
and try to get them, to stay alive.
28
70845
3847
01:26
There have been several close calls.
29
74692
2721
01:29
But luckily, I'm an experimentalist at heart,
30
77413
3699
01:33
so I decided I would experiment
31
81112
3227
01:36
to try to find more optimal dosages
32
84339
2440
01:38
because there really isn't a clear road map
33
86779
1637
01:40
on this that's detailed.
34
88416
1903
01:42
I began to try different mixtures,
35
90319
2151
01:44
and I was blown away by how
36
92470
2872
01:47
tiny changes in dosages
37
95342
2411
01:49
dramatically changed my sense of self,
38
97753
3149
01:52
my sense of who I was, my thinking,
39
100902
1811
01:54
my behavior towards people.
40
102713
2259
01:56
One particularly dramatic case:
41
104972
2049
01:59
for a couple months I actually tried dosages
42
107021
1868
02:00
and chemicals typical of a man in his early 20s,
43
108889
3908
02:04
and I was blown away by how my thoughts changed.
44
112797
3011
02:07
(Laughter)
45
115808
3120
02:10
I was angry all the time,
46
118928
3058
02:13
I thought about sex constantly,
47
121986
1846
02:15
and I thought I was the smartest person
48
123832
2949
02:18
in the entire world, and
49
126781
2051
02:20
—(Laughter)—
50
128832
2263
02:23
of course over the years I'd
met guys kind of like that,
met guys kind of like that,
51
131095
2925
02:26
or maybe kind of toned-down versions of that.
52
134020
2267
02:28
I was kind of extreme.
53
136287
2184
02:30
But to me, the surprise was,
54
138471
2569
02:33
I wasn't trying to be arrogant.
55
141040
2166
02:35
I was actually trying,
56
143206
3209
02:38
with a little bit of insecurity,
57
146415
2360
02:40
to actually fix a problem in front of me,
58
148775
3000
02:43
and it just didn't come out that way.
59
151775
1856
02:45
So I couldn't handle it.
60
153631
1483
02:47
I changed my dosages.
61
155114
1525
02:48
But that experience, I think, gave me
62
156639
2455
02:51
a new appreciation for men
63
159094
1751
02:52
and what they might walk through,
64
160845
1816
02:54
and I've gotten along with men
65
162661
1690
02:56
a lot better since then.
66
164351
1839
02:58
What I was trying to do
67
166190
1545
02:59
with tuning these hormones
68
167735
2028
03:01
and neurotransmitters and so forth
69
169763
2323
03:04
was to try to get my intelligence back
70
172086
3605
03:07
after my illness and surgery,
71
175691
2634
03:10
my creative thought, my idea flow.
72
178325
2635
03:12
And I think mostly in images,
73
180960
2641
03:15
and so for me that became a key metric --
74
183601
2852
03:18
how to get these mental images
75
186453
2330
03:20
that I use as a way of rapid prototyping,
76
188783
2504
03:23
if you will, my ideas,
77
191287
1743
03:25
trying on different new ideas for size,
78
193030
2372
03:27
playing out scenarios.
79
195402
1695
03:29
This kind of thinking isn't new.
80
197097
1913
03:31
Philiosophers like Hume and Descartes and Hobbes
81
199010
3255
03:34
saw things similarly.
82
202265
1528
03:35
They thought that mental images and ideas
83
203793
2737
03:38
were actually the same thing.
84
206530
2331
03:40
There are those today that dispute that,
85
208861
2417
03:43
and lots of debates about how the mind works,
86
211278
3195
03:46
but for me it's simple:
87
214473
1736
03:48
Mental images, for most of us,
88
216209
2532
03:50
are central in inventive and creative thinking.
89
218741
3934
03:54
So after several years,
90
222675
1775
03:56
I tuned myself up and I have lots of great,
91
224450
3233
03:59
really vivid mental images with a lot of sophistication
92
227683
3048
04:02
and the analytical backbone behind them.
93
230731
2269
04:05
And so now I'm working on,
94
233000
1921
04:06
how can I get these mental images in my mind
95
234921
4162
04:11
out to my computer screen faster?
96
239083
2850
04:13
Can you imagine, if you will,
97
241933
2089
04:16
a movie director being able to use
98
244022
2120
04:18
her imagination alone to
direct the world in front of her?
direct the world in front of her?
99
246142
3762
04:21
Or a musician to get the music out of his head?
100
249904
3588
04:25
There are incredible possibilities with this
101
253492
2292
04:27
as a way for creative people
102
255784
1993
04:29
to share at light speed.
103
257777
2233
04:32
And the truth is, the remaining bottleneck
104
260010
1998
04:34
in being able to do this
105
262008
1173
04:35
is just upping the resolution of brain scan systems.
106
263181
3980
04:39
So let me show you why I think
we're pretty close to getting there
we're pretty close to getting there
107
267161
2858
04:42
by sharing with you two recent experiments
108
270029
2387
04:44
from two top neuroscience groups.
109
272416
2587
04:47
Both used fMRI technology --
110
275003
2488
04:49
functional magnetic resonance imaging technology --
111
277491
2279
04:51
to image the brain,
112
279770
1411
04:53
and here is a brain scan set from Giorgio Ganis
113
281181
3257
04:56
and his colleagues at Harvard.
114
284438
1950
04:58
And the left-hand column shows a brain scan
115
286388
3154
05:01
of a person looking at an image.
116
289542
3267
05:04
The middle column shows the brainscan
117
292809
1929
05:06
of that same individual
118
294738
1621
05:08
imagining, seeing that same image.
119
296359
3066
05:11
And the right column was created
120
299425
2048
05:13
by subtracting the middle
column from the left column,
column from the left column,
121
301473
3594
05:17
showing the difference to be nearly zero.
122
305083
2943
05:20
This was repeated on lots of different individuals
123
308026
2894
05:22
with lots of different images,
124
310920
2830
05:25
always with a similar result.
125
313750
1604
05:27
The difference between seeing an image
126
315354
2089
05:29
and imagining seeing that same image
127
317443
2455
05:31
is next to nothing.
128
319898
2155
05:34
Next let me share with you one other experiment,
129
322053
2761
05:36
this from Jack Gallant's lab at Cal Berkeley.
130
324814
4541
05:41
They've been able to decode brainwaves
131
329355
2063
05:43
into recognizable visual fields.
132
331418
2441
05:45
So let me set this up for you.
133
333859
1305
05:47
In this experiment, individuals were shown
134
335164
2333
05:49
hundreds of hours of YouTube videos
135
337497
1995
05:51
while scans were made of their brains
136
339492
2039
05:53
to create a large library of their brain reacting
137
341531
3216
05:56
to video sequences.
138
344747
2649
05:59
Then a new movie was shown with new images,
139
347396
2850
06:02
new people, new animals in it,
140
350246
1952
06:04
and a new scan set was recorded.
141
352198
2711
06:06
The computer, using brain scan data alone,
142
354909
2788
06:09
decoded that new brain scan
143
357697
2024
06:11
to show what it thought the
individual was actually seeing.
individual was actually seeing.
144
359721
4376
06:16
On the right-hand side, you
see the computer's guess,
see the computer's guess,
145
364097
3381
06:19
and on the left-hand side, the presented clip.
146
367478
4007
06:23
This is the jaw-dropper.
147
371485
2319
06:25
We are so close to being able to do this.
148
373804
2687
06:28
We just need to up the resolution.
149
376491
2785
06:31
And now remember that when you see an image
150
379276
3252
06:34
versus when you imagine that same image,
151
382528
2158
06:36
it creates the same brain scan.
152
384686
3475
06:40
So this was done with the highest-resolution
153
388161
2722
06:42
brain scan systems available today,
154
390883
2185
06:45
and their resolution has increased really
155
393068
1784
06:46
about a thousandfold in the last several years.
156
394852
3497
06:50
Next we need to increase the resolution
157
398349
2322
06:52
another thousandfold
158
400671
1977
06:54
to get a deeper glimpse.
159
402648
1789
06:56
How do we do that?
160
404437
1511
06:57
There's a lot of techniques in this approach.
161
405948
2614
07:00
One way is to crack open your
skull and put in electrodes.
skull and put in electrodes.
162
408562
3118
07:03
I'm not for that.
163
411680
1403
07:05
There's a lot of new imaging techniques
164
413083
2955
07:08
being proposed, some even by me,
165
416038
2003
07:10
but given the recent success of MRI,
166
418041
2959
07:13
first we need to ask the question,
167
421000
2068
07:15
is it the end of the road with this technology?
168
423068
2841
07:17
Conventional wisdom says the only way
169
425909
2455
07:20
to get higher resolution is with bigger magnets,
170
428364
2589
07:22
but at this point bigger magnets
171
430953
1842
07:24
only offer incremental resolution improvements,
172
432795
3750
07:28
not the thousandfold we need.
173
436545
2160
07:30
I'm putting forward an idea:
174
438705
1823
07:32
instead of bigger magnets,
175
440528
1963
07:34
let's make better magnets.
176
442491
2450
07:36
There's some new technology breakthroughs
177
444941
2003
07:38
in nanoscience
178
446944
1457
07:40
when applied to magnetic structures
179
448401
1727
07:42
that have created a whole new class of magnets,
180
450128
3013
07:45
and with these magnets, we can lay down
181
453141
2531
07:47
very fine detailed magnetic field patterns
182
455672
2167
07:49
throughout the brain,
183
457839
1355
07:51
and using those, we can actually create
184
459194
3182
07:54
holographic-like interference structures
185
462376
2838
07:57
to get precision control over many patterns,
186
465214
3469
08:00
as is shown here by shifting things.
187
468683
2445
08:03
We can create much more complicated structures
188
471128
3150
08:06
with slightly different arrangements,
189
474278
2071
08:08
kind of like making Spirograph.
190
476349
3033
08:11
So why does that matter?
191
479382
2228
08:13
A lot of effort in MRI over the years
192
481610
2577
08:16
has gone into making really big,
193
484187
2837
08:19
really huge magnets, right?
194
487024
2610
08:21
But yet most of the recent advances
195
489634
2509
08:24
in resolution have actually come from
196
492143
2197
08:26
ingeniously clever encoding and decoding solutions
197
494340
4008
08:30
in the F.M. radio frequency transmitters and receivers
198
498348
3287
08:33
in the MRI systems.
199
501635
2691
08:36
Let's also, instead of a uniform magnetic field,
200
504326
3322
08:39
put down structured magnetic patterns
201
507648
2672
08:42
in addition to the F.M. radio frequencies.
202
510320
3099
08:45
So by combining the magnetics patterns
203
513419
2307
08:47
with the patterns in the F.M. radio frequencies
204
515726
2710
08:50
processing which can massively increase
205
518436
2171
08:52
the information that we can extract
206
520607
1969
08:54
in a single scan.
207
522576
2446
08:57
And on top of that, we can then layer
208
525022
2332
08:59
our ever-growing knowledge
of brain structure and memory
of brain structure and memory
209
527354
4472
09:03
to create a thousandfold increase that we need.
210
531826
3695
09:07
And using fMRI, we should be able to measure
211
535521
2943
09:10
not just oxygenated blood flow,
212
538464
2082
09:12
but the hormones and neurotransmitters
I've talked about
I've talked about
213
540546
2901
09:15
and maybe even the direct neural activity,
214
543447
2345
09:17
which is the dream.
215
545792
1503
09:19
We're going to be able to dump our ideas
216
547295
2234
09:21
directly to digital media.
217
549529
2694
09:24
Could you imagine if we could leapfrog language
218
552223
2711
09:26
and communicate directly with human thought?
219
554934
4209
09:31
What would we be capable of then?
220
559143
3193
09:34
And how will we learn to deal
221
562336
2637
09:36
with the truths of unfiltered human thought?
222
564973
4219
09:41
You think the Internet was big.
223
569192
2567
09:43
These are huge questions.
224
571759
2602
09:46
It might be irresistible as a tool
225
574361
2148
09:48
to amplify our thinking and communication skills.
226
576509
3876
09:52
And indeed, this very same tool
227
580385
2023
09:54
may prove to lead to the cure
228
582408
2126
09:56
for Alzheimer's and similar diseases.
229
584534
3074
09:59
We have little option but to open this door.
230
587608
3512
10:03
Regardless, pick a year --
231
591120
1585
10:04
will it happen in five years or 15 years?
232
592705
2266
10:06
It's hard to imagine it taking much longer.
233
594971
4616
10:11
We need to learn how to take this step together.
234
599587
3695
10:15
Thank you.
235
603282
2174
10:17
(Applause)
236
605456
3974
ABOUT THE SPEAKER
Mary Lou Jepsen - Inventor, entrepreneur, optical physicistMary Lou Jepsen pushes the edges of what's possible in optics and physics, to make new types of devices, leading teams and working with huge factories that can ship vast volumes of these strange, new things.
Why you should listen
Mary Lou Jepsen is one of the world’s foremost engineers and scientists in optics, imaging and display -- inventing at the hairy, crazy edge of what physics allows, aiming to do what seems impossible and leading teams to achieve these in volume in partnership with the world’s largest manufacturers, in Asia. She has more than 200 patents published or issued.
Jepsen is the founder and CEO of Openwater, which aims to use new optics to see inside our bodies. Previously a top technical exec at Google, Facebook, Oculus and Intel, her startups include One Laptop Per Child, where she was CTO and chief architect on the $100 laptop. She studied at Brown, MIT and Rhode Island School of Design, and she was a professor at both MITs -- the one in Cambridge, Mass., and the Royal Melbourne Institute of Tech in Australia.
Mary Lou Jepsen | Speaker | TED.com