TED@BCG Paris
Blaise Agüera y Arcas: How computers are learning to be creative
Filmed:
Readability: 4.4
1,934,067 views
We're on the edge of a new frontier in art and creativity -- and it's not human. Blaise Agüera y Arcas, principal scientist at Google, works with deep neural networks for machine perception and distributed learning. In this captivating demo, he shows how neural nets trained to recognize images can be run in reverse, to generate them. The results: spectacular, hallucinatory collages (and poems!) that defy categorization. "Perception and creativity are very intimately connected," Agüera y Arcas says. "Any creature, any being that is able to do perceptual acts is also able to create."
Blaise Agüera y Arcas - Software architect
Blaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces. Full bio
Blaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces. Full bio
Double-click the English transcript below to play the video.
00:12
So, I lead a team at Google
that works on machine intelligence;
that works on machine intelligence;
0
800
3124
00:15
in other words, the engineering discipline
of making computers and devices
of making computers and devices
1
3948
4650
00:20
able to do some of the things
that brains do.
that brains do.
2
8622
2419
00:23
And this makes us
interested in real brains
interested in real brains
3
11439
3099
00:26
and neuroscience as well,
4
14562
1289
00:27
and especially interested
in the things that our brains do
in the things that our brains do
5
15875
4172
00:32
that are still far superior
to the performance of computers.
to the performance of computers.
6
20071
4042
00:37
Historically, one of those areas
has been perception,
has been perception,
7
25209
3609
00:40
the process by which things
out there in the world --
out there in the world --
8
28842
3039
00:43
sounds and images --
9
31905
1584
00:45
can turn into concepts in the mind.
10
33513
2178
00:48
This is essential for our own brains,
11
36235
2517
00:50
and it's also pretty useful on a computer.
12
38776
2464
00:53
The machine perception algorithms,
for example, that our team makes,
for example, that our team makes,
13
41636
3350
00:57
are what enable your pictures
on Google Photos to become searchable,
on Google Photos to become searchable,
14
45010
3874
01:00
based on what's in them.
15
48908
1397
01:03
The flip side of perception is creativity:
16
51594
3493
01:07
turning a concept into something
out there into the world.
out there into the world.
17
55111
3038
01:10
So over the past year,
our work on machine perception
our work on machine perception
18
58173
3555
01:13
has also unexpectedly connected
with the world of machine creativity
with the world of machine creativity
19
61752
4859
01:18
and machine art.
20
66635
1160
01:20
I think Michelangelo
had a penetrating insight
had a penetrating insight
21
68556
3284
01:23
into to this dual relationship
between perception and creativity.
between perception and creativity.
22
71864
3656
01:28
This is a famous quote of his:
23
76023
2006
01:30
"Every block of stone
has a statue inside of it,
has a statue inside of it,
24
78053
3323
01:34
and the job of the sculptor
is to discover it."
is to discover it."
25
82036
3002
01:38
So I think that what
Michelangelo was getting at
Michelangelo was getting at
26
86029
3216
01:41
is that we create by perceiving,
27
89269
3180
01:44
and that perception itself
is an act of imagination
is an act of imagination
28
92473
3023
01:47
and is the stuff of creativity.
29
95520
2461
01:50
The organ that does all the thinking
and perceiving and imagining,
and perceiving and imagining,
30
98691
3925
01:54
of course, is the brain.
31
102640
1588
01:57
And I'd like to begin
with a brief bit of history
with a brief bit of history
32
105089
2545
01:59
about what we know about brains.
33
107658
2302
02:02
Because unlike, say,
the heart or the intestines,
the heart or the intestines,
34
110496
2446
02:04
you really can't say very much
about a brain by just looking at it,
about a brain by just looking at it,
35
112966
3144
02:08
at least with the naked eye.
36
116134
1412
02:09
The early anatomists who looked at brains
37
117983
2416
02:12
gave the superficial structures
of this thing all kinds of fanciful names,
of this thing all kinds of fanciful names,
38
120423
3807
02:16
like hippocampus, meaning "little shrimp."
39
124254
2433
02:18
But of course that sort of thing
doesn't tell us very much
doesn't tell us very much
40
126711
2764
02:21
about what's actually going on inside.
41
129499
2318
02:24
The first person who, I think, really
developed some kind of insight
developed some kind of insight
42
132780
3613
02:28
into what was going on in the brain
43
136417
1930
02:30
was the great Spanish neuroanatomist,
Santiago Ramón y Cajal,
Santiago Ramón y Cajal,
44
138371
3920
02:34
in the 19th century,
45
142315
1544
02:35
who used microscopy and special stains
46
143883
3755
02:39
that could selectively fill in
or render in very high contrast
or render in very high contrast
47
147662
4170
02:43
the individual cells in the brain,
48
151856
2008
02:45
in order to start to understand
their morphologies.
their morphologies.
49
153888
3154
02:49
And these are the kinds of drawings
that he made of neurons
that he made of neurons
50
157972
2891
02:52
in the 19th century.
51
160887
1209
02:54
This is from a bird brain.
52
162120
1884
02:56
And you see this incredible variety
of different sorts of cells,
of different sorts of cells,
53
164028
3057
02:59
even the cellular theory itself
was quite new at this point.
was quite new at this point.
54
167109
3435
03:02
And these structures,
55
170568
1278
03:03
these cells that have these arborizations,
56
171870
2259
03:06
these branches that can go
very, very long distances --
very, very long distances --
57
174153
2608
03:08
this was very novel at the time.
58
176785
1616
03:10
They're reminiscent, of course, of wires.
59
178779
2903
03:13
That might have been obvious
to some people in the 19th century;
to some people in the 19th century;
60
181706
3457
03:17
the revolutions of wiring and electricity
were just getting underway.
were just getting underway.
61
185187
4314
03:21
But in many ways,
62
189964
1178
03:23
these microanatomical drawings
of Ramón y Cajal's, like this one,
of Ramón y Cajal's, like this one,
63
191166
3313
03:26
they're still in some ways unsurpassed.
64
194503
2332
03:28
We're still more than a century later,
65
196859
1854
03:30
trying to finish the job
that Ramón y Cajal started.
that Ramón y Cajal started.
66
198737
2825
03:33
These are raw data from our collaborators
67
201586
3134
03:36
at the Max Planck Institute
of Neuroscience.
of Neuroscience.
68
204744
2881
03:39
And what our collaborators have done
69
207649
1790
03:41
is to image little pieces of brain tissue.
70
209463
5001
03:46
The entire sample here
is about one cubic millimeter in size,
is about one cubic millimeter in size,
71
214488
3326
03:49
and I'm showing you a very,
very small piece of it here.
very small piece of it here.
72
217838
2621
03:52
That bar on the left is about one micron.
73
220483
2346
03:54
The structures you see are mitochondria
74
222853
2409
03:57
that are the size of bacteria.
75
225286
2044
03:59
And these are consecutive slices
76
227354
1551
04:00
through this very, very
tiny block of tissue.
tiny block of tissue.
77
228929
3148
04:04
Just for comparison's sake,
78
232101
2403
04:06
the diameter of an average strand
of hair is about 100 microns.
of hair is about 100 microns.
79
234528
3792
04:10
So we're looking at something
much, much smaller
much, much smaller
80
238344
2274
04:12
than a single strand of hair.
81
240642
1398
04:14
And from these kinds of serial
electron microscopy slices,
electron microscopy slices,
82
242064
4031
04:18
one can start to make reconstructions
in 3D of neurons that look like these.
in 3D of neurons that look like these.
83
246119
5008
04:23
So these are sort of in the same
style as Ramón y Cajal.
style as Ramón y Cajal.
84
251151
3157
04:26
Only a few neurons lit up,
85
254332
1492
04:27
because otherwise we wouldn't
be able to see anything here.
be able to see anything here.
86
255848
2781
04:30
It would be so crowded,
87
258653
1312
04:31
so full of structure,
88
259989
1330
04:33
of wiring all connecting
one neuron to another.
one neuron to another.
89
261343
2724
04:37
So Ramón y Cajal was a little bit
ahead of his time,
ahead of his time,
90
265293
2804
04:40
and progress on understanding the brain
91
268121
2555
04:42
proceeded slowly
over the next few decades.
over the next few decades.
92
270700
2271
04:45
But we knew that neurons used electricity,
93
273455
2853
04:48
and by World War II, our technology
was advanced enough
was advanced enough
94
276332
2936
04:51
to start doing real electrical
experiments on live neurons
experiments on live neurons
95
279292
2806
04:54
to better understand how they worked.
96
282122
2106
04:56
This was the very same time
when computers were being invented,
when computers were being invented,
97
284631
4356
05:01
very much based on the idea
of modeling the brain --
of modeling the brain --
98
289011
3100
05:04
of "intelligent machinery,"
as Alan Turing called it,
as Alan Turing called it,
99
292135
3085
05:07
one of the fathers of computer science.
100
295244
1991
05:09
Warren McCulloch and Walter Pitts
looked at Ramón y Cajal's drawing
looked at Ramón y Cajal's drawing
101
297923
4632
05:14
of visual cortex,
102
302579
1317
05:15
which I'm showing here.
103
303920
1562
05:17
This is the cortex that processes
imagery that comes from the eye.
imagery that comes from the eye.
104
305506
4442
05:22
And for them, this looked
like a circuit diagram.
like a circuit diagram.
105
310424
3508
05:26
So there are a lot of details
in McCulloch and Pitts's circuit diagram
in McCulloch and Pitts's circuit diagram
106
314353
3835
05:30
that are not quite right.
107
318212
1352
05:31
But this basic idea
108
319588
1235
05:32
that visual cortex works like a series
of computational elements
of computational elements
109
320847
3992
05:36
that pass information
one to the next in a cascade,
one to the next in a cascade,
110
324863
2746
05:39
is essentially correct.
111
327633
1602
05:41
Let's talk for a moment
112
329259
2350
05:43
about what a model for processing
visual information would need to do.
visual information would need to do.
113
331633
4032
05:48
The basic task of perception
114
336228
2741
05:50
is to take an image like this one and say,
115
338993
4194
05:55
"That's a bird,"
116
343211
1176
05:56
which is a very simple thing
for us to do with our brains.
for us to do with our brains.
117
344411
2874
05:59
But you should all understand
that for a computer,
that for a computer,
118
347309
3421
06:02
this was pretty much impossible
just a few years ago.
just a few years ago.
119
350754
3087
06:05
The classical computing paradigm
120
353865
1916
06:07
is not one in which
this task is easy to do.
this task is easy to do.
121
355805
2507
06:11
So what's going on between the pixels,
122
359366
2552
06:13
between the image of the bird
and the word "bird,"
and the word "bird,"
123
361942
4028
06:17
is essentially a set of neurons
connected to each other
connected to each other
124
365994
2814
06:20
in a neural network,
125
368832
1155
06:22
as I'm diagramming here.
126
370011
1223
06:23
This neural network could be biological,
inside our visual cortices,
inside our visual cortices,
127
371258
3272
06:26
or, nowadays, we start
to have the capability
to have the capability
128
374554
2162
06:28
to model such neural networks
on the computer.
on the computer.
129
376740
2454
06:31
And I'll show you what
that actually looks like.
that actually looks like.
130
379834
2353
06:34
So the pixels you can think
about as a first layer of neurons,
about as a first layer of neurons,
131
382211
3416
06:37
and that's, in fact,
how it works in the eye --
how it works in the eye --
132
385651
2239
06:39
that's the neurons in the retina.
133
387914
1663
06:41
And those feed forward
134
389601
1500
06:43
into one layer after another layer,
after another layer of neurons,
after another layer of neurons,
135
391125
3403
06:46
all connected by synapses
of different weights.
of different weights.
136
394552
3033
06:49
The behavior of this network
137
397609
1335
06:50
is characterized by the strengths
of all of those synapses.
of all of those synapses.
138
398968
3284
06:54
Those characterize the computational
properties of this network.
properties of this network.
139
402276
3288
06:57
And at the end of the day,
140
405588
1470
06:59
you have a neuron
or a small group of neurons
or a small group of neurons
141
407082
2447
07:01
that light up, saying, "bird."
142
409553
1647
07:03
Now I'm going to represent
those three things --
those three things --
143
411824
3132
07:06
the input pixels and the synapses
in the neural network,
in the neural network,
144
414980
4696
07:11
and bird, the output --
145
419700
1585
07:13
by three variables: x, w and y.
146
421309
3057
07:16
There are maybe a million or so x's --
147
424853
1811
07:18
a million pixels in that image.
148
426688
1953
07:20
There are billions or trillions of w's,
149
428665
2446
07:23
which represent the weights of all
these synapses in the neural network.
these synapses in the neural network.
150
431135
3421
07:26
And there's a very small number of y's,
151
434580
1875
07:28
of outputs that that network has.
152
436479
1858
07:30
"Bird" is only four letters, right?
153
438361
1749
07:33
So let's pretend that this
is just a simple formula,
is just a simple formula,
154
441088
3426
07:36
x "x" w = y.
155
444538
2163
07:38
I'm putting the times in scare quotes
156
446725
2036
07:40
because what's really
going on there, of course,
going on there, of course,
157
448785
2280
07:43
is a very complicated series
of mathematical operations.
of mathematical operations.
158
451089
3046
07:47
That's one equation.
159
455172
1221
07:48
There are three variables.
160
456417
1672
07:50
And we all know
that if you have one equation,
that if you have one equation,
161
458113
2726
07:52
you can solve one variable
by knowing the other two things.
by knowing the other two things.
162
460863
3642
07:57
So the problem of inference,
163
465158
3380
08:00
that is, figuring out
that the picture of a bird is a bird,
that the picture of a bird is a bird,
164
468562
2873
08:03
is this one:
165
471459
1274
08:04
it's where y is the unknown
and w and x are known.
and w and x are known.
166
472757
3459
08:08
You know the neural network,
you know the pixels.
you know the pixels.
167
476240
2459
08:10
As you can see, that's actually
a relatively straightforward problem.
a relatively straightforward problem.
168
478723
3327
08:14
You multiply two times three
and you're done.
and you're done.
169
482074
2186
08:16
I'll show you an artificial neural network
170
484862
2123
08:19
that we've built recently,
doing exactly that.
doing exactly that.
171
487009
2296
08:21
This is running in real time
on a mobile phone,
on a mobile phone,
172
489634
2860
08:24
and that's, of course,
amazing in its own right,
amazing in its own right,
173
492518
3313
08:27
that mobile phones can do so many
billions and trillions of operations
billions and trillions of operations
174
495855
3468
08:31
per second.
175
499347
1248
08:32
What you're looking at is a phone
176
500619
1615
08:34
looking at one after another
picture of a bird,
picture of a bird,
177
502258
3547
08:37
and actually not only saying,
"Yes, it's a bird,"
"Yes, it's a bird,"
178
505829
2715
08:40
but identifying the species of bird
with a network of this sort.
with a network of this sort.
179
508568
3411
08:44
So in that picture,
180
512890
1826
08:46
the x and the w are known,
and the y is the unknown.
and the y is the unknown.
181
514740
3802
08:50
I'm glossing over the very
difficult part, of course,
difficult part, of course,
182
518566
2508
08:53
which is how on earth
do we figure out the w,
do we figure out the w,
183
521098
3861
08:56
the brain that can do such a thing?
184
524983
2187
08:59
How would we ever learn such a model?
185
527194
1834
09:01
So this process of learning,
of solving for w,
of solving for w,
186
529418
3233
09:04
if we were doing this
with the simple equation
with the simple equation
187
532675
2647
09:07
in which we think about these as numbers,
188
535346
2000
09:09
we know exactly how to do that: 6 = 2 x w,
189
537370
2687
09:12
well, we divide by two and we're done.
190
540081
3312
09:16
The problem is with this operator.
191
544001
2220
09:18
So, division --
192
546823
1151
09:19
we've used division because
it's the inverse to multiplication,
it's the inverse to multiplication,
193
547998
3121
09:23
but as I've just said,
194
551143
1440
09:24
the multiplication is a bit of a lie here.
195
552607
2449
09:27
This is a very, very complicated,
very non-linear operation;
very non-linear operation;
196
555080
3326
09:30
it has no inverse.
197
558430
1704
09:32
So we have to figure out a way
to solve the equation
to solve the equation
198
560158
3150
09:35
without a division operator.
199
563332
2024
09:37
And the way to do that
is fairly straightforward.
is fairly straightforward.
200
565380
2343
09:39
You just say, let's play
a little algebra trick,
a little algebra trick,
201
567747
2671
09:42
and move the six over
to the right-hand side of the equation.
to the right-hand side of the equation.
202
570442
2906
09:45
Now, we're still using multiplication.
203
573372
1826
09:47
And that zero -- let's think
about it as an error.
about it as an error.
204
575675
3580
09:51
In other words, if we've solved
for w the right way,
for w the right way,
205
579279
2515
09:53
then the error will be zero.
206
581818
1656
09:55
And if we haven't gotten it quite right,
207
583498
1938
09:57
the error will be greater than zero.
208
585460
1749
09:59
So now we can just take guesses
to minimize the error,
to minimize the error,
209
587233
3366
10:02
and that's the sort of thing
computers are very good at.
computers are very good at.
210
590623
2687
10:05
So you've taken an initial guess:
211
593334
1593
10:06
what if w = 0?
212
594951
1156
10:08
Well, then the error is 6.
213
596131
1240
10:09
What if w = 1? The error is 4.
214
597395
1446
10:10
And then the computer can
sort of play Marco Polo,
sort of play Marco Polo,
215
598865
2367
10:13
and drive down the error close to zero.
216
601256
2367
10:15
As it does that, it's getting
successive approximations to w.
successive approximations to w.
217
603647
3374
10:19
Typically, it never quite gets there,
but after about a dozen steps,
but after about a dozen steps,
218
607045
3656
10:22
we're up to w = 2.999,
which is close enough.
which is close enough.
219
610725
4624
10:28
And this is the learning process.
220
616302
1814
10:30
So remember that what's been going on here
221
618140
2730
10:32
is that we've been taking
a lot of known x's and known y's
a lot of known x's and known y's
222
620894
4378
10:37
and solving for the w in the middle
through an iterative process.
through an iterative process.
223
625296
3454
10:40
It's exactly the same way
that we do our own learning.
that we do our own learning.
224
628774
3556
10:44
We have many, many images as babies
225
632354
2230
10:46
and we get told, "This is a bird;
this is not a bird."
this is not a bird."
226
634608
2633
10:49
And over time, through iteration,
227
637714
2098
10:51
we solve for w, we solve
for those neural connections.
for those neural connections.
228
639836
2928
10:55
So now, we've held
x and w fixed to solve for y;
x and w fixed to solve for y;
229
643460
4086
10:59
that's everyday, fast perception.
230
647570
1847
11:01
We figure out how we can solve for w,
231
649441
1763
11:03
that's learning, which is a lot harder,
232
651228
1903
11:05
because we need to do error minimization,
233
653155
1985
11:07
using a lot of training examples.
234
655164
1687
11:08
And about a year ago,
Alex Mordvintsev, on our team,
Alex Mordvintsev, on our team,
235
656875
3187
11:12
decided to experiment
with what happens if we try solving for x,
with what happens if we try solving for x,
236
660086
3550
11:15
given a known w and a known y.
237
663660
2037
11:18
In other words,
238
666124
1151
11:19
you know that it's a bird,
239
667299
1352
11:20
and you already have your neural network
that you've trained on birds,
that you've trained on birds,
240
668675
3303
11:24
but what is the picture of a bird?
241
672002
2344
11:27
It turns out that by using exactly
the same error-minimization procedure,
the same error-minimization procedure,
242
675034
5024
11:32
one can do that with the network
trained to recognize birds,
trained to recognize birds,
243
680082
3430
11:35
and the result turns out to be ...
244
683536
3388
11:42
a picture of birds.
245
690400
1305
11:44
So this is a picture of birds
generated entirely by a neural network
generated entirely by a neural network
246
692814
3737
11:48
that was trained to recognize birds,
247
696575
1826
11:50
just by solving for x
rather than solving for y,
rather than solving for y,
248
698425
3538
11:53
and doing that iteratively.
249
701987
1288
11:55
Here's another fun example.
250
703732
1847
11:57
This was a work made
by Mike Tyka in our group,
by Mike Tyka in our group,
251
705603
3437
12:01
which he calls "Animal Parade."
252
709064
2308
12:03
It reminds me a little bit
of William Kentridge's artworks,
of William Kentridge's artworks,
253
711396
2876
12:06
in which he makes sketches, rubs them out,
254
714296
2489
12:08
makes sketches, rubs them out,
255
716809
1460
12:10
and creates a movie this way.
256
718293
1398
12:11
In this case,
257
719715
1151
12:12
what Mike is doing is varying y
over the space of different animals,
over the space of different animals,
258
720890
3277
12:16
in a network designed
to recognize and distinguish
to recognize and distinguish
259
724191
2382
12:18
different animals from each other.
260
726597
1810
12:20
And you get this strange, Escher-like
morph from one animal to another.
morph from one animal to another.
261
728431
3751
12:26
Here he and Alex together
have tried reducing
have tried reducing
262
734221
4614
12:30
the y's to a space of only two dimensions,
263
738859
2759
12:33
thereby making a map
out of the space of all things
out of the space of all things
264
741642
3438
12:37
recognized by this network.
265
745104
1719
12:38
Doing this kind of synthesis
266
746847
2023
12:40
or generation of imagery
over that entire surface,
over that entire surface,
267
748894
2382
12:43
varying y over the surface,
you make a kind of map --
you make a kind of map --
268
751300
2846
12:46
a visual map of all the things
the network knows how to recognize.
the network knows how to recognize.
269
754170
3141
12:49
The animals are all here;
"armadillo" is right in that spot.
"armadillo" is right in that spot.
270
757335
2865
12:52
You can do this with other kinds
of networks as well.
of networks as well.
271
760919
2479
12:55
This is a network designed
to recognize faces,
to recognize faces,
272
763422
2874
12:58
to distinguish one face from another.
273
766320
2000
13:00
And here, we're putting
in a y that says, "me,"
in a y that says, "me,"
274
768344
3249
13:03
my own face parameters.
275
771617
1575
13:05
And when this thing solves for x,
276
773216
1706
13:06
it generates this rather crazy,
277
774946
2618
13:09
kind of cubist, surreal,
psychedelic picture of me
psychedelic picture of me
278
777588
4428
13:14
from multiple points of view at once.
279
782040
1806
13:15
The reason it looks like
multiple points of view at once
multiple points of view at once
280
783870
2734
13:18
is because that network is designed
to get rid of the ambiguity
to get rid of the ambiguity
281
786628
3687
13:22
of a face being in one pose
or another pose,
or another pose,
282
790339
2476
13:24
being looked at with one kind of lighting,
another kind of lighting.
another kind of lighting.
283
792839
3376
13:28
So when you do
this sort of reconstruction,
this sort of reconstruction,
284
796239
2085
13:30
if you don't use some sort of guide image
285
798348
2304
13:32
or guide statistics,
286
800676
1211
13:33
then you'll get a sort of confusion
of different points of view,
of different points of view,
287
801911
3765
13:37
because it's ambiguous.
288
805700
1368
13:39
This is what happens if Alex uses
his own face as a guide image
his own face as a guide image
289
807786
4223
13:44
during that optimization process
to reconstruct my own face.
to reconstruct my own face.
290
812033
3321
13:48
So you can see it's not perfect.
291
816284
2328
13:50
There's still quite a lot of work to do
292
818636
1874
13:52
on how we optimize
that optimization process.
that optimization process.
293
820534
2453
13:55
But you start to get something
more like a coherent face,
more like a coherent face,
294
823011
2827
13:57
rendered using my own face as a guide.
295
825862
2014
14:00
You don't have to start
with a blank canvas
with a blank canvas
296
828892
2501
14:03
or with white noise.
297
831417
1156
14:04
When you're solving for x,
298
832597
1304
14:05
you can begin with an x,
that is itself already some other image.
that is itself already some other image.
299
833925
3889
14:09
That's what this little demonstration is.
300
837838
2556
14:12
This is a network
that is designed to categorize
that is designed to categorize
301
840418
4122
14:16
all sorts of different objects --
man-made structures, animals ...
man-made structures, animals ...
302
844564
3119
14:19
Here we're starting
with just a picture of clouds,
with just a picture of clouds,
303
847707
2593
14:22
and as we optimize,
304
850324
1671
14:24
basically, this network is figuring out
what it sees in the clouds.
what it sees in the clouds.
305
852019
4486
14:28
And the more time
you spend looking at this,
you spend looking at this,
306
856931
2320
14:31
the more things you also
will see in the clouds.
will see in the clouds.
307
859275
2753
14:35
You could also use the face network
to hallucinate into this,
to hallucinate into this,
308
863004
3375
14:38
and you get some pretty crazy stuff.
309
866403
1812
14:40
(Laughter)
310
868239
1150
14:42
Or, Mike has done some other experiments
311
870401
2744
14:45
in which he takes that cloud image,
312
873169
3905
14:49
hallucinates, zooms, hallucinates,
zooms hallucinates, zooms.
zooms hallucinates, zooms.
313
877098
3507
14:52
And in this way,
314
880629
1151
14:53
you can get a sort of fugue state
of the network, I suppose,
of the network, I suppose,
315
881804
3675
14:57
or a sort of free association,
316
885503
3680
15:01
in which the network
is eating its own tail.
is eating its own tail.
317
889207
2227
15:03
So every image is now the basis for,
318
891458
3421
15:06
"What do I think I see next?
319
894903
1421
15:08
What do I think I see next?
What do I think I see next?"
What do I think I see next?"
320
896348
2803
15:11
I showed this for the first time in public
321
899487
2936
15:14
to a group at a lecture in Seattle
called "Higher Education" --
called "Higher Education" --
322
902447
5437
15:19
this was right after
marijuana was legalized.
marijuana was legalized.
323
907908
2437
15:22
(Laughter)
324
910369
2415
15:26
So I'd like to finish up quickly
325
914627
2104
15:28
by just noting that this technology
is not constrained.
is not constrained.
326
916755
4255
15:33
I've shown you purely visual examples
because they're really fun to look at.
because they're really fun to look at.
327
921034
3665
15:36
It's not a purely visual technology.
328
924723
2451
15:39
Our artist collaborator, Ross Goodwin,
329
927198
1993
15:41
has done experiments involving
a camera that takes a picture,
a camera that takes a picture,
330
929215
3671
15:44
and then a computer in his backpack
writes a poem using neural networks,
writes a poem using neural networks,
331
932910
4234
15:49
based on the contents of the image.
332
937168
1944
15:51
And that poetry neural network
has been trained
has been trained
333
939136
2947
15:54
on a large corpus of 20th-century poetry.
334
942107
2234
15:56
And the poetry is, you know,
335
944365
1499
15:57
I think, kind of not bad, actually.
336
945888
1914
15:59
(Laughter)
337
947826
1384
16:01
In closing,
338
949234
1159
16:02
I think that per Michelangelo,
339
950417
2132
16:04
I think he was right;
340
952573
1234
16:05
perception and creativity
are very intimately connected.
are very intimately connected.
341
953831
3436
16:09
What we've just seen are neural networks
342
957611
2634
16:12
that are entirely trained to discriminate,
343
960269
2303
16:14
or to recognize different
things in the world,
things in the world,
344
962596
2242
16:16
able to be run in reverse, to generate.
345
964862
3161
16:20
One of the things that suggests to me
346
968047
1783
16:21
is not only that
Michelangelo really did see
Michelangelo really did see
347
969854
2398
16:24
the sculpture in the blocks of stone,
348
972276
2452
16:26
but that any creature,
any being, any alien
any being, any alien
349
974752
3638
16:30
that is able to do
perceptual acts of that sort
perceptual acts of that sort
350
978414
3657
16:34
is also able to create
351
982095
1375
16:35
because it's exactly the same
machinery that's used in both cases.
machinery that's used in both cases.
352
983494
3224
16:38
Also, I think that perception
and creativity are by no means
and creativity are by no means
353
986742
4532
16:43
uniquely human.
354
991298
1210
16:44
We start to have computer models
that can do exactly these sorts of things.
that can do exactly these sorts of things.
355
992532
3708
16:48
And that ought to be unsurprising;
the brain is computational.
the brain is computational.
356
996264
3328
16:51
And finally,
357
999616
1657
16:53
computing began as an exercise
in designing intelligent machinery.
in designing intelligent machinery.
358
1001297
4668
16:57
It was very much modeled after the idea
359
1005989
2462
17:00
of how could we make machines intelligent.
360
1008475
3013
17:03
And we finally are starting to fulfill now
361
1011512
2162
17:05
some of the promises
of those early pioneers,
of those early pioneers,
362
1013698
2406
17:08
of Turing and von Neumann
363
1016128
1713
17:09
and McCulloch and Pitts.
364
1017865
2265
17:12
And I think that computing
is not just about accounting
is not just about accounting
365
1020154
4098
17:16
or playing Candy Crush or something.
366
1024276
2147
17:18
From the beginning,
we modeled them after our minds.
we modeled them after our minds.
367
1026447
2578
17:21
And they give us both the ability
to understand our own minds better
to understand our own minds better
368
1029049
3269
17:24
and to extend them.
369
1032342
1529
17:26
Thank you very much.
370
1034627
1167
17:27
(Applause)
371
1035818
5939
ABOUT THE SPEAKER
Blaise Agüera y Arcas - Software architectBlaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces.
Why you should listen
Blaise Agüera y Arcas is principal scientist at Google, where he leads a team working on machine intelligence for mobile devices. His group works extensively with deep neural nets for machine perception and distributed learning, and it also investigates so-called "connectomics" research, assessing maps of connections within the brain.
Agüera y Arcas' background is as multidimensional as the visions he helps create. In the 1990s, he authored patents on both video compression and 3D visualization techniques, and in 2001, he made an influential computational discovery that cast doubt on Gutenberg's role as the father of movable type.
He also created Seadragon (acquired by Microsoft in 2006), the visualization technology that gives Photosynth its amazingly smooth digital rendering and zoom capabilities. Photosynth itself is a vastly powerful piece of software capable of taking a wide variety of images, analyzing them for similarities, and grafting them together into an interactive three-dimensional space. This seamless patchwork of images can be viewed via multiple angles and magnifications, allowing us to look around corners or “fly” in for a (much) closer look. Simply put, it could utterly transform the way we experience digital images.
He joined Microsoft when Seadragon was acquired by Live Labs in 2006. Shortly after the acquisition of Seadragon, Agüera y Arcas directed his team in a collaboration with Microsoft Research and the University of Washington, leading to the first public previews of Photosynth several months later. His TED Talk on Seadragon and Photosynth in 2007 is rated one of TED's "most jaw-dropping." He returned to TED in 2010 to demo Bing’s augmented reality maps.
Fun fact: According to the author, Agüera y Arcas is the inspiration for the character Elgin in the 2012 best-selling novel Where'd You Go, Bernadette?
Blaise Agüera y Arcas | Speaker | TED.com