TED2016
Chris Milk: The birth of virtual reality as an art form
Filmed:
Readability: 3.8
905,305 views
Chris Milk uses innovative technologies to make personal, interactive, human stories. Accompanied by Joshua Roman on cello and McKenzie Stubbert on piano, Milk traces his relationship to music and art -- from the first moment he remembers putting on headphones to his current work creating breakthrough virtual reality projects. VR is the last medium for storytelling, he says, because it closes the gap between audience and storyteller. To illustrate, he brought the TED audience together in the world's largest collective VR experience. Join them and take part in this interactive talk by getting a Google Cardboard and downloading the experience at with.in/TED.
Chris Milk - Immersive storyteller
Working at the frontiers of interactive technology, Chris Milk stretches virtual reality into a new canvas for storytelling. Full bio
Working at the frontiers of interactive technology, Chris Milk stretches virtual reality into a new canvas for storytelling. Full bio
Double-click the English transcript below to play the video.
00:14
When I was a kid,
0
2134
1159
00:15
I experienced something so powerful,
1
3317
2508
00:17
I spent the rest of my life
searching for it,
searching for it,
2
5849
3074
00:20
and in all the wrong places.
3
8947
1512
00:23
What I experienced wasn't virtual reality.
4
11327
2593
00:26
It was music.
5
14633
1183
00:28
And this is where the story begins.
6
16251
2296
00:31
That's me,
7
19797
1370
00:33
listening to the Beatles' "White Album."
8
21191
1946
00:35
And the look on my face is the feeling
9
23161
3168
00:38
that I've been searching for ever since.
10
26353
2746
00:41
Music goes straight to the emotional vein,
11
29938
2586
00:44
into your bloodstream
12
32548
1606
00:46
and right into your heart.
13
34178
1331
00:48
It deepens every experience.
14
36157
2521
00:51
Fellas?
15
39717
1166
00:53
(Music)
16
41454
4218
00:59
This is the amazing McKenzie Stubbert
17
47472
2152
01:01
and Joshua Roman.
18
49648
1879
01:03
Music --
19
51551
1163
01:04
(Applause)
20
52738
1194
01:05
Yeah.
21
53956
1158
01:08
Music makes everything
have more emotional resonance.
have more emotional resonance.
22
56630
3825
01:13
Let's see how it does for this talk.
23
61229
1929
01:15
The right piece of music
at the right time fuses with us
at the right time fuses with us
24
63825
3744
01:19
on a cellular level.
25
67593
1325
01:21
When I hear that one song
26
69382
2280
01:24
from that one summer
27
72502
1303
01:26
with that one girl,
28
74488
1177
01:28
I'm instantly transported
back there again.
back there again.
29
76480
2021
01:31
Hey, Stacey.
30
79963
1167
01:34
Here's a part of the story, though,
where I got a little greedy.
where I got a little greedy.
31
82908
3278
01:38
I thought if I added more layers
on top of the music,
on top of the music,
32
86210
3598
01:42
I could make the feelings
even more powerful.
even more powerful.
33
90394
3252
01:45
So I got into directing music videos.
34
93670
2834
01:49
This is what they looked like.
35
97432
1474
01:54
That's my brother, Jeff.
36
102429
1243
01:56
Sorry about this, Jeff.
37
104799
1322
01:58
(Laughter)
38
106145
1087
01:59
Here's me, just so we're even.
39
107256
1947
02:02
Incredible moves.
40
110682
1203
02:04
Should've been a dancer.
41
112444
1341
02:05
(Laughter)
42
113809
1122
02:06
These experiments grew,
43
114955
1312
02:08
and in time, started
to look more like this.
to look more like this.
44
116291
2784
02:13
In both, I'm searching
for the same thing, though,
for the same thing, though,
45
121852
2640
02:16
to capture that lightning in a bottle.
46
124516
2659
02:20
Except, I'm not.
47
128192
1673
02:21
Adding moving pictures over the music
added narrative dimension, yes,
added narrative dimension, yes,
48
129889
3559
02:26
but never quite equated the power
49
134261
2524
02:28
that just raw music had for me on its own.
50
136809
2744
02:32
This is not a great thing to realize
when you've devoted your life
when you've devoted your life
51
140795
3154
02:35
and professional career
to becoming a music video director.
to becoming a music video director.
52
143973
2839
02:38
I kept asking myself,
did I take the wrong path?
did I take the wrong path?
53
146836
2301
02:42
So I started thinking: if I could
involve you, the audience, more,
involve you, the audience, more,
54
150353
3851
02:46
I might be able to make you
feel something more as well.
feel something more as well.
55
154228
3002
02:49
So Aaron Koblin and I began
auditioning new technologies
auditioning new technologies
56
157901
3185
02:53
that could put more of you
inside of the work,
inside of the work,
57
161110
3138
02:56
like your childhood home
in "The Wilderness Downtown,"
in "The Wilderness Downtown,"
58
164272
3634
03:00
your hand-drawn portraits,
in "The Johnny Cash Project,"
in "The Johnny Cash Project,"
59
168750
3760
03:05
and your interactive dreams
60
173581
1400
03:07
in "3 Dreams of Black."
61
175791
1697
03:11
We were pushing beyond the screen,
62
179469
1957
03:13
trying to connect more deeply
63
181450
1981
03:15
to people's hearts and imaginations.
64
183455
2377
03:18
But it wasn't quite enough.
65
186745
1346
03:21
It still didn't have the raw
experiential power of pure music for me.
experiential power of pure music for me.
66
189258
4799
03:27
So I started chasing a new technology
67
195443
2061
03:29
that I only had read about
in science fiction.
in science fiction.
68
197528
2699
03:32
And after years of searching,
I found a prototype.
I found a prototype.
69
200826
2409
03:35
It was a project from Nonny de la Peña
in Mark Bolas's lab in USC.
in Mark Bolas's lab in USC.
70
203871
4224
03:41
And when I tried it, I knew I'd found it.
71
209102
3068
03:45
I could taste the lightning.
72
213095
1492
03:47
It was called virtual reality.
73
215334
1650
03:50
This was it five years ago
when I ran into it.
when I ran into it.
74
218047
2256
03:53
This is what it looks like now.
75
221530
2062
03:56
I quickly started building things
in this new medium,
in this new medium,
76
224846
3905
04:00
and through that process
we realized something:
we realized something:
77
228775
2217
04:03
that VR is going to play
an incredibly important role
an incredibly important role
78
231664
3417
04:07
in the history of mediums.
79
235105
1715
04:09
In fact, it's going to be the last one.
80
237681
2717
04:13
I mean this because it's the first medium
that actually makes the jump
that actually makes the jump
81
241390
3615
04:17
from our internalization
of an author's expression
of an author's expression
82
245029
3288
04:20
of an experience,
83
248341
1670
04:22
to our experiencing it firsthand.
84
250035
2652
04:25
You look confused.
I'll explain. Don't worry.
I'll explain. Don't worry.
85
253691
2164
04:27
(Laughter)
86
255879
1394
04:29
If we go back to the origins of mediums,
87
257830
3190
04:33
by all best guesses,
88
261044
1364
04:34
it starts around a fire,
with a good story.
with a good story.
89
262432
2966
04:37
Our clan leader is telling us
90
265986
1465
04:39
about how he hunted the woolly mammoth
on the tundra that day.
on the tundra that day.
91
267475
3629
04:44
We hear his words
92
272069
1339
04:46
and translate them
into our own internal truths.
into our own internal truths.
93
274090
3273
04:50
The same thing happens
94
278839
1165
04:52
when we look at the cave painting
version of the story,
version of the story,
95
280028
2642
04:56
the book about the mammoth hunt,
96
284241
1583
04:58
the play,
97
286472
1234
05:00
the radio broadcast,
98
288109
1186
05:01
the television show
99
289749
1165
05:03
or the movie.
100
291582
1155
05:05
All of these mediums require
what we call "suspension of disbelief,"
what we call "suspension of disbelief,"
101
293532
3732
05:09
because there's a translation gap
between the reality of the story
between the reality of the story
102
297288
4058
05:13
and our consciousness
interpreting the story
interpreting the story
103
301370
3422
05:16
into our reality.
104
304816
1268
05:19
I'm using the word "consciousness"
as a feeling of reality that we get
as a feeling of reality that we get
105
307081
4076
05:23
from our senses experiencing
the world around us.
the world around us.
106
311181
3336
05:28
Virtual reality bridges that gap.
107
316580
2797
05:31
Now, you are on the tundra
hunting with the clan leader.
hunting with the clan leader.
108
319985
4213
05:36
Or you are the clan leader.
109
324222
2354
05:38
Or maybe you're even the woolly mammoth.
110
326600
2702
05:41
(Laughter)
111
329326
1635
05:46
So here's what special about VR.
112
334218
1612
05:48
In all other mediums,
113
336726
1579
05:50
your consciousness interprets the medium.
114
338329
2270
05:53
In VR, your consciousness is the medium.
115
341044
4170
05:58
So the potential for VR is enormous.
116
346399
2434
06:00
But where are we now?
117
348857
1674
06:02
What is the current state of the art?
118
350555
2586
06:06
Well,
119
354318
1150
06:08
we are here.
120
356746
1189
06:10
We are the equivalent
of year one of cinema.
of year one of cinema.
121
358724
3148
06:13
This is the Lumière Brothers film
122
361896
1626
06:15
that allegedly sent a theater full
of people running for their lives
of people running for their lives
123
363546
3359
06:18
as they thought a train
was coming toward them.
was coming toward them.
124
366929
2293
06:22
Similar to this early stage
of this medium,
of this medium,
125
370136
3276
06:25
in VR, we also have to move
past the spectacle
past the spectacle
126
373436
3541
06:29
and into the storytelling.
127
377001
1886
06:30
It took this medium decades
128
378911
1747
06:32
to figure out its preferred
language of storytelling,
language of storytelling,
129
380682
2658
06:35
in the form of a feature film.
130
383364
1776
06:37
In VR today, we're more learning grammar
131
385164
3354
06:40
than writing language.
132
388542
1291
06:42
We've made 15 films in the last year
at our VR company, Vrse,
at our VR company, Vrse,
133
390880
3551
06:46
and we've learned a few things.
134
394455
1617
06:48
We found that we have a unique,
direct path into your senses,
direct path into your senses,
135
396801
3315
06:52
your emotions, even your body.
136
400140
2809
06:55
So let me show you some things.
137
403897
1675
06:57
For the purpose of this demo,
138
405596
1398
06:59
we're going to take every direction
that you could possibly look,
that you could possibly look,
139
407018
3065
07:02
and stretch it into this giant rectangle.
140
410107
2213
07:05
OK, here we go.
141
413083
1950
07:09
So, first: camera movement
is tricky in VR.
is tricky in VR.
142
417642
4281
07:13
Done wrong, it can actually make you sick.
143
421947
2102
07:16
We found if you move the camera
at a constant speed in a straight line,
at a constant speed in a straight line,
144
424930
4499
07:21
you can actually get away with it, though.
145
429453
2217
07:23
The first day in film school,
146
431694
1727
07:25
they told me you have to learn
every single rule
every single rule
147
433445
2587
07:28
before you can break one.
148
436056
1758
07:29
We have not learned every single rule.
149
437838
2174
07:32
We've barely learned any at all,
150
440036
1528
07:33
but we're already trying to break them
151
441588
1817
07:35
to see what kind of creative things
we can accomplish.
we can accomplish.
152
443429
2529
07:37
In this shot here, where we're moving up
off the ground, I added acceleration.
off the ground, I added acceleration.
153
445982
3712
07:41
I did that because I wanted
to give you a physical sensation
to give you a physical sensation
154
449718
2833
07:44
of moving up off the ground.
155
452575
1383
07:45
In VR, I can give that to you.
156
453982
2493
07:49
(Music)
157
457797
4074
07:55
Not surprisingly, music matters a lot
in this medium as well.
in this medium as well.
158
463882
3547
07:59
It guides us how to feel.
159
467842
1874
08:02
In this project we made
with the New York Times, Zach Richter
with the New York Times, Zach Richter
160
470354
3321
08:05
and our friend, JR,
161
473699
1755
08:07
we take you up in a helicopter,
162
475478
1885
08:09
and even though you're flying
2,000 feet above Manhattan,
2,000 feet above Manhattan,
163
477387
3938
08:13
you don't feel afraid.
164
481349
1588
08:14
You feel triumphant for JR's character.
165
482961
2908
08:19
The music guides you there.
166
487022
1479
08:20
(Music)
167
488525
2000
08:29
Contrary to popular belief,
168
497364
1607
08:30
there is composition in virtual reality,
169
498995
2666
08:33
but it's completely
different than in film,
different than in film,
170
501685
2061
08:35
where you have a rectangular frame.
171
503770
1725
08:37
Composition is now
where your consciousness exists
where your consciousness exists
172
505519
2512
08:40
and how the world moves around you.
173
508055
1947
08:42
In this film, "Waves of Grace,"
which was a collaboration between Vrse,
which was a collaboration between Vrse,
174
510927
3364
08:46
the United Nations, Gabo Arora,
and Imraan Ismail,
and Imraan Ismail,
175
514315
2909
08:49
we also see the changing role
of the close-up in virtual reality.
of the close-up in virtual reality.
176
517248
3242
08:53
A close-up in VR means
you're actually close up to someone.
you're actually close up to someone.
177
521139
4213
08:57
It brings that character inside
of your personal space,
of your personal space,
178
525954
2643
09:00
a space that we'd usually reserve
for the people that we love.
for the people that we love.
179
528621
3845
09:04
And you feel an emotional
closeness to the character
closeness to the character
180
532490
3170
09:07
because of what you feel
to be a physical closeness.
to be a physical closeness.
181
535684
2708
09:16
Directing VR is not like
directing for the rectangle.
directing for the rectangle.
182
544639
4339
09:21
It's more of a choreography
of the viewer's attention.
of the viewer's attention.
183
549002
3084
09:24
One tool we can use
to guide your attention
to guide your attention
184
552864
2594
09:27
is called "spatialized sound."
185
555482
1433
09:28
I can put a sound anywhere
in front of you, to left or right,
in front of you, to left or right,
186
556939
2968
09:31
even behind you,
187
559931
1379
09:33
and when you turn your head,
the sound will rotate accordingly.
the sound will rotate accordingly.
188
561334
2996
09:36
So I can use that to direct your attention
to where I want you to see.
to where I want you to see.
189
564354
3502
09:40
Next time you hear someone
singing over your shoulder,
singing over your shoulder,
190
568308
2578
09:42
it might be Bono.
191
570910
1261
09:44
(Laughter)
192
572195
1207
09:50
VR makes us feel
like we are part of something.
like we are part of something.
193
578539
2755
09:54
For most of human history,
we lived in small family units.
we lived in small family units.
194
582459
3509
09:57
We started in caves,
195
585992
1511
09:59
then moved to clans and tribes,
then villages and towns,
then villages and towns,
196
587527
3433
10:02
and now we're all global citizens.
197
590984
2692
10:06
But I believe that we are still
hardwired to care the most
hardwired to care the most
198
594211
3800
10:10
about the things that are local to us.
199
598035
2497
10:13
And VR makes anywhere
and anyone feel local.
and anyone feel local.
200
601325
4958
10:18
That's why it works as an empathy machine.
201
606307
2508
10:20
Our film "Clouds Over Sidra"
takes you to a Syrian refugee camp,
takes you to a Syrian refugee camp,
202
608839
3224
10:24
and instead of watching a story
about people over there,
about people over there,
203
612087
4551
10:28
it's now a story about us here.
204
616662
3823
10:34
But where do we go from here?
205
622386
1510
10:36
The tricky thing is that
with all previous mediums,
with all previous mediums,
206
624432
2457
10:38
the format is fixed at its birth.
207
626913
2370
10:42
Film has been a sequence of rectangles,
208
630182
1889
10:44
from Muybridge and his horses to now.
209
632095
2839
10:47
The format has never changed.
210
635377
1550
10:49
But VR as a format, as a medium,
211
637654
3447
10:54
isn't complete yet.
212
642370
1249
10:56
It's not using physical celluloid
or paper or TV signals.
or paper or TV signals.
213
644767
3199
10:59
It actually employs what we use
to make sense of the world.
to make sense of the world.
214
647990
3819
11:04
We're using your senses
as the paints on the canvas,
as the paints on the canvas,
215
652642
3903
11:08
but only two right now.
216
656569
1322
11:11
Eventually, we can see if we will have
all of our human senses employed,
all of our human senses employed,
217
659653
3840
11:15
and we will have agency to live
the story in any path we choose.
the story in any path we choose.
218
663517
4195
11:20
And we call it virtual reality right now,
219
668575
2456
11:23
but what happens when we move
past simulated realities?
past simulated realities?
220
671055
2794
11:27
What do we call it then?
221
675345
1430
11:29
What if instead of verbally
telling you about a dream,
telling you about a dream,
222
677503
2836
11:32
I could let you live inside that dream?
223
680363
2774
11:35
What if instead of just experiencing
visiting some reality on Earth,
visiting some reality on Earth,
224
683860
3627
11:39
you could surf gravitational waves
on the edge of a black hole,
on the edge of a black hole,
225
687511
4426
11:43
or create galaxies from scratch,
226
691961
2776
11:46
or communicate with each other
not using words
not using words
227
694761
3084
11:49
but using our raw thoughts?
228
697869
1903
11:53
That's not a virtual reality anymore.
229
701254
2081
11:56
And honestly I don't know
what that's called.
what that's called.
230
704052
2346
11:58
But I hope you see where we're going.
231
706884
2332
12:02
But here I am, intellectualizing
a medium I'm saying is experiential.
a medium I'm saying is experiential.
232
710629
3722
12:06
So let's experience it.
233
714839
1426
12:08
In your hands, you hopefully hold
a piece of cardboard.
a piece of cardboard.
234
716845
3195
12:13
Let's open the flap.
235
721143
1633
12:14
Tap on the power button
to unlock the phone.
to unlock the phone.
236
722800
2533
12:18
For the people watching at home,
237
726145
1624
12:19
we're going to put up a card right now
238
727793
1836
12:21
to show you how to download
this experience on your phone yourself,
this experience on your phone yourself,
239
729653
3230
12:24
and even get a Google cardboard
of your own to try it with.
of your own to try it with.
240
732907
3301
12:29
We played in cardboard boxes as kids,
241
737275
2527
12:31
and as adults, I'm hoping we can all find
a little bit of that lightning
a little bit of that lightning
242
739826
4584
12:36
by sticking our head in one again.
243
744434
2774
12:41
You're about to participate
244
749615
1429
12:43
in the largest collective
VR viewing in history.
VR viewing in history.
245
751068
4452
12:48
And in that classic old-timey
style of yesteryear,
style of yesteryear,
246
756236
3618
12:51
we're all going to watch something
247
759878
2231
12:54
at the exact same time, together.
248
762133
1997
12:57
Let's hope it works.
249
765019
1423
12:58
What's the countdown
look like? I can't see.
look like? I can't see.
250
766466
2185
13:04
Audience: ...15, 14, 13, 12, 11, 10, 9,
251
772564
6994
13:11
8, 7, 6, 5, 4, 3, 2, 1
252
779582
6864
13:22
(Birds singing)
253
790327
3828
13:34
(Train engine)
254
802508
2660
13:45
Audience: (Shreiks)
255
813351
2000
14:03
(Video) JR: Let me tell you
256
831285
1290
14:04
how I shot the cover
of the New York Times Magazine,
of the New York Times Magazine,
257
832599
2477
14:07
"Walking New York."
258
835100
2094
14:18
I just got strapped on
outside the helicopter,
outside the helicopter,
259
846577
3410
14:22
and I had to be perfectly
vertical so I could grab it.
vertical so I could grab it.
260
850011
4244
14:26
And when I was perfectly above --
261
854279
1618
14:27
you know, with the wind,
we had to redo it a few times --
we had to redo it a few times --
262
855921
2737
14:30
then I kept shooting.
263
858682
1794
14:39
(Video) Woman's voice: Dear Lord,
264
867598
1698
14:41
protect us from evil,
265
869900
1320
14:44
for you are the Lord,
266
872654
1211
14:47
the light.
267
875044
1187
14:53
You who gave us life took it away.
268
881824
2383
14:58
Let your will be done.
269
886191
1580
15:00
Please bring peace to the many
who have lost loved ones.
who have lost loved ones.
270
888941
4564
15:05
Help us to live again.
271
893529
1743
15:10
(Music)
272
898564
3119
15:29
(Video) (Children's voices)
273
917066
1461
15:31
Child's voice: There are more kids
in Zaatari than adults right now.
in Zaatari than adults right now.
274
919504
4665
15:41
Sometimes I think
275
929924
2029
15:43
we are the ones in charge.
276
931977
2434
15:51
Chris Milk: How was it?
277
939410
1375
15:52
(Applause)
278
940809
3715
15:56
That was a cheap way of getting you
to do a standing ovation.
to do a standing ovation.
279
944548
2929
15:59
I just made you all stand.
I knew you'd applaud at the end.
I knew you'd applaud at the end.
280
947501
2808
16:02
(Applause)
281
950333
1641
16:03
I believe that everyone on Earth
needs to experience
needs to experience
282
951998
3819
16:07
what you just experienced.
283
955841
1580
16:10
That way we can collectively
start to shape this,
start to shape this,
284
958036
3167
16:13
not as a tech platform
285
961227
1774
16:15
but as a humanity platform.
286
963025
1575
16:17
And to that end, in November of last year,
287
965498
3184
16:20
the New York Times and Vrse made
a VR project called "The Displaced."
a VR project called "The Displaced."
288
968706
3286
16:24
It launched with one million
Google Cardboards
Google Cardboards
289
972016
2346
16:26
sent out to every Sunday subscriber
with their newspaper.
with their newspaper.
290
974386
3473
16:29
But a funny thing happened
that Sunday morning.
that Sunday morning.
291
977883
2518
16:32
A lot of people got them
292
980425
1322
16:33
that were not the intended recipients
on the mailing label.
on the mailing label.
293
981771
4206
16:38
And we started seeing this
all over Instagram.
all over Instagram.
294
986001
3544
16:45
Look familiar?
295
993059
1180
16:48
Music led me on a path
296
996252
2143
16:50
of searching for what seemed
like the unattainable
like the unattainable
297
998419
3049
16:53
for a very long time.
298
1001492
1253
16:55
Now, millions of kids just had
the same formative experience
the same formative experience
299
1003466
3971
16:59
in their childhood
300
1007461
1741
17:01
that I had in mine.
301
1009226
2304
17:04
Only I think this one
302
1012316
2229
17:06
surpasses it.
303
1014569
1158
17:08
Let's see
304
1016358
1150
17:09
where this
305
1017859
1150
17:11
leads them.
306
1019573
1150
17:13
Thank you.
307
1021384
1151
17:14
(Applause)
308
1022559
6482
ABOUT THE SPEAKER
Chris Milk - Immersive storytellerWorking at the frontiers of interactive technology, Chris Milk stretches virtual reality into a new canvas for storytelling.
Why you should listen
Chris Milk is a visual artist who has created music videos for Kanye West, Arcade Fire, Beck, U2, Johnny Cash, Gnarls Barkley and many more. He is known for weaving artistic and technological innovations in pursuit of the next great platform for storytelling. Milk's acclaimed interactive projects include Wilderness Downtown (with Arcade Fire), The Johnny Cash Project and The Treachery of Sanctuary. His interactive installation artworks have been showcased at the MoMA, the Tate Modern and museums around the world.
Milk's most recent contribution to the art and tech frontier is as founder and CEO of the virtual reality company Within (formerly Vrse). In collaboration with the New York Times, Zach Richter and JR, Milk created two VR films, Walking New York and The Displaced, which were distributed along with Google Cardboard viewers to 1 million NYT subscribers in 2015. He has also collaborated on VR projects with the United Nations (Clouds Over Sidra and Waves of Grace), Vice, SNL and U2.
Chris Milk | Speaker | TED.com