sponsored links
TED2016

Chris Milk: The birth of virtual reality as an art form

February 18, 2016

Chris Milk uses innovative technologies to make personal, interactive, human stories. Accompanied by Joshua Roman on cello and McKenzie Stubbert on piano, Milk traces his relationship to music and art -- from the first moment he remembers putting on headphones to his current work creating breakthrough virtual reality projects. VR is the last medium for storytelling, he says, because it closes the gap between audience and storyteller. To illustrate, he brought the TED audience together in the world's largest collective VR experience. Join them and take part in this interactive talk by getting a Google Cardboard and downloading the experience at with.in/TED.

Chris Milk - Immersive storyteller
Working at the frontiers of interactive technology, Chris Milk stretches virtual reality into a new canvas for storytelling. Full bio

sponsored links
Double-click the English subtitles below to play the video.
When I was a kid,
00:13
I experienced something so powerful,
00:15
I spent the rest of my life
searching for it,
00:17
and in all the wrong places.
00:20
What I experienced wasn't virtual reality.
00:23
It was music.
00:26
And this is where the story begins.
00:28
That's me,
00:31
listening to the Beatles' "White Album."
00:33
And the look on my face is the feeling
00:34
that I've been searching for ever since.
00:38
Music goes straight to the emotional vein,
00:41
into your bloodstream
00:44
and right into your heart.
00:45
It deepens every experience.
00:47
Fellas?
00:51
(Music)
00:53
This is the amazing McKenzie Stubbert
00:59
and Joshua Roman.
01:01
Music --
01:03
(Applause)
01:04
Yeah.
01:05
Music makes everything
have more emotional resonance.
01:08
Let's see how it does for this talk.
01:13
The right piece of music
at the right time fuses with us
01:15
on a cellular level.
01:19
When I hear that one song
01:21
from that one summer
01:24
with that one girl,
01:26
I'm instantly transported
back there again.
01:28
Hey, Stacey.
01:31
Here's a part of the story, though,
where I got a little greedy.
01:34
I thought if I added more layers
on top of the music,
01:38
I could make the feelings
even more powerful.
01:42
So I got into directing music videos.
01:45
This is what they looked like.
01:49
That's my brother, Jeff.
01:54
Sorry about this, Jeff.
01:56
(Laughter)
01:57
Here's me, just so we're even.
01:59
Incredible moves.
02:02
Should've been a dancer.
02:04
(Laughter)
02:05
These experiments grew,
02:06
and in time, started
to look more like this.
02:08
In both, I'm searching
for the same thing, though,
02:13
to capture that lightning in a bottle.
02:16
Except, I'm not.
02:20
Adding moving pictures over the music
added narrative dimension, yes,
02:21
but never quite equated the power
02:26
that just raw music had for me on its own.
02:28
This is not a great thing to realize
when you've devoted your life
02:32
and professional career
to becoming a music video director.
02:35
I kept asking myself,
did I take the wrong path?
02:38
So I started thinking: if I could
involve you, the audience, more,
02:42
I might be able to make you
feel something more as well.
02:46
So Aaron Koblin and I began
auditioning new technologies
02:49
that could put more of you
inside of the work,
02:52
like your childhood home
in "The Wilderness Downtown,"
02:56
your hand-drawn portraits,
in "The Johnny Cash Project,"
03:00
and your interactive dreams
03:05
in "3 Dreams of Black."
03:07
We were pushing beyond the screen,
03:11
trying to connect more deeply
03:13
to people's hearts and imaginations.
03:15
But it wasn't quite enough.
03:18
It still didn't have the raw
experiential power of pure music for me.
03:21
So I started chasing a new technology
03:27
that I only had read about
in science fiction.
03:29
And after years of searching,
I found a prototype.
03:32
It was a project from Nonny de la Peña
in Mark Bolas's lab in USC.
03:35
And when I tried it, I knew I'd found it.
03:40
I could taste the lightning.
03:44
It was called virtual reality.
03:47
This was it five years ago
when I ran into it.
03:49
This is what it looks like now.
03:53
I quickly started building things
in this new medium,
03:56
and through that process
we realized something:
04:00
that VR is going to play
an incredibly important role
04:03
in the history of mediums.
04:06
In fact, it's going to be the last one.
04:09
I mean this because it's the first medium
that actually makes the jump
04:13
from our internalization
of an author's expression
04:16
of an experience,
04:20
to our experiencing it firsthand.
04:21
You look confused.
I'll explain. Don't worry.
04:25
(Laughter)
04:27
If we go back to the origins of mediums,
04:29
by all best guesses,
04:32
it starts around a fire,
with a good story.
04:34
Our clan leader is telling us
04:37
about how he hunted the woolly mammoth
on the tundra that day.
04:39
We hear his words
04:43
and translate them
into our own internal truths.
04:45
The same thing happens
04:50
when we look at the cave painting
version of the story,
04:51
the book about the mammoth hunt,
04:56
the play,
04:58
the radio broadcast,
04:59
the television show
05:01
or the movie.
05:03
All of these mediums require
what we call "suspension of disbelief,"
05:05
because there's a translation gap
between the reality of the story
05:09
and our consciousness
interpreting the story
05:13
into our reality.
05:16
I'm using the word "consciousness"
as a feeling of reality that we get
05:18
from our senses experiencing
the world around us.
05:23
Virtual reality bridges that gap.
05:28
Now, you are on the tundra
hunting with the clan leader.
05:31
Or you are the clan leader.
05:36
Or maybe you're even the woolly mammoth.
05:38
(Laughter)
05:41
So here's what special about VR.
05:46
In all other mediums,
05:48
your consciousness interprets the medium.
05:50
In VR, your consciousness is the medium.
05:52
So the potential for VR is enormous.
05:58
But where are we now?
06:00
What is the current state of the art?
06:02
Well,
06:06
we are here.
06:08
We are the equivalent
of year one of cinema.
06:10
This is the Lumière Brothers film
06:13
that allegedly sent a theater full
of people running for their lives
06:15
as they thought a train
was coming toward them.
06:18
Similar to this early stage
of this medium,
06:21
in VR, we also have to move
past the spectacle
06:25
and into the storytelling.
06:28
It took this medium decades
06:30
to figure out its preferred
language of storytelling,
06:32
in the form of a feature film.
06:35
In VR today, we're more learning grammar
06:36
than writing language.
06:40
We've made 15 films in the last year
at our VR company, Vrse,
06:42
and we've learned a few things.
06:46
We found that we have a unique,
direct path into your senses,
06:48
your emotions, even your body.
06:51
So let me show you some things.
06:55
For the purpose of this demo,
06:57
we're going to take every direction
that you could possibly look,
06:58
and stretch it into this giant rectangle.
07:01
OK, here we go.
07:04
So, first: camera movement
is tricky in VR.
07:09
Done wrong, it can actually make you sick.
07:13
We found if you move the camera
at a constant speed in a straight line,
07:16
you can actually get away with it, though.
07:21
The first day in film school,
07:23
they told me you have to learn
every single rule
07:25
before you can break one.
07:27
We have not learned every single rule.
07:29
We've barely learned any at all,
07:31
but we're already trying to break them
07:33
to see what kind of creative things
we can accomplish.
07:35
In this shot here, where we're moving up
off the ground, I added acceleration.
07:37
I did that because I wanted
to give you a physical sensation
07:41
of moving up off the ground.
07:44
In VR, I can give that to you.
07:45
(Music)
07:49
Not surprisingly, music matters a lot
in this medium as well.
07:55
It guides us how to feel.
07:59
In this project we made
with the New York Times, Zach Richter
08:02
and our friend, JR,
08:05
we take you up in a helicopter,
08:07
and even though you're flying
2,000 feet above Manhattan,
08:09
you don't feel afraid.
08:13
You feel triumphant for JR's character.
08:14
The music guides you there.
08:18
(Music)
08:20
Contrary to popular belief,
08:29
there is composition in virtual reality,
08:30
but it's completely
different than in film,
08:33
where you have a rectangular frame.
08:35
Composition is now
where your consciousness exists
08:37
and how the world moves around you.
08:39
In this film, "Waves of Grace,"
which was a collaboration between Vrse,
08:42
the United Nations, Gabo Arora,
and Imraan Ismail,
08:46
we also see the changing role
of the close-up in virtual reality.
08:49
A close-up in VR means
you're actually close up to someone.
08:52
It brings that character inside
of your personal space,
08:57
a space that we'd usually reserve
for the people that we love.
09:00
And you feel an emotional
closeness to the character
09:04
because of what you feel
to be a physical closeness.
09:07
Directing VR is not like
directing for the rectangle.
09:16
It's more of a choreography
of the viewer's attention.
09:20
One tool we can use
to guide your attention
09:24
is called "spatialized sound."
09:27
I can put a sound anywhere
in front of you, to left or right,
09:28
even behind you,
09:31
and when you turn your head,
the sound will rotate accordingly.
09:33
So I can use that to direct your attention
to where I want you to see.
09:36
Next time you hear someone
singing over your shoulder,
09:40
it might be Bono.
09:42
(Laughter)
09:44
VR makes us feel
like we are part of something.
09:50
For most of human history,
we lived in small family units.
09:54
We started in caves,
09:57
then moved to clans and tribes,
then villages and towns,
09:59
and now we're all global citizens.
10:02
But I believe that we are still
hardwired to care the most
10:06
about the things that are local to us.
10:09
And VR makes anywhere
and anyone feel local.
10:13
That's why it works as an empathy machine.
10:18
Our film "Clouds Over Sidra"
takes you to a Syrian refugee camp,
10:20
and instead of watching a story
about people over there,
10:23
it's now a story about us here.
10:28
But where do we go from here?
10:34
The tricky thing is that
with all previous mediums,
10:36
the format is fixed at its birth.
10:38
Film has been a sequence of rectangles,
10:42
from Muybridge and his horses to now.
10:43
The format has never changed.
10:47
But VR as a format, as a medium,
10:49
isn't complete yet.
10:54
It's not using physical celluloid
or paper or TV signals.
10:56
It actually employs what we use
to make sense of the world.
10:59
We're using your senses
as the paints on the canvas,
11:04
but only two right now.
11:08
Eventually, we can see if we will have
all of our human senses employed,
11:11
and we will have agency to live
the story in any path we choose.
11:15
And we call it virtual reality right now,
11:20
but what happens when we move
past simulated realities?
11:22
What do we call it then?
11:27
What if instead of verbally
telling you about a dream,
11:29
I could let you live inside that dream?
11:32
What if instead of just experiencing
visiting some reality on Earth,
11:35
you could surf gravitational waves
on the edge of a black hole,
11:39
or create galaxies from scratch,
11:43
or communicate with each other
not using words
11:46
but using our raw thoughts?
11:49
That's not a virtual reality anymore.
11:53
And honestly I don't know
what that's called.
11:55
But I hope you see where we're going.
11:58
But here I am, intellectualizing
a medium I'm saying is experiential.
12:02
So let's experience it.
12:06
In your hands, you hopefully hold
a piece of cardboard.
12:08
Let's open the flap.
12:12
Tap on the power button
to unlock the phone.
12:14
For the people watching at home,
12:17
we're going to put up a card right now
12:19
to show you how to download
this experience on your phone yourself,
12:21
and even get a Google cardboard
of your own to try it with.
12:24
We played in cardboard boxes as kids,
12:29
and as adults, I'm hoping we can all find
a little bit of that lightning
12:31
by sticking our head in one again.
12:36
You're about to participate
12:41
in the largest collective
VR viewing in history.
12:42
And in that classic old-timey
style of yesteryear,
12:48
we're all going to watch something
12:51
at the exact same time, together.
12:53
Let's hope it works.
12:56
What's the countdown
look like? I can't see.
12:58
Audience: ...15, 14, 13, 12, 11, 10, 9,
13:04
8, 7, 6, 5, 4, 3, 2, 1
13:11
(Birds singing)
13:22
(Train engine)
13:34
Audience: (Shreiks)
13:45
(Video) JR: Let me tell you
14:03
how I shot the cover
of the New York Times Magazine,
14:04
"Walking New York."
14:06
I just got strapped on
outside the helicopter,
14:18
and I had to be perfectly
vertical so I could grab it.
14:21
And when I was perfectly above --
14:26
you know, with the wind,
we had to redo it a few times --
14:27
then I kept shooting.
14:30
(Video) Woman's voice: Dear Lord,
14:39
protect us from evil,
14:41
for you are the Lord,
14:44
the light.
14:46
You who gave us life took it away.
14:53
Let your will be done.
14:58
Please bring peace to the many
who have lost loved ones.
15:00
Help us to live again.
15:05
(Music)
15:10
(Video) (Children's voices)
15:28
Child's voice: There are more kids
in Zaatari than adults right now.
15:31
Sometimes I think
15:41
we are the ones in charge.
15:43
Chris Milk: How was it?
15:51
(Applause)
15:52
That was a cheap way of getting you
to do a standing ovation.
15:56
I just made you all stand.
I knew you'd applaud at the end.
15:59
(Applause)
16:02
I believe that everyone on Earth
needs to experience
16:03
what you just experienced.
16:07
That way we can collectively
start to shape this,
16:09
not as a tech platform
16:13
but as a humanity platform.
16:14
And to that end, in November of last year,
16:17
the New York Times and Vrse made
a VR project called "The Displaced."
16:20
It launched with one million
Google Cardboards
16:23
sent out to every Sunday subscriber
with their newspaper.
16:26
But a funny thing happened
that Sunday morning.
16:29
A lot of people got them
16:32
that were not the intended recipients
on the mailing label.
16:33
And we started seeing this
all over Instagram.
16:37
Look familiar?
16:44
Music led me on a path
16:48
of searching for what seemed
like the unattainable
16:50
for a very long time.
16:53
Now, millions of kids just had
the same formative experience
16:55
in their childhood
16:59
that I had in mine.
17:01
Only I think this one
17:04
surpasses it.
17:06
Let's see
17:08
where this
17:09
leads them.
17:11
Thank you.
17:13
(Applause)
17:14

sponsored links

Chris Milk - Immersive storyteller
Working at the frontiers of interactive technology, Chris Milk stretches virtual reality into a new canvas for storytelling.

Why you should listen

Chris Milk is a visual artist who has created music videos for Kanye West, Arcade Fire, Beck, U2, Johnny Cash, Gnarls Barkley and many more. He is known for weaving artistic and technological innovations in pursuit of the next great platform for storytelling. Milk's acclaimed interactive projects include Wilderness Downtown (with Arcade Fire), The Johnny Cash Project and The Treachery of Sanctuary. His interactive installation artworks have been showcased at the MoMA, the Tate Modern and museums around the world.

Milk's most recent contribution to the art and tech frontier is as founder and CEO of the virtual reality company Within (formerly Vrse). In collaboration with the New York Times, Zach Richter and JR, Milk created two VR films, Walking New York and The Displaced, which were distributed along with Google Cardboard viewers to 1 million NYT subscribers in 2015. He has also collaborated on VR projects with the United Nations (Clouds Over Sidra and Waves of Grace), Vice, SNL and U2.

sponsored links

If you need translations, you can install "Google Translate" extension into your Chrome Browser.
Furthermore, you can change playback rate by installing "Video Speed Controller" extension.

Data provided by TED.

This website is owned and operated by Tokyo English Network.
The developer's blog is here.