EG 2007
John Q. Walker: Great piano performances, recreated
Filmed:
Readability: 3.6
372,416 views
Imagine hearing great, departed pianists play again today, just as they would in person. John Q. Walker demonstrates how recordings can be analyzed for precise keystrokes and pedal motions, then played back on computer-controlled grand pianos.
John Q. Walker - Musician and inventor
Software entrepreneur John Q. Walker uses computers to bring piano legends back to life -- digitally reconstructing their performances from audio tracks and playing them on real instruments, live. Full bio
Software entrepreneur John Q. Walker uses computers to bring piano legends back to life -- digitally reconstructing their performances from audio tracks and playing them on real instruments, live. Full bio
Double-click the English transcript below to play the video.
00:16
Ninety-nine percent of us have the dream of listeners.
0
0
4000
00:20
Not being the musicians -- the listeners, right?
1
4000
2000
00:22
And we crave one thing,
2
6000
2000
00:24
even though we kind of don't know it all the time.
3
8000
2000
00:26
We crave to be in the room with the musician
4
10000
5000
00:31
the day it was recorded, the day it was played.
5
15000
2000
00:33
And we go to live concerts, and we get that as much as we can.
6
17000
3000
00:36
But then we listen to the other 99 percent of our stuff recorded.
7
20000
3000
00:39
And it turns out the further back you go in history,
8
23000
3000
00:42
the little rougher it sounds.
9
26000
2000
00:44
And so we said, there's a solution to this.
10
28000
4000
00:48
Let's separate the performance, as a thing,
11
32000
4000
00:52
out from the recording, which was how it was made.
12
36000
3000
00:55
You know, the thing with microphones in the room and all that day.
13
39000
3000
00:58
But the performance itself was how the musicians worked their fingers,
14
42000
3000
01:01
and what instruments they were using.
15
45000
2000
01:03
And it's the data hidden inside the recording.
16
47000
5000
01:08
In order to do this, it's a lot of hardware and software
17
52000
3000
01:11
that runs in a very high resolution.
18
55000
2000
01:13
And Yamaha makes an incredible thing called the Disklavier Pro
19
57000
2000
01:17
that looks like a nice grand piano there.
20
61000
2000
01:19
And you probably didn't realize it's going to do all these things --
21
63000
2000
01:21
but full of solenoids, and fiber optics, and computers
22
65000
3000
01:24
and all this kind of stuff. The highest resolution out of Japan.
23
68000
3000
01:27
And this just didn't work until we could cross this line that says high-definition.
24
71000
4000
01:31
And we were able to cross this line, called the uncanny valley,
25
75000
3000
01:34
in terms of -- artificial intelligence terms.
26
78000
5000
01:39
We have a process where we, you know,
27
83000
2000
01:41
kind of put it into the computer and digitize it, and then a whole lot of analysis.
28
85000
3000
01:44
And we look at every single note,
29
88000
2000
01:46
and all the attributes of those notes:
30
90000
2000
01:48
how hard they were struck, and how they were held down,
31
92000
2000
01:50
and how you move the fingers.
32
94000
2000
01:52
So we had to develop a whole new science of how you move your fingers.
33
96000
2000
01:54
And, you know, it's a thing your piano teacher teaches you,
34
98000
3000
01:57
but we never had a science behind these kinds of things.
35
101000
3000
02:00
I'm going to start with Glenn Gould.
36
104000
2000
02:02
He died 25 years ago this year, and was born 75 years ago this year.
37
106000
4000
02:06
Was a beloved pianist, maybe the great cult pianist of the twentieth century.
38
110000
4000
02:10
He just got tired of being in front of an audience,
39
114000
2000
02:12
and felt like -- a performing monkey was, in fact, his term.
40
116000
2000
02:14
So he stepped back, and did nothing but the crafting of his work.
41
118000
4000
02:18
And Gould's specialty was playing Bach.
42
122000
2000
02:20
His maybe most famous recording
43
124000
2000
02:22
was something called "The Goldberg Variations."
44
126000
2000
02:24
Bach only wrote themes and variations one time.
45
128000
3000
02:27
He wrote some early pieces,
46
131000
2000
02:29
but late in his life, in his mature period,
47
133000
2000
02:31
he said, "Here's a theme -- 30 variations."
48
135000
3000
02:34
In fact, the theme isn't even the melody, it's the bass line.
49
138000
3000
02:37
And Gould recorded it in two major recordings that you may know about,
50
141000
5000
02:42
one in mono, and one in stereo.
51
146000
2000
02:44
And the one in mono, by the way, he used the pedal,
52
148000
3000
02:47
and as he got older, he said, "No, no, wait a minute.
53
151000
3000
02:50
I'm going to get very scientific about this, and not use the pedal."
54
154000
3000
02:53
What I'd like you to hear live is the 1955 version,
55
157000
4000
02:57
and we'll play the first couple pieces of it.
56
161000
2000
02:59
Glenn Gould, 1955.
57
163000
3000
03:02
(Music)
58
166000
153000
05:35
How about that?
59
319000
2000
05:37
(Applause)
60
321000
4000
05:41
So let me tell you a little bit how this was done.
61
325000
2000
05:43
First of all, let me get you to the end step.
62
327000
2000
05:45
This is -- we have a fairly complex process
63
329000
2000
05:47
that, you know, software and musicians and so on,
64
331000
3000
05:50
but when we're all done, we know that the ear is the final arbiter.
65
334000
5000
05:55
We can play the original in one ear, and a new recording in the other.
66
339000
3000
05:58
So I'm going to do this for you right now, what you just heard.
67
342000
3000
06:01
And in the right speaker is going to be the original recording,
68
345000
2000
06:03
and the left speaker is going to be the new recording,
69
347000
3000
06:06
actually of an instrument just like that one,
70
350000
3000
06:09
and I'm going to play them together at the same time.
71
353000
2000
06:11
(Music)
72
355000
6000
06:17
That's the original. [Unclear] That's the two together.
73
361000
2000
06:19
(Music)
74
363000
36000
06:55
Before "Jurassic Park,"
75
399000
2000
06:57
there was no science for how skin hung off of muscle, right?
76
401000
6000
07:03
So, in the video world,
77
407000
1000
07:04
we've been able to invent, in our lifetimes, natural behavior.
78
408000
3000
07:07
And this is kind of another example
79
411000
2000
07:09
of putting a science behind natural behavior.
80
413000
3000
07:12
And then you heard the original.
81
416000
2000
07:14
Ultimately, I started with the experience.
82
418000
2000
07:16
And the experience is: I want to be in the room and hear the musicians.
83
420000
4000
07:20
Lots of you can afford to buy one of these.
84
424000
2000
07:22
But, if not, there is now high-definition surround sound.
85
426000
6000
07:28
And I got to tell you, if you haven't heard high-definition surround,
86
432000
3000
07:31
go down to your audio dealer, your audiophile dealer.
87
435000
2000
07:33
It's so involving compared to regular stereo.
88
437000
4000
07:37
But if you don't have that, maybe you can listen on your headphones.
89
441000
2000
07:39
And so on the same disk we have five recordings --
90
443000
2000
07:41
Sony has five recordings.
91
445000
2000
07:43
And you could listen in headphones
92
447000
2000
07:45
with this thing called binaural recording.
93
449000
3000
07:48
And it's a dummy head that sits in front of the instrument,
94
452000
4000
07:52
and it's got microphones where the ears are.
95
456000
4000
07:56
And when you put on headphones, and you listen to this,
96
460000
2000
07:58
you're inside of Glenn Gould's body.
97
462000
2000
08:00
And it is a chuckle until, you know, the musicians,
98
464000
3000
08:03
who are musicians who play the piano, listen to this, say,
99
467000
2000
08:05
"I can't believe it! It's just what it's like to play the piano."
100
469000
2000
08:07
Except now you're inside Glenn Gould's body playing the piano,
101
471000
2000
08:09
and it feels like your fingers are making the decisions
102
473000
2000
08:11
and moving through the whole process.
103
475000
2000
08:13
It's a game changer.
104
477000
2000
08:15
Here's now something we know in spectacular quality.
105
479000
2000
08:17
The whole process is very sensitive to temperature and humidity.
106
481000
4000
08:21
What you heard today was not perfect.
107
485000
2000
08:23
It's an amalgam of wood, and cast iron, and felt,
108
487000
4000
08:27
and steel strings, and all these,
109
491000
2000
08:29
and they're all amazingly sensitive to temperature and humidity.
110
493000
3000
08:32
So when you go into the recording session,
111
496000
3000
08:35
you get to stop after every piece and rebuild the piano if you need to.
112
499000
4000
08:39
There's the whole action there, sitting, kind of, on the side,
113
503000
2000
08:41
and the dummy head and our recording engineers
114
505000
2000
08:43
standing around while we rebuild the piano.
115
507000
2000
08:45
Without putting dates next to these things,
116
509000
2000
08:47
step-by-step music will be turned into data,
117
511000
3000
08:50
like every field that's occurred in the past 35 or 40 years.
118
514000
4000
08:54
Audio has come very late to this game --
119
518000
3000
08:57
I'm not talking about digitizing, and bits, and re-mastering.
120
521000
3000
09:00
I'm talking about turn it into the data that it was made from,
121
524000
3000
09:03
which is how it was performed.
122
527000
2000
09:05
And audio came very late because our ears are so hard to fool --
123
529000
3000
09:08
they're high-resolution, and they're wired straight to our emotions,
124
532000
3000
09:11
and you can't trick them very easily.
125
535000
2000
09:13
Your eyes are pretty happy with some color and movement, you know.
126
537000
3000
09:16
All right, there's this episode of "Star Trek."
127
540000
3000
09:19
(Laughter)
128
543000
3000
09:22
I get it -- it was all just laid in for me yesterday there.
129
546000
2000
09:24
The episode of "Star Trek" for me was James Daly played Methuselah --
130
548000
4000
09:28
remember this one?
131
552000
2000
09:30
And at some point he's dancing with his --
132
554000
3000
09:33
and I won't ruin the episode for you, from 1967.
133
557000
2000
09:35
Right, do you know where I'm going?
134
559000
2000
09:37
And Nimoy, I'm sorry, Spock sits down at the piano,
135
561000
2000
09:39
and he starts playing this Brahms waltz, and they all dance to it.
136
563000
4000
09:43
And then Spock turns round, he goes,
137
567000
2000
09:45
"James, I know all of the Brahms waltzes,
138
569000
5000
09:50
and I don't believe this is one of them in the category."
139
574000
2000
09:54
That's where I'm at.
140
578000
2000
09:56
I want to hear the waltzes Brahms didn't write.
141
580000
4000
10:00
I want to hear the pieces that Horowitz didn't play.
142
584000
4000
10:04
But I believe we're on a path now, when we get to data,
143
588000
5000
10:09
that we can distill styles, and templates, and formulas, and all these kinds of things,
144
593000
4000
10:13
again, that you've seen happen in the computer graphics world.
145
597000
3000
10:16
It's now coming in this world.
146
600000
2000
10:18
The transition will be this one.
147
602000
2000
10:20
It says right now, we think music is notes and how they're played.
148
604000
3000
10:23
And I believe this is coming.
149
607000
2000
10:25
Because what you've just heard was a computer playing data --
150
609000
5000
10:30
no Glenn Gould in the room.
151
614000
2000
10:32
But yet, it was human.
152
616000
2000
10:34
And I believe you'll get to the next step, the real dream of listeners.
153
618000
3000
10:37
Every time you listen to a recording today,
154
621000
3000
10:40
every time you take out your iPod and whatever,
155
624000
2000
10:42
every time you listen to it, it's the same thing -- it's frozen.
156
626000
5000
10:47
Wouldn't it be cool if every time you listened, it could be different?
157
631000
4000
10:51
This morning, you're sadder, you want to hear your song,
158
635000
3000
10:54
the same song, played sadder than you did yesterday.
159
638000
2000
10:56
You want to hear it played by different musicians.
160
640000
2000
10:58
You want to hear it in different rooms and whatever.
161
642000
2000
11:01
We've seen all these "Star Treks," and they're all holodeck episodes as well.
162
645000
2000
11:03
Every time I listen to that, I get goose bumps.
163
647000
3000
11:06
It's so amazing, it's so exciting.
164
650000
3000
11:09
Every time I listen to that recording it's like, "Oh my God,
165
653000
2000
11:11
I can't believe I'm in the same room. I can't believe this is happening."
166
655000
2000
11:13
It's a way better experience
167
657000
2000
11:15
than whatever you're used to listening to, in whatever form.
168
659000
2000
11:17
And lastly, I will wrap up with one minute of Art Tatum.
169
661000
5000
11:22
So I've really overshot my budget here.
170
666000
2000
11:24
We made a new recording of him
171
668000
3000
11:27
playing in the Shrine Auditorium in September.
172
671000
3000
11:30
It was a concert he recorded in the Shrine Auditorium in 1949.
173
674000
4000
11:34
And I've got to tell you, we have this lab
174
678000
2000
11:36
where we build and measure everything, back in Raleigh, North Carolina,
175
680000
3000
11:39
and we flew out to Los Angeles.
176
683000
3000
11:42
And as the president of the company,
177
686000
2000
11:44
I didn't feel real comfortable about where we were.
178
688000
3000
11:47
That's a real uncomfortable feeling,
179
691000
2000
11:49
when all the equipment's come out and a whole Sony team,
180
693000
2000
11:51
and people are going to be sitting there in the audience.
181
695000
2000
11:53
And we put the piano on the sweet spot of the stage in the Shrine,
182
697000
4000
11:57
which has not changed since 1949, still seats 6,000 people.
183
701000
4000
12:01
And on the sweet spot on the stage, Tatum starts playing ...
184
705000
3000
12:04
and every note, every beat, every slur, every accent, every pedal
185
708000
6000
12:10
was perfect, because he played it for that room on that day.
186
714000
4000
12:14
And we captured all that data all over again.
187
718000
2000
12:16
And I want you to hear that right now.
188
720000
2000
12:18
And fortunately, it's right in here.
189
722000
2000
12:20
This is an encore he used to do.
190
724000
3000
12:23
It's one minute long.
191
727000
2000
12:25
It's an Irish jig, and I want you to hear his humor.
192
729000
2000
12:27
(Music)
193
731000
56000
13:23
(Applause)
194
787000
1000
13:24
And that's just what the live audience did.
195
788000
3000
13:27
(Applause)
196
791000
2000
13:29
So thank you very much, Michael, thank you for the opportunity.
197
793000
3000
ABOUT THE SPEAKER
John Q. Walker - Musician and inventorSoftware entrepreneur John Q. Walker uses computers to bring piano legends back to life -- digitally reconstructing their performances from audio tracks and playing them on real instruments, live.
Why you should listen
Glenn Gould, Thelonious Monk, Art Tatum: legends of live piano performance lost to time. But John Q. Walker asks us to imagine hearing those great, departed musicians play again today, just as they would in person. Such is the promise of Walker's company, Zenph Studios, which builds technology to re-create live music performances. Piano tracks are converted into precise keystrokes and pedal motions -- then played back on computer-controlled grand pianos.
Before founding Zenph in 2002, Walker was a leading developer of VoIP, and was influential in the creation of the IEEE 802 local-area network (LAN) and the 802.11 wireless LAN (“Wi-Fi”) standards. He is himself a devoted pianist.
John Q. Walker | Speaker | TED.com