TED2012
Peter Norvig: The 100,000-student classroom
Filmed:
Readability: 4.4
1,166,568 views
In the fall of 2011 Peter Norvig taught a class with Sebastian Thrun on artificial intelligence at Stanford attended by 175 students in situ -- and over 100,000 via an interactive webcast. He shares what he learned about teaching to a global classroom.
Peter Norvig - Computer scientist
Peter Norvig is a leading American computer scientist, expert on artificial intelligence and the Director of Research at Google Inc. Full bio
Peter Norvig is a leading American computer scientist, expert on artificial intelligence and the Director of Research at Google Inc. Full bio
Double-click the English transcript below to play the video.
00:17
Everyone is both a learner
0
1135
1976
00:19
and a teacher.
1
3111
1512
00:20
This is me being inspired
2
4623
2456
00:22
by my first tutor,
3
7079
1191
00:24
my mom,
4
8270
945
00:25
and this is me teaching
5
9215
2623
00:27
Introduction to Artificial Intelligence
6
11838
1953
00:29
to 200 students
7
13791
1240
00:30
at Stanford University.
8
15031
1272
00:32
Now the students and I
9
16303
1456
00:33
enjoyed the class,
10
17759
1176
00:34
but it occurred to me
11
18935
1416
00:36
that while the subject matter
12
20351
1711
00:37
of the class is advanced
13
22062
1289
00:39
and modern,
14
23351
688
00:39
the teaching technology isn't.
15
24039
2319
00:42
In fact, I use basically
16
26358
2336
00:44
the same technology as
17
28694
2370
00:46
this 14th-century classroom.
18
31064
2630
00:49
Note the textbook,
19
33694
2800
00:52
the sage on the stage,
20
36494
2944
00:55
and the sleeping guy
21
39438
1899
00:57
in the back. (Laughter)
22
41337
901
00:58
Just like today.
23
42238
2937
01:01
So my co-teacher,
24
45175
3111
01:04
Sebastian Thrun, and I thought,
25
48286
1640
01:05
there must be a better way.
26
49926
1774
01:07
We challenged ourselves
27
51700
1654
01:09
to create an online class
28
53354
1291
01:10
that would be equal or better
29
54645
1631
01:12
in quality to our Stanford class,
30
56276
2336
01:14
but to bring it to anyone
31
58612
2210
01:16
in the world for free.
32
60822
1630
01:18
We announced the class on July 29th,
33
62452
2568
01:20
and within two weeks, 50,000 people
34
65020
3209
01:24
had signed up for it.
35
68229
1407
01:25
And that grew to 160,000 students
36
69636
3105
01:28
from 209 countries.
37
72741
2064
01:30
We were thrilled to have
38
74805
1808
01:32
that kind of audience,
39
76613
1121
01:33
and just a bit terrified that we
40
77734
2639
01:36
hadn't finished preparing the class yet. (Laughter)
41
80373
2311
01:38
So we got to work.
42
82684
1612
01:40
We studied what others had done,
43
84296
1636
01:41
what we could copy and what we could change.
44
85932
2323
01:44
Benjamin Bloom had showed
45
88255
2661
01:46
that one-on-one tutoring works best,
46
90916
1809
01:48
so that's what we tried to emulate,
47
92725
2128
01:50
like with me and my mom,
48
94853
1440
01:52
even though we knew
49
96293
1505
01:53
it would be one-on-thousands.
50
97798
1918
01:55
Here, an overhead video camera
51
99716
2201
01:57
is recording me as I'm talking
52
101917
1832
01:59
and drawing on a piece of paper.
53
103749
1679
02:01
A student said, "This class felt
54
105428
2409
02:03
like sitting in a bar
55
107837
1152
02:04
with a really smart friend
56
108989
1479
02:06
who's explaining something
57
110468
1204
02:07
you haven't grasped, but are about to."
58
111672
2301
02:09
And that's exactly what we were aiming for.
59
113973
2116
02:11
Now, from Khan Academy, we saw
60
116089
2788
02:14
that short 10-minute videos
61
118877
1824
02:16
worked much better than trying
62
120701
1776
02:18
to record an hour-long lecture
63
122477
2071
02:20
and put it on the small-format screen.
64
124548
2417
02:22
We decided to go even shorter
65
126965
2192
02:25
and more interactive.
66
129157
1648
02:26
Our typical video is two minutes,
67
130805
2247
02:28
sometimes shorter, never more
68
133052
1752
02:30
than six, and then we pause for
69
134804
2689
02:33
a quiz question, to make it
70
137493
1367
02:34
feel like one-on-one tutoring.
71
138860
1878
02:36
Here, I'm explaining how a computer uses
72
140738
2402
02:39
the grammar of English
73
143140
1248
02:40
to parse sentences, and here,
74
144388
2177
02:42
there's a pause and the student
75
146565
2064
02:44
has to reflect, understand what's going on
76
148629
2237
02:46
and check the right boxes
77
150866
1640
02:48
before they can continue.
78
152506
1417
02:49
Students learn best when
79
153923
2448
02:52
they're actively practicing.
80
156371
1248
02:53
We wanted to engage them, to have them grapple
81
157619
2296
02:55
with ambiguity and guide them to synthesize
82
159915
2968
02:58
the key ideas themselves.
83
162883
1665
03:00
We mostly avoid questions
84
164548
1712
03:02
like, "Here's a formula, now
85
166260
1711
03:03
tell me the value of Y
86
167971
1075
03:04
when X is equal to two."
87
169046
1212
03:06
We preferred open-ended questions.
88
170258
1817
03:07
One student wrote, "Now I'm seeing
89
172075
3178
03:11
Bayes networks and examples of
90
175253
1862
03:13
game theory everywhere I look."
91
177115
1568
03:14
And I like that kind of response.
92
178683
1738
03:16
That's just what we were going for.
93
180421
1806
03:18
We didn't want students to memorize the formulas;
94
182227
2194
03:20
we wanted to change the way
95
184421
1190
03:21
they looked at the world.
96
185611
1090
03:22
And we succeeded.
97
186701
1326
03:23
Or, I should say, the students succeeded.
98
188027
2400
03:26
And it's a little bit ironic
99
190427
1919
03:28
that we set about to disrupt traditional education,
100
192346
2913
03:31
and in doing so, we ended up
101
195259
2016
03:33
making our online class
102
197275
1584
03:34
much more like a traditional college class
103
198859
2327
03:37
than other online classes.
104
201186
1841
03:38
Most online classes, the videos are always available.
105
203027
3215
03:42
You can watch them any time you want.
106
206242
1817
03:43
But if you can do it any time,
107
208059
2287
03:46
that means you can do it tomorrow,
108
210346
1505
03:47
and if you can do it tomorrow,
109
211851
1408
03:49
well, you may not ever
110
213259
2162
03:51
get around to it. (Laughter)
111
215421
1758
03:53
So we brought back the innovation
112
217179
2192
03:55
of having due dates. (Laughter)
113
219371
2087
03:57
You could watch the videos
114
221458
1312
03:58
any time you wanted during the week,
115
222770
1912
04:00
but at the end of the week,
116
224682
1155
04:01
you had to get the homework done.
117
225837
1717
04:03
This motivated the students to keep going, and it also
118
227554
1913
04:05
meant that everybody was working
119
229467
2856
04:08
on the same thing at the same time,
120
232323
1518
04:09
so if you went into a discussion forum,
121
233841
1497
04:11
you could get an answer from a peer within minutes.
122
235338
2704
04:13
Now, I'll show you some of the forums, most of which
123
238042
2939
04:16
were self-organized by the students themselves.
124
240981
2766
04:19
From Daphne Koller and Andrew Ng, we learned
125
243747
3224
04:22
the concept of "flipping" the classroom.
126
246971
1996
04:24
Students watched the videos
127
248967
1295
04:26
on their own, and then they
128
250262
1588
04:27
come together to discuss them.
129
251850
1710
04:29
From Eric Mazur, I learned about peer instruction,
130
253560
3157
04:32
that peers can be the best teachers,
131
256717
2624
04:35
because they're the ones
132
259341
1432
04:36
that remember what it's like to not understand.
133
260773
2848
04:39
Sebastian and I have forgotten some of that.
134
263621
2681
04:42
Of course, we couldn't have
135
266302
2399
04:44
a classroom discussion with
136
268701
1632
04:46
tens of thousands of students,
137
270333
1546
04:47
so we encouraged and nurtured these online forums.
138
271879
3486
04:51
And finally, from Teach For America,
139
275365
3048
04:54
I learned that a class is not
140
278413
1481
04:55
primarily about information.
141
279894
1496
04:57
More important is motivation and determination.
142
281390
2831
05:00
It was crucial that the students see
143
284221
1840
05:01
that we're working hard for them and
144
286061
1800
05:03
they're all supporting each other.
145
287861
1409
05:05
Now, the class ran 10 weeks,
146
289270
2968
05:08
and in the end, about half of the 160,000 students watched
147
292238
3911
05:12
at least one video each week,
148
296149
1563
05:13
and over 20,000 finished all the homework,
149
297712
2693
05:16
putting in 50 to 100 hours.
150
300405
1664
05:17
They got this statement of accomplishment.
151
302069
1553
05:19
So what have we learned?
152
303622
2119
05:21
Well, we tried some old ideas
153
305741
2816
05:24
and some new and put them together,
154
308557
1657
05:26
but there are more ideas to try.
155
310214
2082
05:28
Sebastian's teaching another class now.
156
312296
1814
05:30
I'll do one in the fall.
157
314110
1368
05:31
Stanford Coursera, Udacity, MITx
158
315478
3760
05:35
and others have more classes coming.
159
319238
2271
05:37
It's a really exciting time.
160
321509
1488
05:38
But to me, the most exciting
161
322997
1472
05:40
part of it is the data that we're gathering.
162
324469
2976
05:43
We're gathering thousands
163
327445
2696
05:46
of interactions per student per class,
164
330141
1656
05:47
billions of interactions altogether,
165
331797
2208
05:49
and now we can start analyzing that,
166
334005
2504
05:52
and when we learn from that,
167
336509
1464
05:53
do experimentations,
168
337973
1241
05:55
that's when the real revolution will come.
169
339214
2240
05:57
And you'll be able to see the results from
170
341454
2776
06:00
a new generation of amazing students.
171
344230
2263
06:02
(Applause)
172
346493
2253
ABOUT THE SPEAKER
Peter Norvig - Computer scientistPeter Norvig is a leading American computer scientist, expert on artificial intelligence and the Director of Research at Google Inc.
Why you should listen
Peter Norvig is a computer scientist and expert in both artificial intelligence and online search. Currently the Director of Research at Google Inc., Norvig was responsible for maintaining and improving the engine's core web search algorithms from 2002 to 2005. Prior to his work at Google, Norvig was NASA's chief computer scientist.
A fellow of the American Association for Artificial Intelligence and the author of the book Artificial Intelligence: A Modern Approach, Norvig (along with Sebastian Thrun) taught the Stanford University class "Introduction to Artificial Intelligence," which was made available to anyone in the world. More than 160,000 students from 209 countries enrolled.
Norvig is also known for penning the world's longest palindromic sentence.
Peter Norvig | Speaker | TED.com