06:12
TED2012

Peter Norvig: The 100,000-student classroom

Filmed:

In the fall of 2011 Peter Norvig taught a class with Sebastian Thrun on artificial intelligence at Stanford attended by 175 students in situ -- and over 100,000 via an interactive webcast. He shares what he learned about teaching to a global classroom.

- Computer scientist
Peter Norvig is a leading American computer scientist, expert on artificial intelligence and the Director of Research at Google Inc. Full bio

Everyone is both a learner
00:17
and a teacher.
00:19
This is me being inspired
00:20
by my first tutor,
00:22
my mom,
00:24
and this is me teaching
00:25
Introduction to Artificial Intelligence
00:27
to 200 students
00:29
at Stanford University.
00:30
Now the students and I
00:32
enjoyed the class,
00:33
but it occurred to me
00:34
that while the subject matter
00:36
of the class is advanced
00:37
and modern,
00:39
the teaching technology isn't.
00:39
In fact, I use basically
00:42
the same technology as
00:44
this 14th-century classroom.
00:46
Note the textbook,
00:49
the sage on the stage,
00:52
and the sleeping guy
00:55
in the back. (Laughter)
00:57
Just like today.
00:58
So my co-teacher,
01:01
Sebastian Thrun, and I thought,
01:04
there must be a better way.
01:05
We challenged ourselves
01:07
to create an online class
01:09
that would be equal or better
01:10
in quality to our Stanford class,
01:12
but to bring it to anyone
01:14
in the world for free.
01:16
We announced the class on July 29th,
01:18
and within two weeks, 50,000 people
01:20
had signed up for it.
01:24
And that grew to 160,000 students
01:25
from 209 countries.
01:28
We were thrilled to have
01:30
that kind of audience,
01:32
and just a bit terrified that we
01:33
hadn't finished preparing the class yet. (Laughter)
01:36
So we got to work.
01:38
We studied what others had done,
01:40
what we could copy and what we could change.
01:41
Benjamin Bloom had showed
01:44
that one-on-one tutoring works best,
01:46
so that's what we tried to emulate,
01:48
like with me and my mom,
01:50
even though we knew
01:52
it would be one-on-thousands.
01:53
Here, an overhead video camera
01:55
is recording me as I'm talking
01:57
and drawing on a piece of paper.
01:59
A student said, "This class felt
02:01
like sitting in a bar
02:03
with a really smart friend
02:04
who's explaining something
02:06
you haven't grasped, but are about to."
02:07
And that's exactly what we were aiming for.
02:09
Now, from Khan Academy, we saw
02:11
that short 10-minute videos
02:14
worked much better than trying
02:16
to record an hour-long lecture
02:18
and put it on the small-format screen.
02:20
We decided to go even shorter
02:22
and more interactive.
02:25
Our typical video is two minutes,
02:26
sometimes shorter, never more
02:28
than six, and then we pause for
02:30
a quiz question, to make it
02:33
feel like one-on-one tutoring.
02:34
Here, I'm explaining how a computer uses
02:36
the grammar of English
02:39
to parse sentences, and here,
02:40
there's a pause and the student
02:42
has to reflect, understand what's going on
02:44
and check the right boxes
02:46
before they can continue.
02:48
Students learn best when
02:49
they're actively practicing.
02:52
We wanted to engage them, to have them grapple
02:53
with ambiguity and guide them to synthesize
02:55
the key ideas themselves.
02:58
We mostly avoid questions
03:00
like, "Here's a formula, now
03:02
tell me the value of Y
03:03
when X is equal to two."
03:04
We preferred open-ended questions.
03:06
One student wrote, "Now I'm seeing
03:07
Bayes networks and examples of
03:11
game theory everywhere I look."
03:13
And I like that kind of response.
03:14
That's just what we were going for.
03:16
We didn't want students to memorize the formulas;
03:18
we wanted to change the way
03:20
they looked at the world.
03:21
And we succeeded.
03:22
Or, I should say, the students succeeded.
03:23
And it's a little bit ironic
03:26
that we set about to disrupt traditional education,
03:28
and in doing so, we ended up
03:31
making our online class
03:33
much more like a traditional college class
03:34
than other online classes.
03:37
Most online classes, the videos are always available.
03:38
You can watch them any time you want.
03:42
But if you can do it any time,
03:43
that means you can do it tomorrow,
03:46
and if you can do it tomorrow,
03:47
well, you may not ever
03:49
get around to it. (Laughter)
03:51
So we brought back the innovation
03:53
of having due dates. (Laughter)
03:55
You could watch the videos
03:57
any time you wanted during the week,
03:58
but at the end of the week,
04:00
you had to get the homework done.
04:01
This motivated the students to keep going, and it also
04:03
meant that everybody was working
04:05
on the same thing at the same time,
04:08
so if you went into a discussion forum,
04:09
you could get an answer from a peer within minutes.
04:11
Now, I'll show you some of the forums, most of which
04:13
were self-organized by the students themselves.
04:16
From Daphne Koller and Andrew Ng, we learned
04:19
the concept of "flipping" the classroom.
04:22
Students watched the videos
04:24
on their own, and then they
04:26
come together to discuss them.
04:27
From Eric Mazur, I learned about peer instruction,
04:29
that peers can be the best teachers,
04:32
because they're the ones
04:35
that remember what it's like to not understand.
04:36
Sebastian and I have forgotten some of that.
04:39
Of course, we couldn't have
04:42
a classroom discussion with
04:44
tens of thousands of students,
04:46
so we encouraged and nurtured these online forums.
04:47
And finally, from Teach For America,
04:51
I learned that a class is not
04:54
primarily about information.
04:55
More important is motivation and determination.
04:57
It was crucial that the students see
05:00
that we're working hard for them and
05:01
they're all supporting each other.
05:03
Now, the class ran 10 weeks,
05:05
and in the end, about half of the 160,000 students watched
05:08
at least one video each week,
05:12
and over 20,000 finished all the homework,
05:13
putting in 50 to 100 hours.
05:16
They got this statement of accomplishment.
05:17
So what have we learned?
05:19
Well, we tried some old ideas
05:21
and some new and put them together,
05:24
but there are more ideas to try.
05:26
Sebastian's teaching another class now.
05:28
I'll do one in the fall.
05:30
Stanford Coursera, Udacity, MITx
05:31
and others have more classes coming.
05:35
It's a really exciting time.
05:37
But to me, the most exciting
05:38
part of it is the data that we're gathering.
05:40
We're gathering thousands
05:43
of interactions per student per class,
05:46
billions of interactions altogether,
05:47
and now we can start analyzing that,
05:49
and when we learn from that,
05:52
do experimentations,
05:53
that's when the real revolution will come.
05:55
And you'll be able to see the results from
05:57
a new generation of amazing students.
06:00
(Applause)
06:02
Translated by Joseph Geni
Reviewed by Morton Bast

▲Back to top

About the Speaker:

Peter Norvig - Computer scientist
Peter Norvig is a leading American computer scientist, expert on artificial intelligence and the Director of Research at Google Inc.

Why you should listen

Peter Norvig is a computer scientist and expert in both artificial intelligence and online search. Currently the Director of Research at Google Inc., Norvig was responsible for maintaining and improving the engine's core web search algorithms from 2002 to 2005. Prior to his work at Google, Norvig was NASA's chief computer scientist.

A fellow of the American Association for Artificial Intelligence and the author of the book Artificial Intelligence: A Modern Approach, Norvig (along with Sebastian Thrun) taught the Stanford University class "Introduction to Artificial Intelligence," which was made available to anyone in the world. More than 160,000 students from 209 countries enrolled.

Norvig is also known for penning the world's longest palindromic sentence.

More profile about the speaker
Peter Norvig | Speaker | TED.com