TEDGlobal 2012

Daphne Koller: What we're learning from online education

Filmed:

Daphne Koller is enticing top universities to put their most intriguing courses online for free -- not just as a service, but as a way to research how people learn. With Coursera (cofounded by Andrew Ng), each keystroke, quiz, peer-to-peer discussion and self-graded assignment builds an unprecedented pool of data on how knowledge is processed.

- Educator
With Coursera, Daphne Koller and co-founder Andrew Ng are bringing courses from top colleges online, free, for anyone who wants to take them. Full bio

Like many of you, I'm one of the lucky people.
00:16
I was born to a family where education was pervasive.
00:19
I'm a third-generation PhD, a daughter of two academics.
00:22
In my childhood, I played around in my father's university lab.
00:26
So it was taken for granted that I attend some of the best universities,
00:30
which in turn opened the door to a world of opportunity.
00:34
Unfortunately, most of the people in the world are not so lucky.
00:38
In some parts of the world, for example, South Africa,
00:42
education is just not readily accessible.
00:45
In South Africa, the educational system was constructed
00:48
in the days of apartheid for the white minority.
00:51
And as a consequence, today there is just not enough spots
00:54
for the many more people who want and deserve a high quality education.
00:56
That scarcity led to a crisis in January of this year
01:00
at the University of Johannesburg.
01:04
There were a handful of positions left open
01:06
from the standard admissions process, and the night before
01:08
they were supposed to open that for registration,
01:11
thousands of people lined up outside the gate in a line a mile long,
01:13
hoping to be first in line to get one of those positions.
01:18
When the gates opened, there was a stampede,
01:21
and 20 people were injured and one woman died.
01:24
She was a mother who gave her life
01:27
trying to get her son a chance at a better life.
01:29
But even in parts of the world like the United States
01:33
where education is available, it might not be within reach.
01:37
There has been much discussed in the last few years
01:41
about the rising cost of health care.
01:44
What might not be quite as obvious to people
01:46
is that during that same period the cost of higher education tuition
01:48
has been increasing at almost twice the rate,
01:52
for a total of 559 percent since 1985.
01:55
This makes education unaffordable for many people.
01:59
Finally, even for those who do manage to get the higher education,
02:04
the doors of opportunity might not open.
02:07
Only a little over half of recent college graduates
02:10
in the United States who get a higher education
02:13
actually are working in jobs that require that education.
02:15
This, of course, is not true for the students
02:19
who graduate from the top institutions,
02:21
but for many others, they do not get the value
02:23
for their time and their effort.
02:25
Tom Friedman, in his recent New York Times article,
02:29
captured, in the way that no one else could, the spirit behind our effort.
02:32
He said the big breakthroughs are what happen
02:36
when what is suddenly possible meets what is desperately necessary.
02:39
I've talked about what's desperately necessary.
02:43
Let's talk about what's suddenly possible.
02:46
What's suddenly possible was demonstrated by
02:48
three big Stanford classes,
02:52
each of which had an enrollment of 100,000 people or more.
02:53
So to understand this, let's look at one of those classes,
02:57
the Machine Learning class offered by my colleague
03:00
and cofounder Andrew Ng.
03:02
Andrew teaches one of the bigger Stanford classes.
03:04
It's a Machine Learning class,
03:06
and it has 400 people enrolled every time it's offered.
03:08
When Andrew taught the Machine Learning class to the general public,
03:11
it had 100,000 people registered.
03:14
So to put that number in perspective,
03:17
for Andrew to reach that same size audience
03:19
by teaching a Stanford class,
03:21
he would have to do that for 250 years.
03:23
Of course, he'd get really bored.
03:27
So, having seen the impact of this,
03:31
Andrew and I decided that we needed to really try and scale this up,
03:33
to bring the best quality education to as many people as we could.
03:36
So we formed Coursera,
03:41
whose goal is to take the best courses
03:42
from the best instructors at the best universities
03:45
and provide it to everyone around the world for free.
03:48
We currently have 43 courses on the platform
03:53
from four universities across a range of disciplines,
03:55
and let me show you a little bit of an overview
03:58
of what that looks like.
04:00
(Video) Robert Ghrist: Welcome to Calculus.
04:03
Ezekiel Emanuel: Fifty million people are uninsured.
04:05
Scott Page: Models help us design more effective institutions and policies.
04:07
We get unbelievable segregation.
04:10
Scott Klemmer: So Bush imagined that in the future,
04:12
you'd wear a camera right in the center of your head.
04:14
Mitchell Duneier: Mills wants the student of sociology to develop the quality of mind ...
04:16
RG: Hanging cable takes on the form of a hyperbolic cosine.
04:21
Nick Parlante: For each pixel in the image, set the red to zero.
04:24
Paul Offit: ... Vaccine allowed us to eliminate polio virus.
04:27
Dan Jurafsky: Does Lufthansa serve breakfast and San Jose? Well, that sounds funny.
04:30
Daphne Koller: So this is which coin you pick, and this is the two tosses.
04:34
Andrew Ng: So in large-scale machine learning, we'd like to come up with computational ...
04:38
(Applause)
04:41
DK: It turns out, maybe not surprisingly,
04:47
that students like getting the best content
04:49
from the best universities for free.
04:51
Since we opened the website in February,
04:54
we now have 640,000 students from 190 countries.
04:57
We have 1.5 million enrollments,
05:01
6 million quizzes in the 15 classes that have launched
05:03
so far have been submitted, and 14 million videos have been viewed.
05:06
But it's not just about the numbers,
05:11
it's also about the people.
05:14
Whether it's Akash, who comes from a small town in India
05:15
and would never have access in this case
05:18
to a Stanford-quality course
05:20
and would never be able to afford it.
05:22
Or Jenny, who is a single mother of two
05:24
and wants to hone her skills
05:26
so that she can go back and complete her master's degree.
05:28
Or Ryan, who can't go to school,
05:32
because his immune deficient daughter
05:35
can't be risked to have germs come into the house,
05:37
so he couldn't leave the house.
05:40
I'm really glad to say --
05:42
recently, we've been in correspondence with Ryan --
05:43
that this story had a happy ending.
05:46
Baby Shannon -- you can see her on the left --
05:48
is doing much better now,
05:49
and Ryan got a job by taking some of our courses.
05:51
So what made these courses so different?
05:55
After all, online course content has been available for a while.
05:57
What made it different was that this was real course experience.
06:01
It started on a given day,
06:05
and then the students would watch videos on a weekly basis
06:06
and do homework assignments.
06:10
And these would be real homework assignments
06:12
for a real grade, with a real deadline.
06:14
You can see the deadlines and the usage graph.
06:17
These are the spikes showing
06:19
that procrastination is global phenomenon.
06:21
(Laughter)
06:25
At the end of the course,
06:28
the students got a certificate.
06:29
They could present that certificate
06:31
to a prospective employer and get a better job,
06:33
and we know many students who did.
06:35
Some students took their certificate
06:37
and presented this to an educational institution at which they were enrolled
06:39
for actual college credit.
06:42
So these students were really getting something meaningful
06:44
for their investment of time and effort.
06:47
Let's talk a little bit about some of the components
06:49
that go into these courses.
06:52
The first component is that when you move away
06:54
from the constraints of a physical classroom
06:56
and design content explicitly for an online format,
06:59
you can break away from, for example,
07:02
the monolithic one-hour lecture.
07:04
You can break up the material, for example,
07:07
into these short, modular units of eight to 12 minutes,
07:08
each of which represents a coherent concept.
07:12
Students can traverse this material in different ways,
07:15
depending on their background, their skills or their interests.
07:17
So, for example, some students might benefit
07:21
from a little bit of preparatory material
07:23
that other students might already have.
07:26
Other students might be interested in a particular
07:28
enrichment topic that they want to pursue individually.
07:31
So this format allows us to break away
07:34
from the one-size-fits-all model of education,
07:37
and allows students to follow a much more personalized curriculum.
07:40
Of course, we all know as educators
07:44
that students don't learn by sitting and passively watching videos.
07:46
Perhaps one of the biggest components of this effort
07:50
is that we need to have students
07:52
who practice with the material
07:55
in order to really understand it.
07:57
There's been a range of studies that demonstrate the importance of this.
08:01
This one that appeared in Science last year, for example,
08:04
demonstrates that even simple retrieval practice,
08:06
where students are just supposed to repeat
08:09
what they already learned
08:12
gives considerably improved results
08:13
on various achievement tests down the line
08:15
than many other educational interventions.
08:18
We've tried to build in retrieval practice into the platform,
08:22
as well as other forms of practice in many ways.
08:25
For example, even our videos are not just videos.
08:27
Every few minutes, the video pauses
08:31
and the students get asked a question.
08:33
(Video) SP: ... These four things. Prospect theory, hyperbolic discounting,
08:36
status quo bias, base rate bias. They're all well documented.
08:38
So they're all well documented deviations from rational behavior.
08:41
DK: So here the video pauses,
08:44
and the student types in the answer into the box
08:45
and submits. Obviously they weren't paying attention.
08:47
(Laughter)
08:51
So they get to try again,
08:52
and this time they got it right.
08:54
There's an optional explanation if they want.
08:56
And now the video moves on to the next part of the lecture.
08:58
This is a kind of simple question
09:03
that I as an instructor might ask in class,
09:04
but when I ask that kind of a question in class,
09:07
80 percent of the students
09:09
are still scribbling the last thing I said,
09:10
15 percent are zoned out on Facebook,
09:12
and then there's the smarty pants in the front row
09:16
who blurts out the answer
09:18
before anyone else has had a chance to think about it,
09:19
and I as the instructor am terribly gratified
09:22
that somebody actually knew the answer.
09:24
And so the lecture moves on before, really,
09:26
most of the students have even noticed that a question had been asked.
09:29
Here, every single student
09:32
has to engage with the material.
09:35
And of course these simple retrieval questions
09:38
are not the end of the story.
09:40
One needs to build in much more meaningful practice questions,
09:41
and one also needs to provide the students with feedback
09:44
on those questions.
09:47
Now, how do you grade the work of 100,000 students
09:48
if you do not have 10,000 TAs?
09:51
The answer is, you need to use technology
09:54
to do it for you.
09:57
Now, fortunately, technology has come a long way,
09:58
and we can now grade a range of interesting types of homework.
10:01
In addition to multiple choice
10:04
and the kinds of short answer questions that you saw in the video,
10:06
we can also grade math, mathematical expressions
10:09
as well as mathematical derivations.
10:12
We can grade models, whether it's
10:14
financial models in a business class
10:17
or physical models in a science or engineering class
10:19
and we can grade some pretty sophisticated programming assignments.
10:22
Let me show you one that's actually pretty simple
10:26
but fairly visual.
10:28
This is from Stanford's Computer Science 101 class,
10:29
and the students are supposed to color-correct
10:32
that blurry red image.
10:33
They're typing their program into the browser,
10:35
and you can see they didn't get it quite right, Lady Liberty is still seasick.
10:37
And so, the student tries again, and now they got it right, and they're told that,
10:41
and they can move on to the next assignment.
10:45
This ability to interact actively with the material
10:47
and be told when you're right or wrong
10:50
is really essential to student learning.
10:52
Now, of course we cannot yet grade
10:55
the range of work that one needs for all courses.
10:57
Specifically, what's lacking is the kind of critical thinking work
11:00
that is so essential in such disciplines
11:03
as the humanities, the social sciences, business and others.
11:05
So we tried to convince, for example,
11:09
some of our humanities faculty
11:11
that multiple choice was not such a bad strategy.
11:13
That didn't go over really well.
11:15
So we had to come up with a different solution.
11:18
And the solution we ended up using is peer grading.
11:20
It turns out that previous studies show,
11:23
like this one by Saddler and Good,
11:26
that peer grading is a surprisingly effective strategy
11:27
for providing reproducible grades.
11:30
It was tried only in small classes,
11:33
but there it showed, for example,
11:35
that these student-assigned grades on the y-axis
11:36
are actually very well correlated
11:39
with the teacher-assigned grade on the x-axis.
11:40
What's even more surprising is that self-grades,
11:42
where the students grade their own work critically --
11:45
so long as you incentivize them properly
11:48
so they can't give themselves a perfect score --
11:50
are actually even better correlated with the teacher grades.
11:51
And so this is an effective strategy
11:55
that can be used for grading at scale,
11:56
and is also a useful learning strategy for the students,
11:58
because they actually learn from the experience.
12:01
So we now have the largest peer-grading pipeline ever devised,
12:03
where tens of thousands of students
12:08
are grading each other's work,
12:11
and quite successfully, I have to say.
12:12
But this is not just about students
12:15
sitting alone in their living room working through problems.
12:17
Around each one of our courses,
12:20
a community of students had formed,
12:22
a global community of people
12:24
around a shared intellectual endeavor.
12:26
What you see here is a self-generated map
12:28
from students in our Princeton Sociology 101 course,
12:31
where they have put themselves on a world map,
12:34
and you can really see the global reach of this kind of effort.
12:37
Students collaborated in these courses in a variety of different ways.
12:40
First of all, there was a question and answer forum,
12:44
where students would pose questions,
12:47
and other students would answer those questions.
12:49
And the really amazing thing is,
12:52
because there were so many students,
12:53
it means that even if a student posed a question
12:55
at 3 o'clock in the morning,
12:57
somewhere around the world,
12:59
there would be somebody who was awake
13:01
and working on the same problem.
13:03
And so, in many of our courses,
13:05
the median response time for a question
13:07
on the question and answer forum was 22 minutes.
13:09
Which is not a level of service I have ever offered to my Stanford students.
13:13
(Laughter)
13:17
And you can see from the student testimonials
13:19
that students actually find
13:20
that because of this large online community,
13:22
they got to interact with each other in many ways
13:25
that were deeper than they did in the context of the physical classroom.
13:27
Students also self-assembled,
13:31
without any kind of intervention from us,
13:34
into small study groups.
13:36
Some of these were physical study groups
13:38
along geographical constraints
13:40
and met on a weekly basis to work through problem sets.
13:42
This is the San Francisco study group,
13:44
but there were ones all over the world.
13:46
Others were virtual study groups,
13:49
sometimes along language lines or along cultural lines,
13:51
and on the bottom left there,
13:54
you see our multicultural universal study group
13:55
where people explicitly wanted to connect
13:59
with people from other cultures.
14:01
There are some tremendous opportunities
14:04
to be had from this kind of framework.
14:06
The first is that it has the potential of giving us
14:09
a completely unprecedented look
14:13
into understanding human learning.
14:15
Because the data that we can collect here is unique.
14:18
You can collect every click, every homework submission,
14:21
every forum post from tens of thousands of students.
14:25
So you can turn the study of human learning
14:29
from the hypothesis-driven mode
14:32
to the data-driven mode, a transformation that,
14:34
for example, has revolutionized biology.
14:37
You can use these data to understand fundamental questions
14:40
like, what are good learning strategies
14:43
that are effective versus ones that are not?
14:45
And in the context of particular courses,
14:48
you can ask questions
14:50
like, what are some of the misconceptions that are more common
14:51
and how do we help students fix them?
14:55
So here's an example of that,
14:57
also from Andrew's Machine Learning class.
14:58
This is a distribution of wrong answers
15:00
to one of Andrew's assignments.
15:02
The answers happen to be pairs of numbers,
15:04
so you can draw them on this two-dimensional plot.
15:06
Each of the little crosses that you see is a different wrong answer.
15:08
The big cross at the top left
15:12
is where 2,000 students
15:14
gave the exact same wrong answer.
15:17
Now, if two students in a class of 100
15:20
give the same wrong answer,
15:22
you would never notice.
15:23
But when 2,000 students give the same wrong answer,
15:25
it's kind of hard to miss.
15:27
So Andrew and his students went in,
15:29
looked at some of those assignments,
15:31
understood the root cause of the misconception,
15:33
and then they produced a targeted error message
15:37
that would be provided to every student
15:39
whose answer fell into that bucket,
15:41
which means that students who made that same mistake
15:44
would now get personalized feedback
15:46
telling them how to fix their misconception much more effectively.
15:48
So this personalization is something that one can then build
15:52
by having the virtue of large numbers.
15:56
Personalization is perhaps
15:59
one of the biggest opportunities here as well,
16:01
because it provides us with the potential
16:04
of solving a 30-year-old problem.
16:06
Educational researcher Benjamin Bloom, in 1984,
16:09
posed what's called the 2 sigma problem,
16:12
which he observed by studying three populations.
16:14
The first is the population that studied in a lecture-based classroom.
16:17
The second is a population of students that studied
16:21
using a standard lecture-based classroom,
16:24
but with a mastery-based approach,
16:26
so the students couldn't move on to the next topic
16:28
before demonstrating mastery of the previous one.
16:30
And finally, there was a population of students
16:33
that were taught in a one-on-one instruction using a tutor.
16:35
The mastery-based population was a full standard deviation,
16:40
or sigma, in achievement scores better
16:43
than the standard lecture-based class,
16:45
and the individual tutoring gives you 2 sigma
16:48
improvement in performance.
16:50
To understand what that means,
16:52
let's look at the lecture-based classroom,
16:53
and let's pick the median performance as a threshold.
16:55
So in a lecture-based class,
16:58
half the students are above that level and half are below.
16:59
In the individual tutoring instruction,
17:03
98 percent of the students are going to be above that threshold.
17:05
Imagine if we could teach so that 98 percent of our students
17:10
would be above average.
17:14
Hence, the 2 sigma problem.
17:16
Because we cannot afford, as a society,
17:20
to provide every student with an individual human tutor.
17:22
But maybe we can afford to provide each student
17:25
with a computer or a smartphone.
17:27
So the question is, how can we use technology
17:29
to push from the left side of the graph, from the blue curve,
17:31
to the right side with the green curve?
17:35
Mastery is easy to achieve using a computer,
17:38
because a computer doesn't get tired
17:40
of showing you the same video five times.
17:41
And it doesn't even get tired of grading the same work multiple times,
17:44
we've seen that in many of the examples that I've shown you.
17:48
And even personalization
17:51
is something that we're starting to see the beginnings of,
17:53
whether it's via the personalized trajectory through the curriculum
17:55
or some of the personalized feedback that we've shown you.
17:58
So the goal here is to try and push,
18:01
and see how far we can get towards the green curve.
18:04
So, if this is so great, are universities now obsolete?
18:07
Well, Mark Twain certainly thought so.
18:12
He said that, "College is a place where a professor's lecture notes
18:15
go straight to the students' lecture notes,
18:18
without passing through the brains of either."
18:20
(Laughter)
18:22
I beg to differ with Mark Twain, though.
18:26
I think what he was complaining about is not
18:29
universities but rather the lecture-based format
18:31
that so many universities spend so much time on.
18:34
So let's go back even further, to Plutarch,
18:37
who said that, "The mind is not a vessel that needs filling,
18:40
but wood that needs igniting."
18:42
And maybe we should spend less time at universities
18:44
filling our students' minds with content
18:47
by lecturing at them, and more time igniting their creativity,
18:49
their imagination and their problem-solving skills
18:53
by actually talking with them.
18:56
So how do we do that?
18:59
We do that by doing active learning in the classroom.
19:00
So there's been many studies, including this one,
19:03
that show that if you use active learning,
19:06
interacting with your students in the classroom,
19:08
performance improves on every single metric --
19:10
on attendance, on engagement and on learning
19:13
as measured by a standardized test.
19:16
You can see, for example, that the achievement score
19:18
almost doubles in this particular experiment.
19:20
So maybe this is how we should spend our time at universities.
19:22
So to summarize, if we could offer a top quality education
19:27
to everyone around the world for free,
19:31
what would that do? Three things.
19:33
First it would establish education as a fundamental human right,
19:36
where anyone around the world
19:40
with the ability and the motivation
19:41
could get the skills that they need
19:43
to make a better life for themselves,
19:45
their families and their communities.
19:46
Second, it would enable lifelong learning.
19:48
It's a shame that for so many people,
19:51
learning stops when we finish high school or when we finish college.
19:53
By having this amazing content be available,
19:56
we would be able to learn something new
19:59
every time we wanted,
20:01
whether it's just to expand our minds
20:03
or it's to change our lives.
20:04
And finally, this would enable a wave of innovation,
20:06
because amazing talent can be found anywhere.
20:09
Maybe the next Albert Einstein or the next Steve Jobs
20:12
is living somewhere in a remote village in Africa.
20:15
And if we could offer that person an education,
20:18
they would be able to come up with the next big idea
20:20
and make the world a better place for all of us.
20:23
Thank you very much.
20:25
(Applause)
20:26
Translated by Morton Bast
Reviewed by Thu-Huong Ha

▲Back to top

About the Speaker:

Daphne Koller - Educator
With Coursera, Daphne Koller and co-founder Andrew Ng are bringing courses from top colleges online, free, for anyone who wants to take them.

Why you should listen

A 3rd generation Ph.D who is passionate about education, Stanford professor Daphne Koller is excited to be making the college experience available to anyone through her startup, Coursera. With classes from 85 top colleges, Coursera is an innovative model for online learning. While top schools have been putting lectures online for years, Coursera's platform supports the other vital aspect of the classroom: tests and assignments that reinforce learning.

At the Stanford Artificial Intelligence Laboratory, computer scientist Daphne Koller studies how to model large, complicated decisions with lots of uncertainty. (Her research group is called DAGS, which stands for Daphne's Approximate Group of Students.) In 2004, she won a MacArthur Fellowship for her work, which involves, among other things, using Bayesian networks and other techniques to explore biomedical and genetic data sets.

More profile about the speaker
Daphne Koller | Speaker | TED.com