sponsored links
TEDGlobal 2012

Shyam Sankar: The rise of human-computer cooperation

June 27, 2012

Brute computing force alone can’t solve the world’s problems. Data mining innovator Shyam Sankar explains why solving big problems (like catching terrorists or identifying huge hidden trends) is not a question of finding the right algorithm, but rather the right symbiotic relationship between computation and human creativity.

Shyam Sankar - Data Intelligence Agent
An advocate of human-computer symbiosis, Shyam Sankar looks for clues in big and disparate data sets. Full bio

sponsored links
Double-click the English subtitles below to play the video.
I'd like to tell you about two games of chess.
00:15
The first happened in 1997, in which Garry Kasparov,
00:18
a human, lost to Deep Blue, a machine.
00:22
To many, this was the dawn of a new era,
00:25
one where man would be dominated by machine.
00:28
But here we are, 20 years on, and the greatest change
00:30
in how we relate to computers is the iPad,
00:34
not HAL.
00:37
The second game was a freestyle chess tournament
00:39
in 2005, in which man and machine could enter together
00:41
as partners, rather than adversaries, if they so chose.
00:44
At first, the results were predictable.
00:49
Even a supercomputer was beaten by a grandmaster
00:51
with a relatively weak laptop.
00:53
The surprise came at the end. Who won?
00:56
Not a grandmaster with a supercomputer,
00:58
but actually two American amateurs
01:01
using three relatively weak laptops.
01:03
Their ability to coach and manipulate their computers
01:07
to deeply explore specific positions
01:09
effectively counteracted the superior chess knowledge
01:12
of the grandmasters and the superior computational power
01:14
of other adversaries.
01:17
This is an astonishing result: average men,
01:19
average machines beating the best man, the best machine.
01:21
And anyways, isn't it supposed to be man versus machine?
01:26
Instead, it's about cooperation, and the right type of cooperation.
01:29
We've been paying a lot of attention to Marvin Minsky's
01:33
vision for artificial intelligence over the last 50 years.
01:36
It's a sexy vision, for sure. Many have embraced it.
01:39
It's become the dominant school of thought in computer science.
01:41
But as we enter the era of big data, of network systems,
01:44
of open platforms, and embedded technology,
01:47
I'd like to suggest it's time to reevaluate an alternative vision
01:50
that was actually developed around the same time.
01:53
I'm talking about J.C.R. Licklider's human-computer symbiosis,
01:56
perhaps better termed "intelligence augmentation," I.A.
02:00
Licklider was a computer science titan who had a profound
02:03
effect on the development of technology and the Internet.
02:06
His vision was to enable man and machine to cooperate
02:09
in making decisions, controlling complex situations
02:12
without the inflexible dependence
02:15
on predetermined programs.
02:17
Note that word "cooperate."
02:20
Licklider encourages us not to take a toaster
02:22
and make it Data from "Star Trek,"
02:25
but to take a human and make her more capable.
02:27
Humans are so amazing -- how we think,
02:31
our non-linear approaches, our creativity,
02:33
iterative hypotheses, all very difficult if possible at all
02:35
for computers to do.
02:37
Licklider intuitively realized this, contemplating humans
02:39
setting the goals, formulating the hypotheses,
02:41
determining the criteria, and performing the evaluation.
02:44
Of course, in other ways, humans are so limited.
02:47
We're terrible at scale, computation and volume.
02:49
We require high-end talent management
02:52
to keep the rock band together and playing.
02:54
Licklider foresaw computers doing all the routinizable work
02:56
that was required to prepare the way for insights and decision making.
02:58
Silently, without much fanfare,
03:01
this approach has been compiling victories beyond chess.
03:04
Protein folding, a topic that shares the incredible expansiveness of chess —
03:07
there are more ways of folding a protein than there are atoms in the universe.
03:10
This is a world-changing problem with huge implications
03:13
for our ability to understand and treat disease.
03:16
And for this task, supercomputer field brute force simply isn't enough.
03:18
Foldit, a game created by computer scientists,
03:22
illustrates the value of the approach.
03:25
Non-technical, non-biologist amateurs play a video game
03:27
in which they visually rearrange the structure of the protein,
03:30
allowing the computer to manage the atomic forces
03:33
and interactions and identify structural issues.
03:35
This approach beat supercomputers 50 percent of the time
03:38
and tied 30 percent of the time.
03:41
Foldit recently made a notable and major scientific discovery
03:43
by deciphering the structure of the Mason-Pfizer monkey virus.
03:46
A protease that had eluded determination for over 10 years
03:50
was solved was by three players in a matter of days,
03:53
perhaps the first major scientific advance
03:55
to come from playing a video game.
03:57
Last year, on the site of the Twin Towers,
04:00
the 9/11 memorial opened.
04:02
It displays the names of the thousands of victims
04:03
using a beautiful concept called "meaningful adjacency."
04:06
It places the names next to each other based on their
04:09
relationships to one another: friends, families, coworkers.
04:11
When you put it all together, it's quite a computational
04:13
challenge: 3,500 victims, 1,800 adjacency requests,
04:16
the importance of the overall physical specifications
04:21
and the final aesthetics.
04:24
When first reported by the media, full credit for such a feat
04:26
was given to an algorithm from the New York City
04:29
design firm Local Projects. The truth is a bit more nuanced.
04:30
While an algorithm was used to develop the underlying framework,
04:34
humans used that framework to design the final result.
04:37
So in this case, a computer had evaluated millions
04:40
of possible layouts, managed a complex relational system,
04:43
and kept track of a very large set of measurements
04:46
and variables, allowing the humans to focus
04:48
on design and compositional choices.
04:51
So the more you look around you,
04:53
the more you see Licklider's vision everywhere.
04:55
Whether it's augmented reality in your iPhone or GPS in your car,
04:56
human-computer symbiosis is making us more capable.
05:00
So if you want to improve human-computer symbiosis,
05:03
what can you do?
05:04
You can start by designing the human into the process.
05:06
Instead of thinking about what a computer will do to solve the problem,
05:08
design the solution around what the human will do as well.
05:10
When you do this, you'll quickly realize that you spent
05:14
all of your time on the interface between man and machine,
05:16
specifically on designing away the friction in the interaction.
05:19
In fact, this friction is more important than the power
05:22
of the man or the power of the machine
05:25
in determining overall capability.
05:27
That's why two amateurs with a few laptops
05:29
handily beat a supercomputer and a grandmaster.
05:31
What Kasparov calls process is a byproduct of friction.
05:33
The better the process, the less the friction.
05:36
And minimizing friction turns out to be the decisive variable.
05:39
Or take another example: big data.
05:43
Every interaction we have in the world is recorded
05:45
by an ever growing array of sensors: your phone,
05:47
your credit card, your computer. The result is big data,
05:50
and it actually presents us with an opportunity
05:53
to more deeply understand the human condition.
05:54
The major emphasis of most approaches to big data
05:57
focus on, "How do I store this data? How do I search
05:59
this data? How do I process this data?"
06:02
These are necessary but insufficient questions.
06:04
The imperative is not to figure out how to compute,
06:06
but what to compute. How do you impose human intuition
06:09
on data at this scale?
06:11
Again, we start by designing the human into the process.
06:13
When PayPal was first starting as a business, their biggest
06:16
challenge was not, "How do I send money back and forth online?"
06:19
It was, "How do I do that without being defrauded by organized crime?"
06:22
Why so challenging? Because while computers can learn
06:26
to detect and identify fraud based on patterns,
06:28
they can't learn to do that based on patterns
06:31
they've never seen before, and organized crime
06:32
has a lot in common with this audience: brilliant people,
06:34
relentlessly resourceful, entrepreneurial spirit — (Laughter) —
06:37
and one huge and important difference: purpose.
06:41
And so while computers alone can catch all but the cleverest
06:43
fraudsters, catching the cleverest is the difference
06:46
between success and failure.
06:49
There's a whole class of problems like this, ones with
06:51
adaptive adversaries. They rarely if ever present with a
06:53
repeatable pattern that's discernable to computers.
06:56
Instead, there's some inherent component of innovation or disruption,
06:59
and increasingly these problems are buried in big data.
07:03
For example, terrorism. Terrorists are always adapting
07:05
in minor and major ways to new circumstances, and despite
07:08
what you might see on TV, these adaptations,
07:10
and the detection of them, are fundamentally human.
07:13
Computers don't detect novel patterns and new behaviors,
07:15
but humans do. Humans, using technology, testing hypotheses,
07:18
searching for insight by asking machines to do things for them.
07:22
Osama bin Laden was not caught by artificial intelligence.
07:26
He was caught by dedicated, resourceful, brilliant people
07:29
in partnerships with various technologies.
07:31
As appealing as it might sound, you cannot algorithmically
07:35
data mine your way to the answer.
07:38
There is no "Find Terrorist" button, and the more data
07:40
we integrate from a vast variety of sources
07:43
across a wide variety of data formats from very
07:45
disparate systems, the less effective data mining can be.
07:47
Instead, people will have to look at data
07:50
and search for insight, and as Licklider foresaw long ago,
07:52
the key to great results here is the right type of cooperation,
07:56
and as Kasparov realized,
07:59
that means minimizing friction at the interface.
08:00
Now this approach makes possible things like combing
08:03
through all available data from very different sources,
08:06
identifying key relationships and putting them in one place,
08:09
something that's been nearly impossible to do before.
08:12
To some, this has terrifying privacy and civil liberties
08:15
implications. To others it foretells of an era of greater
08:17
privacy and civil liberties protections,
08:20
but privacy and civil liberties are of fundamental importance.
08:22
That must be acknowledged, and they can't be swept aside,
08:25
even with the best of intents.
08:27
So let's explore, through a couple of examples, the impact
08:30
that technologies built to drive human-computer symbiosis
08:32
have had in recent time.
08:35
In October, 2007, U.S. and coalition forces raided
08:38
an al Qaeda safe house in the city of Sinjar
08:41
on the Syrian border of Iraq.
08:44
They found a treasure trove of documents:
08:45
700 biographical sketches of foreign fighters.
08:48
These foreign fighters had left their families in the Gulf,
08:50
the Levant and North Africa to join al Qaeda in Iraq.
08:53
These records were human resource forms.
08:56
The foreign fighters filled them out as they joined the organization.
08:58
It turns out that al Qaeda, too,
09:00
is not without its bureaucracy. (Laughter)
09:02
They answered questions like, "Who recruited you?"
09:04
"What's your hometown?" "What occupation do you seek?"
09:06
In that last question, a surprising insight was revealed.
09:09
The vast majority of foreign fighters
09:12
were seeking to become suicide bombers for martyrdom --
09:15
hugely important, since between 2003 and 2007, Iraq
09:17
had 1,382 suicide bombings, a major source of instability.
09:21
Analyzing this data was hard. The originals were sheets
09:26
of paper in Arabic that had to be scanned and translated.
09:28
The friction in the process did not allow for meaningful
09:31
results in an operational time frame using humans, PDFs
09:33
and tenacity alone.
09:36
The researchers had to lever up their human minds
09:38
with technology to dive deeper, to explore non-obvious
09:40
hypotheses, and in fact, insights emerged.
09:43
Twenty percent of the foreign fighters were from Libya,
09:46
50 percent of those from a single town in Libya,
09:48
hugely important since prior statistics put that figure at
09:51
three percent. It also helped to hone in on a figure
09:54
of rising importance in al Qaeda, Abu Yahya al-Libi,
09:56
a senior cleric in the Libyan Islamic fighting group.
09:59
In March of 2007, he gave a speech, after which there was
10:02
a surge in participation amongst Libyan foreign fighters.
10:05
Perhaps most clever of all, though, and least obvious,
10:08
by flipping the data on its head, the researchers were
10:11
able to deeply explore the coordination networks in Syria
10:13
that were ultimately responsible for receiving and
10:16
transporting the foreign fighters to the border.
10:19
These were networks of mercenaries, not ideologues,
10:21
who were in the coordination business for profit.
10:24
For example, they charged Saudi foreign fighters
10:26
substantially more than Libyans, money that would have
10:28
otherwise gone to al Qaeda.
10:30
Perhaps the adversary would disrupt their own network
10:32
if they knew they cheating would-be jihadists.
10:35
In January, 2010, a devastating 7.0 earthquake struck Haiti,
10:38
third deadliest earthquake of all time, left one million people,
10:41
10 percent of the population, homeless.
10:44
One seemingly small aspect of the overall relief effort
10:47
became increasingly important as the delivery of food
10:50
and water started rolling.
10:52
January and February are the dry months in Haiti,
10:54
yet many of the camps had developed standing water.
10:56
The only institution with detailed knowledge of Haiti's
10:59
floodplains had been leveled
11:01
in the earthquake, leadership inside.
11:02
So the question is, which camps are at risk,
11:05
how many people are in these camps, what's the
11:08
timeline for flooding, and given very limited resources
11:10
and infrastructure, how do we prioritize the relocation?
11:12
The data was incredibly disparate. The U.S. Army had
11:15
detailed knowledge for only a small section of the country.
11:18
There was data online from a 2006 environmental risk
11:21
conference, other geospatial data, none of it integrated.
11:23
The human goal here was to identify camps for relocation
11:26
based on priority need.
11:29
The computer had to integrate a vast amount of geospacial
11:31
information, social media data and relief organization
11:34
information to answer this question.
11:36
By implementing a superior process, what was otherwise
11:40
a task for 40 people over three months became
11:42
a simple job for three people in 40 hours,
11:45
all victories for human-computer symbiosis.
11:48
We're more than 50 years into Licklider's vision
11:50
for the future, and the data suggests that we should be
11:52
quite excited about tackling this century's hardest problems,
11:55
man and machine in cooperation together.
11:58
Thank you. (Applause)
12:01
(Applause)
12:03
Translator:Joseph Geni
Reviewer:Morton Bast

sponsored links

Shyam Sankar - Data Intelligence Agent
An advocate of human-computer symbiosis, Shyam Sankar looks for clues in big and disparate data sets.

Why you should listen

Shyam Sankar is a Director at Palantir Technologies, a secretive Silicon Valley company where he oversees deployments of the company's core technology, which helps law enforcement teams and corporations analyze giant, unrelated databases for clues to potential ... anything. Palantir technologies has been used to find missing children, to detect banking fraud, and to uncover the Shadow Network, a cyber-spy ring that stooped so low as to hack the Dalai Lama's email.

As part of his work, Sankar thinks deeply about the place where human and machine intelligence meet. While artificial intelligence (AI) is the dominant paradigm, he is an advocate of JCR Licklider's "intelligence augmentation" (IA) approach, where algorithms and brains work together to solve problems.

The original video is available on TED.com
sponsored links

If you need translations, you can install "Google Translate" extension into your Chrome Browser.
Furthermore, you can change playback rate by installing "Video Speed Controller" extension.

Data provided by TED.

This website is owned and operated by Tokyo English Network.
The developer's blog is here.