sponsored links
TED2011

Aaron Koblin: Visualizing ourselves ... with crowd-sourced data

March 2, 2011

Artist Aaron Koblin takes vast amounts of data -- and at times vast numbers of people -- and weaves them into stunning visualizations. From elegant lines tracing airline flights to landscapes of cell phone data, from a Johnny Cash video assembled from crowd-sourced drawings to the "Wilderness Downtown" video that customizes for the user, his works brilliantly explore how modern technology can make us more human.

Aaron Koblin - Data artist
Aaron Koblin is an artist specializing in data and digital technologies. His work takes real world and community-generated data and uses it to reflect on cultural trends and the changing relationship between humans and technology. Full bio

sponsored links
Double-click the English subtitles below to play the video.
So I think data can actually make us more human.
00:15
We're collecting and creating all kinds of data about how we're living our lives,
00:19
and it's enabling us to tell some amazing stories.
00:22
Recently, a wise media theorist Tweeted,
00:24
"The 19th century culture was defined by the novel,
00:27
the 20th century culture was defined by the cinema,
00:29
and the culture of the 21st century
00:31
will be defined by the interface."
00:33
And I believe this is going to prove true.
00:35
Our lives are being driven by data,
00:37
and the presentation of that data is an opportunity
00:39
for us to make some amazing interfaces that tell great stories.
00:41
So I'm going to show you a few of the projects
00:43
that I've been working on over the last couple years
00:45
that reflect on our lives and our systems.
00:47
This is a project called Flight Patterns.
00:49
What you're looking at is airplane traffic
00:51
over North America for a 24-hour period.
00:53
As you see, everything starts to fade to black,
00:56
and you see people going to sleep.
00:58
Followed by that, you see on the West coast
01:00
planes moving across, the red-eye flights to the East coast.
01:02
And you'll see everybody waking up on the East coast,
01:05
followed by European flights coming in the upper right-hand corner.
01:08
Everybody's moving from the East coast to the West coast.
01:11
You see San Francisco and Los Angeles
01:14
start to make their journeys down to Hawaii in the lower left-hand corner.
01:16
I think it's one thing to say there's 140,000 planes
01:19
being monitored by the federal government at any one time,
01:21
and it's another thing to see that system as it ebbs and flows.
01:24
This is a time-lapse image of that exact same data,
01:29
but I've color-coded it by type,
01:31
so you can see the diversity of aircraft that are in the skies above us.
01:33
And I started making these, and I put them into Google Maps
01:36
and allow you to zoom in and see individual airports
01:39
and the patterns that are occurring there.
01:41
So here we can see the white represents low altitudes,
01:43
and the blue are higher altitudes.
01:46
And you can zoom in. This is taking a look at Atlanta.
01:48
You can see this is a major shipping airport,
01:50
and there's all kinds of activity there.
01:52
You can also toggle between altitude
01:54
for model and manufacturer.
01:57
See again, the diversity.
01:59
And you can scroll around and see
02:01
some of the different airports and the different patterns that they have.
02:03
This is scrolling up the East coast.
02:06
You can see some of the chaos that's happening in New York
02:08
with the air traffic controllers
02:10
having to deal with all those major airports next to each other.
02:12
So zooming back out real quick,
02:17
we see, again, the U.S. -- you get Florida down in the right-hand corner.
02:19
Moving across to the West coast,
02:22
you see San Francisco and Los Angeles --
02:24
big low-traffic zones
02:26
across Nevada and Arizona.
02:28
And that's us down there in L.A. and Long Beach on the bottom.
02:30
I started taking a look as well at different perimeters,
02:36
because you can choose what you want to pull out from the data.
02:38
This is looking at ascending versus descending flights.
02:40
And you can see, over time, the ways the airports change.
02:43
You see the holding patterns that start to develop
02:45
in the bottom of the screen.
02:47
And you can see, eventually the airport actually flips directions.
02:49
So this is another project that I worked on with the Sensible Cities Lab at MIT.
02:53
This is visualizing international communications.
02:56
So it's how New York communicates
02:58
with other international cities.
03:00
And we set this up as a live globe in the Museum of Modern Art in New York
03:02
for the Design the Elastic Mind exhibition.
03:05
And it had a live feed with a 24-hour offset,
03:07
so you could see the changing relationship
03:09
and some demographic info
03:11
coming through AT&T's data and revealing itself.
03:13
This is another project I worked on with Sensible Cities Lab
03:16
and CurrentCity.org.
03:18
And it's visualizing SMS messages being sent in the city of Amsterdam.
03:20
So you're seeing the daily ebb and flow
03:23
of people sending SMS messages from different parts of the city,
03:25
until we approach New Year's Eve, where everybody says, "Happy New Year!"
03:27
(Laughter)
03:30
So this is an interactive tool
03:32
that you can move around and see different parts of the city.
03:34
This is looking at another event. This is called Queen's Day.
03:37
So again, you get this daily ebb and flow
03:40
of people sending SMS messages from different parts of the city.
03:42
And then you're going to see people start to gather in the center of the city
03:45
to celebrate the night before,
03:47
which happens right here.
03:49
And then you can see people celebrating the next day.
03:51
And you can pause it and step back and forth and see different phases.
03:53
So now on to something completely different.
03:56
Some of you may recognize this.
03:58
This is Baron Wolfgang von Kempelen's mechanical chess playing machine.
04:00
And it's this amazing robot that plays chess extremely well,
04:03
except for one thing: it's not a robot at all.
04:05
There's actually a legless man that sits in that box
04:08
and controls this chess player.
04:10
This was the inspiration for a web service by Amazon
04:12
called the Mechanical Turk -- named after this guy.
04:14
And it's based on the premise that there are certain things
04:17
that are easy for people, but really difficult for computers.
04:19
So they made this web service and said,
04:21
"Any programmer can write a piece of software
04:23
and tap into the minds of thousands of people."
04:25
The nerdy side of me thought, "Wow, this is amazing.
04:27
I can tap into thousands of people's minds."
04:29
And the other nerdy side of me thought,
04:31
"This is horrible. This is completely bizarre.
04:33
What does this mean for the future of mankind,
04:36
where we're all plugged into this borg?"
04:38
I was probably being a little extreme.
04:40
But what does this mean when we have no context for what it is that we're working on,
04:42
and we're just doing these little labors?
04:44
So I created this drawing tool.
04:46
I asked people to draw a sheep facing to the left.
04:48
And I said, "I'll pay you two cents for your contribution."
04:50
And I started collecting sheep.
04:52
And I collected a lot, a lot of different sheep.
04:55
Lots of sheep.
04:59
I took the first 10,000 sheep that I collected,
05:01
and I put them on a website called TheSheepMarket.com
05:03
where you can actually buy collections of 20 sheep.
05:06
You can't pick individual sheep,
05:09
but you can buy a single plate block of stamps as a commodity.
05:11
And juxtaposed against this grid,
05:15
you see actually, by rolling over each individual one,
05:17
the humanity behind this hugely mechanical process.
05:19
I think there's something really interesting
05:22
to watching people as they go through this creative toil --
05:24
something we can all relate to,
05:27
this creative process of trying to come up with something from nothing.
05:29
I think it was really interesting to juxtapose this humanity
05:32
versus this massive distributed grid.
05:34
Kind of amazing what some people did.
05:36
So here's a few statistics from the project.
05:39
Approximate collection rate of 11 sheep per hour,
05:41
which would make a working wage of 69 cents per hour.
05:43
There were 662 rejected sheep
05:46
that didn't meet "sheep-like" criteria and were thrown out of the flock.
05:48
(Laughter)
05:51
The amount of time spent drawing ranged from four seconds to 46 minutes.
05:53
That gives you an idea of the different types of motivations and dedication.
05:56
And there were 7,599 people that contributed to the project,
05:59
or were unique IP addresses --
06:02
so about how many people contributed.
06:04
But only one of them out of the 7,599 said this.
06:06
(Laughter)
06:10
Which I was pretty surprised by.
06:14
I expected people to be wondering, "Why did I draw a sheep?"
06:16
And I think it's a pretty valid question.
06:19
And there's a lot of reasons why I chose sheep.
06:21
Sheep were the first animal
06:23
to be raised from mechanically processed byproducts,
06:25
the first to be selectively bred for production traits,
06:27
the first animal to be cloned.
06:29
Obviously, we think of sheep as followers.
06:31
And there's this reference to "Le Petit Prince"
06:33
where the narrator asks the prince to draw a sheep.
06:35
He draws sheep after sheep.
06:37
The narrator's only appeased when he draws a box.
06:39
And he says, "It's not about a scientific rendering of a sheep.
06:41
It's about your own interpretation and doing something different."
06:43
And I like that.
06:46
So this is a clip from Charlie Chaplin's "Modern Times."
06:48
It's showing Charlie Chaplin dealing with some of the major changes
06:50
during the Industrial Revolution.
06:53
So there were no longer shoe makers,
06:55
but now there are people slapping soles on people's shoes.
06:57
And the whole idea of one's relationship to their work changed a lot.
06:59
So I thought this was an interesting clip
07:02
to divide into 16 pieces
07:04
and feed into the Mechanical Turk with a drawing tool.
07:06
This basically allowed -- what you see on the left side is the original frame,
07:09
and on the right side you see that frame
07:12
as interpreted by 16 people
07:14
who have no idea what it is they're doing.
07:16
And this was the inspiration for a project
07:18
that I worked on with my friend Takashi Kawashima.
07:20
We decided to use the Mechanical Turk for exactly what it was meant for,
07:22
which is making money.
07:24
So we took a hundred dollar bill and divided it into 10,000 teeny pieces,
07:26
and we fed those into the Mechanical Turk.
07:29
We asked people to draw what it was that they saw.
07:31
But here there was no sheep-like criteria.
07:33
People, if they drew a stick figure or a smiley face,
07:35
it actually made it into the bill.
07:38
So what you see is actually a representation of how well people did
07:40
what it was they were asked to do.
07:42
So we took these hundred dollar bills,
07:44
and we put them on a website called TenThousandsCents.com,
07:46
where you can browse through and see all the individual contributions.
07:48
And you can also trade real hundred-dollar bills for fake hundred-dollar bills
07:51
and make a donation to the Hundred Dollar Laptop Project,
07:54
which is now known as One Laptop Per Child.
07:57
This is again showing all the different contributions.
08:00
You see some people did beautiful stipple renderings,
08:02
like this one on top --
08:04
spent a long time making realistic versions.
08:06
And other people would draw stick figures or smiley faces.
08:09
Here on the right-hand side in the middle
08:12
you see this one guy writing, "$0.01!!! Really?"
08:14
That's all I'm getting paid for this?
08:17
(Laughter)
08:21
So the last Mechanical Turk project I'm going to talk to you about
08:23
is called Bicycle Built for 2000.
08:25
This is a collaboration with my friend Daniel Massey.
08:27
You may recognize these two guys.
08:29
This is Max Mathews and John Kelly from Bell Labs in the '60s,
08:31
where they created the song "Daisy Bell,"
08:34
which was the world's first singing computer.
08:36
You may recognize it from "2001: A Space Odyssey."
08:38
When HAL's dying at the end of the film he starts singing this song,
08:40
as a reference to when computers became human.
08:43
So we resynthesized this song.
08:46
This is what that sounded like.
08:48
We broke down all the individual notes
08:50
in the singing as well as the phonemes in the singing.
08:52
Daisy Bell: ♫ Daisy, Daisy ... ♫
08:55
Aaron Koblin: And we took all of those individual pieces,
08:59
and we fed them into another Turk request.
09:01
This is what it would look like if you went to the site.
09:03
You type in your code,
09:05
but you first test your mic.
09:07
You'd be fed a simple audio clip.
09:09
(Honk)
09:11
And then you'd do your best to recreate that with your own voice.
09:13
After previewing it and confirming it's what you submitted,
09:22
you could submit it into the Mechanical Turk with no other context.
09:25
And this is what we first got back from the very first set of submissions.
09:28
Recording: ♫ Daisy, Daisy ♫
09:31
♫ give me your answer do ♫
09:36
♫ I'm half crazy ♫
09:41
♫ all for the love of you ♫
09:45
♫ It can't be a stylish marriage ♫
09:50
♫ I can't afford a carriage ♫
09:55
♫ But you'll look sweet upon the seat ♫
09:59
♫ of a bicycle built for two ♫
10:04
AK: So James Surowieki has this idea of the wisdom of crowds,
10:09
that says that a whole bunch of people are smarter than any individual.
10:12
We wanted to see how this applies to collaborative, distributed music making,
10:15
where nobody has any idea what it is they're working on.
10:18
So if you go to the BicycleBuiltforTwoThousand.com
10:21
you can actually hear what all this sounds like together.
10:23
I'm sorry for this.
10:25
(Noise)
10:27
Chorus: ♫ Daisy, Daisy ♫
10:32
♫ Give me your answer do ♫
10:36
♫ I'm half crazy ♫
10:41
♫ all for the love of you ♫
10:46
♫ It can't be a stylish marriage ♫
10:50
♫ I can't afford a carriage ♫
10:55
♫ But you'd look sweet upon the seat ♫
10:59
♫ of a bicycle built for two ♫
11:04
AK: So stepping back for a quick second,
11:10
when I was at UCLA going to grad school,
11:13
I was also working at a place called the Center for Embedded Network Sensing.
11:15
And I was writing software to visualize laser scanners.
11:18
So basically motion through 3D space.
11:21
And this was seen by a director in L.A. named James Frost
11:23
who said, "Wait a minute.
11:25
You mean we can shoot a music video without actually using any video?"
11:27
So we did exactly that.
11:29
We made a music video for one of my favorite bands, Radiohead.
11:31
And I think one of my favorite parts of this project
11:33
was not just shooting a video with lasers,
11:35
but we also open sourced it,
11:37
and we made it released as a Google Code project,
11:39
where people could download a bunch of the data and some source code
11:41
to build their own versions of it.
11:43
And people were making some amazing things.
11:45
This is actually two of my favorites:
11:47
the pin-board Thom Yorke and a LEGO Thom Yorke.
11:49
A whole YouTube channel of people submitting really interesting content.
11:51
More recently, somebody even 3D-printed Thom Yorke's head,
11:54
which is a little creepy, but pretty cool.
11:57
So with everybody making so much amazing stuff
12:00
and actually understanding what it was they were working on,
12:02
I was really interested in trying to make a collaborative project
12:05
where people were working together to build something.
12:07
And I met a music video director named Chris Milk.
12:09
And we started bouncing around ideas
12:11
to make a collaborative music video project.
12:13
But we knew we really needed the right person
12:15
to kind of rally behind and build something for.
12:17
So we put the idea on the back burner for a few months.
12:20
And he ended up talking to Rick Rubin,
12:22
who was finishing up Johnny Cash's final album
12:24
called "Ain't No Grave."
12:26
The lyrics to the leading track are "Ain't no grave can hold my body down."
12:28
So we thought this was the perfect
12:31
project to build a collaborative memorial
12:33
and a virtual resurrection for Johnny Cash.
12:35
So I teamed up with my good friend Ricardo Cabello, also known as Mr. doob,
12:37
who's a much better programmer than I am,
12:40
and he made this amazing Flash drawing tool.
12:42
As you know,
12:44
an animation is a series of images.
12:46
So what we did was cross-cut a bunch of archival footage of Johnny Cash,
12:48
and at eight frames a second,
12:51
we allowed individuals to draw a single frame
12:53
that would get woven into
12:55
this dynamically changing music video.
12:57
So I don't have time to play the entire thing for you,
12:59
but I want to show you two short clips.
13:01
One is the beginning of the music video.
13:03
And that's going to be followed by a short clip
13:05
of people who have already contributed to the project
13:07
talking about it briefly.
13:09
(Music)
13:12
(Video) Johnny Cash: ♫ There ain't no grave ♫
13:18
♫ can hold my body down ♫
13:20
♫ There ain't no grave ♫
13:24
♫ can hold body down ♫
13:26
♫ When I hear the trumpet sound ♫
13:30
♫ I'm going to ride right out of the ground ♫
13:34
♫ Ain't no grave ♫
13:36
♫ can hold my body ... ♫
13:39
(Applause)
13:42
AK: What better way to pay tribute to the man
13:44
than to make something for one of his songs.
13:47
Collaborator: I felt really sad when he died.
13:50
And I just thought it'd be wonderful,
13:52
it'd be really nice to contribute something to his memory.
13:54
Collaborator Two: It really allows
13:57
this last recording of his
13:59
to be a living, breathing memorial.
14:02
Collaborator Three: For all of the frames to be drawn by fans,
14:07
each individual frame,
14:10
it's got a very powerful feeling to it.
14:12
Collaborator Four: I've seen everybody
14:14
from Japan, Venezuela, to the States,
14:16
to Knoxville, Tennessee.
14:18
Collaborator Five: As much as is different from frame to frame,
14:20
it really is personal.
14:23
Collaborator Six: Watching the video in my room,
14:25
I could see me not understanding at the beginning of it.
14:27
And I just worked and worked through problems,
14:30
until my little wee battles that I was fighting within the picture
14:33
all began to resolve themselves.
14:37
You can actually see the point when I know what I'm doing,
14:40
and a lot of light and dark comes into it.
14:42
And in a weird way,
14:45
that's what I actually like about Johnny Cash's music as well.
14:47
It's the sum total of his life,
14:49
all the things that had happened --
14:51
the bad things, the good things.
14:53
You're hearing a person's life.
14:55
AK: So if you go to the website JohnnyCashProject.com,
15:01
what you'll see is the video playing above.
15:03
And below it are all the individual frames
15:05
that people have been submitting to the project.
15:07
So this isn't finished at all,
15:09
but it's an ongoing project where people can continue to collaborate.
15:11
If you roll over any one of those individual thumbnails,
15:13
you can see the person who drew that individual thumbnail
15:15
and where they were located.
15:17
And if you find one that you're interested in,
15:19
you can actually click on it and open up an information panel
15:21
where you're able to rate that frame,
15:23
which helps it bubble up to the top.
15:25
And you can also see the way that it was drawn.
15:27
Again, you can get the playback and personal contribution.
15:29
In addition to that, it's listed, the artist's name, the location,
15:31
how long they spent drawing it.
15:34
And you can pick a style. So this one was tagged "Abstract."
15:36
But there's a bunch of different styles.
15:39
And you can sort the video a number of different ways.
15:41
You can say, "I want to see the pointillist version
15:43
or the sketchy version or the realistic version.
15:45
And then this is, again, the abstract version,
15:47
which ends up getting a little bit crazy.
15:49
So the last project I want to talk to you about is another collaboration with Chris Milk.
15:54
And this is called "The Wilderness Downtown."
15:57
It's an online music video for the Arcade Fire.
15:59
Chris and I were really amazed
16:02
by the potential now with modern web browsers,
16:04
where you have HTML5 audio and video
16:06
and the power of JavaScript to render amazingly fast.
16:08
And we wanted to push the idea of the music video that was meant for the Web
16:11
beyond the four-by-three or sixteen-by-nine window
16:14
and try to make it play out and choreograph throughout the screen.
16:17
But most importantly, I think,
16:20
we really wanted to make an experience that was unlike the Johnny Cash Project,
16:22
where you had a small group of people spending a lot of time
16:25
to contribute something for everyone.
16:28
What if we had a very low commitment,
16:30
but delivered something individually unique to each person who contributed?
16:33
So the project starts off by asking you to enter the address
16:36
of the home where you grew up.
16:38
And you type in the address --
16:40
it actually creates a music video specifically for you,
16:42
pulling in Google maps and Streetview images
16:44
into the experience itself.
16:46
So this should really be seen at home with you typing in your own address,
16:48
but I'm going to give you a little preview of what you can expect.
16:51
(Video) Win Butler: ♫ Now our lives are changing fast ♫
16:54
♫ Now our lives are changing fast ♫
16:58
♫ Hope that something pure can last ♫
17:02
♫ Hope that something pure can last ♫
17:06
♫ Ooh we used to wait ♫
17:13
♫ Ooh we used to wait ♫
17:17
♫ Ooh we used to wait ♫
17:21
♫ Sometimes it never came ♫
17:24
♫ Sometimes it never came ♫
17:27
♫ Still moving through the pain ♫
17:31
♫ We used to wait for it ♫
17:34
♫ We used to wait for it ♫
17:38
♫ We used to wait for it ♫
17:42
AK: So I think, if there's one thing to take away from my talk today,
17:50
it's that an interface can be a powerful narrative device.
17:53
And as we collect more and more personally and socially relevant data,
17:55
we have an opportunity, and maybe even an obligation,
17:59
to maintain the humanity and tell some amazing stories
18:01
as we explore and collaborate together.
18:03
Thanks a lot.
18:05
(Applause)
18:07

sponsored links

Aaron Koblin - Data artist
Aaron Koblin is an artist specializing in data and digital technologies. His work takes real world and community-generated data and uses it to reflect on cultural trends and the changing relationship between humans and technology.

Why you should listen

Aaron Koblin finds art through the unlikely confluence of massive data sets and personal intimacy. His work ranges from animating the paths of every North American airline flight, to using Amazon’s Mechanical Turk crowdsourcing platform to pay workers to “draw a sheep facing left,” which were then placed in "The Sheep Market."

Koblin was creative director for Johnny Cash's final music video, "Ain't No Grave," and worked on Radiohead’s video “House of Cards,” both of which received a Grammy nomination. He is now the Creative Director of the Data Arts team in Google's Creative Lab. His team collaborated with Arcade Fire to produce an online music video that allows viewers to incorporate images of their home neighborhood into the experience using Google Street View.

sponsored links

If you need translations, you can install "Google Translate" extension into your Chrome Browser.
Furthermore, you can change playback rate by installing "Video Speed Controller" extension.

Data provided by TED.

This website is owned and operated by Tokyo English Network.
The developer's blog is here.