sponsored links
TED Fellows Retreat 2013

James Patten: The best computer interface? Maybe ... your hands

August 20, 2013

"The computer is an incredibly powerful means of creative expression," says designer and TED Fellow James Patten. But right now, we interact with computers, mainly, by typing and tapping. In this nifty talk and demo, Patten imagines a more visceral, physical way to bring your thoughts and ideas to life in the digital world, taking the computer interface off the screen and putting it into your hands.

James Patten - Interaction designer
James Patten, a TED Fellow, imagines new ways for us to play with computers. Full bio

sponsored links
Double-click the English subtitles below to play the video.
A computer is an incredibly powerful means
00:12
of creative expression,
00:15
but for the most part,
00:16
that expression is confined to the screens
00:18
of our laptops and mobile phones.
00:20
And I'd like to tell you a story about
00:23
bringing this power of the computer
00:25
to move things around and interact with us
00:27
off of the screen and into the physical world
00:29
in which we live.
00:32
A few years ago, I got a call from
00:33
a luxury fashion store called Barneys New York,
00:35
and the next thing I knew,
00:37
I was designing storefront kinetic sculptures
00:39
for their window displays.
00:41
This one's called "The Chase."
00:43
There are two pairs of shoes,
00:44
a man's pair and a woman's pair,
00:45
and they play out this slow, tense chase
00:47
around the window
00:50
in which the man scoots up behind the woman
00:51
and gets in her personal space,
00:53
and then she moves away.
00:55
Each of the shoes has magnets in it,
00:57
and there are magnets underneath the table
00:59
that move the shoes around.
01:01
My friend Andy Cavatorta was building
01:03
a robotic harp for Bjork's Biophilia tour
01:05
and I wound up building the electronics
01:08
and motion control software
01:11
to make the harps move and play music.
01:13
The harp has four separate pendulums,
01:15
and each pendulum has 11 strings,
01:17
so the harp swings on its axis and also rotates
01:19
in order to play different musical notes,
01:22
and the harps are all networked together
01:24
so that they can play the right notes
01:26
at the right time in the music.
01:28
I built an interactive chemistry exhibit
01:31
at the Museum of Science and Industry in Chicago,
01:33
and this exhibit lets people use physical objects
01:36
to grab chemical elements off of the periodic table
01:39
and bring them together to cause
01:42
chemical reactions to happen.
01:43
And the museum noticed that people
01:45
were spending a lot of time with this exhibit,
01:47
and a researcher from a science education center
01:49
in Australia decided to study this exhibit
01:52
and try to figure out what was going on.
01:55
And she found that the physical objects
01:57
that people were using were helping people
02:00
understand how to use the exhibit,
02:01
and were helping people learn in a social way.
02:03
And when you think about
it, this makes a lot of sense,
02:06
that using specialized physical objects
02:08
would help people use an interface more easily.
02:11
I mean, our hands and our minds are optimized
02:14
to think about and interact with tangible objects.
02:17
Think about which you find easier to use,
02:20
a physical keyboard or an onscreen keyboard
02:22
like on a phone?
02:24
But the thing that struck me
02:27
about all of these different projects
02:28
is that they really had to be built from scratch,
02:30
down to the level of the electronics
02:33
and the printed circuit boards and
02:34
all the mechanisms all the way up to the software.
02:36
I wanted to create something
02:38
where we could move objects
under computer control
02:40
and create interactions around that idea
02:43
without having to go through this process
02:46
of building something from scratch
02:48
every single time.
02:49
So my first attempt at this
02:51
was at the MIT Media Lab
02:53
with Professor Hiroshi Ishii,
02:54
and we built this array of
02:56
512 different electromagnets,
02:58
and together they were able to move objects around
03:00
on top of their surface.
03:03
But the problem with this
03:04
was that these magnets
03:06
cost over 10,000 dollars.
03:07
Although each one was pretty small,
03:10
altogether they weighed so much
03:11
that the table that they were on
03:13
started to sag.
03:14
So I wanted to build something
03:16
where you could have this kind of interaction
03:17
on any tabletop surface.
03:19
So to explore this idea,
03:21
I built an army of small robots,
03:23
and each of these robots has
what are called omni wheels.
03:26
They're these special wheels
03:28
that can move equally easily in all directions,
03:30
and when you couple these robots
03:32
with a video projector,
03:35
you have these physical tools
03:37
for interacting with digital information.
03:39
So here's an example of what I mean.
03:41
This is a video editing application
03:43
where all of the controls
03:45
for manipulating the video are physical.
03:47
So if we want to tweak the color,
03:49
we just enter the color mode,
03:51
and then we get three different dials
03:52
for tweaking the color,
03:53
or if we want to adjust the audio,
03:55
then we get two different dials
for that, these physical objects.
03:57
So here the left and right channel stay in sync,
04:00
but if we want to, we can override that
04:03
by grabbing both of them at the same time.
04:05
So the idea is that we get the speed
04:08
and efficiency benefits of using these physical dials
04:09
together with the flexibility and versatility
04:13
of a system that's designed in software.
04:15
And this is a mapping application
04:19
for disaster response.
04:21
So you have these physical objects
04:23
that represent police, fire and rescue,
04:25
and a dispatcher can grab them
04:27
and place them on the map
04:28
to tell those units where to go,
04:30
and then the position of the units on the map
04:32
gets synced up with the position
04:34
of those units in the real world.
04:36
This is a video chat application.
04:40
It's amazing how much emotion you can convey
04:41
with just a few simple movements
04:44
of a physical object.
04:45
With this interface, we open up
a huge array of possibilities
04:47
in between traditional board games
04:50
and arcade games,
04:53
where the physical possibilities of interaction
04:54
make so many different styles of play possible.
04:56
But one of the areas that I'm most excited
05:00
about using this platform for
05:03
is applying it to problems that are difficult
05:04
for computers or people to solve alone.
05:07
One example of those is protein folding.
05:09
So here we have an interface
05:12
where we have physical handles onto a protein,
05:13
and we can grab those handles
05:17
and try to move the protein and
try to fold it in different ways.
05:18
And if we move it in a way that
doesn't really make sense
05:21
with the underlying molecular simulation,
05:24
we get this physical feedback where we can
05:26
actually feel these physical handles
05:29
pulling back against us.
05:31
So feeling what's going on
05:32
inside a molecular simulation
05:34
is a whole different level of interaction.
05:36
So we're just beginning to explore
05:39
what's possible when we use software
05:42
to control the movement
05:44
of objects in our environment.
05:46
Maybe this is the computer of the future.
05:48
There's no touchscreen.
05:51
There's no technology visible at all.
05:53
But when we want to have a video chat
05:55
or play a game
05:57
or lay out the slides to our next TED Talk,
05:58
the objects on the table come alive.
06:01
Thank you.
06:03
(Applause)
06:04

sponsored links

James Patten - Interaction designer
James Patten, a TED Fellow, imagines new ways for us to play with computers.

Why you should listen

James Patten is an interaction designer, inventor and visual artist working at the intersection of the physical and digital worlds. Patten is a TED Fellow and the founder and principal of the design firm Patten Studio, where his clients have included Björk, Barneys New York, General Electric, Steelcase and Autodesk. Patten earned his doctorate at the MIT Media Lab where he studied in the Tangible Media Group under Hiroshi Ishii. His work has been exhibited or performed in venues such as the Museum of Modern Art, the Transmediale festival in Berlin, the Museo Guggenheim Bilbao and the Ars Electronica Center in Linz, Austria. Patten's work has been recognized in several international design competitions including the International Design Magazine's Annual Design Review, and the International Design Excellence Awards. Tour his Brooklyn workshop.

sponsored links

If you need translations, you can install "Google Translate" extension into your Chrome Browser.
Furthermore, you can change playback rate by installing "Video Speed Controller" extension.

Data provided by TED.

This website is owned and operated by Tokyo English Network.
The developer's blog is here.