TED Fellows Retreat 2013
James Patten: The best computer interface? Maybe ... your hands
Filmed:
Readability: 5
1,092,626 views
"The computer is an incredibly powerful means of creative expression," says designer and TED Fellow James Patten. But right now, we interact with computers, mainly, by typing and tapping. In this nifty talk and demo, Patten imagines a more visceral, physical way to bring your thoughts and ideas to life in the digital world, taking the computer interface off the screen and putting it into your hands.
James Patten - Interaction designer
James Patten, a TED Fellow, imagines new ways for us to play with computers. Full bio
James Patten, a TED Fellow, imagines new ways for us to play with computers. Full bio
Double-click the English transcript below to play the video.
00:12
A computer is an incredibly powerful means
0
721
2667
00:15
of creative expression,
1
3388
1654
00:17
but for the most part,
2
5042
1571
00:18
that expression is confined to the screens
3
6613
2165
00:20
of our laptops and mobile phones.
4
8778
2418
00:23
And I'd like to tell you a story about
5
11196
2200
00:25
bringing this power of the computer
6
13396
1956
00:27
to move things around and interact with us
7
15352
2370
00:29
off of the screen and into the physical world
8
17722
2913
00:32
in which we live.
9
20635
1189
00:33
A few years ago, I got a call from
10
21824
1618
00:35
a luxury fashion store called Barneys New York,
11
23442
2486
00:37
and the next thing I knew,
12
25928
1264
00:39
I was designing storefront kinetic sculptures
13
27192
2693
00:41
for their window displays.
14
29885
1392
00:43
This one's called "The Chase."
15
31277
1395
00:44
There are two pairs of shoes,
16
32672
1346
00:46
a man's pair and a woman's pair,
17
34018
1625
00:47
and they play out this slow, tense chase
18
35643
2904
00:50
around the window
19
38547
1278
00:51
in which the man scoots up behind the woman
20
39825
2225
00:54
and gets in her personal space,
21
42050
1573
00:55
and then she moves away.
22
43623
1935
00:57
Each of the shoes has magnets in it,
23
45558
1841
00:59
and there are magnets underneath the table
24
47399
2003
01:01
that move the shoes around.
25
49402
1964
01:03
My friend Andy Cavatorta was building
26
51366
2032
01:05
a robotic harp for Bjork's Biophilia tour
27
53398
3542
01:08
and I wound up building the electronics
28
56940
2392
01:11
and motion control software
29
59332
2189
01:13
to make the harps move and play music.
30
61521
2466
01:15
The harp has four separate pendulums,
31
63987
2011
01:17
and each pendulum has 11 strings,
32
65998
2126
01:20
so the harp swings on its axis and also rotates
33
68124
2488
01:22
in order to play different musical notes,
34
70612
2203
01:24
and the harps are all networked together
35
72815
2162
01:26
so that they can play the right notes
36
74977
1800
01:28
at the right time in the music.
37
76777
2626
01:31
I built an interactive chemistry exhibit
38
79403
2424
01:33
at the Museum of Science and Industry in Chicago,
39
81827
2599
01:36
and this exhibit lets people use physical objects
40
84426
2981
01:39
to grab chemical elements off of the periodic table
41
87407
3036
01:42
and bring them together to cause
42
90443
1408
01:43
chemical reactions to happen.
43
91851
1766
01:45
And the museum noticed that people
44
93617
1924
01:47
were spending a lot of time with this exhibit,
45
95541
2226
01:49
and a researcher from a science education center
46
97767
3147
01:52
in Australia decided to study this exhibit
47
100914
2805
01:55
and try to figure out what was going on.
48
103719
2240
01:57
And she found that the physical objects
49
105959
2293
02:00
that people were using were helping people
50
108252
1664
02:01
understand how to use the exhibit,
51
109916
2014
02:03
and were helping people learn in a social way.
52
111930
2943
02:06
And when you think about
it, this makes a lot of sense,
it, this makes a lot of sense,
53
114873
2266
02:09
that using specialized physical objects
54
117139
2240
02:11
would help people use an interface more easily.
55
119379
3272
02:14
I mean, our hands and our minds are optimized
56
122651
2839
02:17
to think about and interact with tangible objects.
57
125490
3269
02:20
Think about which you find easier to use,
58
128759
2062
02:22
a physical keyboard or an onscreen keyboard
59
130821
2319
02:25
like on a phone?
60
133140
2100
02:27
But the thing that struck me
61
135240
1025
02:28
about all of these different projects
62
136265
1999
02:30
is that they really had to be built from scratch,
63
138264
3117
02:33
down to the level of the electronics
64
141381
1790
02:35
and the printed circuit boards and
65
143171
1106
02:36
all the mechanisms all the way up to the software.
66
144277
2745
02:39
I wanted to create something
67
147022
1762
02:40
where we could move objects
under computer control
under computer control
68
148784
3319
02:44
and create interactions around that idea
69
152103
2249
02:46
without having to go through this process
70
154352
2215
02:48
of building something from scratch
71
156567
1383
02:49
every single time.
72
157950
1375
02:51
So my first attempt at this
73
159325
1991
02:53
was at the MIT Media Lab
74
161316
1684
02:55
with Professor Hiroshi Ishii,
75
163000
1581
02:56
and we built this array of
76
164581
1835
02:58
512 different electromagnets,
77
166416
2257
03:00
and together they were able to move objects around
78
168673
2627
03:03
on top of their surface.
79
171300
1536
03:04
But the problem with this
80
172836
1548
03:06
was that these magnets
81
174384
1656
03:08
cost over 10,000 dollars.
82
176040
2160
03:10
Although each one was pretty small,
83
178200
1666
03:11
altogether they weighed so much
84
179866
1709
03:13
that the table that they were on
85
181575
1255
03:14
started to sag.
86
182830
1558
03:16
So I wanted to build something
87
184388
1332
03:17
where you could have this kind of interaction
88
185720
1616
03:19
on any tabletop surface.
89
187336
2643
03:21
So to explore this idea,
90
189979
2081
03:24
I built an army of small robots,
91
192060
2150
03:26
and each of these robots has
what are called omni wheels.
what are called omni wheels.
92
194210
2765
03:28
They're these special wheels
93
196975
1305
03:30
that can move equally easily in all directions,
94
198280
2666
03:32
and when you couple these robots
95
200946
2589
03:35
with a video projector,
96
203535
1910
03:37
you have these physical tools
97
205445
1808
03:39
for interacting with digital information.
98
207253
2900
03:42
So here's an example of what I mean.
99
210153
1842
03:43
This is a video editing application
100
211995
1922
03:45
where all of the controls
101
213917
1486
03:47
for manipulating the video are physical.
102
215403
2307
03:49
So if we want to tweak the color,
103
217710
1582
03:51
we just enter the color mode,
104
219292
1352
03:52
and then we get three different dials
105
220644
1296
03:53
for tweaking the color,
106
221940
1378
03:55
or if we want to adjust the audio,
107
223318
1961
03:57
then we get two different dials
for that, these physical objects.
for that, these physical objects.
108
225279
3341
04:00
So here the left and right channel stay in sync,
109
228620
2846
04:03
but if we want to, we can override that
110
231466
1743
04:05
by grabbing both of them at the same time.
111
233209
2999
04:08
So the idea is that we get the speed
112
236208
1928
04:10
and efficiency benefits of using these physical dials
113
238136
3117
04:13
together with the flexibility and versatility
114
241253
2790
04:16
of a system that's designed in software.
115
244043
3331
04:19
And this is a mapping application
116
247374
1935
04:21
for disaster response.
117
249309
2127
04:23
So you have these physical objects
118
251436
1945
04:25
that represent police, fire and rescue,
119
253381
2227
04:27
and a dispatcher can grab them
120
255608
1553
04:29
and place them on the map
121
257161
1326
04:30
to tell those units where to go,
122
258487
1989
04:32
and then the position of the units on the map
123
260476
2376
04:34
gets synced up with the position
124
262852
1969
04:36
of those units in the real world.
125
264821
3627
04:40
This is a video chat application.
126
268448
1612
04:42
It's amazing how much emotion you can convey
127
270060
2352
04:44
with just a few simple movements
128
272412
1213
04:45
of a physical object.
129
273625
2425
04:48
With this interface, we open up
a huge array of possibilities
a huge array of possibilities
130
276050
3052
04:51
in between traditional board games
131
279102
2078
04:53
and arcade games,
132
281180
1414
04:54
where the physical possibilities of interaction
133
282594
2547
04:57
make so many different styles of play possible.
134
285141
3713
05:00
But one of the areas that I'm most excited
135
288854
2389
05:03
about using this platform for
136
291243
1814
05:05
is applying it to problems that are difficult
137
293057
2543
05:07
for computers or people to solve alone.
138
295600
2254
05:09
One example of those is protein folding.
139
297854
2622
05:12
So here we have an interface
140
300476
1412
05:13
where we have physical handles onto a protein,
141
301888
3514
05:17
and we can grab those handles
142
305402
1751
05:19
and try to move the protein and
try to fold it in different ways.
try to fold it in different ways.
143
307153
2987
05:22
And if we move it in a way that
doesn't really make sense
doesn't really make sense
144
310140
3000
05:25
with the underlying molecular simulation,
145
313140
1967
05:27
we get this physical feedback where we can
146
315107
2202
05:29
actually feel these physical handles
147
317309
1973
05:31
pulling back against us.
148
319282
1608
05:32
So feeling what's going on
149
320890
1531
05:34
inside a molecular simulation
150
322421
2239
05:36
is a whole different level of interaction.
151
324660
2946
05:39
So we're just beginning to explore
152
327606
2981
05:42
what's possible when we use software
153
330587
2511
05:45
to control the movement
154
333098
1491
05:46
of objects in our environment.
155
334589
2243
05:48
Maybe this is the computer of the future.
156
336832
2904
05:51
There's no touchscreen.
157
339736
1616
05:53
There's no technology visible at all.
158
341352
2031
05:55
But when we want to have a video chat
159
343383
1844
05:57
or play a game
160
345227
1461
05:58
or lay out the slides to our next TED Talk,
161
346688
2713
06:01
the objects on the table come alive.
162
349401
2273
06:03
Thank you.
163
351674
1352
06:05
(Applause)
164
353026
2123
ABOUT THE SPEAKER
James Patten - Interaction designerJames Patten, a TED Fellow, imagines new ways for us to play with computers.
Why you should listen
James Patten is an interaction designer, inventor and visual artist working at the intersection of the physical and digital worlds. Patten is a TED Fellow and the founder and principal of the design firm Patten Studio, where his clients have included Björk, Barneys New York, General Electric, Steelcase and Autodesk. Patten earned his doctorate at the MIT Media Lab where he studied in the Tangible Media Group under Hiroshi Ishii. His work has been exhibited or performed in venues such as the Museum of Modern Art, the Transmediale festival in Berlin, the Museo Guggenheim Bilbao and the Ars Electronica Center in Linz, Austria. Patten's work has been recognized in several international design competitions including the International Design Magazine's Annual Design Review, and the International Design Excellence Awards. Tour his Brooklyn workshop.
James Patten | Speaker | TED.com