TED2009
David Merrill: Toy tiles that talk to each other
Filmed:
Readability: 4.4
1,947,080 views
MIT grad student David Merrill demos Siftables -- cookie-sized, computerized tiles you can stack and shuffle in your hands. These future-toys can do math, play music, and talk to their friends, too. Is this the next thing in hands-on learning?
David Merrill - MIT grad student
David Merrill works on Siftables, tiny computer blocks that interact with each other to make networks (and music). Full bio
David Merrill works on Siftables, tiny computer blocks that interact with each other to make networks (and music). Full bio
Double-click the English transcript below to play the video.
00:12
I want to start out by asking you to think back to when you were a kid,
0
0
2000
00:14
playing with blocks.
1
2000
2000
00:16
As you figured out how to reach out and grasp,
2
4000
2000
00:18
pick them up and move them around,
3
6000
2000
00:20
you were actually learning how to think and solve problems
4
8000
2000
00:22
by understanding and manipulating spatial relationships.
5
10000
3000
00:25
Spatial reasoning is deeply connected
6
13000
3000
00:28
to how we understand a lot of the world around us.
7
16000
2000
00:30
So, as a computer scientist
8
18000
2000
00:32
inspired by this utility of our interactions with physical objects --
9
20000
3000
00:35
along with my adviser Pattie, and my collaborator Jeevan Kalanithi --
10
23000
4000
00:39
I started to wonder -- what if when we used a computer,
11
27000
3000
00:42
instead of having this one mouse cursor that was a like a digital fingertip
12
30000
3000
00:45
moving around a flat desktop,
13
33000
2000
00:47
what if we could reach in with both hands and grasp information physically,
14
35000
4000
00:51
arranging it the way we wanted?
15
39000
2000
00:53
This question was so compelling that we decided to explore the answer,
16
41000
3000
00:56
by building Siftables.
17
44000
2000
00:58
In a nutshell, a Siftable is an interactive computer
18
46000
3000
01:01
the size of a cookie.
19
49000
2000
01:03
They're able to be moved around by hand,
20
51000
2000
01:05
they can sense each other, they can sense their motion,
21
53000
2000
01:07
and they have a screen and a wireless radio.
22
55000
3000
01:10
Most importantly, they're physical,
23
58000
2000
01:12
so like the blocks, you can move them just by reaching out and grasping.
24
60000
3000
01:15
And Siftables are an example of a new ecosystem
25
63000
3000
01:18
of tools for manipulating digital information.
26
66000
2000
01:20
And as these tools become more physical,
27
68000
3000
01:23
more aware of their motion, aware of each other,
28
71000
2000
01:25
and aware of the nuance of how we move them,
29
73000
2000
01:27
we can start to explore some new and fun interaction styles.
30
75000
3000
01:30
So, I'm going to start with some simple examples.
31
78000
2000
01:32
This Siftable is configured to show video,
32
80000
3000
01:35
and if I tilt it in one direction, it'll roll the video this way;
33
83000
2000
01:37
if I tilt it the other way it rolls it backwards.
34
85000
3000
01:40
And these interactive portraits are aware of each other.
35
88000
3000
01:43
So if I put them next to each other, they get interested.
36
91000
3000
01:46
If they get surrounded, they notice that too,
37
94000
4000
01:50
they might get a little flustered.
38
98000
2000
01:52
And they can also sense their motion and tilt.
39
100000
4000
01:56
One of the interesting implications on interaction, we started to realize,
40
104000
5000
02:01
was that we could use everyday gestures on data,
41
109000
2000
02:03
like pouring a color the way we might pour a liquid.
42
111000
3000
02:06
So in this case, we've got three Siftables configured to be paint buckets
43
114000
4000
02:10
and I can use them to pour color into that central one,
44
118000
3000
02:13
where they get mixed.
45
121000
2000
02:15
If we overshoot, we can pour a little bit back.
46
123000
3000
02:19
There are also some neat possibilities for education,
47
127000
3000
02:22
like language, math and logic games
48
130000
2000
02:24
where we want to give people the ability to try things quickly,
49
132000
2000
02:26
and view the results immediately.
50
134000
3000
02:29
So here I'm --
51
137000
2000
02:31
(Applause)
52
139000
2000
02:33
This is a Fibonacci sequence that I'm making with a simple equation program.
53
141000
4000
02:37
Here we have a word game that's kind of like a mash-up between Scrabble and Boggle.
54
145000
4000
02:41
Basically, in every round
55
149000
2000
02:43
you get a randomly assigned letter on each Siftable,
56
151000
2000
02:45
and as you try to make words it checks against a dictionary.
57
153000
3000
02:48
Then, after about 30 seconds, it reshuffles,
58
156000
3000
02:51
and you have a new set of letters and new possibilities to try.
59
159000
3000
02:54
(Applause)
60
162000
1000
02:55
Thank you.
61
163000
1000
02:56
(Applause)
62
164000
2000
02:58
So these are some kids that came on a field trip to the Media Lab,
63
166000
2000
03:00
and I managed to get them to try it out, and shoot a video.
64
168000
3000
03:10
They really loved it.
65
178000
2000
03:12
And, one of the interesting things about this kind of application
66
180000
3000
03:15
is that you don't have to give people many instructions.
67
183000
3000
03:18
All you have to say is, "Make words,"
68
186000
2000
03:20
and they know exactly what to do.
69
188000
2000
03:22
So here's another few people trying it out.
70
190000
2000
03:31
That's our youngest beta tester, down there on the right.
71
199000
3000
03:36
Turns out, all he wanted to do was to stack the Siftables up.
72
204000
5000
03:42
So to him, they were just blocks.
73
210000
3000
03:46
Now, this is an interactive cartoon application.
74
214000
3000
03:49
And we wanted to build a learning tool for language learners.
75
217000
5000
03:54
And this is Felix, actually.
76
222000
2000
03:57
And he can bring new characters into the scene,
77
225000
2000
03:59
just by lifting the Siftables off the table that have that character shown on them.
78
227000
4000
04:03
Here, he's bringing the sun out.
79
231000
3000
04:07
Video: The sun is rising.
80
235000
2000
04:09
David Merrill: Now he's brought a tractor into the scene.
81
237000
3000
04:13
Video: The orange tractor.
82
241000
2000
04:15
Good job! Yeah!
83
243000
2000
04:18
DM: So by shaking the Siftables and putting them next to each other
84
246000
3000
04:21
he can make the characters interact --
85
249000
2000
04:23
Video: Woof!
86
251000
1000
04:24
DM: inventing his own narrative.
87
252000
2000
04:26
Video: Hello!
88
254000
1000
04:27
DM: It's an open-ended story,
89
255000
2000
04:29
and he gets to decide how it unfolds.
90
257000
2000
04:34
Video: Fly away, cat.
91
262000
2000
04:42
DM: So, the last example I have time to show you today
92
270000
3000
04:45
is a music sequencing and live performance tool
93
273000
2000
04:47
that we've built recently,
94
275000
2000
04:49
in which Siftables act as sounds
95
277000
3000
04:52
like lead, bass and drums.
96
280000
2000
04:54
Each of these has four different variations,
97
282000
2000
04:56
you get to choose which one you want to use.
98
284000
2000
04:58
And you can inject these sounds into a sequence
99
286000
3000
05:01
that you can assemble into the pattern that you want.
100
289000
3000
05:04
And you inject it by just bumping up the sound Siftable against a sequence Siftable.
101
292000
4000
05:08
There are effects that you can control live, like reverb and filter.
102
296000
4000
05:12
You attach it to a particular sound and then tilt to adjust it.
103
300000
3000
05:15
And then, overall effects like tempo and volume
104
303000
3000
05:18
that apply to the entire sequence.
105
306000
2000
05:20
So let's have a look.
106
308000
2000
05:22
Video: (Music)
107
310000
2000
05:24
DM: We'll start by putting a lead
108
312000
2000
05:26
into two sequence Siftables, arrange them into a series,
109
314000
5000
05:31
extend it, add a little more lead.
110
319000
2000
05:34
Now I put a bass line in.
111
322000
2000
05:36
Video: (Music)
112
324000
5000
05:41
DM: Now I'll put some percussion in.
113
329000
2000
05:43
Video: (Music)
114
331000
5000
05:48
DM: And now I'll attach the filter to the drums, so I can control the effect live.
115
336000
3000
05:51
Video: (Music)
116
339000
2000
05:53
DM: I can speed up the whole sequence
117
341000
2000
05:55
by tilting the tempo one way or the other.
118
343000
3000
05:58
Video: (Music)
119
346000
2000
06:00
DM: And now I'll attach the filter to the bass for some more expression.
120
348000
3000
06:03
Video: (Music)
121
351000
5000
06:08
DM: I can rearrange the sequence while it plays.
122
356000
3000
06:11
So I don't have to plan it out in advance, but I can improvise,
123
359000
2000
06:13
making it longer or shorter as I go.
124
361000
2000
06:16
And now, finally, I can fade the whole sequence out
125
364000
3000
06:19
using the volume Siftable, tilted to the left.
126
367000
3000
06:22
(Applause)
127
370000
5000
06:27
Thank you.
128
375000
2000
06:29
So, as you can see,
129
377000
3000
06:32
my passion is for making new human-computer interfaces
130
380000
3000
06:35
that are a better match to the ways our brains and bodies work.
131
383000
3000
06:38
And today, I had time to show you one point in this new design space,
132
386000
4000
06:42
and a few of the possibilities that we're working to bring out of the laboratory.
133
390000
3000
06:45
So the thought I want to leave you with
134
393000
2000
06:47
is that we're on the cusp of this new generation of tools
135
395000
3000
06:50
for interacting with digital media
136
398000
3000
06:53
that are going to bring information into our world
137
401000
3000
06:56
on our terms.
138
404000
2000
06:58
Thank you very much.
139
406000
1000
06:59
I look forward to talking with all of you.
140
407000
2000
07:01
(Applause)
141
409000
2000
ABOUT THE SPEAKER
David Merrill - MIT grad studentDavid Merrill works on Siftables, tiny computer blocks that interact with each other to make networks (and music).
Why you should listen
David Merrill is currently the VP of Product at 3D Robotics, where he builds new, easy-to-use flying robots. He was formerly president and co-founder of Sifteo, a San Francisco startup spun out of his research at the MIT Media Lab. This research helped build an award-winning physical+digital play system and catalog of unique games. He also directed HW and SW on Sifteo products.
David constantly tinkers with next-generation technologies to make physical+digital tools that give people new powers (for good).
He is a human-computer interaction expert, frequent public speaker and aerial photography enthusiast. David's work has been featured by MoMA, the Disocvery Channel, Popular Science and Wired.
His main interest now is the Siftables project, the subject of his TEDTalk, on which he works with Jeevan Kalanithi. In another field of inquiry, Merrill is looking at ways to access digital information in the wider world, when we are away from a traditional computer.
A musician himself, Merril also has a deep interest in new musical interfaces. His Adaptive Music Controller is an instrument that learns its player (rather than the player learning the instrument). And working musicians should check out his AudioPint, a portable case that consolidates instruments, effects and audio processors into a gig-safe rugged housing.
David Merrill | Speaker | TED.com