TED2009

David Merrill: Toy tiles that talk to each other

Filmed:

MIT grad student David Merrill demos Siftables -- cookie-sized, computerized tiles you can stack and shuffle in your hands. These future-toys can do math, play music, and talk to their friends, too. Is this the next thing in hands-on learning?

- MIT grad student
David Merrill works on Siftables, tiny computer blocks that interact with each other to make networks (and music). Full bio

I want to start out by asking you to think back to when you were a kid,
00:12
playing with blocks.
00:14
As you figured out how to reach out and grasp,
00:16
pick them up and move them around,
00:18
you were actually learning how to think and solve problems
00:20
by understanding and manipulating spatial relationships.
00:22
Spatial reasoning is deeply connected
00:25
to how we understand a lot of the world around us.
00:28
So, as a computer scientist
00:30
inspired by this utility of our interactions with physical objects --
00:32
along with my adviser Pattie, and my collaborator Jeevan Kalanithi --
00:35
I started to wonder -- what if when we used a computer,
00:39
instead of having this one mouse cursor that was a like a digital fingertip
00:42
moving around a flat desktop,
00:45
what if we could reach in with both hands and grasp information physically,
00:47
arranging it the way we wanted?
00:51
This question was so compelling that we decided to explore the answer,
00:53
by building Siftables.
00:56
In a nutshell, a Siftable is an interactive computer
00:58
the size of a cookie.
01:01
They're able to be moved around by hand,
01:03
they can sense each other, they can sense their motion,
01:05
and they have a screen and a wireless radio.
01:07
Most importantly, they're physical,
01:10
so like the blocks, you can move them just by reaching out and grasping.
01:12
And Siftables are an example of a new ecosystem
01:15
of tools for manipulating digital information.
01:18
And as these tools become more physical,
01:20
more aware of their motion, aware of each other,
01:23
and aware of the nuance of how we move them,
01:25
we can start to explore some new and fun interaction styles.
01:27
So, I'm going to start with some simple examples.
01:30
This Siftable is configured to show video,
01:32
and if I tilt it in one direction, it'll roll the video this way;
01:35
if I tilt it the other way it rolls it backwards.
01:37
And these interactive portraits are aware of each other.
01:40
So if I put them next to each other, they get interested.
01:43
If they get surrounded, they notice that too,
01:46
they might get a little flustered.
01:50
And they can also sense their motion and tilt.
01:52
One of the interesting implications on interaction, we started to realize,
01:56
was that we could use everyday gestures on data,
02:01
like pouring a color the way we might pour a liquid.
02:03
So in this case, we've got three Siftables configured to be paint buckets
02:06
and I can use them to pour color into that central one,
02:10
where they get mixed.
02:13
If we overshoot, we can pour a little bit back.
02:15
There are also some neat possibilities for education,
02:19
like language, math and logic games
02:22
where we want to give people the ability to try things quickly,
02:24
and view the results immediately.
02:26
So here I'm --
02:29
(Applause)
02:31
This is a Fibonacci sequence that I'm making with a simple equation program.
02:33
Here we have a word game that's kind of like a mash-up between Scrabble and Boggle.
02:37
Basically, in every round
02:41
you get a randomly assigned letter on each Siftable,
02:43
and as you try to make words it checks against a dictionary.
02:45
Then, after about 30 seconds, it reshuffles,
02:48
and you have a new set of letters and new possibilities to try.
02:51
(Applause)
02:54
Thank you.
02:55
(Applause)
02:56
So these are some kids that came on a field trip to the Media Lab,
02:58
and I managed to get them to try it out, and shoot a video.
03:00
They really loved it.
03:10
And, one of the interesting things about this kind of application
03:12
is that you don't have to give people many instructions.
03:15
All you have to say is, "Make words,"
03:18
and they know exactly what to do.
03:20
So here's another few people trying it out.
03:22
That's our youngest beta tester, down there on the right.
03:31
Turns out, all he wanted to do was to stack the Siftables up.
03:36
So to him, they were just blocks.
03:42
Now, this is an interactive cartoon application.
03:46
And we wanted to build a learning tool for language learners.
03:49
And this is Felix, actually.
03:54
And he can bring new characters into the scene,
03:57
just by lifting the Siftables off the table that have that character shown on them.
03:59
Here, he's bringing the sun out.
04:03
Video: The sun is rising.
04:07
David Merrill: Now he's brought a tractor into the scene.
04:09
Video: The orange tractor.
04:13
Good job! Yeah!
04:15
DM: So by shaking the Siftables and putting them next to each other
04:18
he can make the characters interact --
04:21
Video: Woof!
04:23
DM: inventing his own narrative.
04:24
Video: Hello!
04:26
DM: It's an open-ended story,
04:27
and he gets to decide how it unfolds.
04:29
Video: Fly away, cat.
04:34
DM: So, the last example I have time to show you today
04:42
is a music sequencing and live performance tool
04:45
that we've built recently,
04:47
in which Siftables act as sounds
04:49
like lead, bass and drums.
04:52
Each of these has four different variations,
04:54
you get to choose which one you want to use.
04:56
And you can inject these sounds into a sequence
04:58
that you can assemble into the pattern that you want.
05:01
And you inject it by just bumping up the sound Siftable against a sequence Siftable.
05:04
There are effects that you can control live, like reverb and filter.
05:08
You attach it to a particular sound and then tilt to adjust it.
05:12
And then, overall effects like tempo and volume
05:15
that apply to the entire sequence.
05:18
So let's have a look.
05:20
Video: (Music)
05:22
DM: We'll start by putting a lead
05:24
into two sequence Siftables, arrange them into a series,
05:26
extend it, add a little more lead.
05:31
Now I put a bass line in.
05:34
Video: (Music)
05:36
DM: Now I'll put some percussion in.
05:41
Video: (Music)
05:43
DM: And now I'll attach the filter to the drums, so I can control the effect live.
05:48
Video: (Music)
05:51
DM: I can speed up the whole sequence
05:53
by tilting the tempo one way or the other.
05:55
Video: (Music)
05:58
DM: And now I'll attach the filter to the bass for some more expression.
06:00
Video: (Music)
06:03
DM: I can rearrange the sequence while it plays.
06:08
So I don't have to plan it out in advance, but I can improvise,
06:11
making it longer or shorter as I go.
06:13
And now, finally, I can fade the whole sequence out
06:16
using the volume Siftable, tilted to the left.
06:19
(Applause)
06:22
Thank you.
06:27
So, as you can see,
06:29
my passion is for making new human-computer interfaces
06:32
that are a better match to the ways our brains and bodies work.
06:35
And today, I had time to show you one point in this new design space,
06:38
and a few of the possibilities that we're working to bring out of the laboratory.
06:42
So the thought I want to leave you with
06:45
is that we're on the cusp of this new generation of tools
06:47
for interacting with digital media
06:50
that are going to bring information into our world
06:53
on our terms.
06:56
Thank you very much.
06:58
I look forward to talking with all of you.
06:59
(Applause)
07:01

▲Back to top

About the Speaker:

David Merrill - MIT grad student
David Merrill works on Siftables, tiny computer blocks that interact with each other to make networks (and music).

Why you should listen

David Merrill is currently the VP of Product at 3D Robotics, where he builds new, easy-to-use flying robots. He was formerly president and co-founder of Sifteo, a San Francisco startup spun out of his research at the MIT Media Lab. This research helped build an award-winning physical+digital play system and catalog of unique games. He also directed HW and SW on Sifteo products.

David constantly tinkers with next-generation technologies to make physical+digital tools that give people new powers (for good).

He is a human-computer interaction expert, frequent public speaker and aerial photography enthusiast. David's work has been featured by MoMA, the Disocvery Channel, Popular Science and Wired.

His main interest now is the Siftables project, the subject of his TEDTalk, on which he works with Jeevan Kalanithi. In another field of inquiry, Merrill is looking at ways to access digital information in the wider world, when we are away from a traditional computer.

A musician himself, Merril also has a deep interest in new musical interfaces. His Adaptive Music Controller is an instrument that learns its player (rather than the player learning the instrument). And working musicians should check out his AudioPint, a portable case that consolidates instruments, effects and audio processors into a gig-safe rugged housing.

More profile about the speaker
David Merrill | Speaker | TED.com