sponsored links
TED2013

Jinha Lee: Reach into the computer and grab a pixel

February 27, 2013

The border between our physical world and the digital information surrounding us has been getting thinner and thinner. Designer and engineer Jinha Lee wants to dissolve it altogether. As he demonstrates in this short, gasp-inducing talk, his ideas include a pen that penetrates into a screen to draw 3D models and SpaceTop, a computer desktop prototype that lets you reach through the screen to manipulate digital objects.

Jinha Lee - Interface designer, engineer
Jinha Lee works at the boundary of the physical and digital world. The TED Fellow, designer and engineer is exploring ways to weave digital information into physical space. Full bio

sponsored links
Double-click the English subtitles below to play the video.
Throughout the history of computers
00:12
we've been striving to shorten the gap between us
00:15
and digital information,
00:18
the gap between our physical world
00:20
and the world in the screen
00:22
where our imagination can go wild.
00:23
And this gap has become shorter,
00:26
shorter, and even shorter,
00:30
and now this gap is shortened down
00:33
to less than a millimeter,
00:34
the thickness of a touch-screen glass,
00:36
and the power of computing
00:38
has become accessible to everyone.
00:39
But I wondered, what if there could be no boundary at all?
00:44
I started to imagine what this would look like.
00:49
First, I created this tool
00:53
which penetrates into the digital space,
00:55
so when you press it hard on the screen,
00:58
it transfers its physical body into pixels.
01:00
Designers can materialize their ideas
01:03
directly in 3D,
01:05
and surgeons can practice on virtual organs
01:07
underneath the screen.
01:10
So with this tool, this boundary has been broken.
01:12
But our two hands still remain outside the screen.
01:17
How can you reach inside and interact
01:20
with the digital information
01:22
using the full dexterity of our hands?
01:24
At Microsoft Applied Sciences,
01:27
along with my mentor Cati Boulanger,
01:29
I redesigned the computer
01:31
and turned a little space above the keyboard
01:32
into a digital workspace.
01:35
By combining a transparent display and depth cameras
01:37
for sensing your fingers and face,
01:40
now you can lift up your hands from the keyboard
01:42
and reach inside this 3D space
01:44
and grab pixels with your bare hands.
01:47
(Applause)
01:49
Because windows and files have a position in the real space,
01:52
selecting them is as easy as grabbing a book off your shelf.
01:56
Then you can flip through this book
02:00
while highlighting the lines, words
02:02
on the virtual touch pad below each floating window.
02:04
Architects can stretch or rotate the models
02:06
with their two hands directly.
02:09
So in these examples,
02:11
we are reaching into the digital world.
02:13
But how about reversing its role
02:15
and having the digital information reach us instead?
02:18
I'm sure many of us have had the experience
02:23
of buying and returning items online.
02:24
But now you don't have to worry about it.
02:27
What I got here is an online augmented fitting room.
02:29
This is a view that you get from
02:33
head-mounted or see-through display
02:35
when the system understands the geometry of your body.
02:37
Taking this idea further, I started to think,
02:42
instead of just seeing these pixels in our space,
02:46
how can we make it physical
02:49
so that we can touch and feel it?
02:51
What would such a future look like?
02:55
At MIT Media Lab, along with my advisor Hiroshi Ishii
02:58
and my collaborator Rehmi Post,
03:02
we created this one physical pixel.
03:04
Well, in this case, this spherical magnet
03:07
acts like a 3D pixel in our space,
03:10
which means that both computers and people
03:12
can move this object to anywhere
03:15
within this little 3D space.
03:17
What we did was essentially canceling gravity
03:19
and controlling the movement by combining
03:23
magnetic levitation and mechanical actuation
03:25
and sensing technologies.
03:27
And by digitally programming the object,
03:30
we are liberating the object from constraints
03:32
of time and space, which means that now,
03:35
human motions can be recorded and played back
03:39
and left permanently in the physical world.
03:43
So choreography can be taught physically over distance
03:46
and Michael Jordan's famous shooting can be replicated
03:49
over and over as a physical reality.
03:52
Students can use this as a tool
03:55
to learn about the complex concepts
03:57
such as planetary motion, physics,
03:59
and unlike computer screens or textbooks,
04:03
this is a real, tangible experience
04:06
that you can touch and feel, and it's very powerful.
04:09
And what's more exciting
04:12
than just turning what's currently in the computer physical
04:14
is to start imagining how programming the world
04:18
will alter even our daily physical activities.
04:21
(Laughter)
04:27
As you can see, the digital information
04:29
will not just show us something
04:32
but it will start directly acting upon us
04:34
as a part of our physical surroundings
04:36
without disconnecting ourselves from our world.
04:38
Today, we started by talking about the boundary,
04:42
but if we remove this boundary,
04:46
the only boundary left is our imagination.
04:49
Thank you.
04:53
(Applause)
04:54

sponsored links

Jinha Lee - Interface designer, engineer
Jinha Lee works at the boundary of the physical and digital world. The TED Fellow, designer and engineer is exploring ways to weave digital information into physical space.

Why you should listen

Jinha Lee is an interface designer, engineer, and researcher working at the boundary of the physical and digital world. A 2013 TED Fellow, he is exploring ways to harness humans' innate kinesthetic and sensory skills to impact the way we interact with the world of digital data.

Jinha is currently on leave from his Ph.D. studies at MIT Media Lab and is leading a team to design TV interfaces at Samsung Electronics in lieu of his national military duty. His recent inventions include the SpaceTop gesture-controlled 3D desktop OS, a levitating tangible interface, and a pen that penetrates into the screen.

Previously, Jinha co-founded eone-timepieces, a tactile watch brand for "everyone," including the visually impaired, and conducted research at Sony Computer Science Lab (CSL) and Microsoft Applied Sciences. He received his B.E. in Electronic Engineering from the University of Tokyo. His works have been presented and exhibited at academic conferences and museums, and have received recognition from design and innovation awards.

The original video is available on TED.com
sponsored links

If you need translations, you can install "Google Translate" extension into your Chrome Browser.
Furthermore, you can change playback rate by installing "Video Speed Controller" extension.

Data provided by TED.

This website is owned and operated by Tokyo English Network.
The developer's blog is here.