SixthSense

Summary

SixthSense is a wearable gestural interface developed and popularized by Pranav Mistry at the MIT Media Lab, building on earlier wearable computing experiments by Steve Mann, that explores how digital information can be seamlessly integrated into the physical world. Using a wearable projector, camera, and computer vision system, SixthSense transforms everyday surfaces, objects, and gestures into interactive computational interfaces. Instead of confining information to screens, the system projects digital media directly onto walls, paper, hands, books, newspapers, or physical objects, allowing users to interact with information through natural gestures.

At its core, SixthSense proposes a different relationship between people and computation: rather than pulling humans into virtual environments, computation should augment the world people already inhabit. The system recognizes gestures, tracks objects, and overlays contextual information onto the surrounding environment in real time. Users can draw on any surface, navigate maps with hand gestures, project a watch onto their wrist, interact with printed media, or retrieve information about objects simply by looking at them.

What makes SixthSense especially significant is its vision of computation as something ambient, embodied, and spatial. The interface dissolves the boundary between digital and physical interaction, treating the world itself as an interactive surface. In doing so, SixthSense anticipated many later developments in augmented reality, spatial computing, wearable interfaces, and ubiquitous computing, while remaining grounded in a strongly human-centered philosophy: technology should augment human perception and action without demanding constant attention to screens or devices.

I think that integrating information to everyday objects will not only help us to get rid of the digital divide, the gap between these two worlds, but will also help us, in some way, to stay human, to be more connected to our physical world. And it will actually help us not end up being machines sitting in front of other machines.

Key concepts

• The world as interface SixthSense turns walls, paper, tables, books, newspapers, and even the human body into interactive computational surfaces by projecting information directly onto the physical environment.
• Natural gestural interaction Users interact with digital information through intuitive hand gestures, freehand drawing, and physical movements rather than keyboards, mice, or touchscreens.
• Augmented physical reality Instead of replacing the real world with virtual environments, SixthSense augments existing objects and spaces with contextual digital information in real time.
• Embodied computing The interface is wearable and mobile, allowing computation to move with the user and integrate directly into everyday activity rather than remaining fixed to a desktop or screen.
• Information beyond the screen SixthSense challenges the assumption that digital information must live inside dedicated displays, proposing instead that information should appear wherever and whenever it is useful.

SixthSense represents a powerful reimagining of human-computer interaction in which computation becomes woven into the fabric of everyday life rather than confined to isolated devices. By combining wearable sensing, projection, gesture recognition, and contextual information, the project transforms the physical world itself into a dynamic computational medium. More than a technological prototype, SixthSense articulates a broader vision of humane computing: one where digital information augments human perception and activity without separating people from their environment, attention, or embodied experience.

This idea was originally published as WUW – Wear Ur World – A Wearable Gestural Interface. See also the TED talks Meet the SixthSense Interaction by Pattie Maes, and The Thrilling Potential of SixthSense Technology by Pranav Mistry.