Computers today barely connect with people. The human body evolved as a whole to sense and interact with the world, but computers sense us only at our fingertips. Even the fingertips aren’t allowed to do all they can; a computer that was designed to interact with us holistically would feel different from moment to moment in order to convey information. For more than two decades, I’ve been working on the grand project of virtual reality (VR) to bring the whole body into computing.
You can think of an ideal virtual reality setup as a sensory-motor mirror opposite of the human body. Wherever the body has a sensor, like an eye or an ear, a VR system must present a stimulus to that body part to create a corresponding illusory world. The eye needs a visual display, for instance, and the ear needs an audio speaker. That’s the easy part. Most of the body is covered in skin, so the ideal VR setup would be largely devoted to touch and feel. The term of art for this side of VR is "haptics."
I have high expectations for haptics. People think in deeper, more intuitive ways when they express themselves physically. Like anyone else who has learned to improvise at the piano, I’ve been amazed to find that my hands can solve mathematically rich problems in harmony more quickly than I could think about them intellectually. I’ve always been fascinated by this potential and have tried letting kids experience DNA molecules and abstract geometry problems inside fancy VR systems that allow for full-body interaction. My friend Brian Eno once said that computers need more “Africa” in them. I took him to mean interaction with the whole body.
Alas, development in this area is all about being patient. Virtual reality technology needs a lot of computing power, and it is still waiting for Moore’s law to catch up. Nevertheless, this year has seen two notable breakthroughs. The online world Second Life has received a huge amount of attention, and deservedly so, because it demonstrates that people can be enthralled with creative uses of VR that go far beyond canned gaming. But there is another digital-world pop phenomenon—a haptic one—that says at least as much about the future of VR, even though it hasn’t been widely considered in that light: the Wii controller. This gadget will be remembered as the beginning of a haptic revolution.
Back in the early 1980s, my colleagues and I spent a lot of time exploring how the human body could best interact with the virtual world. The iconic VR device that emerged from that work was the DataGlove, a sensor-filled glove that would direct a real-time virtual model of your hand into a computer-generated world.
If only there were a way to convey how exciting virtual reality research was in those days. I remember being in the Palo Alto garage where VPL Research started (I also lived there), handling a virtual object for the first time. I think it was Tom Zimmerman, the inventor of that first glove, who said, “This is definitely the most exciting room to be in, in the whole world at this moment.” We built a lot more than gloves. We built whole body suits! If you want to see what they looked like, rent the science fiction movie The Lawnmower Man, which showcases authentic VR equipment of the period (although we didn’t have a problem with autonomous, bloodthirsty lawn mowers).
