Credit: Pulkit Budhiraja / University of Illinois at Urbana-Champaign How do you physically grab a drink while wearing an Oculus Rift headset that has you immersed in virtual reality? Figuring out how to navigate smoothly between the physical and virtual worlds could greatly improve virtual reality experiences as VR headsets become more widespread in both entertainment and workplaces. Researchers at the University of Illinois at Urbana-Champaign decided to test out several possible scenarios that would allow people to see just enough of the physical world to take a sip of coffee or soda without removing their virtual reality headsets. Researchers figured out how to create "renderings" of the physical world within virtual reality experiences by using two cheap webcams to capture the real-world action. They then recruited 10 participants who tried out each of the renderings under various virtual reality scenarios: watching a movie, playing a first-person shooter video game, and playing a car racing game. While wearing an Oculus Rift, participants engaged in a total of 300 grasping trials that involved trying to physically pick up a cup, take a drink of water and put the cup back down, according to the paper published on the arXiv preprint server. They also rated each of the renderings based on factors such as how easy it was to take a drink or how distracted they were from the virtual reality experiences. "While being engaged with the virtual world, people naturally perform small tasks like picking up their game controller, looking for their mouse or drinking water from a cup which require awareness of the real world," said Pulkit Budhiraja, a Ph.D. candidate in computer science at the University of Illinois at Urbana-Champaign and lead author on the new paper. "Currently, the only existing solution is removing the head mounted display, which is disruptive to game play and breaks immersion."
A demonstration of an "augmented virtuality" display that allows people wearing virtual reality headsets to still see some of the physical world. Credit: Pulkit Budhiraja / University of Illinois at Urbana-Champaign Participant ratings overwhelmingly favored one winner: an "Object, Hands & Context" option that showed the physical cup, the users' hands and the outlines of other physical objects nearby. "The extra lines helped me find the cup and put it back, and it wasn't distracting, it didn't break my concentration on the game,” according to one study participant cited in the paper. Other options included: seeing only the physical cup and user hands floating in the virtual world; seeing a large windowed view of the physical world smack dab in the center of the virtual reality experience; seeing the physical world as a small picture in the lower right hand corner of the virtual reality experience; and simply lifting the Oculus Rift DK1 headset up to see the physical world. Unsurprisingly, the last option received the lowest ratings. "Removing the goggles is immersion breaking," said one study participant. This is still far from a definitive survey of people's preferences for virtual reality experiences. After all, the study had relatively few participants overall and very few females from the start. Of the original 16 people recruited for the study, 13 were male and three were female. Six of the original 16 participants also had to drop out after experiencing simulator sickness, which remains a big concern for virtual reality pioneers who hope to make the technology more widespread. As a result, only one female ended up in the final group of 10 people who took part in the study. Past work with military simulators has suggested that women experience simulator sickness more often than men. https://www.youtube.com/watch?v=DTiw0A_jdsM Budhiraja described some of the other challenges facing virtual reality experiences with head mounted displays (HMDs) in an email:
"Simulator sickness is still a major hindrance. However, apart from the usually talked about hardware limitations and lack of mature content, there is a host of other challenges that need to be addressed. Usability is one such issue which we tried to address in our work. Ergonomics is also a major factor. No one wants to wear a headset that starts becoming uncomfortable with prolonged use. Safety is a concern as well. You need HMDs to be smart enough that can warn the user before they run into a wall!"
Still, such research could prove a useful pilot study for larger studies about how physical interactions can work seamlessly with virtual reality experiences. That could enable physical toys such as guns or wands to become controllers within a virtual game or movie experience, according to the research team. Military simulators could incorporate physical objects held by virtual reality users, such as weapons, to better immerse people in the experience. Virtual reality designers could also make physical interactions with the real world play a role in the virtual reality experience; the paper gives the example of drinking a real glass of water to recharge a player's health in a first person shooter game. "Currently, I am interested in exploring the continuum between reality and virtual reality," Budhiraja said. "Bringing parts of one world into the other and exploring novel applications and interactions that can be made possible doing it."
