Only a heartbeat away

I find myself at the edge of a virtual forest clearing, looking around. In the distance I see an avatar who is speaking to me; that's where I need to go. As I get closer, I can hear the person better; he is talking about how we should take better care of the earth. Beforehand, I was instructed to stand near the avatar, at a distance where I feel comfortable. I choose my spot and listen for a moment, then I confirm my position and find myself back again at the edge of the clearing to meet the next avatar. This way, I am sent into the virtual world twenty-four times in total.

But not every time is the same. Occasionally, as I approach the avatar, I feel the wristband with sensors on my forearm warming up. Sometimes I feel vibrations in the controller in my hand, or warmth and vibrations, and sometimes nothing at all. After each session, I answer questions in the virtual world on what I felt and how I experienced this encounter.

Simulated heartbeat

Now that virtual worlds are becoming more detailed and VR headsets are improving, efforts are being made to make the experience more realistic. This aligns with the study of Simone Ooms, where participants visit a virtual forest. "What can you add besides 'seeing' and 'hearing'? That quickly leads to the sense of 'feeling'. In an earlier study, I asked people to watch news videos on the computer while their arm rested on a device. Then, I had them feel warmth and vibrations through that device - that's what we call haptic feedback. Just that alone adds an extra dimension. Now, I'm doing the same in VR. I want to know how a simulated heartbeat and mimicked body warmth influence the virtual experience."

"In the real world, a significant part of communication is non-verbal. You can, so to speak, tell from someone's gait whether they're happy or not. You don't have that in VR. That's why we're trying to add elements to convey that non-verbal aspect," explains Ooms. "Do people, for instance, link a simulated heartbeat to a certain emotion?"

Adding a sense of feeling to the virtual experience is being researched worldwide in various ways. Researchers often engage in tinkering for this purpose. Experiments have been conducted with tubes sewn into clothing, through which you can make cold and warm liquids flow, Ooms notes. "They've come quite far with that. Now, the tubes are filled mixes of cold and warm water to achieve precise temperature feedback."

Designing for different senses

Due to her background in industrial design, Ooms was enthusiastic to make prototypes to conduct research at the Distributed and Interactive Systems (DIS) group, simulating physical proximity. For these experiments, she had to devise something with sensors that could emit vibrations and warmth. "One sensor I used was originally in thermal clothing in earlier DIS work. It emitted heat or cold while test subjects listened to a voice message. They were then asked how they experienced the message. The idea behind this is that cold leads them to perceive a message more negatively, and warmth has the opposite effect. I added the vibration feedback for the later experiments with news videos and virtual reality."

A next step in the research could be to record someone's actual heartbeat and use it for the experiments - to see how it affects the virtual experience. Ooms: "There are also studies that work with smell or taste, adding a new dimension to the experience. Other studies utilize the sense of touch differently. For example, you might feel vibrations when petting a cat or warmth when standing close to a fire."

Manipulation

The ultimate goal of the researchers from the DIS group is to gather as much knowledge as possible before this kind of technology becomes available to everyone. "You need to know precisely what you can do with it," clarifies Ooms. "Imagine: you present people with a negative news feed and simultaneously have sensors emit warmth so that the experience of that feed becomes more positive. That's manipulation. And if that works, can you also use the same method to lure people to an advertisement? These are the kinds of things you want to figure out before commercializing the technology. And that's what we're here for."

^{Author: Irene van Elzakker}

Read more in this series:

• How do we use space in VR?