Understanding Spatial Awareness in XR

Imagine you’re playing a video game on your TV and a water balloon is thrown at you in the game. You might laugh as it splashes and the screen and gets blurry for a moment. Now imagine that you’re playing this game in VR. The water balloon is thrown at your face and you instinctively dodge it as it gets too close to you. Why do you have the reflex to dodge one virtual balloon and not the other? The answer is spatial awareness.  

‍

What is Spatial Awareness?

Spatial awareness is the ability to perceive two or more objects relative to each other and to yourself. Your brain perceives virtual objects very differently when they are on a screen versus in a headset. When you are watching something on a screen, your brain recognizes the screen itself as the object. Your spatial awareness is focused on the device you are watching, not the thing you are seeing displayed. But when you interact with a digital avatar in a VR setting, for instance, your brain recognizes that avatar as being in the same space as you. Your brain experiences it as if it is in the same physical space as you are. That changes the relationship between viewer and avatar, and you need to factor that in when creating spatial apps and experiences. For example, if the avatar gets too close it can make the viewer uncomfortable. That’s just not something you need to contend with if you’re building something for a flat screen. 

That spatial relationship is also part of the reason that memories created in XR are often more intense than memories created through playing a video game, or watching something on a 2D screen. XR has famously made a huge impact in mental health and a wide variety of therapy options, partly because of its ability to connect with people in a way that is both real, but also safe and curated. 

That connection to experiences in XR includes how people react to objects on a subconscious level. If, for example, you think you see a rat run across your kitchen counter, even if you know for a fact that the rat is just an AR creation, your brain may associate it with a real rat and you’ll remember that visceral feeling. Or as in the Holonlens game Fragments, you can create memories of your game characters sitting in people’s real furniture. 

Hear a bit more from Paul Hoover, our head of design at ShapesXR.

‍

User Safety & Proxemics

That brings with a responsibility for teams and individuals creating spatial apps. Especially when multiple remote people can interact with each other and feel perceived like they are in the same physical space. This special subset of spatial awareness is called proxemics: “the study of how people perceive the proximity of others”. As seen below, people generally think of the space around themselves in the four categories of intimate, personal, social, and public space.

‍

Best Practices

A few best practices have emerged in social XR from studying proxemics.

1. Keeping users safe: Don’t let someone enter another user’s intimate space without consent. Early in the days of social VR a lot of people were groped and mistreated, creating real trauma. Today avatars either bounce off each other in VR or disappear completely when they get too close to others. 
2. Sharing: Many applications match sharing modes to the zones in proxemics. Many user interfaces live in the intimate space and are private. The personal space typically is where you interact with content. The social space is where you expect to interact with others and their content. And public space is where you can see, but sometimes not hear other users and content.

Proximity chat and microphones: Most people expect audio in virtual space to act like they do in real spaces. The closer people get to each other the clearer and louder their voices are to each other. This allows for groups of people to hold multiple simultaneous conversations in larger virtual spaces (proximity chat is another name for this concept). But just like in large conference rooms, there is typically a microphone so that one person can command the attention of everyone. And a clear space where this person stands to present. Consider re-using these patterns in the virtual spaces you create so that your users can build on their real-world experiences.

‍

Try it for yourself

Experience: To get a feeling for this yourself jump into ShapesXR and experience the lecture from Paul Hoover, our head of design here at ShapesXR:
(Open in VR in ShapesXR to play Paul's holonotes)

Experiment: You can also experiment with multiplayer spatial audio in ShapesXR to see how it impacts user behavior in large groups. Try creating conversation areas and changing the audio falloff distance to enable proximity chat.