Navigating 3d Scatter Plots in
Immersive Virtual Reality 

A thesis submitted in partial fulfillment of the requirements for the degree of: Master of Design, Interaction Design (MDes IxD) at the University of Washington, 2016. The thesis is comprised of three main parts: the design and development of a working prototype, an interactive exhibit and a comprehensive documentation book.

Abstract

Virtual reality reveals spatially complex structures behind 3d data in an easily explorable way. The difficulty is in navigating this space while wearing a head-mounted display (HMD). One viable option for view control is a combination of a 3d mouse for camera location and head tracking of the HMD for the control of view direction. The scenes shown here display semi-randomized 3d scatter plots, each of which emphasizes a type of navigation that leads the viewer to discover structure behind the data. The three types of navigation are rotation, scale and alignment.

Introduction

I must admit that working with Virtual Reality (VR) was never a consideration prior to my graduate studies. In all honesty, I’m still not entirely sure how I ended up there. Looking back, however, VR did satisfy many of my grad school “wants”.

Foremost, VR was and in many ways still is an emerging technology. Sure, VR had been around for decades—I believe the first mention of VR was in 1962—though cost and technical limitations left it quite inaccessible. For a few months prior to my thesis project, I kept hearing how 2016 was going to be the “year of VR”. Always dubious of these types of grandiose claims, I started searching through academic journals and publications for studies involving VR. I was amazed at the diversity of the studies I found! It seemed that academics in all fields had studied potential applications of the technology. From market analysis to treating vertigo, I was quickly becoming convinced of VR’s potential.

During the exploratory problem statement phase of the thesis, I experienced a sort of “Aha!” moment after reading a number of publications regarding data visualization in VR. Most were authored by computer scientists or human-computer interaction (HCI) specialists, though the emphasis was often on “computer” than interaction. Not to detract from the quality of these studies but I noticed a general lack of visual design / perception principles along with clunky, ovewhelming interfaces. There was clearly an opportunity for design here. And considering most of the first year of grad school was focused on data viz—my topic area was becoming very clearly defined.

Lastly, I had also acquired a fairly strong foundation in computer science in anticipation of the thesis. I knew that I wanted to go beyond envisioning. I wanted to design and develop a working prototype. So I bought an Oculus Rift DK2, built a PC to run it and started learning Unity.

Research

Being a graduate-level thesis, research was a major part of the project. As there were no established “best practices” for data visualizations in virtual reality nor VR in general, I focused my research on the following three areas:

Visual Perception

I knew I would be venturing a bit beyond the field of design for this project. I had studied Gestalt principles during my undergraduate education but I knew a deeper understanding of visual perception and visual design principles would be critical. 

Aside from a couple-few dozen journal publications, there were four seminal books in particular that were fundamental to this thesis:

(1) The Ecological Approach to Visual Perception by James J. Gibson. 1979.
(2) Vision Science: Photons to Phenomenology by Stephen E. Palmer. 1999.
(3) Information Visualization: Perception for Design by Colin Ware. 2012.
(4) Visual Thinking for Design by Colin Ware. 2008.

Interactive Data Visualization

Literature here was much more sparse. Most of what I found pertained to 2d data visualizations and the 3d examples I found were being displayed on a 2d screen. Still, there were a number publications that provided interaction principles and examples that would be relevant to interactive data visualizations on any platform. They were:

(1) d3js.org / Mike Bostock
(2) Information Dashboard Design: Displaying Data for at-a-glance Monitoring by Stephen Few. 2013.
(3) Now You See It: Simple Visualization Techniques for Quantitative Analysis by Stephen Few. 2009.

One type of interaction in particular—navigation—was almost universally absent from literature. This meant that the opportunity space here would be much larger, especially as it pertains to Virtual Reality applications. For this reason, I decided that navigation would be a reasonable focus of the thesis. 

Developing for the Oculus Rift / Unity

Being new to both VR and Unity, I most certainly would not have been able to build a functional prototype in such a short time without the following resources:

(1) Oculus Rift in Action by Bradley Austin Davis, Philips Alexander Benton and Karen Bryla. 2015.
(2) U-Demy
(3) Unity forums

Research Findings & Synthesis / Principal Considerations

After reading such dense literature, I noticed a common thread among all of my sources—Ben Shneiderman’s Visual Information-Seeking Mantra:

(1) Overview first
(2) Zoom and filter
(3) Details-on-demand

These simple yet profound guidelines would be at the forefront of this project. I was curious to see if it would hold up for data vis in VR (it does). Additional guidelines suggested by other authors are:

(4) View relationships
(5) View history
(6) Extract information

Scene 1: Rotation

In this scene, the viewer is inside of an enclosed cube. The walls of the cube serve as axes. The initial view appears to show that the cube is uniformly filled with data points, represented by shaded spheres. The size of the spheres is arbitrary to the data set and has been set at a scale that prevents the most distant data points from disappearing due to their distance from the camera.

In order to introduce the viewer to the 3d mouse, navigation in this scene is limited—the scene, not the camera, can only be rotated around the center xy-plane of the enclosing cube.

This means that directional illumination will alter shadows on each data point. This illustrates the deficits of moving the environment rather than the camera. Scenes 2 and 3 move the camera which adds to the feeling of presence and immersion.

Once the viewer has rotated the scene 90 degrees (+/- approx. 10 degrees), the pattern of “banding” becomes apparent; data points are relatively constrained along the halfway point along one axis (in this case, the y-axis). Additionally, the head-mounted display provides depth cues through motion parallax as the viewer moves their head.

Scene 2: Scale

In this scene, the three axes are intersected rather than positioned to form an enclosure. This layout is especially helpful for any data set with both positive and negative values, or if the origin (intersection of three axes) is needed to be positioned at a point other than (0, 0, 0).

Control of the camera location is no longer constrained to a particular path as in scene 1. Rather, the viewer is allowed to move freely in three dimensions. In order to prevent the viewer from turning the camera upside down—this can cause the viewer to feel lost—the camera is only allowed to rotate 90 degrees along the vertical (z) axis. Additionally, directional lighting assists in preserving the viewer’s sense of their position in relation to the horizon, informing them which direction is up. The top of each sphere receives directional illumination from above the scene, similar to the sun shining from above.

In the initial view, the data set appears to occupy a sphere who’s center is at the origin. Data points appear less dense toward the sphere’s apparent boundary. Navigating the camera around the outside of the sphere confirms this structure. Once the viewer moves the camera near the origin, the hollowness of the data set becomes obvious—there are no data points within a certain radial distance from the origin.

As in the first scene, the head-mounted display provides depth cues through motion parallax as the viewer’s head moves. Additionally, head movement hides and reveals data points that would otherwise obscure the majority of the view. In computer graphics, this is known as “clipping”. Another technique to solve for this would be making the data points semi-transparent.

Scene 3: Alignment

As in scene 2, the three axes are intersected. Building upon the theme of preserving the viewers sense of the vertical—which direction is up—the xy-plane has been extended into the horizon, serving as a “ground”. It is important to note that the viewer is still able to navigate the camera below the ground plane which then becomes a “ceiling” plane. Still, both planes are easily to located by looking up or down.

In the center of the viewer’s view direction, a small reticle is displayed. This serves as yet another reference point, though the reticle is not completely static. Rather, the reticle will move slightly depending on the geometry of the object it hits—the reticle is a ray cast from the camera location. When a data point is “hit” with the reticle, precise coordinates are displayed as well as perspective lines originating from the data points center. These aid in quickly gaining a sense of how far the point is from the origin.

Scene 3 also moves away from spheres as representations of individual data points. Instead, scene 3 uses cubes. As the cubes are also rendered in perspective, the edges provide trailing lines to the axis. This assist in estimating where along a particular axis a data point would lie. Direction lighting from above remains, though it is somewhat harder to discern from certain viewpoints.

While these visual cues seem rather obvious, a surprising majority of 3d scatter plots ignore these visual cues as a provider of natural orientation. They are particularly important in VR as each cue makes extending viewing more comfortable.

Discovered Principles & Affordances

Through user testing, revisits to the literature and a great deal of trial-and-error, these findings could be considered the primary contribution to the practice of designing data visualizations for VR.

Motion Parallax

The greatest advantage of VR in viewing 3d scatter plots is motion parallax. The ability to make slight adjustments to the camera location by moving one’s head slightly is a natural cue for relative distances. This affords the user a better sense of a data point’s position in space. Those moving faster are intuitively closer than those moving slower. I conclude that this alone is reason for viewing 3d data vis in VR.

Ambulatory and Ambient Vision

While the 3d mouse allows the viewer to “get up and fly around” the virtual environment, the headset simultaneously allows changes in visual display through minor adjustments of the head in ways that 3d projections on 2d screens can not. These two types of vision-based navigation—vision being inherently linked to movement—are referred to as ambulatory and ambient vision, respectively. Motion parallax is an inherent characteristic of ambient vision the real world and in VR.

Visual Cues 

Visual cues such as illumination, shadows, textures, transparency/opacity, occlusion, the inclusion of a ground plane and motion parallax all improve the sense of immersion, feeling physically present in a non-physical yet plausible environment.

Motion Sickness

VR is notorious for its capacity to induce motion sickness. This is mostly due to visual senses receiving input that is in conflict with vestibular input. The effect is similar to reading while in a car. There are, however, a few design principles that can reduce the likelihood of motion sickness. Some of the most important considerations are:

- Avoid high-contrast flickering
- Use real-world scales
- No objects take up majority of user’s view
- No floating in a void
- Provide static reference points
- Points of interest are in center of view
- Sufficient frame rate & latency 

Exhibition

In addition to the above discovered principles, part of the core output of this thesis was an interactive exhibit. For many visitors, the exhibit had been their first experience with VR and for even more visitors, their first time using a 3d mouse. The 3d mouse is, for the most part, intuitive though there is a slight learning curve. For myself, it took about an hour before I felt that I could navigate the camera exactly where I wanted it to go.

During the opening night, I stood near the exhibit and offered minor pointers (pun intented) on how to use the 3d mouse. Expectedly, some visitors—younger ones especially—caught on immediately. These younger visitors (under 25 years old or so) were also less likely to complain of motion sickness. The most overlooked and seemingly difficult interaction visitors had with the installation was in changing scenes. The 3d mouse had two small buttons on either side of the base. Finding these buttons while wearing the headset proved surprisingly difficult for some. 

Perhaps the most surprising observation was how well older visitors used the system. With a sample size of around 200 visitors, the 25-60 year old demographic generally experienced difficulty not with the headset but with the 3d mouse. This is understandable as it is intended primarily for people working with 3d modeling software.