2D to 3D Mixed Reality Transformations

This is the repository for the publication:

A Design Space For Data Visualisation Transformations Between 2D And 3D In Mixed-Reality Environments
By Benjamin Lee, Maxime Cordeil, Arnaud Prouzeau, Bernhard Jenny, and Tim Dwyer
To be published in ACM CHI '22

The Unity project contains a number of demonstrative visualisation transformation techniques that were described in the aforementioned paper (pre-print found here). A built version that is ready for deployment on the Microsoft HoloLens 2 is provided in the Releases section of this repository.

The supplemental video for the submission can be found in the following Youtube link. It contains video versions of all of the techniques with voice-over explanations.

Supplemental video

Running the Protoype

To try the prototype techniques on your desktop using a simulator, you will need Unity 2019.4.20f1. Open the MainScene.unity scene and simply press Play to use the MRTK Simulator. Controls for the simulator can be found here.

To try the techniques on a Microsoft HoloLens 2, you can download an .appx file in the Releases section which you can use to install the application. I personally use the Windows Device Portal to install HoloLens 2 applications.

I have not tested the Unity project using the Holographic Remoting feature, and so your mileage may vary here.

Using the Prototype

There is an in-game window that provides some basic instructions on how to use the prototype. I recommend you read this before closing the window.

It is important to note that the transformation techniques are not explicitly listed or stated by the prototype. If you want to try a specific transformation, you will need to refer to the paper to determine which visualisation needs to be instantiated first for you to access said transformation.

The prototype contains some additional techniques that were not described in the paper. These have been kept in the prototype for posterity. I suggest either playing around yourself, or looking directly through the code in order to find these.

There also exists several additional functionalities that were not mentioned in the paper, such as the ability to change visualisation dimensions by grabbing the axis labels, or to resize them by grabbing the axis tips. You may use these to assist in your exploration of the prototype.

Notes on the Code

The code is provided as-is using an MIT License. It was developed over many iterations and has not been extensively documented or organised. You are more than welcome to look through and dissect the code, but I cannot guarantee its readability or sensibility.

Several naming conventions in the code will differ from that used in the design space of the paper (e.g., splatting instead of flattening). Wherever these inconsistencies exist, the naming scheme used in the paper takes priority.

Third Party Libraries

This project relies on a number of libraries and plugins created by third-parties: