Skip to Main Content

Augmented Reality (AR) Sandbox

AR Sandbox Instruction Guide

How Does the AR Sandbox Work?

The AR Sandbox integrates physical interaction with real-time digital augmentation by combining depth-sensing hardware, projection technology, and open-source software. Designed primarily as an educational tool, it creates an interactive 3D topographic model by projecting contour lines, elevation shading, and water simulation effects directly onto a manipulable sand surface. The AR Sandbox typically includes three main components
• A Depth Sensor (usually a Microsoft Kinect): Positioned above the sandbox, the depth sensor uses infrared light to measure the distance from the camera to the sand surface at thousands of points per frame. This data generates a real-time 3D height map of the sand’s surface. 
• Projector: Mounted next to the depth camera, the projector is calibrated to align perfectly with the sandbox. It overlays digital content (e.g., color-coded elevation data, contour lines, and simulated water) directly onto the sand. This alignment ensures that the projections track precisely with the sand’s physical topography.

• Computer and Open-Source Software: The system runs on a computer equipped with a capable GPU to handle real-time 3D rendering. The most widely used software, developed by Oliver Kreylos at UC Davis, processes the incoming depth data and generates the graphical output. Functions include:

• Interpreting elevation data from the Kinect to create a digital terrain model. 

• Applying a dynamic color gradient based on relative elevation (e.g., green for lowlands, brown for hills, white for peaks). 

• Drawing real-time contour lines to emphasize changes in elevation. 

• Simulating water flow using fluid dynamics, including features like rainfall generation and watershed accumulation.

Step-by-Step Guide to Using the AR Sandbox

Turning on the AR Sandbox:

Step 1:

Plug in the outlet to the left of the sandbox if unplugged.

 

Step 2:

Open the metal doors under the sandbox and turn on the computer console by pressing the circular power button on the front of the unit, top left of the panel.

 

Step 3:

Use the white BENQ remote to turn on the projector, pressing the green button.

 

Step 4:

Use the dark gray remote to turn on the TV, press the red power button (screen should indicate HDMI 3).

 

Step 5:

Click on the red AR Sandbox application on the navigation bar to open the software.

 

***To turn off the sandbox, perform the above steps in reverse order.***

AR Sandbox Settings and Shortcuts

Shortcuts for FLOODING (1) and DRAINING (2)

1. Press and Hold down the Key 1 to make the menu appear and keep it on the screen

Continue to hold Key 1 until the end of the next step

2. Hover the mouse over the menu item “Manage Water” then release Key 1

3. Press Key 2

4. You have now set the following shortcuts:

Key 1 can be used to flood the sandbox

Key 2 can be used to drain the sandbox


Note: Technically any keys (numbers/letters) can be used to set these shortcuts. The key used to open the menu will become the Flooding Shortcut. The key pressed after selecting “Manage Water” will become the Draining Shortcut.


Switching between WATER and LAVA

The code file that determines whether the liquid that appears as WATER or LAVA in the AR Sandbox is called SurfaceAddWaterColor.

There are two folders (WATER and LAVA) on the desktop where these code files are stored so you can copy them as needed. THE CONTENTS OF THESE FOLDERS SHOULD NEVER BE CHANGED–ONLY COPIED.

To change from one liquid to the other, the current file in the AR Sandbox folder must be replaced with the opposite code file of the same name. Here is the process:

1. First, copy the file that you need.

IF: The Sandbox is currently producing Water and you want it to produce Lava open the desktop folder LAVA.

OR

IF: The Sandbox is currently producing Lava and you want it to produce Water open the desktop folder WATER.

Copy the file called SurfaceAddWaterColor inside that folder. DO NOT drag/delete/edit the file in this folder.

Close the window.

2. Next, replace the existing “SurfaceAddWaterColor” file in the Sandbox folder with the one you just copied. To get the file you need to replace, follow this path:

• Open the desktop folder called “ARSandbox’s Home

Open the folder called “src

Open the folder “SARndbox-2.6

Open the folder “share

Open the folder “SARndbox-2.6

Finally, open the folder “Shaders

Locate the file in this folder called SurfaceAddWaterColor (this is the file code you need to replace)

Delete the file.

Now paste the file you copied earlier in its place

AR Sandbox Graphic