back

Use Your Melon

A gesture-controlled installation challenging Georgia Aquarium guests to a game of echolocation charades

Partner
Timeline
  • 2019–2020
Role
  • UX Design
  • Software Development
Tools
  • Unity
  • Kinect
  • Figma
Outcomes
  • Completed Georgia Tech MS-HCI thesis
  • Shipped a Unity + Kinect pilot
  • Ran nine stakeholder and expert evaluations
A child mid-transformation into a lemon shark in front of the installation screen

summary

The Georgia Aquarium’s Dolphin Coast opens onto a pre-show atrium where guests wait — often the better part of an hour — in front of blank walls and a small tank view. For our Georgia Tech master’s thesis, we asked how that space could be better leveraged to teach people about the animals they were about to meet.

Use Your Melon turns the wait into a game of charades. Guests strike a pose to mimic a sea creature, and a digital dolphin echolocates to identify them and explain their relationship in the wild.

The walls were pretty blank… it would’ve been nice to have a little more to do

Aquarium guest
Guests crowded into the Dolphin Coast pre-show atrium
“On days where we only have 4 shows, guests might have to arrive an hour early to get a seat.” — Georgia Aquarium staff member

my role

I pitched in on the guest research, sitting in on the observations and interviews and helping sort it all onto the affinity map.

Once we landed on a concept for a gesture-based installation, I designed the interactions and the general flow, then built technical spikes to test the capabilities of our hardware. From there I integrated all of the UI into a working pilot in Unity, all from the comfort of my apartment during lockdown.

how to play

step right up

Approach the screen and a dolphin companion greets you beside a bubblized version of yourself.

strike a pose

Find the hidden gesture for one of six creatures and the dolphin starts echolocating to identify you.

transform

Hold the pose, burst into bubbles, and learn how that creature lives alongside dolphins in the wild.

On-screen gesture key showing creature poses and the motions to copy

process

Affinity map covering a wall, with sorted research cards laid out on tables

guest research

We interviewed 14 participants across guests, floor staff, and managers, then processed their feedback on an affinity map.

Aquarium stakeholders sketching concepts during a participatory design workshop

participatory design

Five aquarium stakeholders helped brainstorm interaction concepts in a group session.

gestural study

We asked some friends to mimic different sea creatures on camera, and six came through consistently enough to base the game on.

technical spikes

I built quick prototypes to test our core interactions against Microsoft Kinect’s gesture recognition engine.

Use Your Melon title screen: the logo and echolocation mark over an undersea scene, with the six creature icons beneath
Common octopus stat card with a bilingual fact about dolphins
Every creature carries an aquarium-approved stat card and fact in English and Spanish.
Graceful failure states activate when players step away from the camera.

outcomes

  • — Completed the work as a Georgia Tech MS-HCI master’s thesis advised by Dr. Carrie Bruce
  • — Partnered with the Georgia Aquarium to design for their Dolphin Coast pre-show atrium
  • — Shipped a feature-complete Unity and Kinect demo recognizing six creature poses
  • — Evaluated remotely with four aquarium stakeholders and five physical-interaction experts; 12 of 13 heuristics averaged only cosmetic or minor issues
Hand-cut paper sea creatures and echolocation waves laid out on a table beside a legal pad of prototype notes

Credits

Special thanks