Use Your Melon
Challenging aquarium guests to a game of echolocation charades
Duration: Aug. 2019 - May 2021
Team: 1 UX researcher, 1 UX designer
Tools: Sketch, Unity, Kinect
My contributions
Researched guests’ waiting behavior at the Georgia Aquarium through observations and interviews
Designed a gesture-based installation to educate and entertain guests in the dolphin pre-show area
Programmed a polished, testable demo using Unity and the Microsoft Kinect
Built in partnership with the Georgia Aquarium
A sight to sea! 🌊
At the Georgia Aquarium's "Dolphin Coast," guests can enjoy a spectacular live performance of dolphin acrobatics. The pre-show atrium, however, is less-than-spectacular, with few activities to help pass the longer wait times.
For my master’s thesis, we designed a playful installation to entertain waiting guests by teaching them about the different creatures in dolphin ecosystems.
Objectives
01
Identify and address the needs of guests waiting in the dolphin pre-show area
02
Support the aquarium's mission to inspire awareness and preservation of aquatic animals
03
Produce a polished installation pilot that could be deployed in guest-facing areas for testing
Process
This project had WAY too many steps to pack into one webpage (shown below), so I’ve done my best to pull out the highlights!
Research · 10 weeks
Literature review
Case studies
Observational studies
Interviews
Affinity mapping
Design · 5 weeks
Participatory design
Paper prototypes
Storyboarding
Tech feasibility
Gestural studies
Build & Evaluate · 8 weeks
Technical spikes
Unity demo
Heuristic evaluation
Stakeholder evaluation
If you’re interested in the full breakdown, check out our 137-page process book or feel free to ask me 😬
01
Research
Observations
We began our research with 5 days of guest observations, where we passively studied guests’ behaviors in the dolphin show atrium.
Interviews
We then conducted 14 in-person interviews with guests, floor staff members, and managers to unpack the waiting experience more granularly.
“On days where we only have 4 shows, guests might have to arrive an hour early to get a seat”
“The walls were pretty blank...it would’ve been nice to have a little more to do”
Key findings
Our research revealed that the dolphin atrium was dull and confusing compared to the aquarium’s other spectacular exhibits. To remedy that, we developed the following opportunity statements as guides for our concepting:
Helping guests navigate the dolphin area
Warming up guests for the live dolphin performance
Teaching guests more about the aquarium’s dolphins
Providing guests with alternative plans when they can’t access the show
02
Design
Participatory design session
To kick off our design phase, we invited 5 stakeholders from the aquarium to brainstorm ideas based on our researched pain points and opportunity areas.
Paper prototypes
We riffed on our stakeholders’ concepts by creating simple prototypes to test out the interactions.
Our conclusion was that a gesture-based installation had a high potential for entertaining guests without requiring too much hardware.
Storyboarding
The final concept allowed guests to play with digital dolphins by physically mimicking different sea creatures.
As guests struck a pose, the dolphins would echolocate to interpret their gestures and respond accordingly.
03
Build
Gestural study
We asked some friends to try mimicking different creatures to learn which gestures would work best for our recognition model.
Ultimately, we chose to include the six creatures that our testers mimicked most consistently: a crab, diver, fish, octopus, shark, and kelp cluster.
Technical spikes
Using the results of the gestural study, I built this quick prototype to test our poses with Microsoft Kinect’s gesture recognition engine.
Unity demo
Through constant iteration and spectacular visual design support—thanks Rex!—I built out a feature-complete demo of the installation in the Unity engine.
04
Evaluation
COVID-19 impact 🦠
This installation was originally intended to be assembled in the aquarium for live testing with guests. However, due to the global pandemic we pivoted to a remote evaluation plan where testers reviewed video footage of the demo.
Stakeholder evalutations
We asked 4 aquarium stakeholders to complete a survey about the how the installation addressed the project’s goals. Overall they were pleased with its educational value and encouraged us to pursue more long-term interactions.
Expert evaluations
We also reached out to 5 experts in the physical interaction industry to collect feedback on the installation’s design. This was structured around UX heuristics published by the Nielsen-Norman group and video game journals.
Example heuristic
Feedback should be given at appropriate times to the user (Pinelle & Brown, 2008)
Expert’s response
[1 - Low severity issue]
“I read relevant descriptions about each character and felt those were provided at right times.”
Final design
How to play
Step right up
Once a player approaches the installation, they’re greeted by a dolphin companion and a bubblized version of themselves.
Strike a pose
Players then must discover a hidden gesture corresponding to one of the six creatures shown onscreen (left). Once they do, the dolphin will start to identify them.
Transform!
If the player holds a creature pose, they’ll transform into that creature in a flurry of bubbles and see facts about how that creature interacts with dolphins in the wild.
How it works
A Microsoft Kinect detects users’ gestures by running depth camera data through a machine learning model. This information feeds into a Unity game, which displays a responsive undersea environment in real time.
Design features
Deep learning
Guests can transform into six creatures, each of which is accompanied by an aquarium-approved stat card and fact.
Reactive gesture tips
If a player is having trouble discovering the hidden gestures, tips will appear to nudge them in the right direction.
Play boundary warnings
When players step out of the designated play area, they’re gently notified to re-enter the Kinect’s view.
Captions
The interface features English and Spanish captions—the two most common languages spoken at the aquarium—to ensure that guests can still enjoy the experience when the atrium is loud and crowded.
Wrap-up
Despite COVID-19’s impact on our deployment, I had a blast getting to dust off my Unity skills to build this playful and educational experience.
Thank you to Jahmar Hannans from the Georgia Aquarium for making this partnership possible, Dr. Carrie Bruce for overseeing our work, and my wonderful teammates Rex and Courtney for putting in the time to create something special.
Thanks for reading this! 🙌