Designing XR exhibition experience, creating augmented storytelling for Gen Z with embodied interaction.
Unity
Blender & Rhino
After Effects
Premiere Pro
Figma
3D Design
XR UX Interaction Design
Concept Development
Physical Prototyping
Yawen Yang
Yanqing Yang
XIaoyu Huang
OVERVIEW
Dimono, designed for CMNH Dinosaur in their Times exhibition, offers a multi-sensory experience where patrons embody their chosen dinosaur, partaking in an evolutionary narrative across eras. Through interactive projection, users engage in real-time conversations, exploring the Mesozoic Era with mixed reality devices. This immersive experience bridges the gap between Gen Z visitors and museum exhibition stories, allowing patrons to appreciate authentic dinosaurs and spatially connect with narratives. Afterward, patrons can review their journey through phone-based AR, transforming passive learning into a personalized exploration of their own lives.
OBJECTIVES
How can we engage and educate Generation Z visitors, who have short attention spans and high digital literacy, in a way that connects them to exhibition narratives?
OVERVIEW
Dimono, designed for CMNH Dinosaur in their Times exhibition, offers a multi-sensory experience where patrons embody their chosen dinosaur, partaking in an evolutionary narrative across eras. Through interactive projection, users engage in real-time conversations, exploring the Mesozoic Era with mixed reality devices. This immersive experience bridges the gap between Gen Z visitors and museum exhibition stories, allowing patrons to appreciate authentic dinosaurs and spatially connect with narratives. Afterward, patrons can review their journey through phone-based AR, transforming passive learning into a personalized exploration of their own lives.
OBJECTIVES
How can we engage and educate Generation Z visitors, who have short attention spans and high digital literacy, in a way that connects them to exhibition narratives?
We have designed for an experience that prioritizes task optimization, customer-helper synergy, and immediate information access. Sammy epitomizes an intelligent agent, facilitating multitasking through hands-free dialogue for movers.
Scenario 1
Book the right service effortlessly
Tedious booking steps
Scenario 2
Make alteration easily
Repetitive steps to rebook
Scenario 3
Smooth connection between customer & third-party helpers
Inefficient communication & collaboration
Scenario 1
Time management for organized
multi-tasking
Stressful multi-tasking
Scenario 2
Make driving safe
Unfamiliar with vehicle operation
Scenario 3
Accompany driving
Stressful driving experience
The roles and value of Sammy is designed as personable for customized needs, reliable for supporting possible chaotic multi-tasking scenarios, and energized for providing mental support.
Different motion states for Sammy are designed communicate the different states of Sammy to an audience and to visualize responses to a user's input.
Dimono has two parts - a "participatory museum experience" to transform distant historical facts into engaging and relatable narratives for Gen Z, and "path collector" as a tangible reminder of their journey to encourage reflection and revisit to reinforce the delivery of museum's message to their audience.
01 Embodiment
The visitors can choose a dinosaur active in Triassic period they are most resonated to and mimic their posture to start their journey. We let the visitors to become dinosaur to know about dinosaur.
02 Onboarding
The visitors receive their headsets and path collector, going through onboarding to understand the basic interaction.
03 Interact with Paintings
Approaching and lingering at a painting for over 2.5 seconds activates an informative display. Information appears in the head-up display.
04 Transition
When entering new period of time, visitors will be stopped by virtual boundary. It will introduce how the previous period end and what dinosaur they evolved into to acknowledge their progress.
05 Interact with Fossils
When interacting with the fossils, visitors can appreciate the authentic dinosaur and the information and stories spatially together by using guided hand gestures. This is the fundamental interaction for all the fossils in the exhibition.
06 Path Collector
Each fossil display is paired with an incubator. Visitors use the collector to trigger the interaction and collect unique features of their dinosaur fellows. This creates a personalized path through the exhibit.
07 Talk with Iconic Dinosaurs
In addition to the basic interaction, the iconic dinosaurs like t-rex and Dippy will be brought to life and talk to the visitor proactively. Their mood depends on the relationship between them and the dinosaur the visitor is embodying. The storytelling is designed to inspire the resilience of life with museum’s core value.
08 Talk with Iconic Dinosaurs
when visitors leave the exhibit, they can retrieve their experience by scanning unique marks on their collector.These marks represent the diverse features of the dinosaurs they encountered, creating a lasting memory of their journey.
We used laser cutter and 3D printer to create the physical path collector and incubator. We also used Unity and Meta Quest 3 to create workable MR prototypes for testing and iterating.
Our client, CMNH, is committed to deepening the understanding of our natural world, to foster a sense of collective responsibility for our future. We’re focusing on their Dinosaur in Their Times exhibition, known for its famous holotypes specimens & reconstruction of the dinosaur habitats.
Our target audience, Gen Z visitors, whom the museum is currently struggling to retain. They were observed to often just pass through the exhibit and marvel at the wonders. We also conducted one-on-one interviews to understand their needs and values.
Gen Z visitors are characterized by shorter attention span and high digital literacy supported.
It may be more challenging for them to stay focused on traditional, less interactive museum, potentially impacting their ability to retain information presented in these formats.
The Gen Z are actually very interested in dinosaurs. They grew up with Jurassic parks. The dinosaur stories often impress them a lot once explained or revealed.
However, the rich stories within the exhibition are not reaching Gen Z visitors. The static displays don’t quite capture their fleeting interests, leading them miss the chance for deeper engagement in the exhibition narratives.
We discovered that the designed route is chronological, the earliest dinosaurs are shown first and follow the path of evolution. However, the curations of the spatial narratives of evolution are often neglected due to the scattered and overwhelming information.
By incorporating XR interactions and augmented reality, we aim to spark curiosity and enhance learning through immersive storytelling. Our goal is to convert passing interests into sustained engagement ultimately envisioning CMNH as an interactive, participatory museum.
Is moving a headache for you? We feel overwhelmed by the planning process. We struggle with loading and packing. We have trouble with moving logistics and coordination. What's more, it’s something so many people are doing all the time.
33%
of people in their 20s move in a given year
17-20 Million
people may move this summer
11
average number of times a person moves in a lifetime
1 out of 5
people move every year
U-Haul is one of the biggest transportation and storage solution company in North America. Their movers, mostly do-it-yourselfers are those embarking on this moving journey, who is bound to encounter moments of stress.
75%
of movers are “do-it-yourselfers”
costs 50% less
than full service moving solutions
127,000
number of trailers in U-Haul’s fleet
176,000
number of trucks in U-Haul’s fleet
We conducted primary research on U-Haul's exiting service. Their service is comprehensive but overwhelming. Therefore, we did an interface audit on their U-Haul website & app and ecosystem analysis to learn and find insights into the current user flow. We discovered that:
The navigation is rather overwhelming. We observed chaotic information architecture, wordy & confusing description and repetitive but hard-to-find access towards instructions.
U-Haul's service is very comprehensive. However, it also makes the customers lost in the searching. What's more, the connection between customers with adding-on service like Moving Helper is very loose and unguided.
Through 8 + one-on-one user interviews and online research, we found common themes in grievances and complaints.
Qualified Interviewees
21+ Years Old Do-It-Yourself Movers
"I'm afraid of driving a big truck. There is no rear mirror!"
"We were all not familiar with trucks and the interior and accidentally broke a handle."
Anxiety about driving trucks & misuse of equipment
Design for anxiety relief and instruction
"None of the records reflect the movers I ordered, no emails or text nor the website. Does this mean they are not booked and I need to book someone else?"
Challenges in accessing information & alternation
Design for accessible information in real-time
"The way U-haul helps me choose the truck isn't effective. The customer service makes me wait for hours."
"I had to cancel my reservation and rebook just to change the moving date."
Challenges in choosing the right service
Design for an agent could assist in choosing and altering order without redundant steps
"I tried to plan everything ahead but the moving day is still very stressful. Things always go wrong."
Reveal mental stress in multi-tasking
Design for schedule alerts and reminders for users
CUI can improve user experience, streamlined interactions, accessibility, and task automation for enhanced efficiency.
Speculating how CUI assistant would improve the situations
To address our research question in our design, we followed these four design principles:
This system blends various technologies such as digital projection mapping, Mixed Reality (MR) interactions, and natural language processing.
Digital Projections Mapping
XR Interactions
Nature Language Processing(NLP)
To seamlessly guide our visitors, we created an integrated visual system.
01 Information Panel
Visitors initiate interaction with gaze control which will trigger the info to appear within 2.5 seconds.
Eye Control
To interact with the fossil, hover over to reveal the info panel, pinch for more information, and grab to examine fossils closely. We designed the hierarchy to avoid info overload.
Hover
Pinch
Grab
02 Navigation Bar
Navigation bar has three distinct stages. We designed it appear as eggs to relate to the ‘life’ topic. Pat to switch among three stages—default, unselected, and selected—to activate information panels and secondary selections.
Pat
Pinch
03 Conversational UI
Distinct dialogue bubbles for iconic dinosaurs and visitors for visual clarity. And pinch the hint bubbles for suggested conversation starter.
Speak to Text
Pinch
When design the sound space in MR experience, we used the following principles:
Bring the users back to the dinosaurs’ time and environment with 3D spatial audio; Aural feedback to user’s interactions; Enhance the emotional and non-intrusive narrative events; Maintain immersion and consistency across platform.
We decided to focus the design to create an experience that prioritizes task optimization, customer-helper synergy, and immediate information access, ensuring a seamless and stress-free experience for all users.
We identify our core customer as the do-it-yourselfers who feel anxious and overwhelmed by the moving process. The Moving Helpers as the U-Haul contractors are our secondary users, providing professional accessibility service to those in need and getting extra income.
Based on our immersive research, we identified 3 main roles that Sammy acts as, which are guide, reminder, and cheerleader. The percentage of the 3 roles and personalities that Sammy exhibits when interacting with customers and moving helpers varies depending on the current scenario.
More detailed storyboard & user journey map are created to refine the narrative, incorporating feedback, and enhancing visual storytelling to align better with the design focus.
Overall
Detailed - Reservation
Detailed - Contacting Moving Helper
‘Sammy’ is the intelligent conversational assistant we design to integrate in the U-Haul app. The mechanism and system of Sammy is designed according to the design focus we generated based on our research.
U-Haul introduced it's first version of its "Sammy U" character in 1953, named after the company's founder.
Style Identification
The visual design is informed by U-Haul’s brand. It takes cues from U-Haul’s boxy, yet rounded equipment and incorporates core values like reliability and stability into its tone.
Final Form
Our final visual design began with a more abstract representation of a box and landed at a more literal design that felt more on-brand with U-Haul. The idea of an active opened box speaks to the core user experience for U-Haul service.
Motion Design
Curated visual cues and motion states, fostering more natural and engaging tech interactions. Sammy's varied motions add personality and clarify messages to users.
Our emphasis was on ensuring optimal reading legibility specifically tailored for mobile users, maintaining alignment with U-Haul's brand identity throughout the design system.
We developed a UI design system that incorporates U-Haul's branding elements, encompassing a cohesive color palette, information cards, icons, and buttons.
The app's design features two distinct user paths: one for users with upcoming orders and one for starting brand new orders.
When a user starts a conversation, a compact pop-up screen captures and exhibits the inquiry, preventing abrupt transitions to new pages. This pop-up then expands to a mid-height page, showcasing all dialogues, arranged from the bottom upward.
Here are the final interfaces for the redesigned U-Haul app. These screens cover the customers' pre-moving experience, Moving Helper collaboration & moving day (driving) experience.
We tested UI components using paper prototypes to fast iterate the positions and angles of UI components. And then we built a digital demo with Unity. The design of path collector and incubator keep iterating when we updated spatial UI component.
Paper Prototypes
The rapid prototyping helped us test the spatial postion and angle of the UI components.
Unity Prototypes
Unity demo helped us test the scale and position of Egg UI in spatial space; Test the “pat” and “distance grab” gesture usability; Test the usability of interaction with skeleton and user flow.
Physical Prototypes
The path collector and the incubator are the key to enter the AR interactions. They should not only act as a working prop but also an attractive piece of designed product.
The arrangement of UX components aims to minimize visual disruption and easy and ergonomic reaching.
To transform the natural history museum journey for Gen Z visitors, empowering them with knowledge that bridges the past and present, inspiring stewardship for our collective future.
Our new design makes U-Haul’s services not only more efficient, but also more inclusive and accessible, which in turn supports more diverse users and aligns with the principle of social responsibility. An expanded customer base also supports the growth of the shared economy. What’s more, the feasibility of Sammy is high, leveraging existing new technology like generative AI, and GPT-4.
Child PPE customization App, which bridges the function & the heritage, forming a new way to both care about the children & our tradition.
A systematic and biology-based solution to waste management in urban dog parks, turning waste into electricity to grow microgreens.
