Dimono

Designing XR exhibition experience, creating augmented storytelling for Gen Z with embodied interaction.

Timeframe

6 weeks (Fall 2023)

tools

Unity

Blender & Rhino

After Effects

Premiere Pro

Figma

my roles

3D Design

XR UX Interaction Design

Concept Development

Physical Prototyping

collaborators

Yawen Yang

Yanqing Yang

XIaoyu Huang

OVERVIEW

Dimono, designed for CMNH Dinosaur in their Times exhibition, offers a multi-sensory experience where patrons embody their chosen dinosaur, partaking in an evolutionary narrative across eras. Through interactive projection, users engage in real-time conversations, exploring the Mesozoic Era with mixed reality devices. This immersive experience bridges the gap between Gen Z visitors and museum exhibition stories, allowing patrons to appreciate authentic dinosaurs and spatially connect with narratives. Afterward, patrons can review their journey through phone-based AR, transforming passive learning into a personalized exploration of their own lives.

OBJECTIVES

How can we engage and educate Generation Z visitors, who have short attention spans and high digital literacy, in a way that connects them to exhibition narratives?

OVERVIEW

Dimono, designed for CMNH Dinosaur in their Times exhibition, offers a multi-sensory experience where patrons embody their chosen dinosaur, partaking in an evolutionary narrative across eras. Through interactive projection, users engage in real-time conversations, exploring the Mesozoic Era with mixed reality devices. This immersive experience bridges the gap between Gen Z visitors and museum exhibition stories, allowing patrons to appreciate authentic dinosaurs and spatially connect with narratives. Afterward, patrons can review their journey through phone-based AR, transforming passive learning into a personalized exploration of their own lives.

OBJECTIVES

How can we engage and educate Generation Z visitors, who have short attention spans and high digital literacy, in a way that connects them to exhibition narratives?

concept video
key moments
Sammy is an agent to smooth the whole moving journey

We have designed for an experience that prioritizes task optimization, customer-helper synergy, and immediate information access. Sammy epitomizes an intelligent agent, facilitating multitasking through hands-free dialogue for movers.

Before the Move

Scenario 1
Book the right service effortlessly
Tedious booking steps

Scenario 2
Make alteration easily
Repetitive steps to rebook

Scenario 3
Smooth connection between customer & third-party helpers
Inefficient communication & collaboration

During the Move

Scenario 1
Time management for organized
multi-tasking
Stressful multi-tasking

Scenario 2
Make driving safe
Unfamiliar with vehicle operation

Scenario 3
Accompany driving
Stressful driving experience

DESIGN FOR CONVERSATION
Sammy's tone is personable, reliable and energized

The roles and value of Sammy is designed as personable for customized needs, reliable for supporting possible chaotic multi-tasking scenarios, and energized for providing mental support.

Personable: Personal Helper

Reliable: Amid Chaos

Energized: Uplift for Stress

Reserving Truck
Updating Infor
Helper integration
loading
Driving truck
Returning truck
motion states for Sammy
Sammy’s greeting is active, bright, playful, and friendly

Different motion states for Sammy are designed communicate the different states of Sammy to an audience and to visualize responses to a user's input.

key moments
Creating embodied experience to facilitate spatial narratives

Dimono has two parts - a "participatory museum experience" to transform distant historical facts into engaging and relatable narratives for Gen Z, and "path collector" as a tangible reminder of their journey to encourage reflection and revisit to reinforce the delivery of museum's message to their audience.

Mapping Out the Experience

Key Interactions

01 Embodiment

The visitors can choose a dinosaur active in Triassic period they are most resonated to and mimic their posture to start their journey. We let the visitors to become dinosaur to know about dinosaur.

02 Onboarding

The visitors receive their headsets and path collector, going through onboarding to understand the basic interaction.

03 Interact with Paintings
Approaching and lingering at a painting for over 2.5 seconds activates an informative display. Information appears in the head-up display.

04 Transition

When entering new period of time, visitors will be stopped by virtual boundary. It will introduce how the previous period end and what dinosaur they evolved into to acknowledge their progress.

05 Interact with Fossils

When interacting with the fossils, visitors can appreciate the authentic dinosaur and the information and stories spatially together by using guided hand gestures. This is the fundamental interaction for all the fossils in the exhibition.

06 Path Collector

Each fossil display is paired with an incubator. Visitors use the collector to trigger the interaction and collect unique features of their dinosaur fellows. This creates a personalized path through the exhibit.

07 Talk with Iconic Dinosaurs

In addition to the basic interaction, the iconic dinosaurs like t-rex and Dippy will be brought to life and talk to the visitor proactively. Their mood depends on the relationship between them and the dinosaur the visitor is embodying. The storytelling is designed to inspire the resilience of life with museum’s core value.

08 Talk with Iconic Dinosaurs

when visitors leave the exhibit, they can retrieve their experience by scanning unique marks on their collector.These marks represent the diverse features of the dinosaurs they encountered, creating a lasting memory of their journey.

feasible prototypes
How to assess the utility of multi-sensory interactions?

We used laser cutter and 3D printer to create the physical path collector and incubator. We also used Unity and Meta Quest 3 to create workable MR prototypes for testing and iterating.

Path Collector and Incubator Prototypes

Unity Prototype

efficient operational workflow
How does the new injector system elevate medical workflow?

The new multi-control system streamlines complex procedures, enabling faster and tailored operations that significantly boost clinical efficiency.

Intuitive Protocol Management

Old Injectors

Inconsistent settings with analog and digital controls across units

Our New Injector

Standardize setting process across touchscreens for ease of use

Seamless Cooperation

Old Injectors

Fully rely on control room to remotely set up protocols

Our New Injector

Tailored access permissions to support Injector-led workflow possibility

Patient data integration

Old Injectors

Transfer patient data manually, causing delays and errors.

Our New Injector

Synchronize patient data across all displays for accessibility

Brand Identity
What sets our injector apart in the competitive landscape?

High standard visual design boosted brand recognition, establishing our client as a leader in medical device aesthetics and user engagement.

Design System

The comprehensive design system ensures visual consistency and strengthens brand identity across all control units to facilitate scalability and future enhancements.

It's show time

Attention to ID + UIUX enhances the product’s aesthetic appeal and functionality, leading to the successful product launch in 2024.

Enhanced Clinical Accuracy and Reliability
How can the on-machine screen improve clinical performance?

The system utilizes digital simulation to enhance comprehension of the injection process, helping to reduce errors and increase confidence.

Graphic Illustration of Operational Structure

The graphics provide a clear representation of the physical operational structure of the injector, allowing users quickly understand internal mechanisms.

Animation of the Injection Process

The animations demonstrate the fluid dynamics and  injection progress, enhancing the comprehension of the injection procedure.

No items found.
A glimpse of the process
understanding the client
The misson of CMNH (Carnegie Museum of Nature History)

Our client, CMNH, is committed to deepening the understanding of our natural world, to foster a sense of collective responsibility for our future. We’re focusing on their Dinosaur in Their Times exhibition, known for its famous holotypes specimens & reconstruction of the dinosaur habitats.

Why Dinosaur in their Time?

Target audience
Why we are focusing on Gen Z visitors?

Our target audience, Gen Z visitors, whom the museum is currently struggling to retain. They were observed to often just pass through the exhibit and marvel at the wonders. We also conducted one-on-one interviews to understand their needs and values.

According to ...

Gen Z visitors are characterized by shorter attention span and high digital literacy supported.

It may be more challenging for them to stay focused on traditional, less interactive museum, potentially impacting their ability to retain information presented in these formats.

Value Mapping

The Gen Z are actually very interested in dinosaurs. They grew up with Jurassic parks. The dinosaur stories often impress them a lot once explained or revealed

Interview Synthesis

However, the rich stories within the exhibition are not reaching Gen Z visitors. The static displays don’t quite capture their fleeting interests, leading them miss the chance for deeper engagement in the exhibition narratives.

Understanding the space
How is the spatial narratives currently used?

We discovered that the designed route is chronological, the earliest dinosaurs are shown first and follow the path of evolution. However, the curations of the spatial narratives of evolution are often neglected due to the scattered and overwhelming information.

challenge & opportunity
How can a new design help?

By incorporating XR interactions and augmented reality, we aim to spark curiosity and enhance learning through immersive storytelling. Our goal is to convert passing interests into sustained engagement ultimately envisioning CMNH as an interactive, participatory museum.

A glimpse of the process
understanding the client
The misson of CMNH (Carnegie Museum of Nature History)

Our client, CMNH, is committed to deepening the understanding of our natural world, to foster a sense of collective responsibility for our future. We’re focusing on their Dinosaur in Their Times exhibition, known for its famous holotypes specimens & reconstruction of the dinosaur habitats.

Why Dinosaur in their Time?

Target audience
Why we are focusing on Gen Z visitors?

Our target audience, Gen Z visitors, whom the museum is currently struggling to retain. They were observed to often just pass through the exhibit and marvel at the wonders. We also conducted one-on-one interviews to understand their needs and values.

According to ...

Gen Z visitors are characterized by shorter attention span and high digital literacy supported.

It may be more challenging for them to stay focused on traditional, less interactive museum, potentially impacting their ability to retain information presented in these formats.

Value Mapping

The Gen Z are actually very interested in dinosaurs. They grew up with Jurassic parks. The dinosaur stories often impress them a lot once explained or revealed

Interview Synthesis

However, the rich stories within the exhibition are not reaching Gen Z visitors. The static displays don’t quite capture their fleeting interests, leading them miss the chance for deeper engagement in the exhibition narratives.

Understanding the space
How is the spatial narratives currently used?

We discovered that the designed route is chronological, the earliest dinosaurs are shown first and follow the path of evolution. However, the curations of the spatial narratives of evolution are often neglected due to the scattered and overwhelming information.

challenge & opportunity
How can a new design help?

By incorporating XR interactions and augmented reality, we aim to spark curiosity and enhance learning through immersive storytelling. Our goal is to convert passing interests into sustained engagement ultimately envisioning CMNH as an interactive, participatory museum.

A GLIMPSE OF THE PROCESS
why design for moving
Moving is stressful and closely tied to significant life changes

Is moving a headache for you? We feel overwhelmed by the planning process. We struggle with loading and packing. We have trouble with moving logistics and coordination. What's more, it’s something so many people are doing all the time.

33%
of people in their 20s move in a given year

17-20 Million
people may move this summer

11
average number of times a person moves in a lifetime

1 out of 5
people move every year

why design for u-haul
U-Haul's core customers are do-it-yourself movers

U-Haul is one of the biggest transportation and storage solution company in North America. Their movers, mostly do-it-yourselfers are those embarking on this moving journey, who is bound to encounter moments of stress.

75%
of movers are “do-it-yourselfers”

costs 50% less
than full service moving solutions

127,000
number of trailers in U-Haul’s fleet

176,000
number of trucks in U-Haul’s fleet

UNDERSTANDING THE CLIENT AND THEIR APPLICATIONS
U-Haul's design opportunities are plentiful

We conducted primary research on U-Haul's exiting service. Their service is comprehensive but overwhelming. Therefore, we did an interface audit on their U-Haul website & app and ecosystem analysis to learn and find insights into the current user flow. We discovered that:

Interface Audit

The navigation is rather overwhelming. We observed chaotic information architecture, wordy & confusing description and repetitive but hard-to-find access towards instructions.

Ecosystem Analysis

U-Haul's service is very comprehensive. However, it also makes the customers lost in the searching. What's more, the connection between customers with adding-on service like Moving Helper is very loose and unguided.

user interview
Interviews as key to informing opportunities for design

Through 8 + one-on-one user interviews and online research, we found common themes in grievances and complaints.

Interview Samples

Qualified Interviewees
21+ Years Old Do-It-Yourself Movers

What We Heard
What We Learned
What This Means

"I'm afraid of driving a big truck. There is no rear mirror!"
"We were all not familiar with trucks and the interior and accidentally broke a handle."

Anxiety about driving trucks & misuse of equipment

Design for anxiety relief and instruction

"None of the records reflect the movers I ordered, no emails or text nor the website. Does this mean they are not booked and I need to book someone else?"

Challenges in accessing information & alternation

Design for accessible information in real-time

"The way U-haul helps me choose the truck isn't effective. The customer service makes me wait for hours."
"I had to cancel my reservation and rebook just to change the moving date."

Challenges in choosing the right service

Design for an agent could assist in choosing and altering order without redundant steps

"I tried to plan everything ahead but the moving day is still very stressful. Things always go wrong."

Reveal mental stress in multi-tasking

Design for schedule alerts and reminders for users

Insights

Reserving Truck
Updating Infor
Helper integration
loading
Driving truck
Returning truck
Conversational user interface
We see CUI as a possible solution

CUI can improve user experience, streamlined interactions, accessibility, and task automation for enhanced efficiency.

Storyboard 1.0

Speculating how CUI assistant would improve the situations

No items found.
No items found.
No items found.
No items found.
No items found.
Design principles
We design our system according to...

To address our research question in our design, we followed these four design principles:

system diagram
How are we using the technologies?

This system blends various technologies such as digital projection mapping, Mixed Reality (MR) interactions, and natural language processing.

Digital Projections Mapping

XR Interactions

Nature Language Processing(NLP)

Visual system
Have you dreamed about being a dinosaur in their times?

To seamlessly guide our visitors, we created an integrated visual system.

Branding

Spatial UI for an intuitive, low-learning curve user experience

01 Information Panel

Visitors initiate interaction with gaze control which will trigger the info to appear within 2.5 seconds.

Eye Control

To interact with the fossil, hover over to reveal the info panel, pinch for more information, and grab to examine fossils closely. We designed the hierarchy to avoid info overload.

          Hover

          Pinch

            Grab

02 Navigation Bar

Navigation bar has three distinct stages. We designed it appear as eggs to relate to the ‘life’ topic. Pat to switch among three stages—default, unselected, and selected—to activate information panels and secondary selections.

             Pat

           Pinch

03 Conversational UI

Distinct dialogue bubbles for iconic dinosaurs and visitors for visual clarity. And pinch the hint bubbles for suggested conversation starter.

    Speak to Text

          Pinch

AR App Design

Sound in MR Space

When design the sound space in MR experience, we used the following principles:
Bring the users back to the dinosaurs’ time and environment with 3D spatial audio; Aural feedback to user’s interactions; Enhance the emotional and non-intrusive narrative events; Maintain immersion and consistency across platform.

"Malia led the team through a tough start with stakeholders new to the field, skillfully navigating challenges to deliver a standout product."
– Quan, Design Director at WILDDESIGN
Challenge 1: filling the gap
How to design a new UX in a market with few examples?

In a market dominated by CT injectors with control room displays and limited on-machine controls, our challenge was to innovate a user experience with advanced on-machine touchscreens to meet and exceed current clinical procedures.

Market Research for Opportunities

Even with few direct examples, market research showed growth potential for syringeless injectors. It also showed the need for intuitive on-machine controls to streamline workflows for better user interaction and efficiency.

Insight 1: Look into the future trend

We should focus on design UX for the new syringeless injectors, which are poised for growth due to their efficiency, safety, and waste reduction.

Insight 2: User habits matter

Preserving conventions, such as color coding (blue for saline, green for contrast medium), enhances usability and eases transitions.

Insight 3: Explore workflow integration

Recent injector systems allow customized patient protocols and usage tracking to improve operational efficiency. Visibility on the machine can further enhance accessibility.

Insight 4: Screen-Driven Clarity

On-machine touch screens should be designed to offer intuitive controls and simulation capabilities, simplifying  training and reduce dependency on remote systems.

Design Collaboration

I led discussions across teams to improve the input information architecture and wireframes by asking straightforward and critical questions about feedback and requests. We also utilized paper prototypes for quick, remote cross-team evaluations, enabling efficient design enhancements.

Collaboration Adjustment




Fast Prototyping




Refined Information Architecture & Wireframes

Design focus
Establishing the scope of the project

We decided to focus the design to create an experience that prioritizes task optimization, customer-helper synergy, and immediate information access, ensuring a seamless and stress-free experience for all users.

Persona
Who will use Sammy?

We identify our core customer as the do-it-yourselfers who feel anxious and overwhelmed by the moving process. The Moving Helpers as the U-Haul contractors are our secondary users, providing professional accessibility service to those in need and getting extra income.

Sammy's values
Sammy is a ...

Based on our immersive research, we identified 3 main roles that Sammy acts as, which are guide, reminder, and cheerleader. The percentage of the 3 roles and personalities that Sammy exhibits when interacting with customers and moving helpers varies depending on the current scenario.

interaction design
How will Sammy be used?

More detailed storyboard & user journey map are created to refine the narrative, incorporating feedback, and enhancing visual storytelling to align better with the design focus.

Storyboard 2.0 & User journey

Overall

Detailed - Reservation

Detailed - Contacting Moving Helper

No items found.
Challenge 2: High impact, low strain
How to create intuitive simulations without straining resources?

As we learnt that incorporating digital simulations into the CT injector enhances the intuitiveness of the injection process. The challenge was to achieve this without overloading system resources, ensuring high reliability crucial in a medical context.

Identifying essential Simulations

I collaborated closely with clients and engineers to determine the optimal amount of essential animated simulation through technical testing and iterative feedback.

Find the right animation method

After determining the key animated simulations, I started to explore desirable ways to both communicate impactful message and efficiently utilize resources, especially for injection and fluid simulation.

🥇Frame-by-frame

✅Pros
Memory-efficient, simple & best for low-memory systems

❎Cons

Less dynamic

🥈Embedded vector

✅Pros
Flexible, interactive & great for dynamic content

❎Cons

Requires careful memory management

🥉GIF

✅Pros
Simple & requires minimal coding effort

❎Cons

Low resolution, memory-heavy & less flexible or interactive for real-time updates

XR Prototype
Studying XR interactions during a museum visit

We tested UI components using paper prototypes to fast iterate the positions and angles of UI components. And then we built a digital demo with Unity. The design of path collector and incubator keep iterating when we updated spatial UI component.

Paper Prototypes
The rapid prototyping helped us test the spatial postion and angle of the UI components.

Unity Prototypes
Unity demo helped us test the scale and position of Egg UI in spatial space; Test the “pat”  and “distance grab” gesture usability; Test the usability of interaction with skeleton and user flow.

Physical Prototypes

The path collector and the incubator are the key to enter the AR interactions. They should not only act as a working prop but also an attractive piece of designed product.

XR Component arrangement
Generating optimal XR component arrangement  

The arrangement of UX components aims to minimize visual disruption and easy and ergonomic reaching.

Defining Identity
Introducing Sammy

‘Sammy’ is the intelligent conversational assistant we design to integrate in the U-Haul app. The mechanism and system of Sammy is designed according to the design focus we generated based on our research.

Why "Sammy"?

U-Haul introduced it's first version of its "Sammy U" character in 1953, named after the company's founder.

How should Sammy look?

Style Identification
The visual design is informed by U-Haul’s brand. It takes cues from U-Haul’s boxy, yet rounded equipment and incorporates core values like reliability and stability into its tone.

Final Form
Our final visual design began with a more abstract representation of a box and landed at a more literal design that felt more on-brand with U-Haul. The idea of an active opened box speaks to the core user experience for U-Haul service.

Motion Design
Curated visual cues and motion states, fostering more natural and engaging tech interactions. Sammy's varied motions add personality and clarify messages to users.

interface Design
Integrating Sammy to UI

Our emphasis was on ensuring optimal reading legibility specifically tailored for mobile users, maintaining alignment with U-Haul's brand identity throughout the design system.

Branding

We developed a UI design system that incorporates U-Haul's branding elements, encompassing a cohesive color palette, information cards, icons, and buttons.

Redesign Homepage

The app's design features two distinct user paths: one for users with upcoming orders and one for starting brand new orders.

CUI Pattern

When a user starts a conversation, a compact pop-up screen captures and exhibits the inquiry, preventing abrupt transitions to new pages. This pop-up then expands to a mid-height page, showcasing all dialogues, arranged from the bottom upward.

GUI Overview

Here are the final interfaces for the redesigned U-Haul app. These screens cover the customers' pre-moving experience, Moving Helper collaboration & moving day (driving) experience.

Challenge 3: Designing for Distinction
How to develop a visual style maximizing our brand's impact?

Designing a unique visual interface involved navigating the client’s cautious decision-making process. The client often leaned towards familiar, traditional styles, seeking safety in what was known. Our challenge was to find the sweet spot to elevate the product and maintain competitiveness.

Keep learning and iterating

Conducted iterative design adjustments based on client feedback, maintaining and learning from high visual standards while introducing innovative elements.

Initialize Design System

The development team expanded the interface using initial components, but inconsistencies in screens revealed the need for standardized guidelines. I proposed a design system across all device to ensure consistency and scalability in future designs.




Engineering Team Inquiry




Design System Templates

No items found.
impact
What can Dimono bring?

To transform the natural history museum journey for Gen Z visitors, empowering them with knowledge that bridges the past and present, inspiring stewardship for our collective future.

meet our team!
Reflection
Things I've learned...

As one of the biggest projects I was in charge of, the CT injector project strengthened both my technical and interpersonal skills, making me a strong team player.

Collaboration & Iteration 🔃

Working in medical design requires close cooperation with industrial design, human factors, and engineering teams. Maintaining tight collaboration across disciplines and open for iteration ensures that the design meets both high standard market and functional regulatory requirements.

Client Education 🧐

Educating clients requires skillful communication, as they may not always realize that their requests might not be feasible. I discovered that using their language and visualizing ideas effectively can bridge understanding gaps, leading to better decision-making and outcomes.

Always ask for clarity 🙋

Leading this project gave me the opportunity to work closely with developers, project managers, and top decision-makers. I learned that asking questions clearly stating requests and expectations to clarify uncertainties are essential for team efficiency and ensures everyone is aligned.  

My piles of paper prototypes

Design that last
How will Sammy affect U-Haul ecosystem?

Our new design makes U-Haul’s services not only more efficient, but also more inclusive and accessible, which in turn supports more diverse users and aligns with the principle of social responsibility. An expanded customer base also supports the growth of the shared economy. What’s more, the feasibility of Sammy is high, leveraging existing new technology like generative AI, and GPT-4.

Maybe you're also interested in...

play-time

Learning Experience Design
service design

An anti-bullying guardian system utilizes the concept of non-directive play therapy to evoke students' empathy & self-worth through workshops & play.

→ View Project

JOJO

Animation
service design

A companion system that aims to make child life service more tangible with pre-admission guidance through interactive cartoon series & on-site psychological support.

→ View Project