CMU Mixed Reality Tour Experience

Thien Le
12 min readOct 3, 2021

--

Environments Studio I

Project Brief: While campus tours are a wonderful way to introduce the university, the information is broad, focused on what’s on campus, and questions of student life and individual programs are limited to the knowledge of the Tartan Ambassador providing the tour. Provide two items: (a) an overview of the whole tour experience and (b) a low-fi prototype of a single tour stop. Through this presentation you will explain the stellar user experience afforded by your approach to AR and they will give you sign off to design the other stops on the tour.

In order to better understand what the problems are with campus tours, we created fictional personas. In this way, we can identify needs of potential users of our mixed reality tour service.

Personas:

For our personas, we decided to have a range of people so that we can approach our possible experience from multiple different perspectives. We considered personas with different ethnicities, interests, ages, and backgrounds.

Thien Self Reflection: As the prevalence of digital media in our physical environments increases daily, what is the role and/or responsibility of designers in shaping our environments?

I think that the technology and engineering exists for every facet of our lives to be digital. We have the capabilities for everything to be virtual but as designers, we are humanists in that we have to consider what is essential and necessary. As seen with Zoom and learning online, people do not need everything to be virtual because we know that human to human connection is essential in our development and learning. It was only a solution because there was no other means necessary, but designers have the capabilites to see that hybrid environments can be enhance our experiences in some cases. I think that if it were up to engineers, every experience would be virtual but designers know that it wouldn’t enhance interactions or our lives in most cases so I think that it is our responsibility to look at the bigger picture and only enhance what is necessary.

Development of Concept

Issues of Current Campus Tours:

  • Not very personalized, have to follow tour leader.
  • Not very fun or interactive since people just walk around and look at stuff.
  • Could be intimidating for people scared to ask questions to tour guide.
  • Time constraints

Potential Solution: The initial tour would start at the UC since that is a common space that every student will interact with and learn to use but to branch out and get visitors to walk through the campus, there will be a scotty dog avatar that guides them through how to use the HoloLens tour experience. The avatar will then give them categories through which they can experience certain aspects of CMU i.e. food, art, architecture, sports, work spaces, etc. At each location they go to, there can be an interactive experience along with information about the history and the school so that visitors can picture themselves there.

Visiting Location

Pictures of stadium and soccer field

Questions to consider:

  • What would AR afford that you can’t experience exclusively online or in person? What would it enhance? Create an experience that exists on a spectrum between being purely on the screen and purely in person.
  • How can AR make the experience enjoyable and informative?
  • In the space, how do people know where to go? What directs their path?
  • How can we take full advantage of what the HoloLens has to offer including visuals and sound?
  • What interactions can be designed to make users feel that they can be a student here?
  • What interactions with the physical world can be designed? Users can interact with the physical world, not just experience everything through the HoloLens screen. Students will engage with the school more authentically if they experience something and not just look at something.
  • What are key stories that need to be told about CMU? How can this be done in the space?
  • When users approach a stop, how do they know there is an interaction there? How do they know when to leave?
  • Why should this be created? What are we hoping to achieve?

Concept:

  • Users can choose what events that occur at the stadium they want to see, for example club and intramural sports, varsity sports like football and track and field, kiltie band, cheerleading, and commencement.
  • Looking at the scoreboard, there can be an icon that allows users to gesture and see historic games and sports history at CMU.
  • Collection of points throughout the experience to make it more fun.
  • UI to ask virtual tour guide verbal questions or given a set of FAQ’s.
  • One idea was letting visitors physically interact with the world by playing or learning how to play a sport, but we don’t think showing them how to play a sport would be an effective way to introduce users to the university and to the culture of CMU. An idea could be laying out objects and as they interact with the object, information about that activity appears.
  • Action Items: Think of a way to make users follow a specific path throughout the stadium, how to capture their attention at a hotspot, how to make the experience more fun through collection of points.

Storyboard of Concepts:

Sports History

Problem: Visitors can learn about CMU sports history online but they can’t imagine themselves experiencing it because they are not in person.

Solution: Being physically in the space of history and seeing information, visitors of the campus can imagine themselves partaking in sports culture at CMU.

As users approach the scoreboard, they can gesture and information about historic CMU games will appear.

Stadium Events

Problem: When visitors visit campus, the most activity they can see is what is currently happening ie. a sports practice or game.

Solution: Giving them options of what activities that are there and overlaying it on top of the field gives them a broader glimpse as to what the stadium is used for and the different activities CMU has to offer.

Variations of viewing different activities that occur at the stadium.
Physical interaction idea

Although these storyboards and concepts aren’t fully refined, it gives us a visual to better understand where we are and how we want to expand on these interactions.

One idea for physical interaction was dribbling a ball to collect points along side learning the information, however we feel as though it would not be very accessible and even though the demographic may be sports enthusiasts, the disconnect between naturally dribbling a ball physically is too drastic to dribbling it in virtual space so we may just be giving them a half ass experience because we aren’t able to give them an experience that they are naturally used to. To expand on the idea of points, we thought about placing digital icons that display how many points the user would earn as they walk through the space. We figured that this would solve the questions of how users know information is available at a certain hotspot, how they would know to move throughout the space, and also make the experience more memorable through the collection of points.

Points icons to show visitors that they collect points through learning information. As users get closer, the icon turns red and stands out more.
Initial Aero exploration

Peter and Mihika Feedback:

  • What is the information architecture? What information is presented to the user and at what time? Think about what order of information makes sense and what the user can do with this information.
  • Sport icons can be more informative and visually engaging.
  • Action items: Storyboard the interactions in more detail, take videos of space, prototype UI in Aero, physical, and AfterEffects.
Beginning of tour
Information architecture of activity viewing interaction

Interaction: Seeing different activities

Sketching out and thinking more about the interaction of seeing different activities on the stadium led us to rethink how users would interact with this idea. Our previous idea was to have text and 5 icons that appear and the user would choose one, however we did not think that it would be intuitive because they would have to stand at one spot and gesture through each activity if they wanted to view all of them, so we decided to have the 5 icons be placed in different spots throughout the field and have eye trackers and a gaze function that displays information and videos as the user looks at an icon.

Interaction: Scotty dog Q&A

Another idea that we expanded on was the ask feature. We recognize that not everyone is comfortable asking a tour guide questions about the school and also not all questions can be answered by a single tour guide, so our idea was to create a scotty dog avatar that can answer questions of all types and of most languages for visitors who are not fluent in English. The way that this would work is that the HoloLens would recognize when the user is asking a question and the scotty dog will appear in a corner and provide applicable answers to the question.

Interaction transition throughout the stadium

We realized that though the idea of the point system would be a good idea, calling it points may cause users to believe that they would get a reward at the end of the tour when that is not the case. Because we don’t really have a point system, we also have to figure out how to direct people from one point to another. A solution we came up with was having a status badge that shows what percentage of that part of the tour stop they are finished with. Even though this is not the same as collecting points, it still will incentivize people to finish the tour and also visually shows them what they have accomplished thus far. To direct people’s path from one part of the stadium to another, we decided that arrows would be an effective way to accomplish this, so every time the user is finished at one hotspot, an updated status badge will appear, a map of where they are will appear, and arrows that lead them to another hotspot will appear. Although this makes the tour somewhat less autonomous, it gives visitors a more guided tour so that they are not wandering aimlessly in the stadium trying to learn about CMU.

Prototype with Aero
Prototype in Illustrator

After creating all of the assets for the sketch video, our next step is to print them all out and recreate the video with the physical print outs to understand what gaps are missing and how to make the experience better.

Physical Prototypes:

Intro to Gesling Stadium prototype
User on bleachers football interaction
Scoreboard Interaction
Ask Scotty question Interaction

After physically prototyping our interactions, we got a better understanding of how the users will get the information to appear and what changes need to be made. We realized that the text for football information was too dense and that including more visual elements like icons will make the information easier to digest. By doing this, we also had to think about the path users would have to take and if information will already be there or if it would only appear after stepping on a hotspot. We decided that to make the experience less overwhelming and utilize the space more, information and icons will only appear when users are close to it.

Action steps: Redesign the information so that it is more digestible and visual, start placing arrows into the first few frames to show where users will be guided, finish making icons for the different activities, finish filming more scenes for the sketch video.

First Iteration of Graphics:

Information boxes
Icons for activities

After putting these graphics into Aero and physically prototyping, we saw that the information was too text heavy and could be more graphic to convey the information a lot faster.

Second Iteration of Graphics:

Activity Information Icon

When thinking about our interaction and graphics, we wanted to have a feature that allows users to gesture to expand the graphic and have more information and for them to somehow save this information to access again later. The activity information graphics that we created for the brief overview had too much stuff on it so we decided to have most of these elements on the second graphic for them to read if they decide to expand to get more info.

Final Graphics:

(Left) First graphic users will see, (Right) Expanded information if users want to have more info
(Left) First graphic users will see for track and field, (Right) Expanded information if users want to have more info
Final activity and attention icons

After creating these assets on Illustrator, our next step is to film the videos. One way we might approach this to make it easier is to import the graphics into Adobe Aero and film the videos on the app just to knock two birds out with one stone. We also think this is much easier than adjusting the perspective in Aftereffects.

Final Presentation

Final Prototype

Thien Self Reflection: How were the skills you developed in the first project similar and/or different from the second project? What is your understanding of the role of an Environments designer?

I think that in the first project, I was more limited in the space that I could design interactions for so it was a little easier than this project. However for both projects, I really had to think about how visitors would know that an interaction was occurring at a particular spot and how visuals could aid in that. For the last project because there were walls, I only designed interactions that would display on them or move along with the user so my interactions were more limiting but because this project was entirey in space, I had to think more about how to effectively convey the significance of the space and why an interaction would be used there. My view on environments designers is that we have to think about how users would interact with our designs in a space. Because our medium is the idea of space, that can exist anywhere in physical and digital forms which is exciting because it makes the work more accessible. Additionally, environments designers have the ability to use visuals and graphics to aid usability in spaces rather than have it existing purely for informational or aesthetic purposes. Moving forward, I hope to develop skills that will help me to design spaces that are physical, digital, or somewhere in between, to engage people and provide them with experiences that can change the way they interact with themselves, others, and the world.

--

--