Interactive neighborhood journeys with On the Streets
for a personalized, gamified touring experience
Team
with Vanshika Shah and Ruhee Shah
Time
4 weeks
Role
Design Research, Conceptual Design, 2D/3D Prototyping, Video Editing
Tools
WebAR, GitHub, Figma, Blender, Figma, VS Code, Adobe Premiere Pro
The Opportunity
During a recent neighborhood walking tour in New York City, I noticed that I along and several other attendees were unable to hear the guide and keep up with the rest of the group while getting absorbed in learning more about the history.
​
I saw this as an opportunity to create a more interactive and flexible experience using Augmented Reality (AR). Users can experience an interactive and immersive tour of the neighborhood, where they can get information about the neighborhood's history, architecture, and local businesses, making it an excellent tool for tourism and community engagement - something that the diverse city of New York thrives on.
The Process
03
Validate
-
User Testing
-
High Fidelity Prototype
02
Design
-
Task/User Flows
-
Style & UI Design
-
Mid-Fidelity Prototype
01
Research
-
Competitive Analysis
-
Secondary Research
Project Objectives
How might we design an interactive, personalized neighborhood walking experience to increase the connection between users and their surroundings?
Experiential Learning
By providing interactive content, visitors are more likely to remember the information presented in the AR tour leading to a more meaningful and educational experience that allows them to feel more included and engaged.
Gamification
By allowing users to share their achievements and progress on social media or connect with other users, the AR tour can foster a sense of belonging and encourage visitors to explore the neighborhood together.
The Outcome
On the Streets is an augmented reality experience using a combination of smart glasses and mobile phones that aims to encourage curious minds to explore neighborhoods and culture through experiential learning, gamification and the power of participation.
Demo Video
Features
Problem
HMW give our users the best chance to follow along?
The main issues we saw in the current tour experience is that the users can often not follow the guide due to two reasons:
-
inability to hear the guide or see where they're pointing due to environmental factors
-
they find it difficult to catch up with the guide after spending more time at certain places
01
Onboarding and neighborhood selection
We explored how to solve this problem by splitting up the main user flow into two:
​
-
with a tour guide: where the app is used as an aid for guides to visually indicate spots with the help of AR elements
​
-
by themselves: for a self-paced, geotagged succession of AR elements allowing the user to spend time as they wish
​
We also provided the affordance to select a neighborhood they would like to begin the tour from, and guided through the gamification process - where and how to win points.
Problem
HMW make sure our users are engaged but not unaware?
Once we split the user flow into two types, we further explored the how we could better the feasibility of the application without the hassle of holding a phone out for long time periods, and prevent safety hazard like users not paying attention to environmental factors around them.
02
Multi-modal experience
To introduce a safer way and clearer range of vision for the users, we opted to allow for a multi modal experience where the user can select to continue the viewing experiencing of the tour through their phone - or migrate to a cross-platform experience, including smart glasses.
​
Here we integrated design considerations like:
​
1. accessibility through audio input and haptic output
2. safety features like warning symbols to help users distinguish between reality and AR during navigation
Problem
HMW create a seamless narrative through AR?
Once we created the user flow and narrative for the touring experience, we then explored the best way to introduce the AR elements at each location to the users with natural succession and least cognitive load.
03
Geo-tagged AR elements
To create a seamless experience for the users, we explored the option of using LIDAR scanning where we are able to map and pin the exact location for AR elements using geo-tagging.
This way, when the user enters a specific landmark's geofence, the application will then identify the building through the real-time camera feed and trigger the relevant AR elements and content - including illustrations, 3D objects, and facts.
Problem
HMW create an engaging experience throughout the tour?
Though the interactive elements and content delivered to the user during the tour would be engaging, we found that that wasn't enough to keep them motivated to complete the entire tour nor revisit the application. We then began to think of an incentivized feature to help foster a sense of belonging for the users.
04
Gamification and interactive elements
To keep the users engaged, as they navigate through the tour, they are prompted at specific locations to 'complete tasks' and win more points.
​
These interactive AR elements thereby keep the user engaged throughout the tour process - rather than the show-and-tell process that most generic tours tend to be.
​
At the end of the tour, the user is then given the ability to use the points they won during the tour to go to the nearest coffee shop for a well-deserved drink - keeping them engaged till the very end.
Prototyping
Keeping all the solutions in mind, we created a user flow diagram to direct our prototyping. For this project, we stuck to prototyping the self-guided tour and not the tour guide aid as we believed this would be more valuable for the users.
Using a mix of Figma, WebAR, and video editing, we were able to create a prototype that we could further test through a role-play user testing method.
Takeaways
Though the application was relatively simple, we had to overcome many challenges just to create a demo - something that I had not anticipated when I began this project. Throughout the journey of creating the app we learnt more and more about how difficult and yet experimental designing for emerging technologies really is, where everyone is still 'figuring it out' and the best way to show value is to showcase it in any way possible.
We had to come up with some really scrappy ways to prototype and test our flow and yet that worked absolutely perfectly. If anything, what I learnt from this project is that sometimes the method doesn't matter as long as you think creatively enough.
​
For the next steps in this project, I'd like to give more thought to these couple of things:
​
-
Accessibility: XR gives us the opportunity to engage different senses, not only sight. This could be an interesting exploration of the guiding experience with sound, smell and touch!
-
Feasibility: I'd love to work with developers to really understand the feasibility of building this model and how this project could be made into smaller digestible parts in a product management setting.
-
Integration of AI image processing: LIDAR scanning takes a long time and scanning entire neighborhoods seems a bit of a stretch; Using AI for auto recognition of buildings and using that as a trigger for AR seems more feasibile.