Sridarshini

Usability Evaluation

Discover usability issues with a social-networking app prior to product launch

  • Client: POP, a startup
  • Timeframe: 12 weeks
  • My Role: UX Researcher
  • Team: 3 Researchers
  • Methods: Usability Testing, Prototyping, Heuristic Evaluation, Competitive Analysis, Subject-Matter-Expert Interview
  • Tools: Figma, Miro, Zoom

Project Overview

Client Kickoff

The POP app facilitates friendships based on shared interests for University students. POP’s long-term goal was to improve the mental health of university students by providing meaningful social connections.

Objectives

  • Understand the pain points of students and their social network at college
  • Assess the usability and user experience of the POP app
  • Prioritize recommendations so POP can address users’ deepest pain points

Notes

At the time of our assessment, POP had a working prototype in Figma where we conducted our heuristic evaluation as a team. Shortly after, POP released their app on TestFlight

Methodology

Heuristic Evaluation

We evaluated two different “sections” of POP’s app:

  1. The onboarding process that every user will go through one time to create their account
  2. The matchmaking and chatting process that users will go through repeatedly as they interact with the app

Our team went through the prototype together and took notes using Airtable so we could tag notes by the possible 10 heuristics violated and mark the level of severity.

Heuristic evaluation findings

The most heuristic violations were:

  • Visibility of system status (29 issues)
  • Consistency and standards (18 issues)
  • User control and freedom (18 issues).

1. Keep users informed

POP should provide feedback to clear up issues and help users know they’re on the right track.

POP should have very clear indicators around their most important tasks (getting a match and messaging a match) otherwise users may stay confused and abandon the app.

  • We recommend POP add features like a progress bar and confirmation feedback to keep users aware of what’s going on.
  • Why? Humans like to be in the know. Predictable actions and providing confirmation can help build trust between the user and an app.
  • Heuristic referenced: Visibility of system status

2. Give users control

POP should provide users the option to quickly fix mistakes or make changes later.

POP should allow users to change their mind (un-matching with someone) at any time of a user’s interaction.

  • We also recommend POP allow users to edit as much of their profile as possible, including avatars and codenames.
  • Why? Humans make mistakes or change their minds. When an app doesn’t allow changes, users can feel trapped and dissatisfied.
  • Heuristic referenced: User control and freedom

3. Be consistent

Especially as users are learning a new app, it’s important to be consistent so users know what to expect and feel safe to explore.

POP should follow industry conventions with match-making as many of its target users are familiar with apps like Tinder and Bumble.

  • We recommend POP use a similar information hierarchy on profiles and only allow messaging after matching.
  • Why? Humans prefer not to work. When an app doesn’t follow what users expect, it can be hard to learn or a barrier to use.
  • Heuristic referenced: Consistency and standards

Competitive Analysis

Our team looked at typical matchmaking apps like Bumble. At the time of the study, they were the only dating app offering platonic connections.

Competitive analysis findings

The competitive analysis showed POP had a unique proposition but a very similar idea was previously tried and saw limited success. Research on the desirability of the app and assessing product-market fit should be major priorities.

1. Exclusive membership

POP stands apart in its membership being exclusive to university students

  • Both direct and indirect competitors had an open community approach
  • Nextdoor has seen success by keeping things exclusive based on neighborhood and POP could see similar benefits

2. Anonymous profiles

Anonymity was the biggest differentiator across all competitors but may bring novel issues

  • Anonymity increases the cognitive load users will need for remembering information about matches
  • Anonymity has effects on people’s behaviors and should look into especially around how individuals will be treated online and safety for offline meeting

User Interviews

Methodology

  • Screener – We selected a diverse group of individuals that varied in their age (freshmen-seniors), their residency (in & out – of state, international students), and transfer students
  • Remote Moderation – The interviews had to be conducted online using Zoom and Figma prototype

The Process

  • We chose six tasks for users to do during the usability test
  • Since we were a team of four, we all covered four sessions, two moderating and two sessions of note-taking
  • We created an interview script, a deck for users to reference tasks and likert scale choices, and a codebook for the notetaker

Analysis – Affinity Mapping

  • We grouped things by which task/screen it was associated with. I made sure each sticky referenced the task and participant id before importing them to miro
  • Within each task, we grouped things by their impact: positive, negative, and neutral.
  • After discovering themes across multiple interviews, we worked together to synthesize the data and agree on insights. From these insights, we created our first suggestions that would later become more polished recommendations for the POP team.

Findings

  • POP was a fairly usable app since it was pretty similar to matchmaking apps. Since POP’s target user was young university students, most were pretty advanced tech users.
  • Users struggled most with new features that were different from other matchmaking app conventions.
  • Usability and user interface issues were minor inconveniences for users. Users struggled with the purpose of the app.

Recommendations

Overall project findings

  • POP was a fairly usable app
  • Some novel issues may come up with features like anonymous profiles

1. Conduct further research on the desirability of POP

  • Direct further research on desirability over usability – a perfectly usable app does no good if users don’t want to use it
  • Focus on pain points of users to assess product-market fit
  • Explore further revenue stream – the liability universities would need to take on may keep them from partnering officially with POP

2. Design for the effects of anonymity

  • Allow users to create meaning and uniqueness in their profiles to encourage an emotional connection to the app
  • Lower the effort needed to make and keep connections with other users
  • Reduce the users’ cognitive load associated with anonymous profiles to keep them engaged

3. Follow industry standards

  • Provide confirmation screens for user actions to reduce confusion
  • Avoid information overload by reducing content displayed to users
  • Refer to conventional patterns of feature placement to enhance learnability

4. Include a comprehensive onboarding experience

  • Comprehensive onboarding should clarify issues users had with anonymity and privacy
  • Highlight differentiators to increase user engagement and desirability of POP

Reflection

Industry Takeaways

POP was a fairly usable app so our recommendations to improve the user experience felt nonessential. The problems we discovered during our user interviews were rooted in problems with the desirability of the app and not understanding users’ real pain points.

This project demonstrated the limitations of evaluative research when generative research was not done to assess the initial product-market fit.

Sridarshini