Process

Through a research-driven approach, we aimed to evaluate and analyze CapMetro’s overall usability.

Heuristic Evaluation

We began with an audit of the CapMetro app. By conducting a heuristic evaluation, we were able to understand what challenges the app is currently facing and discover potential solutions.

Challanges

We identified several issues facing the app:

• Match Between System and Real World - the app fails to use familiar language that a user would expect to see when on a transportation app, especially when it comes to purchasing tickets/bus fare.

• Consistency and Standards - the layout of certain elements changes when viewing from different entry points.

• Aesthetic and Minimalist Design - Instead of focusing on the most important information, the app requires users to navigate through multiple steps to access essential or expected details. It prioritizes providing all available information over emphasizing what is truly necessary.

Recommendations

Similar tabs can be combined. The language and icons used on the app should meet users' expectations by introducing more familiar language.

• Use a single consistent design and layout for components on different tabs to ease users' cognitive load.

• Prioritize important information. Create a more minimalist interface that does not require users to do extra, unnecessary steps.

Competitive Analysis

We conducted a competitive analysis, comparing CapMetro’s app to both direct and indirect competitors to assess their features and user experience. Using our heuristic findings as a guide, we identified the strengths and weaknesses of each competitor. Some areas we focused on included:

• Convenience and availability of an interactive and visually engaging route map.

• Overall ease of use, including for product selection and checkout process.

• User friendliness and consistent design throughout the interfaces.

• Geographic availability for user convenience.

Research Design

Recruitment

Screened participants utilizing an 18-question survey. Participants' demographics included:

• Mix of age and gender.

• Living in the greater Austin area.

• Mix of those with the CapMetro app experience and those without.

Research Methods

Six moderated in-person usability tests.

• Users were asked to complete seven task scenarios on the CapMetro app.

• Applied a think-aloud protocol to gather users' thoughts, feelings, and opinions towards the app.

• Participants' actions using the app were screen-recorded on a smartphone. Audio and video recordings of the participant were also taken.

Interview Structure

The test was conducted over four sections:

• Four pre-test questions to understand the user's background.

• Seven usability tasks to evaluate the CapMetro app flow.

• Post-task questions after completion of each task.

• Eight Post-Test questions to evaluate users' overall thoughts of the app.

Task Flow

Analysis

We utilized the qualitative and quantitative data we collected to assess users' interactions with the CapMetro app:

• Task success rate (Success, Partial Success, Partial Failure, Failure)

• Time-on-task

• Post-task difficulty scale (measured after the completion of each task)

• Post-test Likert scale

Findings

Our evaluations led us to several findings.

• The task of finding a route to H-E-B and navigating to the bus stop had the two highest time-on-task.

• The task of navigating to the bus stop only had 2 out of 6 participants completely succeed.

• During the post-test survey, 4 out of 6 participants agreed they would not use the CapMetro app frequently, and were confused by the labels used throughout the app.

From this evaluation, we synthesized our data through affinity mapping and statistical analysis to find commonalities and to provide actionable and impactful recommendations. Additionally, we developed a user persona and mapped out the user journey to illustrate their experience during the usability test.

Insights

After thoroughly analyzing our findings, we uncovered key insights:

• There are multiple different ways to perform each task, often resulting in unnecessary clutter within the app and confusion regarding the specific pathways the user should take.

• Some essential functions feel “hidden” in the app and are only findable by expert users, such as finding the map view of a route and directions to get to the closest bus stop.

• The ticket-purchasing flow is lengthy and requires additional steps, and knowledge of ambiguous terminology in order to purchase the correct ticket.

• Labels were especially confusing for the participants, and sometimes were too similar that they offered no clear distinction between them.

Recommendations

These insights led us to our final recommendations:

• Create simple, clear pathways for the user to take to complete each major task within the app.

• Prioritizing the map view so users can access it quickly. In addition, the app can provide a quick tutorial when users first start exploring the app, so these features are highlighted to the user.

• We recommend simplifying the ticket-purchasing experience by having all prices and ticket options listed on one screen, and shown before needing to create an account.

• Consider combining similar functions to reduce confusion. Additionally, we recommend simplifying information and highlighting key features to ensure clear and easy navigation.