UX Research
Usability Evaluation for PAWS Montclair
Usability Evaluation for PAWS Montclair
A summative usability evaluation for a non-profit animal shelter to improve usability, accessibility, and drive engagement towards organization and user goals.
Team
Gabriel Castro (Me
Valerie Kolb
Alyssa Odeste
Michelle Pham
Role
UX Research
Skills
Usability Testing
Heuristic Evaluation
Cognitive Walkthrough
User Interviews
Timeline
10 weeks
Project Overview
About the client
PAWS Montclair (Pound Animal Welfare Society of Montclair) serves Montclair, NJ and its surrounding communities. The organization’s goal is to find animals appropriate forever homes and prevent the warehousing and/or euthanizing of adoptable animals.
Our objectives
- Understand the pain points of users of the current website
- Assess the usability and user experience of the website
- Prioritize recommendations so PAWS Montclair can address usability issues
Research questions
-What obstacles do users face when navigating and completing the adoption form?
-How do users feel about the adoption process in terms of time and fulfillment?
-How relevant to the process does the pre-adoption application feel for the user?
-How well does the site support the goals of the users?
-What is the overall satisfaction of users when using the product?
Our evaluation process
Heuristic Evaluation
Cognitive Walkthrough
Test Design
Testing Method
Task Scenarios
Findings
Recommendations
Takeaways
Preliminary research
To kickoff our usability evaluation, we started started with a heuristic evaluation and a cognitive walkthrough to quickly identify potential issues for the website.
Heuristic Evaluation
We started with a heuristic evaluation of the website–with the goal of quickly identifying usability issues based on violations against Nielsen’s 10 usability heuristics. We conducted an individual evaluation using Nielsen’s “UI inspection” method–which includes: identifying and documenting usability issues, outlining where they occur, identifying which heuristic was violated, and prescribing a rating of the severity of the issue. This was done on all pages of the website.
Early insights from our evaluation
We had 4 key findings that were consistent throughout the entirety of the website. We had organized them by findings, heuristic violated, and level of severity.
Typography, language, and information hierarchy was inconsistent through all pages
Heuristic Violated -
Consistency and standards
Severity -
3
Language does not yield to goal.
Heuristic Violated -
Matching between system and the real world
Severity -
3
System is not clear about what links will do.
Heuristic Violated -
User control and freedom
Severity -
3
Website content is wordy, lengthy, and cluttered with irrelevant or redundant information.
Heuristic Violated -
Aesthetic and minimal design
Severity -
3
Cognitive Walkthrough
In our next step, we conducted a cognitive walkthrough to find usability issues that users might face when trying to complete tasks on the website. Again we followed Nielsen Norman Group’s cognitive walkthrough method to evaluate this. This includes: walking through each step of a defined task flow and answering a set of defined analysis questions to identify aspects that could be challenging to a new user.
Defining primary user tasks
For this, we identified three primary tasks for the website: adopting, volunteering and donating and defined the corresponding happy paths for each task.
Early insights from our walkthrough
As for our findings, we found that users WOULD be able to complete each task, but not without failures at certain steps. Failures at certain steps potentially happened in steps where users would need to either seek information or navigate to different pages.
"New users will struggle with content discovery"
From the results of our preliminary research we found that, in general, it might be challenging for new users to successfully complete action steps where finding and learning website information is important. This calls back to our findings from our heuristic evaluation and gave us a direction for usability evaluation in our next steps.
Usability Testing
In order to test the usability of the website, we recruited 5 participants for testing. We conducted an online moderated usability test where the participants carried out 3 tasks scenarios requested by the moderator. These scenarios were comprised of specific action tasks on the website.
Test Design
We recruited 5 participants to undergo 1 usability test per session. The usability test sessions were administered remotely via Zoom, where participants screen’s were recorded to produce visual and audio descriptions.
Roles
We served as both recorders and moderators for our individual sessions of the usability test. Each researcher moderated and recorded tests for 1-2 participants each, totaling five participants.
Recruitment
We determined criteria of the potential primary users of the PAWS Montclair website including: prospective pet adopters, current pet owners, people who like pets, prospective volunteers, and prospective donators. We recruited 5 participants through the research team’s connections. We surveyed research participants to determine their pet ownership status, volunteer status, and animal shelter donation history.
Testing Method
Testing protocol
Before starting the usability test, we administered a pre-test questionnaire to verify eligibility from our screener and to help collect data for demographics.
During the testing sessions, we asked participants to “think out loud” and verbally walkthrough each step while they completed 3 task scenarios.
At the end of each test, we a brief verbal survey about the participant’s overall experience using the website for 5 subjective measurements:
- Ease of use
- Learnability – perceived time to complete tasks
- Perceived frustration
- Clarity of naming and labels
- Look and feel appeal
Task Scenarios
For the usability test, we identified 3 tasks that represent the core goals for new users of the website: starting the adoption process, volunteering for the shelter, and donating to the shelter. We gave instructions at each step. In addition, each task was broken into 3 parts to ensure that participants did need to recall instructions.
Data Analysis
As a team, we grouped our individual findings using a spreadsheet to find commonalities in our data, perform quantitative analyses where relevant, and identify themes.
Once all testing sessions were completed, we conducted a team debrief session to discuss our findings and observations from each testing session.
Tasks Results
The results from our usability test showed that the website was sufficiently usable and without any catastrophic failures. All of the participants successfully navigated the website with little to no errors. Additionally, all participants commented that the website was easy-to-use.
Insights from participants' experiences
“It is okay...I think it’s a lot of text, but I would just skim it...definitely not read the whole thing.”
Participants 3, 4, and 5 made comments about the lengthy paragraph style content featured on the website. One participant directly mentioned that they would not read the entire page, even if it was important.
“The adoption form could be labelled better. It's mixed in with other words.”
Three of five participants struggled to find links to important links such as the pre-adoption application form. Three of five participants commented how they were hard to see on pages with long passages.
“The website looks like its as old as 2005!”
Three of five participants made comments directly about the overall look and feel of the website. These participants pointed the different typefaces, backgrounds, and images on the website.
“It is difficult to interact with the current design system”
We found participant’s were able to use complete the primary tasks with minimal errors and most reported the website as easy-to-use. However, where most participant’s had a negative experience was in quickly finding information on pages because of the lack of a design system. In addition, many participants reported that the overall look and feel of the website felt outdated.
Synthesis
Our recommendations based on our findings from our usability test centered around 3 themes: improving visibility of important information for task completion, updating and standardizing a purposeful design system, and improving information architecture and navigation items.
Improving visibility of important information for task completion
Updating and standardizing a purposeful design system
Improving information architecture and navigation items
Center the homepage around user goals
Most participants first action was to look through the website’s homepage before navigating to other pages when completing a task. This shows that the current homepage does not support user and organization goals. Centering the homepage around adoptions, community involvement, and donations would improve the user experience by leading users to their goals.
Be concise with content
Participant’s were less likely to read through pages when met with lengthy blocks of text. Some participant’s mentioned feeling overwhelmed with the amount of text to read. Writing the website’s content with more purpose and intent will help users quickly read and understand important policies.
Use typography and hyperlink styles with intention
Typography
Participants struggled with finding information due to the lack of an organized type hierarchy. Participants commented about the website’s use of varying font families, styles, and sizes made it difficult to find important information. Standardizing a type hierarchy would help users find what they are looking for quicker and easier.
Hyperlinks
Participants struggled finding links to important adoption and volunteer forms due to the varying hyperlink styles. Participants commented about how hyperlinks varied in font styles and colors that made it difficult to recognize links. Standardizing hyperlink styles would make it easier for users to find these forms.
Further information architecture testing
The current website has different menu items that lead to the same page. Because of this, participant’s took varying paths to complete tasks. This could lead to potential issues for users navigating the website. We suggest testing the website’s information architecture to help create simpler pathways for users.
Takeaways
Some successes
The PAWS website was highly functional and rated as easy-to-use by most participants. Most users easily understood how to use the different components of the website. All web pages of the website functioned properly.
The “talk aloud” method and limitations
In our tests, we asked participant’s to talk out loud as they walked through each step to help us understand user intent. Because of this we weren’t able to accurately measure task completion time. Given that users mainly struggled with content discovery, a quantified measurement of task completion would help analyze issues in the website’s text content.
Focusing future tests on user satisfaction
The PAWS website was highly functional, without any catastrophic usability issues. Where most negative experiences occurred was in participant’s subjective experience. Future tests focusing on user satisfaction and sentiment could play a bigger role in improving usability. It would also open the opportunity for testing a future design system.