Usability Testing of a SaaS Application for Healthcare Systems

Sandeeb Adhikari

YWCC 307

April 2025
Abstract
This report presents an in-depth usability testing approach for a healthcare Software as a
Service (SaaS) application. The core objective is to ensure that the application provides an
intuitive, accessible, and efficient user experience for a wide range of users, including
patients, healthcare professionals, and administrative personnel. Given the critical nature of
healthcare services, usability issues can have serious implications, from delayed patient
care to compromised data integrity. Through task-based testing, persona-driven scenarios,
and accessibility evaluations, this project identifies usability challenges and proposes
recommendations for improvement. The methodology integrates both quantitative metrics
and qualitative feedback to ensure a user-centered design approach that promotes safety,
efficiency, and satisfaction.

Introduction
The rise of cloud-based healthcare solutions has transformed how medical services are
delivered, providing flexible access to critical tools like patient portals, appointment
scheduling systems, and telehealth platforms. However, for these systems to be effective,
they must be not only functional but also user-friendly and accessible to all user groups,
regardless of their technical expertise.

Usability testing serves as a key strategy to evaluate the effectiveness of software interfaces
by observing real users as they interact with the system. It highlights where users
encounter obstacles, what confuses them, and how efficiently they can complete essential
tasks.

In healthcare, these challenges carry even greater weight. Poor usability can lead to clinical
mistakes, user frustration, and system abandonment. This project focuses on evaluating the
usability of a healthcare SaaS application to uncover these pain points and provide
actionable recommendations for system improvement.

Research and Test Plan Summary

Target User Groups:

User Role Description

Patients Include elderly patients and those with limited
technical skills, accessing the system via mobile
 devices or tablets.
Healthcare Providers Physicians, nurses, and specialists using desktops and
tablets for daily clinical workflows.
Administrative Staff System administrators and healthcare IT professionals
managing system configurations and reporting tools.

Testing Goals:

 Assess the ease of use across various user roles.

 Evaluate the system’s compliance with accessibility standards (WCAG).
 Measure user satisfaction through validated instruments like the System Usability Scale
(SUS).
 Identify areas where user errors or inefficiencies may occur and suggest design
improvements.

Testing Methods:

Method Description Purpose

Moderated Usability Testing Facilitated sessions where Gather real-time feedback
users complete tasks with a and observe pain points.
moderator observing and
guiding as needed.
Unmoderated Usability Independent user sessions Scalable testing across
Testing without a moderator. broader audiences.
Remote Testing Conducted via online Cost-effective, convenient
platforms to reach data collection.
geographically dispersed
users.
A/B Testing Comparing two design Data-driven decisions on UI
versions for task success design.
and satisfaction.
Accessibility Evaluation Automated tools and Ensure inclusive design for
manual testing (screen users with disabilities.
readers, high-contrast
modes).

Sample Personas and Scenarios

Persona Name Role Key Pain Points
Characteristics
 Margaret Johnson Patient, 72 years old Limited tech skills, Struggles with small
uses mobile app fonts and complex
occasionally navigation.
Dr. James Chen Cardiologist, 45 Highly skilled, time- Needs fast access to
constrained, uses patient records,
desktop/tablet dislikes cluttered
interfaces.
Sarah Williams System IT professional, Seeks efficient
Administrator, 34 manages user dashboards and
permissions and clear error
system messages for
configurations troubleshooting.

Task Scenarios Table:

Scenario Purpose Expected Outcome

Book a follow-up Test appointment User books an appointment,
appointment scheduling usability. sets a reminder, and adds it
to their calendar.
View test results Assess information retrieval User successfully finds CBC
and interpretation. results and checks
hemoglobin levels.
Upload health documents Check document upload and User uploads and
sharing functionality. categorizes vaccination
record, shares it with the
provider.

Metrics for Measuring Usability

Metric Definition Purpose
Task Completion Rate Percentage of users Measures effectiveness.
successfully completing
tasks.
Time on Task Time taken by users to Assesses efficiency and ease
complete specific tasks. of use.
Error Rate Number of mistakes made Identifies usability
during task performance. problems.
Satisfaction Score (SUS) User-reported experience Evaluates subjective user
using surveys. satisfaction.

Mock Test Results Example

Task Completion Time on Task Error Rate (%) Satisfaction

 Rate (%) (seconds) Score (out of 5)
Login & 95 30 2 4.5
Authentication
Patient Record 88 45 10 4.0
Access
Appointment 92 60 8 4.2
Scheduling
Prescription 85 70 12 3.8
Ordering

Usability Testing Process Flow Diagram

Challenges and Limitations

 Diverse user roles and skill levels require adaptable design strategies.
 Regulatory constraints (like HIPAA) limit flexibility in user interface adjustments.
 Integration challenges with existing Electronic Health Record (EHR) systems.
 High cognitive load in clinical workflows increases the risk of errors if the system is not
optimized for efficiency.

Recommendations for Improvement

 Enhance User Training: Provide role-specific onboarding tutorials and in-app guidance
for different user groups.
 Optimize Integration: Strengthen compatibility with EHR systems via standardized APIs
to streamline workflows.
 Simplify Navigation and Workflows: Reduce cognitive load by minimizing steps
required to perform critical tasks.
 Improve Accessibility: Conduct regular accessibility testing, focusing on screen readers,
keyboard navigation, and high-contrast modes.
 Implement Continuous Usability Testing: Establish usability testing as an ongoing
process to adapt to user needs and regulatory changes.
Conclusion
Usability testing is not just a one-time activity but a continuous process that ensures
healthcare SaaS applications remain effective, safe, and user-friendly. By focusing on real-
world scenarios and user feedback, this project identifies key areas of improvement that
directly influence patient care quality, user satisfaction, and system adoption. A user-
centered approach combined with regular usability evaluations will drive long-term success
and compliance for healthcare SaaS solutions.

References
 ISO 9241-11. Ergonomics of human-system interaction — Guidelines for usability.
International Organization for Standardization, 1998.
 HIPAA Compliance Resources. U.S. Department of Health & Human Services.
 Usability.gov. Best practices and standards for usability testing methodologies. U.S.
General Services Administration.
 Healthcare IT News. Industry insights and updates on healthcare SaaS innovations and
challenges. HIMSS Media.
 Journal of Medical Internet Research. Research studies on the impact of usability on
patient care outcomes. JMIR Publications.