Get user insights fast, early, and often so you can make data-informed product and design decisions.

Get user insights fast, early, and often so you can make data-informed product and design decisions.

Eduard Kuric discusses the significance and role of context in the creation of relevant follow-up questions for unmoderated usability testing, how an AI tasked with interactive follow-up should be validated for its capability to incorporate such context, and what the potential — along with the risks — of AI interaction in usability testing.

You're Already an Expert. It's Time to Get Paid for It. Think about how many websites and apps you use every single day. You scroll through social media, you order…

Make sure every usability test goes off without a hitch. We’ve built a robust checklist to ensure you’re prepared for everything prior, during, and after…

Flat design hides calls to action, and swiping around the edges can interfere with carousels and scrolling.

This article is based on Abid Virani and Alwar Pillai’s talk at A11yNYC. Alwar’s career started as a user experience (UX) designer. She attended the Masters of Inclusive Design program at OCAD…

Relying on WCAG isn't enough to ensure accessibility. Conducting usability testing with disabled users is the only way. And is required by regulations such as the European Accessibility Act.

Background: Immune checkpoint inhibitors are widely used in oncology but can cause immune-related adverse events (irAEs), which may be severe or life-threatening if not detected early. Electronic patient-reported outcome (ePRO) symptom monitoring systems may facilitate timely recognition and management of irAEs. #usability testing is a critical stage in ePRO system development, yet no published examples of formal #usability and acceptance testing exist. Objective: This study aims to assess the #usability and acceptance of a co-designed ePRO symptom monitoring prototype for irAEs embedded within the Epic electronic medical record. Methods: Testing was conducted at an Australian quaternary #Cancer center. Eligible participants were patients who had received or were receiving immune checkpoint inhibitors, their caregivers, or clinicians (oncologists and nurse specialists). Participants completed baseline digital literacy assessments (16-item Mobile Device Proficiency Questionnaire [MDPQ-16] and 12-item Computer Proficiency Questionnaire [CPQ-12]) before a structured testing session. Each session involved role-specific tasks using the patient-facing Health Hub or the clinician-facing Epic electronic medical record. #usability was assessed using the System #usability Scale (SUS). Acceptance was assessed using a customized Unified Theory of Acceptance and Use of Technology (UTAUT) questionnaire. Semistructured interviews were used to capture qualitative feedback. Results: A total of 30 participants (7 patients, 3 caregivers, 10 oncologists, and 10 nurse specialists) completed 10 testing sessions. Median MDPQ-16 and CPQ-12 scores were higher for clinicians compared to patients and caregivers. Median SUS scores indicated high #usability—patients and caregivers: 77.5% (IQR 70.0%‐86.3%), oncologists: 82.5% (IQR 80.0%‐90.0%), and nurse specialists: 80.0% (IQR 75.6%‐94.4%). Median UTAUT scores demonstrated strong user acceptance—patients or caregivers: 4.27 (IQR 4.09‐4.58), oncologists: 4.33 (IQR 4‐4.63), and nurse specialists: 4.23 (IQR 3.87‐4.57). Health Hub #usability themes highlighted overall ease of navigation and efficiency of reporting, but a need for clearer survey navigation, simplification of the actions page, and improved organization of trend graphs. For clinicians, themes included efficient side effect capture and intuitive system design, but a need to improve navigation to results, optimize data display, and facilitate team-based alert management. Health Hub acceptance themes highlighted patient empowerment to self-manage, enhanced patient-clinician communication, and reinforcement of existing care. However, concerns were raised about digital equity for vulnerable groups. Clinicians reported that the system streamlined side effect management between visits, aligned with existing Epic workflows, and could be tailored to personal preferences. Concerns remained regarding additional workload and medico-legal responsibilities associated with real-time alerts. Conclusions: The ePRO prototype demonstrated high levels of #usability and acceptance across patients, caregivers, and clinicians. Limitations around navigation and data visualization, alongside equity and workload concerns, will guide refinements prior to implementation. These findings emphasize the value of rigorous formative #usability and acceptance testing to optimize ePRO systems prior to deployment in routine #Cancer care.

Background: US veterans with mental health disorders have high rates of smoking and low rates of smoking cessation. Objective: This study aims to focus on an adaptation of a web-based acceptance and commitment therapy (ACT) tobacco cessation intervention (Vet WebQuit) for veterans with mental health disorders who use tobacco and used a qualitative approach to test its #usability (n=16). Methods: Participants were asked to walk through the site during laboratory-based #usability testing and “think aloud” about the features of the intervention. A trained facilitator used semistructured interview questions to assess participants’ experiences with Vet WebQuit and obtain feedback on their impressions of the site. Qualitative analyses identified themes regarding participants’ experiences with the intervention, #usability concerns, and recommendations for improving Vet WebQuit. Results: Overall, veterans found that the Vet WebQuit layout was simple and easy to navigate and use. Veterans reported that several features of the program were useful, including the quit plan, identification of triggers, content that targets mental health concerns (eg, dealing with anger), information on the health effects of smoking, tools for managing triggers (eg, urge surfing), and involving others in their quit plan. Veterans reported that particular features of the ACT approach for tobacco cessation were appealing to them, including the distinction between internal and external smoking triggers, the inclusion of the serenity prayer, and mindfulness exercises, which they could use as a tool reduce the intensity of cravings. Veterans reported wanting more information on the health aspects of smoking (ie, effects on breathing and lung capacity) as a way to motivate them to quit smoking. In addition, they suggested targeting specific mental health concerns that serve as triggers for smoking, including nightmares, boredom, and social isolation. Conclusions: Overall, results from this project identified important elements of ACT digital tobacco cessation interventions for veterans with mental health disorders.

Background: Digital short cognitive tests administered by medical assistants (MAs) in general practitioners’ (GPs) practices have great potential for the timely identification of patients with dementia, because they can lead to targeted specialist referrals or to immediate reassurance of patients regarding their perceived concerns. However, integration of this testing approach into clinical practice requires good #usability for the test itself, especially for cognitively impaired older adults. Objective: In this implementation study, the digital version of the Montreal Cognitive Assessment (MoCA) Duo was conducted by MAs in general practice. We tested if the interaction with the test is associated with #usability problems for the patients and aimed to find additional relevant constructs that should be considered for the potential implementation of such digital tests into clinical practice. We focused the study on subjective success, #usability, and workload as well as their association with the result of the cognitive test to assess whether the MoCA Duo can be implemented into general practice. Methods: In total, 10 GPs took part in the study. Within their practices, 299 GP patients (aged 51-97 years) with cognitive concerns completed the MoCA Duo administered by MAs. Subsequently, patients and MAs completed digital questionnaires regarding the interaction with the app. #usability was measured using the adapted System #usability Scale, and perceived workload using the National Aeronautics and Space Administration Task Load Index. For the perceived workload, we included an assessment of the patient by the MA. Results of the MoCA Duo were supplied to the GPs for their consultation with the patient. Results: The results indicated good #usability for the MoCA Duo. Self-assessment by the patients indicated that 64% (191/299) could perform in the test to the best of their ability, affected by their MoCA score. We found significant higher #usability ratings by patients with better MoCA scores as well as by younger patients. Furthermore, the perceived workload showed overall medium workload. We found significant correlations between the subjective perceived workload of the patients and the assessment by MAs. Self-assessments as well as assessments by the MAs were significantly influenced by the MoCA scores and the age of the participants. Conclusions: The results indicate good #usability of the digital MoCA within the sample, supporting the idea that the resulting scores are adequate to assess cognitive status without dependence on technological affinity. Furthermore, the results highlight the relevance of heterogenous samples for comparable evaluation studies, based on the significant effect of cognitive status and age on #usability and workload.

Background: Canada’s legalization of recreational cannabis use (CU) has further highlighted the need for innovative interventions that promote lower-risk CU. Young adults aged 18-25 years represent the age group with the highest prevalence of CU. Protective behavioral strategies (PBSs) have been shown to help manage CU and reduce its negative consequences. To date, only a few interventions have focused on PBSs. To address this gap, a mobile app prototype using PBSs to influence CU was developed with and for young adults. Objective: This study aims to describe the development process and #usability testing of Joint Effort, a CU self-management mobile app prototype centered on promoting the use of PBSs among young adults with any past 30-day CU. Methods: Intervention mapping (IM) and a co-design approach were used. Six steps were followed: (1) focus groups were conducted to identify needs and preferences regarding CU interventions; (2) a matrix of change objectives was used to select target behaviors and determinants; (3) theory-based intervention methods and practical applications were selected; (4) focus groups were held to validate the intervention structure and examples of tailored messages; (5) preliminary intervention content was created; and (6) the intervention content was transposed into a mobile app prototype. #usability was assessed through qualitative semistructured interviews and the User Version of the Mobile Application Rating Scale (uMARS), completed by a sample of 20 university students with a mean age of 21.8 (median 22) years, 14 (70%) of whom were women and 15 (75%) were undergraduates. Qualitative data were analyzed using thematic analysis. Results: Four themes were identified from the interviews: Joint Effort was visually pleasing and easy to use; the content was well-adapted to the target audience and nonjudgmental; customization functions were appreciated; and the app was perceived as helpful and relevant for initiating behavior change. The prototype received a mean quality score of 4.43/5.0 (SD 0.53) per item on the uMARS. The mean scores on the 5 subscales were as follows: engagement (4.14, SD 0.53), functionality (4.60, SD 0.47), aesthetics (4.53, SD 0.52), information quality (4.44, SD 0.61), and subjective quality (3.36, SD 0.53). Conclusions: Our findings highlight the added value of IM and a co-design approach, underscoring the importance of incorporating user feedback in the development of mobile apps. Building on the strong #usability results, the Joint Effort prototype has since been developed into an iOS mobile app, and larger-scale evaluations are currently underway to assess its acceptability, #feasibility, and efficacy.

Background: Game-based cognitive assessments (GBCAs) have the potential to transform the field of cognitive testing by enabling more effective screening of age-related cognitive decline. However, we lack a strong understanding of the usability and overall user experience of these games. This is a risk because the primary target users for GBCAs, older people, are seldom involved in game design research and development. Objective: This study aims to address this gap by investigating the usability, acceptability, and enjoyability of GBCAs for older people. Methods: This study followed established practices for undertaking evidence-based systematic reviews. Results: The initial database search returned 15,232 records. After a thorough screening process, 8 studies remained for extraction and analysis. A synthesis of the included papers identified 2 overlapping yet distinct areas of focus: system usability and subjective user experience. Usability scores were mostly positive across the studies included. However, in several of the game studies, older adults and those with cognitive impairment tended to find GBCAs less usable. This trend was observed even when the games were explicitly designed for these populations, and the tasks were simplistic and representative of basic daily activities. In our second focus area, user experience, we identified the importance of perceived challenge in mediating gameplay experience across groups. That is, generating the appropriate level of difficulty for each user is important for positive user experiences, specifically enjoyment. Conclusions: On the basis of these findings, we identified key learnings for researchers interested in designing and developing GBCAs. These include (1) recognizing that validity is essential but not sufficient on its own; (2) clearly defining the intended user; (3) designing games that align with the unique preferences and needs of older people; and (4), whenever possible, providing each user with their optimal level of challenge. Clinical Trial: PROSPERO CRD42023433298; https://www.crd.york.ac.uk/PROSPERO/view/CRD42023433298

Background: Clinical trial success depends on efficient participant recruitment and retention, yet low enrollment and high attrition remain persistent challenges. To address these issues, we developed RecruitGPS, a scalable dashboard that provides…