Browsed by
Tag: Usability Testing

STAC-T Usability Testing Results

STAC-T Usability Testing Results

A research team from Boise State University and Klein Buendel have published a paper in JMIR Human Factors entitled, “Usability Testing of a Bystander Bullying Intervention App (STAC-T) for Rural Middle Schools: A Mixed-Methods Study.” The paper reports on the conduct and outcomes of the usability testing of a bystander bullying intervention app for rural middle school students.

Students who are targets of bullying are at high risk for negative mental health outcomes including depression, anxiety, and suicidal ideation. Implementing school-based bullying prevention programs reduces bullying. Bullying prevention programs are particularly important in rural schools, as bullying is more prevalent in rural schools compared to urban schools. Comprehensive, school-wide bullying prevention programs, however, require resources that create significant barriers to implementation for rural schools. Because technology-based programs can reduce some of these implementation barriers, the development of a technology-based program to address bullying increases access to bullying prevention for students in rural settings.

This study assessed the usability and acceptability of the STAC-T app and differences in usability between school personnel and students. Qualitative feedback related to usability, program features, and feasibility was also obtained and analyzed. A sample of 21 participants (10 school personnel and 11 students) recruited from two middle schools in rural, low-income communities in two states completed usability testing followed by a qualitative interview. We used descriptive statistics and independent sample t-tests to assess usability and program satisfaction. We used consensual qualitative research (CQR) as a framework to extract themes related to usefulness, relevance, needs, barriers, and feedback for intervention development.

Usability testing indicated the app was easy to use, acceptable, and feasible. Both school personnel and students rated the app well above the standard cutoff score for above-average usability (68.0) and both school personnel and students gave the app high user-friendliness ratings (0-7 scale, with 7 high user-friendliness). Overall ratings also suggested school personnel and students were satisfied with the program. Among school personnel, 100% said they would recommend the program to others and 10%, 50%, and 40% rated the program as 3, 4, and 5 stars, respectively. Among students, 90.9% said they would recommend the program to others and 27.3% and 72.7% rated the program as 4 stars and 5 stars, respectively. There were no statistically significant differences in ratings between school personnel and students.

Qualitative data revealed that both school personnel and students found the STAC-T app useful, relevant, and appropriate, while providing feedback related to the importance of narration of the text and the need for teacher and parent trainings to accompany the student program. Data also showed school personnel and students would find a tracker useful, in which students could report the different types of bullying they witnessed and strategies they used to intervene. Data from school personnel also indicated the program was perceived as practical and very likely to be adopted by schools, with time, cost, and accessibility being noted as potential barriers for schools in rural communities. The results demonstrate high usability and acceptability of the STAC-T app and provide support for implementing a full-scale randomized controlled trial to test the efficacy of STAC-T.

This research is funded by an STTR Phase II grant to Klein Buendel from the National Institute on Minority Health and Health Disparities at the National Institutes of Health (MD014943; Dr. Aida Midgett, Principal Investigator, Boise State University). Co-authors include Dr. Diana Doumas, Dr. Claudia Peralta, and Dr. Blaine Reilly from Boise State University; Dr. Matthew Peck from the University of Arkansas; and Ms. Mary Buller from Klein Buendel. The STAC-T modules were graphically designed and programmed by Klein Buendel’s Creative Team.

E-learning modules for peer end-of-life caregivers living in prison

E-learning modules for peer end-of-life caregivers living in prison

A research team led by Dr. Susan Loeb from Penn State University and Dr. Barbara Walkosz from Klein Buendel, has published results from a recent study in the Journal of Offender Rehabilitation. The paper reports findings from the usability testing of the Just Care electronic learning program for training carefully vetted people living in prison to assist staff with geriatric and end-of-life care. 

Aging is accompanied by an increased probability of health problems and subsequent need for health care. According to the authors, “The growing number of older adults living in prison, who will likely age and die in prison, calls attention to the challenge of how correctional facilities will care for this population’s health needs, which includes the increasing demand for geriatric and end-of-life care.” Training men and women who live in prison to assist corrections staff with the care of growing numbers of older, sicker, cognitively impaired, and dying people in prison is an innovative and viable option to help meet pressing care demands. 

Two rounds of usability testing were conducted via videoconference. Prison personnel (such as corrections specialists, supervisors, nurses, and chaplains) and people living in prison were recruited from one men’s prison and one women’s prison. Fifteen participants completed the testing in each of two rounds. Methods and analyses are described in the paper.

Results include participant demographics, concerns and areas for improvement by severity level of programming, and mean scores for both participant groups on the System Usability Scale.  Overall, both the incarcerated and staff participants found the program easy to navigate and the interactive content was useful, engaging, and relevant to caring for incarcerated aging and dying people. The findings helped refine Just Care before additional evaluation was conducted with a larger sample.

This research was funded by an STTR grant to Klein Buendel from the National Institute on Aging (AG057239; Dr. Susan Loeb and Dr. Barbara Walkosz, Multiple Principal Investigators). The lead author on this Journal of Offender Rehabilitation paper is Dr. Erin Kitt-Lewis from Penn State University. Additional authors are Dr. Susan Loeb and Mr. Sherif Olanrewaju from Penn State University; and Dr. Barbara Walkosz, Mr. Brandon Herbeck, and Mr. Steve Fullmer from Klein Buendel. 

Usability Testing of Just Care Modules in Prisons

Usability Testing of Just Care Modules in Prisons

Prison population demographics are shifting such that many people will grow old and spend their final days in prison. Research evidence supports using peer caregivers to assist prison staff with providing supportive care. However, the training received by peer caregivers varies widely. Evidenced-based, accessible, and contextually relevant learning materials are needed to effectively prepare peer caregivers for their role.

A research team from The Penn State University Ross and Carol Nese College of Nursing and Klein Buendel have developed Just Care, a six-module e-learning program designed to augment the face-to-face training that is typically provided to peer caregivers. Small-scale usability testing of the Just Care program was conducted with twenty men and women, who are incarcerated, and ten staff members at one men’s and one women’s state prison in a southeastern state. Two rounds of usability tests were conducted via a video-conferencing platform. The results of the usability tests were recently presented at the annual conference of Academic Consortium on Criminal Justice Health (ACCJH) in Raleigh, North Carolina on April 13-14, 2023.

Most users, who were incarcerated, did activities without aid and no tasks or programming issues were identified that made it impossible to use the application as intended. However, a few usability and content issues were noted including: (a) some users did find the x-ray scanner activity challenging when trying to drag the scanner across the body image to reveal symptoms as death approaches; and (b) a few staff users noted that the returning citizens’ video testimonials in the Staff Module lacked racial diversity.​

The System Usability (SUS) scores for each round respectively were 87.5 (incarcerated), 74.5 (staff); and 85.28 (incarcerated), 83.75 (staff). A SUS score of 68 is considered above average. Overall, both participant groups found Just Care easy to navigate with content that is interactive, useful, engaging, and relevant. Prison staff noted that Just Care raised awareness about peer caregivers’ need for training to help care for older people in prisons.

Next steps for the research team is to address the issues raised in the usability tests and prepare for large-scale usability testing to be conducted in state prisons in the summer and fall of 2023. This research was funded by an STTR grant to Klein Buendel from the National Institute on Aging [AG057239; Dr. Susan Loeb (Penn State) and Dr. Barbara Walkosz (Klein Buendel), Multiple Principal Investigators]. Collaborators on the ACCJH poster presentation also included Dr. Erin Kitt-Lewis, Mr. Sherif Olanrewaju, and Ms. Katherine Aiken from The Penn State University; and Ms. Amanda Brice and Mr. Steve Fullmer from Klein Buendel.