Relationship Between Efficiency, Effectiveness, and Learnability of Home Connected Medical Device in Ambulatory Surgery

Background:Currently, usability assessments of home connected medical devices do not systematically take into account learnability metrics. In case of the Smart Angel device—designed for monitoring ambulatory surgery patients—users are trained at the hospital and have to use the device at home to monitor their health remotely.

Objective:The aim of this study was to better understand the relationships between two metrics of usability—efficiency and effectiveness—and learnability of the Smart Angel device.

Materials and Methods:Twenty-eight participants were trained in a simulated hospital (SimUsanté), and then we filmed the participant using the device three times. Between each session, the participant had to complete questionnaires (sociodemographic and health literacy).

Results:The results of a between-subject analysis [χ²(2) = 18.969, p < 0.001] and a within-subject analysis [F(2.28) = 13.34, p < 0.001, η² = 0.35] showed that efficiency (manipulation time) significantly improved with learnability (number of sessions). Conversely, effectiveness (number of manipulation errors) stagnated over the three sessions with a between-subject analysis [F(2.75) = 1.628, p = 0.203], while the within-subject analysis revealed that users seemed to significantly correct their errors with the number of sessions [F(2.28) = 6, p = 0.005, η² = 0.19]. By analyzing the errors, we observed that some errors could appear at any time (e.g., moving during the measurement) and others were systematic (e.g., the wrist blood pressure on the right arm).

Conclusions:While the “first attempt” at using a home medical device is a major revealing step for the study of manipulation errors, learnability is an equally useful metric to include in usability studies, as well as in the very definition itself of usability.