Title
With how many users should you test a medical infusion pump? Sampling strategies for usability tests on high-risk systems
Author
Schmettow, M.
Vos, W.
Schraagen, J.M.
Publication year
2013
Abstract
Usability testing is recognized as an effective means to improve the usability of medical devices and prevent harm for patients and users. Effectiveness of problem discovery in usability testing strongly depends on size and representativeness of the sample. We introduce the late control strategy, which is to continuously monitor effectiveness of a study towards a preset target.A statistical model, the LNBzt model, is presented, supporting the late control strategy. We report on a case study, where a prototype medical infusion pump underwent a usability test with 34 users. On the data obtained in this study, the LNBzt model is evaluated and compared against earlier prediction models.The LNBzt model fits the data much better than previously suggested approaches and improves prediction. We measure the effectiveness of problem identification, and observe that it is lower than is suggested by much of the literature. Larger sample sizes seem to be in order. In addition, the testing process showed high levels of uncertainty and volatility at small to moderate sample sizes, partly due to users' individual differences. In reaction, we propose the idiosyncrasy score as a means to obtain representative samples. Statistical programs are provided to assist practitioners and researchers in applying the late control strategy. © 2013 Elsevier Inc.
Subject
Human
HOI - Human Behaviour & Organisational Innovations
BSS - Behavioural and Societal Sciences
Biomedical Innovation
Biology
Healthy Living
Ergonomics
Infusion pump
Patient safety
Sample size
Usability
User testing
Infusion pump
Patient safety
Sample sizes
Usability
User testing
Biomedical equipment
Safety testing
Sampling
Usability engineering
awareness
food and drug administration
geometry
hazard ratio
health care quality
intensive care unit
medical devices
patient monitoring
patient safety
problem identification
quality control
statistical model
To reference this document use:
http://resolver.tudelft.nl/uuid:6a952c4e-d22f-46ce-afa0-472577092401
DOI
https://doi.org/10.1016/j.jbi.2013.04.007
TNO identifier
477628
ISSN
1532-0464
Source
Journal of Biomedical Informatics, 46 (4), 626-641
Bibliographical note
Tradenames: Arsena Alaris infusion pump; Braun infusion pump
Document type
article