An extended protocol for usability validation of medical devices: Research design and reference model

An extended protocol for usability validation of medical devices: Research design and reference model

Schmettow, M.
Schnittker, R.
Schraagen, J.M.

2017

This paper proposes and demonstrates an extended protocol for usability validation testing of medical devices. A review of currently used methods for the usability evaluation of medical devices revealed two main shortcomings. Firstly, the lack of methods to closely trace the interaction sequences and derive performance measures. Secondly, a prevailing focus on cross-sectional validation studies, ignoring the issues of learnability and training. The U.S. Federal Drug and Food Administration's recent proposal for a validation testing protocol for medical devices is then extended to address these shortcomings: (1) a novel process measure ‘normative path deviations’ is introduced that is useful for both quantitative and qualitative usability studies and (2) a longitudinal, completely within-subject study design is presented that assesses learnability, training effects and allows analysis of diversity of users. A reference regression model is introduced to analyze data from this and similar studies, drawing upon generalized linear mixed-effects models and a Bayesian estimation approach. The extended protocol is implemented and demonstrated in a study comparing a novel syringe infusion pump prototype to an existing design with a sample of 25 healthcare professionals. Strong performance differences between designs were observed with a variety of usability measures, as well as varying training-on-the-job effects. We discuss our findings with regard to validation testing guidelines, reflect on the extensions and discuss the perspectives they add to the validation process. © 2017 Elsevier Inc.

Human & Operational Modelling
HOI - Human Behaviour & Organisational Innovations
ELSS - Earth, Life and Social Sciences
Generalized linear mixed-effects models
Human Factors
Longitudinal research
Medical device design
Patient safety
Usability testing
Bayesian networks
Graphical user interfaces
Human engineering
Regression analysis
Safety testing
Usability engineering
Generalized linear mixed-effects models
Longitudinal research
Medical device design
Patient safety
Usability testing
Biomedical equipment

http://resolver.tudelft.nl/uuid:fdb572aa-84b4-4ea8-9606-7eead7d1aa8d

https://doi.org/10.1016/j.jbi.2017.03.010

756691

1532-0464

Journal of Biomedical Informatics, 69, 99-114

article