Summative usability is not a pass or fail event, it is proof you understood and controlled your device’s real risks
By Horacio M Pace-Bedetti, PhD, Principal Human Factors Engineer
Question this answers
Is medical device summative usability testing a pass or fail test?
Summative usability testing is not a pass or fail event based on error counts. Real users inevitably make mistakes, so success relies on analysing root causes, demonstrating that identified use risks are adequately controlled, and providing evidence that design decisions throughout development have minimised risk for the intended setting.
Summative usability is not a pass or fail event, it is proof you understood and controlled your device’s real risks.
Summative testing in medical devices often gets treated like an exam. Teams line up for that one big moment, hoping to count up the use errors and see if they “pass.” That thinking often backfires. You are not supposed to end up with a perfect, error-free study. In fact, if every user breezes through without a single hiccup, it usually means you missed something, because real users make real mistakes, even with good design.
The focus should be on what you do with those findings. If someone makes an error, what exactly happened? Was the interface unclear? Did the instructions create confusion? Did users rely on habits from a different device? Each event gives you a chance to look at the root cause and ask whether the risk is already handled, or if you need to rethink the design, instructions, or training. The total number alone means almost nothing.
I have seen cases where just a couple of use errors led to a red flag and others with more errors still moved forward. The difference comes down to how well the team understood the risks, how clearly they addressed them through design and iteration, and how convincingly they could show that the issues left were truly controlled for their actual user group.
Good summative studies are just the wrap-up to a much longer process. You should have identified your critical tasks and high-risk steps up front, tested and learned with real users through formative studies, and used all that feedback as early as possible. By the time you reach summative, you should be confirming a story that is already built, not searching for last-minute fixes.
Regulators do not just want a set of numbers. They want a clear path from identifying risks to making design decisions, supported by actual evidence. Summative usability is not about getting a perfect score, it is about showing that you understand how your device will really be used and that you have put in the work to make that as safe as possible for the intended setting.
Anyone else run into projects that get stuck at the summative stage because they skipped the early work?