,

Dress Rehearsal vs. Opening Night: Formative vs. Summative Usability Testing for Medical Devices

No theater company opens a show without rehearsing. As rehearsals go on, lines get cut, blocking changes, lighting cues get updated, and someone inevitably finds an issue with the stage design. By the time opening night arrives, the surprises are supposed to be done.

Medical device usability testing works the same way. Formative testing is the dress rehearsal, where you find problems and fix them. Summative testing, also called human factors validation testing, is opening night, where you confirm the design is safe and effective in the hands of real users.

The two have a lot in common and are easy to confuse. Mixing them up can cost months of progress, so it is worth understanding how they differ.

What Is the Difference Between Formative and Summative Usability Testing?

Formative usability testing happens during development to find and fix use problems. Summative usability testing happens at the end of development to validate that the final design can be used safely and effectively. One method is built for learning, while the other is built to prove the design is safe and effective to use.

 

Formative (Dress Rehearsal) Summative (Opening Night)
Purpose Find problems and improve the design Confirm the design is safe and effective to use
Timing Throughout development, before the design is finalized After the design is finalized, as part of design validation
Device tested Sketches, mockups, predicates, and early prototypes Final or production-equivalent device
Participants Often five to eight per user group, per round At least 15 per distinct user group (FDA recommendation)
Mindset Break it, fix it, repeat Confirm it, document it
Typical output Updated design requirements, design changes, and an updated risk analysis Human factors validation report for the submission

 

Formative Usability Testing: The Dress Rehearsal

In a dress rehearsal, the director can stop a scene at any time, rewrite a line, and swap the props. Formative testing gives design teams that same level of freedom.

You can run it on almost anything: a napkin sketch, a foam mockup, a 3D-printed prototype, or an early software screen. The goal is to see where real users hesitate, misread, or reach for the wrong control. Then you change the design before those problems get expensive.

Formative studies are usually small and run in rounds. Many teams test with five to eight participants per user group, make changes, then test again. There is no fixed participant requirement, because the sample only needs to be big enough to reveal patterns.

With formative testing, you can produce:

  • Specific design changes tied to what users did, not just what they said
  • Updates to the use-related risk analysis
  • A record of how and why the design evolved

That record matters later. It shows reviewers that the design was refined based on evidence.

Curious what early testing looks like in practice? See our case studies.

Summative Usability Testing: Opening Night

Opening night is different. The set is built, the script is finalized, and nobody stops the show to coach the actors.

Summative testing, which the FDA calls human factors validation testing, evaluates the final design under realistic simulated use. Representative users complete critical tasks without any assistance, and observers document what happens, including close calls, use errors, and the reasons behind them.

The FDA’s human factors guidance recommends at least 15 participants from each distinct user group. A device used by both nurses and home caregivers, for example, needs 15 of each.

Summative results are documented in a human factors report that becomes part of the regulatory submission. It covers the testing, what went right, and any residual risk.

Where Each Fits in the Design Controls Timeline

Formative work lives in the early design stages, feeding user needs, requirements, and the risk analysis. Summative testing lands at design validation, once the design is finalized and the device represents what will actually ship.

Run summative testing too early, and you are testing a design that is still changing. Change the design after summative testing, and you may have to repeat the study.

The Most Common Mistake: Treating Opening Night Like an Audition

The costliest mistake is walking into summative testing hoping to pass. That turns a confirmation into a risk.

A summative study should confirm what formative testing already proved. If the team is still discovering how users interact with a device, it is not ready. A problem found in formative testing costs a design iteration. A problem found in summative testing can cost a redesign, a repeat study, and a delayed launch.

The fix is to rehearse more than you think is necessary. Test early, test often, and treat every formative round as a chance to find the problems first. That includes foreseeable misuse, which is why we recommend designing with misuse in mind from the beginning.

Who Can Run a Human Factors Validation Study?

Usually a human factors engineering team with experience in FDA submissions. Look for a partner that can plan the study, recruit representative users, run the sessions, analyze root causes, and write the report.

Kablooe’s human factors team supports the entire arc, from early formative research through summative validation and the documentation that goes into your submission. The same team that helps shape the design can help prove it works.

Planning a usability study, or not sure your device is ready for opening night? Let’s talk.

Frequently Asked Questions

What is the difference between formative and summative usability testing for medical devices?

Formative testing is done during development to find and fix issues, using sketches and early prototypes. Summative testing is done on the final design to confirm it can be used safely and effectively, and it produces the evidence for an FDA submission.

How many participants do you need for a human factors validation study?

The FDA recommends at least 15 participants for every distinct user group. Formative studies are smaller, often five to eight participants per group in each round.

Who can run a usability validation study for an FDA submission?

A human factors engineering team with experience planning, running, and documenting validation studies can run one. Kablooe offers human factors consulting that covers both formative and summative testing.