Manual vs automated testing for medical software

Does an assessor read an automated result differently from a manual one?

Neither word is a category in the document that supplies the vocabulary. FDA's computer software assurance guidance, final and issued on 3 February 2026, takes its testing definitions from IEC/IEEE/ISO 29119-1, second edition 2022-01, and divides them once at the top into scripted testing and unscripted testing. Scripted testing is defined as "Testing in which test cases are recorded (e.g., document in a test management tool or in a spreadsheet) and can then be executed manually or executed automatically using an automated testing tool." Manual and automatic execution appear in the same sentence, as two ways of running one recorded case.

The other branch is defined by an absence. In the guidance's words, unscripted testing is dynamic testing in which a tester's actions "are not prescribed by written instructions in a test case", and written instructions of that kind are what a script is. Scenario testing, error guessing and exploratory testing sit under that branch, and how FDA nests the unscripted methods and what each one has to record sets the hierarchy out in the guidance's own order. FDA also states the list is not an exhaustive list of software testing methods and principles.

A plan reported as a share of tests automated has therefore described the execution mode and said nothing about the branch. Two teams can report the same share, with one of them holding scripted cases and unscripted sessions and the other holding scripted cases alone.

One boundary belongs at the top of this page, because it decides which document answers the question for you. Where the software is the product, the guidance is not that document: its scope section puts "device software functions" outside its own recommendations on design and development verification and validation. The requirements that bind your product testing are the ones in your quality system and in IEC 62304, and the clauses below are where this question is settled for them.

Which requirements can no script close?

The ones IEC 62304 asks somebody to evaluate, inspect or characterise. Edition 1.1 carries seven such requirements in six subclauses, and none of them sits in an annex or in an optional route.

Seven IEC 62304 Edition 1.1 requirements that call for evaluation or inspection no automated test can supply, with each subclause and its safety classes.

What the clause asks for	Where it sits	Which classes
An evaluation of the appropriateness of the verification strategies and the test procedures, before the results are read	5.7.4	A, B and C
Verification that every software requirement has been tested or otherwise verified, with the traceability recorded	5.7.4	A, B and C
Known residual anomalies documented	5.8.2	A, B and C
Verification that units were integrated according to the integration plan, which the standard's own note says is most likely implemented by some form of inspection	5.6.2	B and C
Established strategies, methods and procedures for verifying software units, and, where that verification is done by testing, the test procedures evaluated for adequacy	5.5.2	B and C
Integration test procedures evaluated for adequacy	5.6.5	B and C
A documented procedure for identifying categories of defects the selected programming technology may introduce, with evidence that those defects do not contribute to unacceptable risk	5.1.12	B and C

Adequacy and appropriateness are opinions about the test, and a passing test cannot hold either. Each of those rows ends in an artefact somebody signs, and a suite that grows without them adds assertions to the file and no judgement. Which IEC 62304 clauses bind at which software safety class covers the markers themselves, which changed with Amendment 1 in 2015.

clause 5.1.12 names an artefact that a human method already produces. It asks for a procedure that identifies categories of defect the chosen programming technology can introduce, and for evidence that those categories do not contribute to unacceptable risk. FDA's definition of error guessing describes the same object from the other side: test cases derived from the tester's knowledge of past failures or general knowledge of failure modes, knowledge that "can be encapsulated in, for example, a defects database or a 'bug taxonomy.'" A team that runs error-guessing sessions and writes down the taxonomy behind them has started an artefact that subclause asks for at classes B and C. A team that recorded those sessions as scripted cases instead holds a regression suite and an empty procedure.

Which half gets automated, and which half carries the evidence?

The half that carries the evidence is the clause list above, and none of it has a scripted form: the evaluation at clause 5.7.4, the residual anomaly list at clause 5.8.2, the error-guessing taxonomy behind 5.1.12, and the inspection the note to clause 5.6.2 describes. Work left in the unscripted branch does not get faster, and it goes missing when nothing collects it. Give the unscripted session the same template as the scripted run: what was tried, what was found, and who decided the result was acceptable.

The record obligations do not soften to compensate. The description of what was tested, the issues found, the conclusion of acceptability and the name and date behind it survive every method, and what a lighter validation method does and does not remove from an assurance file sets out which column of the file actually gets shorter. The suite that produces those records is itself software carrying a validation obligation, which is what a test automation engagement for regulated software builds and, inside a system already under validation, what automation owes when the system it drives is already validated.

A person doing unscripted work also raises a question a scheduled job never raises, because the person sees whatever the environment holds while deciding where to look next. We do not need production PHI to test. Environments run on synthetic and de-identified data, on the terms set out at how we work with protected health information.

When does your safety class decide what to automate first?

When the clauses you are automating towards are not the clauses that bind you. Amendment 1 to IEC 62304 changed 5.7.1, 5.7.2 and 5.7.3 from classes B and C to classes A, B and C, and the replacement texts it wrote for 5.7.4 and 5.7.5 carry classes A, B and C as well. Under the 2006 first edition those five subclauses did not reach class A at all. Everything in the table above that is marked B and C stayed there.

Read the two lists together and the effort lands in different places by class.

• At class A, the whole of clause 5.7 binds and the unit and integration verification clauses do not. The system test and its record are what an assessor has to be given, so automation that produces a system-level run with a complete record is buying something a clause names.
• At classes B and C, four further clauses appear, and three of them ask for an evaluation of the test procedures rather than for more test executions. Deep unit automation with no evaluation of the unit verification procedures answers 5.5.2 with the wrong half of the subclause.

Which runs a given release reopens is a separate decision with its own method, and it governs how often any of this is executed at all: how to choose and defend the regression set for a regulated release.

How do you split the suite without moving the evidence?

Six decisions, taken once and reused. The first three are about requirements and the last three about the record.

Step 6 is the one teams skip. A pipeline reports what it ran and has no way of reporting what stopped being run by a person, and the shift happens one test at a time. Where the two halves are counted separately in the plan, the change is a line item somebody approves. Where they are not, the file gets thinner in a way that only shows up when a reader asks who evaluated the test procedures and the answer is the procedures themselves.

Sources for this page. The testing definitions, the scripted and unscripted split, the 29119-1 attribution, the exploratory and error-guessing wording and the exclusion of device software functions are read in the FDA guidance Computer Software Assurance for Production and Quality Management System Software, issued 3 February 2026, checked on 2 September 2026, and nonbinding on its own face. The clause numbers, class markers and clause contents for 5.1.12, 5.5.2, 5.6.2, 5.6.5, 5.7.1 to 5.7.5 and 5.8.2 are read in IEC 62304 Edition 1.1, checked against IEC's own preview of the amendment on the same date.