SaMD testing services

What breaks in software that is itself the device?

The output is information, and a wrong answer looks exactly like a right one. There is no leak and nothing that fails loudly. A value, a label or a ranking arrives in the same format and at the same speed whether the reasoning behind it held or not, and the person reading it has no signal that anything went wrong.

The clinical claim is made about a population and the product is used one patient at a time. A case the rule set or the training data never covered still receives an answer, because returning nothing is rarely a state anyone implemented. The question a test programme has to answer is what the software does at the edge of the population named in its own intended use, and whether the edge is visible to the user standing in it.

The product cannot refuse its input, so the behaviour under wrong input is part of the product. A record system can reject a malformed message and make the sender resend it. Software whose intended purpose is to return an interpretation answers anyway, and the answer is a clinical output regardless of what arrived: units that followed the sending site rather than the specification, a reference range set by one laboratory, a timestamp with no zone on it.

Degradation is silent. When an input distribution moves away from what the software was built against, output keeps arriving with the same presentation and the same apparent confidence. Nothing in the software has failed in a sense a monitor recognises.

The platform belongs to someone else. A product placed on the market without dedicated hardware runs on an operating system that updates without asking, in a browser that reflows, on a screen whose contrast decides whether a marginal result reads as marginal.

The artefact that shipped is often not the artefact that was tested. A model file, a rule table or a threshold set travels separately from the application build, which means the release identifier on the report and the thing actually evaluated can drift apart without anybody noticing. That last failure costs more on this class of product than on any other, because of what a test record is used for here.

Which class is your software, and who decides it?

Your software carries two classifications at once, decided by different inputs and neither of them by how complicated the code is.

Software as a medical device carries an EU MDR Rule 11 risk class and an IEC 62304 safety class, each decided from different inputs.

The question	Decided by	What the decision is made from	What it changes
Which EU risk class?	MDR Annex VIII Rule 11, with implementing rules 3.3 and 3.5	The intended purpose, which Article 2(12) builds from the label, the instructions for use and the promotional or sales materials	Whether an outside assessor enters your development records, under Article 52
Which IEC 62304 safety class?	IEC 62304 clause 4.3, per software item	Whether unacceptable risk remains once risk control measures outside the software are taken into account, and how serious the possible harm then is	Which clauses of the standard bind, through the class marker each requirement carries

Rule 11 is quoted in full, with the route each class takes, on EU MDR software requirements. Reading it is rarely what settles the question. Teams arrive already able to recite the rule and still unable to place themselves, because the rule turns on what the software is for, and that is a sentence they wrote themselves.

MDCG 2019-11 Rev.1, the coordination group's guidance on qualification and classification, splits Rule 11 into sub-rules of its own numbering, 11a, 11b and 11c, and says of the first that it "describes, in very general terms, the 'mode of action' which is characteristic of all MDSW. Therefore, this sub-rule is generally applicable to all MDSW (excluding those MDSW that have no medical purpose)." The guidance states of itself that its views are not legally binding. Three of its positions locate a product faster than the rule text does:

• Where software both achieves its own intended purpose and drives or influences a hardware device, MDCG reads it as classified on its own, and the class "shall not be lower than the risk class of the hardware medical device".
• Rule 11 is not the only rule to check. Because software is an active device, MDCG names rules 9, 10, 11, 12, 13, 15 and 22 as the ones to consider, with implementing rule 3.5 taking the strictest applicable rule or sub-rule.
• Location does not classify. MDCG states that MDSW "should be classified in the same way, regardless of the software's location or the type of interconnection between the software and a (hardware) device", so moving inference from a device to a server changes the architecture and not the class.

The MDR itself defines none of the vocabulary the industry uses. Its Article 2 definitions run to seventy-one entries and none of them defines software, SaMD, MDSW, algorithm, artificial intelligence or machine learning. Software is reached only through Article 2(1), Article 2(4), implementing rule 3.3 and Rule 11. What is left carrying the weight is the intended purpose statement, and MDCG says so directly: a precise one "ensures that the MDSW aligns with the appropriate qualification and classification rules", and ambiguity leads to misclassification. Two teams with identical code and different intended purpose statements sit in different classes. Whether you are inside the definition at all is the prior question, worked through in is my health app a medical device.

The IEC 62304 side answers to something else entirely. The class is assigned per software item, after the risk control measures outside the software have been taken into account, and there is no default class A: software with no documented classification answers to the full clause set. A product that ships without dedicated hardware has few controls available outside its own code, and the one most often relied on is a person who checks the output before acting on it.

Which standards reach software that is the device?

Five documents do most of the work: IEC 62304, IEC 82304-1, ISO 14971, IEC 62366-1, and the EU AI Act where a model is involved. What each of them adds to a SaMD file differs from what it adds to software that ships inside a box.

IEC 62304 applies unchanged, and one sentence in its scope decides how it is used. The standard "does not cover validation and final release of the medical device, even when the medical device consists entirely of software". For a SaMD the boundary that sentence draws runs straight through a single codebase: the life cycle records on one side, device validation and release on the other, both produced by the same team from the same repository, and filed in different places. What IEC 62304 asks of software testing covers the clause set and the classification behind it.

IEC 82304-1 is the standard written for exactly this shape of product: health software designed to operate on general computing platforms and placed on the market without dedicated hardware. It sits on top of IEC 62304 and adds product level clauses, verification of use requirements at 4.3 and of system requirements at 4.6, validation at clause 6, and re-validation at clause 8.3. Its own Introduction sets the limit of what testing can do: "Testing of the finished product is not, by itself, adequate to address the safety of health software." The clause list and the documents it names are on IEC 82304-1 health software testing requirements.

ISO 14971 names software as a medical device inside its own scope sentence, which closes the argument that the risk process belongs to whoever integrates you. It also defines harm to include damage to property, extended by the Introduction to objects, data and other equipment, so a corrupted or lost result belongs in the risk file. What testing feeds that file is set out in ISO 14971 risk management and software testing.

IEC 62366-1 matters more for SaMD than the clause count suggests, because the whole product is a user interface over a decision. Summative evaluation of the selected hazard-related use scenarios is the validation step, and amended clause 5.7.3 asks how the characteristics of the test participants are representative of the intended user profiles, and how those participants are grouped into distinct user groups for the purpose of determining how many of them are needed.

The EU AI Act reaches a SaMD through Article 6(1), which needs both conditions met: the product is covered by Annex I legislation, and it is required to undergo third-party conformity assessment. Annex I Section A lists Regulation (EU) 2017/745 at entry 11. Chapter III Sections 1, 2 and 3 apply on that route from 2 August 2028, a date Regulation (EU) 2026/1744 moved back by a year from 2 August 2027. Article 9(8) is the provision that binds a test campaign: testing is carried out "against prior defined metrics and probabilistic thresholds that are appropriate to the intended purpose". A metric chosen after the run does not satisfy it.

Teams cross one US boundary by accident. FDA's computer software assurance guidance covers production and quality management system software, and its Section III excludes design and development verification and validation for device software functions, meaning software that meets the device definition under section 201(h) of the FD&C Act. That is your product. Scoping device software under CSA scopes it under a guidance that states it does not reach it. FDA does recognise the standards above: IEC 82304-1 Edition 1.0 under recognition number 13-97, ISO 14971:2019 under 5-125, and the consolidated IEC 62366-1 Edition 1.1 under 5-129.

Why does a test run here have to be reconstructable later?

Because the run is evidence in a regulatory file, and evidence nobody can reconstruct is not evidence. In every other product class the testing is a check performed on the software before it ships. Here the software is the device, so the record of the check is part of the thing being assessed. IEC 82304-1 makes the point structurally: compliance with it is determined by inspecting the documentation it requires, which means what gets assessed is the record rather than the run that produced it.

That imposes two directions of traceability on every result.

One direction runs backwards, to a hazard. ISO 14971 requires traceability for each identified hazard through the risk analysis, the risk evaluation, the implementation and verification of risk control measures, and the evaluation of residual risk, as ISO/TR 24971 subclause 4.5 restates it. A test case that covers a feature and cannot be joined to a hazard leaves a row of that trace empty, and the row is walked by the assessor rather than by you.

The other runs forwards, to a release. IEC 62304 expects the build identifier each run was executed on. MDR section 6.1(b) of Annex II requires the summary results of verification, validation and testing performed both in-house and in a simulated or actual user environment before final release, and requires them to address all the different hardware configurations and, where applicable, operating systems identified in the information supplied by the manufacturer. The platform list you print in the instructions for use is therefore a test matrix you have already committed to filling.

The risk file and the test evidence are the same argument told twice. The risk file asserts that a control reduces a risk to an acceptable level; the test evidence is what shows it does. Where the two disagree, an assessor reads the disagreement before reading either document on its own merits, and the usual form of the disagreement is a risk file that names a control the test suite never exercised.

Then a change reopens the argument. This is where teams shipping continuously get caught, because the release process was built to move code and the evidence obligations were written for a product that is placed on the market once.

What has to be exercised in a SaMD test programme?

Everything the class binds, plus the boundary conditions the product cannot refuse. The list below is what the standards above turn into work.

• Each hazard-related use scenario selected for summative evaluation, with the representativeness of the participants written down as amended IEC 62366-1 clause 5.7.3 asks.
• The behaviour at the edges of the population named in the intended use, and whether a user can tell they are standing at one.
• Input the product will meet and does not control: absent fields, values outside a reference range, units following a local convention, a stale or zoneless timestamp, a record that belongs to another patient.
• The degraded state. What the product presents when it cannot answer, and whether that presentation is distinguishable from an answer.
• Every hardware configuration and operating system named in the information supplied with the product, because Annex II Section 6.1(b) names that coverage explicitly.
• A run in a simulated or actual user environment as well as in-house, which the same indent names beside in-house testing.
• Each risk control measure twice, once for implementation and once for effectiveness, under ISO 14971 clause 7.2.
• The change path itself: what re-validation covers when the model artefact moves and the application code does not.

What does a SaMD connect to, and where does the evidence thin?

It connects to whatever the setting it is dropped into already speaks, which is why this class carries no characteristic protocol of its own. An imaging algorithm meets one interchange format, a decision support component inside a record system meets another, and a product used directly by a lay person may meet none. The qualifying question is the intended purpose rather than the interface, so no interchange standard follows from the class the way it follows from a record system or a monitoring device.

That is also what makes the evidence expensive. Annex II Section 6.2(g) asks for a description of the combination or configuration a device is used in, "including proof that it conforms to the general safety and performance requirements when connected to any such device(s)", so proof runs per configuration. A record system or a monitoring device can bound that list by the protocol it implements: it speaks one interchange standard, and the configurations worth claiming are the ones that standard admits. This class has no such boundary. The list is bounded only by the intended purpose statement, which is written in prose by you, so a sentence added to widen the market adds configurations that then have to be proven.

Getting data that exercises those boundaries is the hardest part of building the environment. What can be accepted from a client, and on what terms, is a separate negotiation, raised at how we work with protected health information.

Where does SaMD test evidence fail an assessment?

• The safety class and the risk class are recorded as one number in one place, so a change that moves one of them is never checked against the other.
• Test cases are named after features and the risk file is organised by hazard, and no artefact joins the two, which leaves the ISO 14971 trace to be assembled during the assessment.
• The risk control is a warning on a screen. Effectiveness evidence would require a user who reads it under the conditions the warning was written for, and nobody scheduled that session.
• The platform list in the instructions for use is longer than the platform list in the test matrix, which is the coverage Annex II Section 6.1(b) requires.
• The report cites the application release and the model artefact carries a version of its own, so nothing on file says which weights were evaluated.
• A retrain ships under an unchanged application version, and the re-validation clause 8.3 asks for has no release to attach itself to.
• Accuracy is reported as a figure without the metric it was measured under, where Article 15(3) of the AI Act requires the levels and the relevant metrics to be declared in the instructions for use.
• The device software is scoped under the FDA computer software assurance guidance, whose Section III excludes device software functions from its coverage.

What do we run against a SaMD?

We start at the classification, because it sets the size of everything after it. The intended purpose statement is read first and the Rule 11 reasoning is written against it; the IEC 62304 class is assigned per software item alongside it, and the two are kept in one place so a change to either is visible. Only then does a test plan get written, against the clause set those two decisions bind.

From there the work is the trace, and it is built in the direction an assessor reads it. Every risk control carries a criterion for whether it is present and a separate criterion for whether the risk moved, and every result carries the build identifier and the artefact version it executed against. Platform coverage is derived from the instructions for use rather than from the development machines, and the simulated user environment is treated as a separate run rather than as a description of the same one. Each post-market change is assessed for what it reopens before it ships.

What that produces is a file that can be entered at any hazard and read through to the residual risk that remains, with nobody standing beside the reader to explain it. How those runs are executed is covered under medical device software testing. The protocols, the traceability matrices and the reports that hold them together are covered under validation documentation.

Every clause number, article number and date on this page was checked against the primary text of the standard or regulation it cites on 2 September 2026. Statements attributed to MDCG 2019-11 Rev.1 are that group's guidance, which states of itself that it is not legally binding.