What does validating your product actually involve?

What reaches your product?

Pick a market and a product type to see the standards that reach you. Until then this page lists every standard the site covers, which is what you get with JavaScript switched off.

Which standards reach this product in these markets?

• DICOMDICOM Standards Committee, with the National Electrical Manufacturers Association and its Medical Imaging and Technology Alliance division as secretariat
• EU AI ActEuropean Parliament and Council of the European Union
• EU MDREuropean Parliament and Council of the European Union
• 21 CFR Part 11U.S. Food and Drug Administration
• FDA CSAFDA, Center for Devices and Radiological Health and Center for Biologics Evaluation and Research
• FHIR R4Health Level Seven International (HL7). The conformance machinery is maintained by the FHIR Infrastructure Work Group, the terminology and conformance resources by the Vocabulary Work Group, and package composition by the FHIR Management Group (FMG)
• HIPAA Security RuleU.S. Department of Health and Human Services, Office for Civil Rights
• HITRUSTHITRUST Services LLC, a Delaware limited liability company
• HL7 v2Health Level Seven International (HL7). Chapter work sits with HL7 work groups, including Infrastructure and Messaging for Chapter 2 Control, Patient Administration for Chapters 3 and 10, and Orders and Observations for Chapters 4 and 7. The V2 Management Group sponsors version 2.9.1.
• ICD-10World Health Organization for ICD-10 itself. In the United States, the Department of Health and Human Services, through CDC's National Center for Health Statistics for ICD-10-CM and the Centers for Medicare & Medicaid Services for ICD-10-PCS
• IEC 62304IEC
• IEC 62366-1IEC, prepared by a joint working group of IEC/SC 62A with ISO/TC 210 and published as a double logo IEC/ISO standard
• IEC 82304-1IEC and ISO, prepared by IEC subcommittee 62A with ISO technical committee 215 and published as a double logo standard
• ISO 13485International Organization for Standardization
• ISO 14971ISO, prepared by ISO/TC 210 jointly with IEC/SC 62A
• IVDREuropean Parliament and Council of the European Union
• ONC Health IT Certification ProgramOffice of the National Coordinator for Health Information Technology (ONC), U.S. Department of Health and Human Services
• WCAG 2.1 Level AAWorld Wide Web Consortium (W3C), Web Accessibility Initiative, Accessibility Guidelines Working Group

Which artefacts do these standards ask you to produce?

The same artefact is often asked for by two standards for different reasons. Each one below is listed once, with the standard that asks for it.

Conformance Statement review against the Annex N template

Shows which of the 14 template sections carry content, which are marked N/A, and which were deleted, so the document can be compared with a peer vendor's

DICOM

Storage SOP Class and Transfer Syntax matrix from Table N.1-1

Records the roles and Transfer Syntax sets declared for each Storage SOP Class next to the ones actually exercised

DICOM

Statement to behaviour test report

Covers the gap PS3.2 Section 1 leaves open, where no procedure is specified for checking an implementation against its own Conformance Statement

DICOM

Data Set validation log in PS3.5 terms

Names each protocol violation with the tag, VR and VM from PS3.6 and the module usage from the PS3.3 IOD table

DICOM

DICOMweb transaction results against PS3.18 Table 10.3-2

Separates the resources mandatory for an origin server from the optional ones, so a missing resource reads as a gap rather than a preference

DICOM

De-identification results against the PS3.15 Annex E profile

Records which attributes were protected or retained, and which identifying values survived outside Table E.1-1

DICOM

Peer interoperability log

Records each exchange with a named peer system, the SOP Class and Transfer Syntax used, and the response, which is the validation PS3.2 Annex N.3.3 assigns to the user

DICOM

Annex IV content merged into the MDR or IVDR technical file

Answers Article 11(2), which requires one single set of technical documentation rather than a second file for the AI Act

EU AI Act

Test logs and test reports, dated and signed

Fills Annex IV point 2(g), which names the reports, the validation and testing data used, and the metrics applied

EU AI Act

Data set characterisation record

Shows an assessor how the training, validation and testing data sets were judged against Article 10(3)

EU AI Act

Accuracy declaration written for the instructions for use

Carries the levels and the metrics that Article 15(3) requires to be declared to the deployer

EU AI Act

AI risk management record folded into the existing risk file

Evidence for Article 9, including the fundamental rights risks that Article 9(2), point (a), adds to health and safety

EU AI Act

Robustness and adversarial test report

Covers the five attack classes named in Article 15(5), which a platform penetration test does not reach

EU AI Act

Human oversight evidence

Shows an assessor that the five capabilities listed in Article 14(4) are available to the person assigned oversight

EU AI Act

Rule 11 classification rationale

Gives the notified body the reasoning that Annex II Section 1.1(f) requires for the classification rule applied

EU MDR

GSPR conformity table with evidence locations

Answers Annex II Section 4 items (a) to (d), including where in the file each record of conformity sits

EU MDR

Software verification and validation summary

Supplies the process description and the summary results that Annex II Section 6.1(b) asks for by name

EU MDR

Coverage matrix for hardware configurations and operating systems

Shows that every platform named in the information supplied by the manufacturer was tested

EU MDR

Test report from a simulated or actual user environment

Covers the environment Annex II Section 6.1(b) names beside in-house testing

EU MDR

Verification record for the minimum IT environment

Checks the hardware, IT network and IT security minimums the manufacturer states under GSPR 17.4 and repeats at Annex I 23.4(ab)

EU MDR

Use error test record

Evidence for Annex I Section 5, and for Section 22 where the intended user is a lay person

EU MDR

Written rationale for testing not performed

Answers the closing sentence of Annex II Section 6.1(b) for each subject area left empty

EU MDR

Part 11 record scope decision, per predicate rule

Shows an investigator which records on the system are Part 11 records, which predicate rule requires each one, and whether you rely on the electronic record or on a paper printout

21 CFR Part 11

Validation package against the four objectives of 11.10(a)

Lets an investigator trace each of accuracy, reliability, consistent intended performance and the detection of altered records to the test that exercised it

21 CFR Part 11

Record copy demonstration in human readable and electronic form

Lets an investigator take away an accurate and complete copy under 11.10(b) and confirm the copy preserves the content and meaning of the original

21 CFR Part 11

Audit trail test evidence with the retention setting attached

Shows the trail is computer generated and time stamped, that a change does not obscure the previous value, and that the trail is retained for at least as long as the records it covers under 11.10(e)

21 CFR Part 11

Authority check matrix, role against privilege

Lets an investigator pick one role and see which of the five privileges named in 11.10(g) it holds and which negative test proved the refusal

21 CFR Part 11

Signature manifestation and linking evidence

Shows the printed name, the date and time and the meaning of the signing appear in the human readable record under 11.50, and that a signature cannot be moved to another record by ordinary means under 11.70

21 CFR Part 11

Identification code and password control records

Covers the five controls in 11.300, including the initial and periodic testing of tokens or cards that 11.300(e) requires on a recurring basis

21 CFR Part 11

Systems documentation change control log

Answers 11.10(k)(2) with a time sequenced record of how the system documentation itself was developed and modified

21 CFR Part 11

Intended use inventory per feature, function and operation

Shows an investigator which parts of a system the validation requirement was applied to, and the reasoning for the parts left out

FDA CSA

Process risk determination record

Shows an investigator why each feature was judged high process risk or not, against the foreseeable-safety wording the guidance uses

FDA CSA

Assurance activity record per feature

Gives an investigator the intended use, the risk result, the testing description, the issues found and the acceptability conclusion in one place

FDA CSA

Unscripted session records with pass or fail objectives

Lets an investigator accept an exploratory or error-guessing session that has no step-by-step procedure behind it

FDA CSA

Supplier assurance file

Shows an investigator which of the vendor's own development and quality work was leveraged, and against which supplier procedure it was judged

FDA CSA

Digital evidence map

Points an investigator at the system logs and audit trails that hold the result, instead of at screenshots pasted into a protocol

FDA CSA

CapabilityStatement conformance report

Records what each environment returned from the capabilities interaction, and whether that document satisfies the ten cpb invariants R4 attaches to the resource

FHIR R4

Version and profile pin record

Names the fhirVersion each endpoint declares and the canonical URL of every StructureDefinition listed in rest.resource.profile and rest.resource.supportedProfile

FHIR R4

Must Support interpretation matrix

Shows which meaning of support the profile author assigns to each flagged element, since the base specification assigns none and the word is otherwise untestable

FHIR R4

Validation report with the raw OperationOutcome

Lets a reader separate the fatal, error, warning and information issues instead of reading a single pass mark that hides the split

FHIR R4

Maturity exposure register

Lists every resource and element the product depends on that sits below Normative, which is where HL7 reserves the right to make a breaking change

FHIR R4

TestScript suite with its TestReport output

Gives a repeatable run whose fixtures, profiles and assertions are declared in the artefact itself rather than in a test framework nobody else can execute

FHIR R4

ePHI inventory with safeguard coverage

Lets an investigator follow one item of ePHI from creation to disposal and see which 164.312 standard covers each hop

HIPAA Security Rule

Technical safeguard test report against 45 CFR 164.312

Shows which access control, audit control, integrity, authentication and transmission check ran, on which build, and what it returned

HIPAA Security Rule

Addressable specification decision record

Supplies the written reason 164.306(d)(3)(ii)(B) requires for each addressable specification left unimplemented, and names the alternative measure

HIPAA Security Rule

Audit control evidence set

Shows that the mechanism required by 164.312(b) records the events an information system activity review under 164.308(a)(1)(ii)(D) has to examine

HIPAA Security Rule

Evaluation report for 164.308(a)(8)

Serves as the periodic technical evaluation the standard requires, carrying the scope, the date and the system version it covered

HIPAA Security Rule

Test data provenance record

Shows an investigator that the test environment held no live ePHI, or under which written contract it did

HIPAA Security Rule

Scope boundary listing the physical facilities and IT platforms assessed

Tells a relying party what the certificate covers, which HITRUST's own report says they have to evaluate

HITRUST

Evidence for every evaluative element inside each requirement statement

Criterion 8.1.2 requires all elements to be addressed, not the statement as a whole

HITRUST

Policy and procedure documents for each requirement in an r2

Policy and Procedure are two of the five maturity levels an r2 scores per requirement

HITRUST

Corrective action plans carrying a progress status

At the interim the assessor concludes whether half or more of the required CAPs are started or complete

HITRUST

Management Representation Letter

Carries the report date, and the report date is the day the certificate starts running

HITRUST

A HIPAA risk analysis held outside the assessment

HITRUST states its assessments are not risk assessments and leaves that obligation with management

HITRUST

Conformance profile per interaction, exported as XML

Gives a validator a machine-computable definition of the interface to test message instances against, which the base standard on its own does not provide

HL7 v2

Message inventory of interactions with acknowledgement choreography

Lets a reviewer see one profile per interaction and the immediate and application acknowledgements expected for each trigger event

HL7 v2

Usage difference matrix, base standard against site profile

Names every element the receiving site requires that the base standard leaves optional, which is the list a conforming sender fails on

HL7 v2

Z-segment and local extension register

Records each locally defined segment, field and message the interface depends on, so a version upgrade can be scoped before it is booked

HL7 v2

Validation report per message instance

Names each finding by element location, so a defect can be reproduced from the exact message that produced it

HL7 v2

Test message set with positive and negative cases

Gives the receiving site the instances its own regression run replays, including the invalid ones that exercise the rejection codes

HL7 v2

Version compatibility record across the endpoints in scope

Records the version each endpoint sends in MSH-12 and the elements that differ between them, so an upgrade on one side has a baseline

HL7 v2

Code set version inventory per environment

Names the release file each service loaded, by file name and effective date, so a reader can tell which code set produced a given verdict

ICD-10

Date of service validation suite

Shows the system reaches the verdict the code set in force on that date would reach, which is the criterion 45 CFR 162.1000(a) sets

ICD-10

Changeover regression report

Records what the 190 additions, 30 deletions and 4 description revisions in FY2027 ICD-10-CM did to stored data and to the validator

ICD-10

Structural boundary case set

Holds a worked case for each construction rule, including the placeholder X, the mandatory seventh character, a valid three character code and both written forms

ICD-10

Mapping results against the NCHS Conversion Table

Shows which remapped codes have more than one predecessor and what the system did with the predecessors it did not keep

ICD-10

ICD-10-PCS table row validity evidence

Shows the validator rejects a seven character combination whose values come from different rows of the same PCS table

ICD-10

Software safety classification record

Shows an auditor why the clauses you skipped were allowed to be skipped

IEC 62304

Requirements to test traceability matrix

Lets an auditor pick any software requirement and find the test that covers it

IEC 62304

Software unit verification records

Evidence for clause 5.5, including the acceptance criteria each unit was checked against

IEC 62304

Integration and system test reports

Evidence for clauses 5.6 and 5.7, with the build identifier each run was executed on

IEC 62304

SOUP inventory with anomaly evaluation

Shows which published defects in third party components were assessed and why they were accepted

IEC 62304

Problem reports with resolution and verification

Closes clause 9 by showing each problem was investigated, changed under control, and retested

IEC 62304

Usability engineering file

The document set every other record on this list sits inside, and the object of the compliance checks at clause 4.1.2 and the amended clause 5.5

IEC 62366-1

Use specification

Fixes the indication, patient population, user profiles and intended use environment that every later decision on the page is read against

IEC 62366-1

Record of safety-related interface characteristics and potential use errors

Lets an auditor see what was considered before any scenario was chosen for testing, as amended clause 5.2 requires

IEC 62366-1

Selection scheme for summative evaluation, with rationale and results

Gives an auditor the written basis for the scenarios that went into the summative test and the ones that stayed out

IEC 62366-1

User interface specification

The prospective description of the interface that the built product and the test protocol are both compared against

IEC 62366-1

User interface evaluation plan, formative and summative parts

Shows that participants, environment and data collection were fixed before the summative sessions rather than reconstructed afterwards

IEC 62366-1

Summative evaluation analysis with root causes

Answers amended clause 5.9 by listing every use error and use difficulty observed and the root cause of those that can lead to a hazardous situation

IEC 62366-1

Validation plan for the health software product

The document clause 6.1 is titled after, written before the runs so an assessor sees what was planned as well as what passed

IEC 82304-1

Validation report for the health software product

The document clause 6.3 is titled after, and the artefact an assessor opens because clause 1.3 decides compliance by inspecting documentation

IEC 82304-1

Verification records for product use requirements

Lets a reviewer take any use requirement and find the check that covered it, which is what clause 4.3 is titled after

IEC 82304-1

Verification records for system requirements

Names the platform and build each check ran on, for a product placed on the market without dedicated hardware

IEC 82304-1

Review notes on the instructions for use and the technical description

Records where clauses 7.2.2 and 7.2.3 describe a product that differs from the one we tested

IEC 82304-1

Re-validation records per release

Shows that a post-market change was revalidated under clause 8.3 rather than closed with a regression run

IEC 82304-1

Software inventory split across clauses 4.1.6, 7.5.6 and 7.6

Lets an auditor see which obligation each tool was validated under

ISO 13485

Intended use statement per software item

Fixes what the validation is measured against before anything is run

ISO 13485

Validation protocol with predetermined acceptance criteria

Shows the criteria existed ahead of the run, which is what MDSAP Task 15 checks first

ISO 13485

Records of the validation activities the protocol describes

Evidence that each activity in the protocol was executed and what it returned

ISO 13485

Change control records for every validated tool

Shows an upgrade was assessed and the affected validation repeated

ISO 13485

Justification for off the shelf software

Explains why the code was not reviewed and what was checked in its place

ISO 13485

Risk management plan with both verification methods stated

Shows an auditor how you decided to verify implementation and effectiveness before testing started, rather than after

ISO 14971

Hazard traceability record

An auditor picks one hazard and walks it to the residual risk that remains, through the verification of its control

ISO 14971

Verification records for implementation of risk control measures

Shows that each control chosen in clause 7 is present in the product that shipped

ISO 14971

Verification records for effectiveness of risk control measures

Shows that the risk each control was chosen for actually went down, which is the half of clause 7.2 most files are missing

ISO 14971

Overall residual risk evaluation against the criteria in the plan

Closes clause 8 by comparing the result with the method and criteria the plan committed to in advance

ISO 14971

Risk management report

Evidence for clause 9 that the plan was executed and reviewed before commercial distribution

ISO 14971

Production and post-production information records

Closes clause 10 by showing field information was collected actively and reviewed for relevance to safety

ISO 14971

Classification rationale under Annex VIII

Lets an assessor follow the rule you applied to the class you claim, which Annex II section 1.1(f) asks for by name

IVDR

GSPR conformity matrix with evidence cross-references

Takes an assessor from each applicable Annex I requirement to the controlled document that satisfies it, as Annex II section 4 requires

IVDR

Scientific validity report

Names which of the five sources allowed by Annex XIII, Part A, section 1.2.1 the analyte association rests on

IVDR

Analytical performance report

Gives an assessor every parameter listed in Annex I section 9.1(a), or the written justification for omitting one

IVDR

Clinical performance report

Carries the Annex I section 9.1(b) parameters and the source relied on for each, per Annex XIII, Part A, section 1.2.3

IVDR

Performance evaluation report

Wraps the three reports and the assessment of them that Article 56(5) makes part of the technical documentation

IVDR

Minimum IT security requirements specification

Answers GSPR 16.4 and supplies the instructions-for-use item at Annex I section 20.4.1(ah)

IVDR

PMPF plan and PMPF evaluation report

Shows how the performance evidence is kept current, or carries the justification Annex XIII, Part B, section 8 requires when PMPF is left out

IVDR

Certification scope map, criterion by criterion

Shows which criteria in section 170.315 the product is presented for, and which further criteria section 170.550(g) and (h) attach to that scope whether or not they were asked for

ONC Health IT Certification Program

Real World Testing plan draft carrying all seven required elements

Gives the ONC-ACB a plan with elements (A) through (G) of section 170.405(b)(1)(iii) in time to publish the CHPL hyperlink by December 15

ONC Health IT Certification Program

Real World Testing results evidence with its measurements

Supplies the six elements of section 170.405(b)(2)(ii), including the at least one measurement the results report has to carry per criterion in scope

ONC Health IT Certification Program

API conformance record for section 170.315(g)(10)

Lets a tester confirm that every data element marked mandatory and every element marked must support responds, for a single patient and for a patient group

ONC Health IT Certification Program

Safety-enhanced design usability report in the NISTIR 7742 format

Carries the participant description, the task to criterion mapping and the five metrics itemised in section 170.315(g)(3)(iv), for at least the ten test participants (g)(3)(ii) requires

ONC Health IT Certification Program

EHI export demonstration record

Shows a user creating a single patient export and a patient population export without developer assistance, with the publicly accessible hyperlink of the export format attached

ONC Health IT Certification Program

Privacy and security coverage matrix

Shows which of the criteria listed in section 170.550(h)(3) each certified capability was tested against, and where a separate test to section 170.315(d)(9) was required

ONC Health IT Certification Program

Non-conformity log with discovery dates

Supports the report to the ONC-ACB within 30 days that section 170.405(b)(2)(i) requires once real world testing finds a non-conformity

ONC Health IT Certification Program

Applicable-rule memo naming the instrument, the WCAG version and the level

Records which federal instrument reaches this product, because they name three different WCAG versions and two different levels, and a customer asking for WCAG 2.1 AA is usually quoting one of them

WCAG 2.1 Level AA

Scope list of full pages and of complete processes

Shows which pages the claim covers and which multi-step sequences were exercised end to end, since section 5.2.2 refuses conformance where part of a page is excluded and section 5.2.3 fails a whole process if one page in it fails

WCAG 2.1 Level AA

Results table with one row per success criterion, 50 rows for Level AA

Gives a result against the 5 June 2018 text for all 30 Level A and 20 Level AA criteria, including Success Criterion 4.1.1 Parsing, which the current W3C text treats as always satisfied and the regulations still require

WCAG 2.1 Level AA

Responsive variant evidence, one set per breakpoint

Answers Note 3 to section 5.2.2, under which each variation a page presents for a different screen size has to conform in its own right

WCAG 2.1 Level AA

Non-interference evidence for the four always-applicable criteria

Covers 1.4.2 Audio Control, 2.1.2 No Keyboard Trap, 2.3.1 Three Flashes or Below Threshold and 2.2.2 Pause, Stop, Hide across all content on the page, including content the claim does not rely upon

WCAG 2.1 Level AA

Accessibility support record for every technology relied upon

Documents the two-part test in the WCAG glossary, that the way the technology is used is supported by users' assistive technology and that accessibility-supported user agents are available to users

WCAG 2.1 Level AA

Conforming alternate version register with the limitation behind each one

Names the technical or legal limitation behind every alternate version, which is the only ground on which 28 CFR 35.202(a) and 45 CFR 84.86(a) permit one, and identifies the single version that is fully conformant

WCAG 2.1 Level AA

Conformance claim carrying the five required components

Supplies the date, the guidelines title with version and URI, the level satisfied, the page scope including whether subdomains are covered, and the technologies relied upon, which section 5.3.1 requires as soon as any claim or conformance logo is published

WCAG 2.1 Level AA

Which of our services cover these standards?

• FDA validation
• Data migration testing
• Mobile app testing
• QA outsourcing
• Test automation
• HIPAA testing
• Interoperability testing
• Device software testing
• PHI security testing
• Validation documentation