Healthcare interoperability testing

What does interoperability testing cover?

The behaviour of the exchange, measured at both ends. A conformance run asks whether a message satisfies a specification. An interoperability run asks whether the system on the other side did with that message what the exchange existed to do. HL7's conformance methodology defines interoperability as "the ability of implementations to exchange data and to use that data as intended to accomplish a desired task", and places it a step past the specifications themselves: "in theory, compatibility is a prerequisite for interoperability". Two specifications can be compatible on paper while the two implementations built from them still fail to accomplish the task.

A validator settles whether an instance is well formed against a document it was given. It cannot settle whether the receiving system stored the value against the right patient, or acted on it at all, because neither fact is visible from the sending side. Those answers are read out of the partner system after the exchange, which is why an exchange case here ends in a query against the partner rather than in a status code.

Four things sit inside the scope of a run. The first is the set of messages, resources and objects that cross the boundary. The second is the acknowledgement and error path back, including the codes a receiver is entitled to return. The third is the record the receiving system holds once the exchange has finished, read out of that system rather than off the transport. The fourth is the version each of those is pinned to, which is a different question for every standard involved.

Why does a message that passes a validator get rejected at a customer site?

Because the specification left the element open and the partner closed it. Optionality is designed into these standards, and each standard hands the closing decision to a different document.

What HL7 v2, FHIR R4, DICOM and 45 CFR part 162 transaction standards leave open, and which document or date closes each gap.

Where the exchange runs	What the specification leaves open	What closes it
HL7 v2 messaging	The usage of every element the base standard marks optional or conditional	The receiving site's message profile
FHIR R4	What "support" means for an element flagged Must Support, which the base specification does not define	The profile author, in writing
DICOM	Which SOP Classes and roles an implementation carries, and which of the four published types each one is	The vendor's Conformance Statement
45 CFR part 162 transactions	Which adopted version governs, since each standard is adopted for a stated range of dates	The date the transaction is conducted

PS3.2 section 7.3, Rules Governing Types of SOP Classes states that "each SOP Class published in a Conformance Statement is one of four basic types", abbreviated in the template tables as S for Standard, SE for Standard Extended, SP for Specialized and P for Private. A Standard Extended SOP Class "shall: be a proper super set of one Standard SOP Class; not change the semantics of any Standard Attribute of that Standard SOP Class". So a partner can extend an object, stay inside the rules, and send you attributes your reader has never met.

A disagreement between two conformant systems rarely announces itself, because both ends accepted the traffic. It surfaces as clinical data in the wrong place: a value filed under a heading nobody reads, or an identifier that was rewritten in transit so the record cannot be found under the one you sent. DICOM PS3.1 section 1.4.3 permits the second in terms: "while minor changes may be made to the attributes of an instance to facilitate its handling within a particular organization (e.g., by coercing a Patient ID to the value used in a local context), the semantic content of an instance does not change." Coercion is legitimate, and it means the identifier your test asserted on is not always the identifier the receiver stored.

Is the capability claim the same as the behaviour?

They are separate objects, so a run against one settles nothing about the other. HL7 sets out the chain as a sequence: an implementer makes a conformance claim based on the implementation, a message instance is validated against that claim, and "a Tester can analyze the results of the validation." Nothing in that sequence checks that the claim describes the system that is running.

The comparison is cheap and it is almost never in a supplier's suite, because a claim reads like documentation. It is the first thing worth running against an imaging peer, and the shape of a full programme around one is in the medical device interoperability test plan.

Does the specification under the test move?

It does, and unevenly inside a single document. Part of a standard can be settled while another part is free to change, so a system built across that boundary ages at two speeds.

DICOM keeps retired material alive on the wire and stops maintaining it on paper. PS3.1 section 1.4.2, Continuous Maintenance states: "Retirement does not imply that these features cannot be used. However, the DICOM Standards Committee will not maintain the documentation of retired features. The reader is referred to earlier editions of the Standard." A retired feature still in traffic therefore has a reference text, and it is an edition nobody on the project has open.

FHIR R4 runs the same mechanism with a published clock on it. Exactly one artefact is marked deprecated in R4, OperationOutcome.issue.location, and the policy attached to the marking reads: "Deprecated materials are eligible to be balloted to be withdrawn two years after their deprecated status is published." The two states behave differently: deprecated is defined as "Systems should continue to support the artifact/feature/concept, but are discouraged from making use of it", while withdrawn is "Documented for historical purposes, no longer supported". A suite that treats them as one state gets a surprise on one of them.

In a test plan, each interface carries the release of each specification it was run against, and the maturity marking of the parts it depends on, so that a passing result from last year can be read for what it still proves.

Which version of a transaction standard is conformant?

Whichever one part 162 adopts for the day the transaction is conducted. The regulation pins each adopted guide to a range of dates, so a conformance question with no date attached to it cannot be answered either way. 45 CFR 162.103 defines a standard transaction as one "that complies with an applicable standard and associated operating rules adopted under this part", which puts two documents behind the acceptance criterion.

Two of those ranges change what a partner test has to cover. Between 14 August 2027 and 14 April 2028 the pharmacy claim has two lawfully adopted NCPDP versions at once, under 45 CFR 162.1102(e). During those eight months, which version a partner is on is a question about the partner, and a suite that answers it once has covered one partner. The second range runs the other way. The claims attachments standards at 45 CFR 162.2002 are adopted only "for the period on and after May 26, 2028", so a partner asking for attachments in the adopted format before that day is asking for something the regulation has not yet required. That request is settled in the trading partner agreement, and the final rule at 91 FR 14350 of 24 March 2026 fixes the same compliance date.

Nothing on this page describes the contents of any implementation guide adopted under part 162. Those documents are sold rather than published, and what the regulation makes public is the designation of each one: its title, version, date and document number, and the section that adopts it.

Which products does this work sit inside?

Four on this site, and the boundary looks different in each.

An EHR or EMR is where most of the traffic terminates, so the assertion has to end at a field a clinician can read rather than at an interface engine that accepted a message. A medical billing product sits between a charge raised upstream and a payer response that arrives weeks later, so the exchange test follows one identifier across systems that were built by different teams. A patient portal exposes an API to software nobody in the project controls, which makes the published capability the contract. A telemedicine platform produces an encounter record that leaves the session and enters somebody else's chart, and the record is checked in the receiving system.

Which standards does the work close?

Five, and each one is closed by a different artefact.

An HL7 v2 interface is closed by the message profile the receiving site issued. The run reports every instance against that profile and exercises the acknowledgement path when an instance fails it. HL7 v2 conformance testing requirements sets out what a profile has to say before it can carry an assertion at all.

A FHIR interface is closed by the capabilities and profiles the server declares, compared with what the server answers, plus the validator output for each instance. FHIR R4 conformance testing requirements covers which parts of a profile are testable and which are a statement of intent.

A DICOM interface is closed by the Conformance Statement together with the peer equipment the contract actually names, since scope follows from both. DICOM conformance testing requirements covers what can be tested when the Standard itself specifies no test procedure.

Coded content is closed by the code set release valid for the date of service rather than by the release loaded today, which is why the date of service travels with every validation call that crosses a boundary.

The ONC certification criteria are closed by API behaviour that matches the standards those criteria adopt. Neither the certification nor the compliance position is something a testing supplier holds. What a run produces is the evidence a certification body and an internal auditor each read before they decide.

What do you receive?

Six documents, and each one answers a question that gets asked after the engagement ends.

The interface inventory names every interaction that crosses the boundary and attaches one specification to each: a message profile, a profile canonical URL, a SOP Class with its role, or the section of part 162 that adopts the transaction. An interaction with no document attached is reported as such, because that is the finding, and the register below is where it goes.

The capability claim reconciliation report holds the two lists from the run above, side by side, with the date and build of each. It is what a buyer's own auditor opens to find out whether the vendor's statement was ever checked against anything.

The validation reports carry one file per interface: the tool and its version, the profile or Conformance Statement the run was made against, the build identifier, and every finding with its severity as the tool graded it. A finding that was accepted carries the reason and the name of the person who accepted it.

The optionality and local extension register lists every element the specification left open, the decision taken on it, the partner it was agreed with, and the date. Local extensions belong here too, since an extension is legitimate only while it is written down somewhere both sides can read.

The version and adopted period matrix records, per interface, the release of each specification it was run against together with the code set releases loaded, and the period the adopted standard covers. It exists so that a result from an earlier run can be read against the dates it actually covered.

The partner defect record separates findings against the specification from findings against a partner's local rule, and says which is which on the line. That one column is what keeps a trading partner disagreement out of a development backlog it can never be closed from.

Where does interoperability testing go wrong?

• The assertion stops at the interface engine, so a message that was accepted and never written to the chart is recorded as a pass.
• The vendor's conformance statement is filed rather than run, so a declared capability is first exercised by a customer.
• One message structure is tested and every trigger event routed through it is marked covered.
• The specification is signed off with elements still open, so each partner closes them differently and no assertion in the suite holds across two of them.
• Test fixtures are pinned to a release family rather than to a version string, so a change inside a trial-use part of the specification lands at runtime.
• Only the happy path across the boundary is exercised, so the rejection codes a receiver can return are never produced.
• Identifier coercion at the receiving site is unknown to the suite, so a record is looked up by the identifier that was sent instead of the one that was stored.
• Only the current code set release is deployed, so a message replayed from an earlier date of service gets its verdict from a file that postdates it.
• A partner disagreement and a product defect land in one backlog under one priority, and the disagreement is still open at the next release.
• De-identified imaging fixtures are treated as safe because a published attribute list was applied. PS3.15 Annex E says otherwise: the attributes it lists "may not be sufficient to guarantee confidentiality of patient identity", since identifying information may sit in private attributes, retired attributes and attributes used in Standard Extended SOP Classes. The fixture is checked for identifiers outside that list before it leaves the imaging system.

How does it fit the release you already run?

Test data is settled first, because it fixes the shape of everything built on top of it. An exchange fixture is a message, a resource or an imaging object carrying an identifier and a name, so it is either generated from the profile it has to satisfy or lifted from somewhere it should not be lifted from. The terms that would have to cover a captured corpus are their own question, and the page on working with protected health information is where it is raised. The choice also decides how the suite ages: a generated corpus can be regenerated against a tightened profile, while a captured one stays fixed to the site and the release it came from.

From the second release onward the exchange cases run as regression, and the part worth automating is the part with a stable oracle: instance validation against a profile, and code set checks that take the date of service as an input. The FHIR specification is explicit that full validation is expensive, warning that it "can be quite computationally demanding" and "can impose unacceptable time delays in a production system", so the run belongs at a control point in the pipeline rather than inside the request path.

What stays manual is the judgement about a difference: whether a partner's behaviour contradicts the specification or exercises an option it allows. That question is answered once, written into the register, and after that it is a test like any other.