How to test with synthetic PHI data

Is a record you generated still protected health information?

Nobody stands behind it, so the removal work described by the safe harbor method has nothing to act on. 45 CFR 164.514(b)(2)(i) opens by naming whose identifiers are at issue: "The following identifiers of the individual or of relatives, employers, or household members of the individual, are removed", and then lists eighteen categories lettered (A) through (R). A generated patient has no relatives, no employer and no household. The standard the section serves, at section 164.514(a), reads in full: "Health information that does not identify an individual and with respect to which there is no reasonable basis to believe that the information can be used to identify an individual is not individually identifiable health information."

The open question is therefore about the generator, not about the output. Two paragraphs of the same section describe the shapes that answer it. The safe harbor method closes at 164.514(b)(2)(ii) with a condition on knowledge: "The covered entity does not have actual knowledge that the information could be used alone or in combination with other information to identify an individual who is a subject of the information." And 164.514(c) permits a re-identification code only where the code "is not derived from or related to information about the individual and is not otherwise capable of being translated so as to identify the individual". A lookup table that pairs each generated medical record number with the real record it was modelled on is capable of exactly that translation, and it usually lives in the same repository as the generator.

Getting this right has a stated effect. 45 CFR 164.502(d)(2) says that information meeting the standard and implementation specifications at 164.514(a) and (b) "is considered not to be individually identifiable health information", and that the requirements of the Privacy Rule subpart do not apply to it, subject to two provisos on re-identification. What is signed and who may reach an environment stays a contractual question, and ours is set out at how we work with protected health information. That page still carries two questions with no answer recorded against them, and both are about the environment itself: who stands one up, and who holds access once it exists.

Nothing in the Security Rule tells you to work this way. Read 45 CFR 164.302 through 164.318 in full and the words test environment, test data and de-identified test data do not appear, alongside the absence of any named testing technique described under HIPAA testing requirements for software. Generated fixtures are a decision about exposure that the regulated entity makes for itself.

Where the dataset begins life as an export of real records, this page is the wrong one. 164.514 applies to that export in full, and the method belongs to de-identification for test environments. Everything below is about data that was never real.

How do you build a corpus that exercises anything?

Start from the contract the system is measured against. A dataset generated from a mental model of a patient produces plausible records and covers whatever the author happened to imagine. A dataset generated from the conformance artefact covers what a receiving system is entitled to reject.

What does a generated HL7 v2 feed leave out?

A generator written against the current release emits clean messages, and clean messages are the part of an interface that already works. The failures cluster at the encoding layer, which most generators sidestep by never producing a character that needs escaping. MSH-2 carries five characters in a fixed order: the component separator, the repetition separator, the escape character, the subcomponent separator and the truncation character. The base standard states that when delimiters are included in any component "the delimiters SHALL be escaped", and defines escape sequences for six characters. A corpus with no apostrophes, no ampersands and no clinician name containing a caret never reaches that code path. The segment terminator is the one delimiter you cannot vary: it is always a carriage return, ASCII 13, hex 0D, and the standard says the value cannot be changed by implementers.

Message control identifiers are the second omission. MSH-10 holds "a number or other identifier that uniquely identifies the message", and its value comes back from the far side inside MSA. A generator that emits eight random characters per message will eventually repeat one, and when it does the acknowledgement can no longer be tied to the message that caused it, which is the assertion the test was built on.

Coverage by trigger event is where a generated corpus quietly under-tests. Structures are shared: HL7 Table 0354 records that ADT_A01 is the structure for A01, A04, A08 and A13, and that SIU_S12 serves fourteen events. Generating one message per structure covers four admission-family events with one message and calls it done. Table 0003 holds 384 trigger events and Table 0076 holds 159 message types, so the inventory has to come from the interface specification.

Two things a real feed carries that no generator invents on its own: Z-segments, which the standard reserves for local definition and forbids itself from defining, and messages the standard has already withdrawn. The order entry chapter marks ORM "withdrawn as of v2.7", and the V2-to-FHIR guide nonetheless carries a map for it, because components in that state are still known to be used. The site at the other end of the cable probably speaks 2.5.1, the version 45 CFR 170.205 adopts in four places. Pinning the version an interface is actually speaking, and reading what a profile constrains, is covered by HL7 v2 conformance testing requirements.

What does a FHIR generator have to aim at?

FHIR R4 defines only four cardinalities in the base specification, 0..1, 0.., 1..1 and 1.., and profiles tighten them within those limits. Generating to cardinality alone produces resources that validate and prove very little, and the specification says so directly: "setting an element to a minimum cardinality of 1 does not ensure that valid data will be present; specific FHIRPath constraints are required to ensure that the required data will be present."

The constraints are the generation targets. Every invariant carries a key, a human description and an expression, "a FHIRPath expression that must evaluate to true when run on the element", at one of three severities. Error is "a rule that all resources must conform to". Warning means the resource is still valid and can be processed normally. Guideline is a warning carrying the best-practice extension, treated as an error only when the implementation context asks for it. A corpus needs a resource that trips each error invariant and a resource that trips a guideline, because two validators can legitimately disagree on the second.

Terminology is the other target, and the required binding strength is unusually specific about what a fixture must contain. For a required binding on a CodeableConcept, at least one Coding element SHALL be present, one of the Coding values SHALL be from the specified value set, and text "is always recommended, but is not an acceptable substitute for the required code". A generator that fills CodeableConcept.text and leaves the coding array empty produces records that read correctly to a human and fail a conformance run.

The hardest case to generate is absence. US Core states that when information on a data element is not present and the reason is unknown, responders "SHALL NOT include the data elements in the resource instance returned as part of the query results", that requestors SHALL read a missing element as data not present in the responder's system, and that requestors SHALL be able to process instances asserting missing information. A generator that populates every element in the profile produces a corpus in which none of those three rules is ever exercised, while 45 CFR 170.315(g)(10)(i)(A) measures a certifying system on exactly the elements marked mandatory and Must Support. Which parts of R4 carry that weight, and which are still Trial Use, belong to FHIR R4 conformance testing requirements.

Which codes belong in the fixtures?

Diagnosis and procedure codes are the one part of a synthetic record where the interesting values are known in advance, because validity is a property of the release file. Sampling the code list at random reproduces the shape of the list and misses the edges. The FY2027 ICD-10-CM release holds 74,879 codes valid for submission, of which 215 are three characters long and valid on their own, and 51,587 are the full seven. 14,411 of them carry the placeholder X in position 5 or position 6.

The fixtures a corpus needs are therefore chosen against the construction rules: a three-character code that stands alone, a six-character stem that is invalid without its seventh character, a seven-character code where X pads position 6, a procedure code assembled from individually legal characters drawn from two different rows of the same table, and the same diagnosis code in both written forms, dotted and undotted. The rules those cases come from, and what a validator does with each, are tabulated under ICD-10 code mapping testing requirements.

Dates belong in the same fixture. The clock that decides which release applies is set at 45 CFR 162.1000(a), which pins the medical data code sets to those "valid at the time the health care is furnished". The encounter governs, and the day the claim went out does not. FY2027 deleted 30 codes that were valid on 30 September 2026, among them D691, which was split into D6911 and D6919. A single record carrying D691 with a September date of service and a second carrying it with an October date is the whole test, and a corpus with one date on every row cannot express it.

What has to ship alongside the data?

The records themselves are the smallest part of the deliverable. A dataset with no provenance cannot answer the question that made it worth generating.

The six artefacts that ship alongside a synthetic PHI test dataset, from generator specification to refresh trigger list, with what each settles.

Artefact	What it settles
Generator specification	Which constraint in which contract artefact each family of records was written against
Version and seed pin	That a run can be reproduced on the same generator, the same input artefacts and the same seed
Provenance statement	That no production record, and no derivative of one, entered the generator
Coverage matrix	Which profile elements, invariants and code set classes have a record, and which have none
Negative fixture register	Which record is expected to fail, at which constraint, with which error
Refresh trigger list	Which upstream release dates invalidate the corpus, with the date of the last regeneration

The negative register is the one most often missing. A validation run over a generated corpus that reports zero errors is ambiguous between a conformant system and a corpus that never tested anything, and only the register tells the two apart. FHIR makes the ambiguity concrete: the $validate operation "returns a 200 OK whether or not the resource is valid", while a 4xx or 5xx means the validation itself could not be performed and the resource's validity is unknown. A harness that treats HTTP status as the assertion passes on every resource it sends, including the ones the corpus wrote to fail.

Which defects does synthetic data hide?

Six classes recur, and each one follows from the way generated data is made.

Distribution. A generator produces the volumes the author asked for. Real estates are skewed, and the skew is where pagination, timeouts and memory behaviour live. FHIR warns against reading the base cardinalities as a guide to what turns up in practice: "The minimum cardinalities should not be taken as a guide to what elements are expected to be present in any particular use of the resource, including their normal/primary usage purpose." A patient carrying several thousand observations is a fixture somebody has to decide to build.

Local dialect. Interfaces in the field carry local extensions. The base standard reserves every segment identifier beginning with Z for local definition and states that Z codes SHALL NOT be defined within the standard itself, and it records that extending an HL7 segment with locally defined fields, "while not prohibited, is ill-advised", with the caveat that doing so "will likely cause conformance problems with the next release". None of it sits in any published table, so no generator emits it. It turns up on the first day of testing against a real site.

Optional-but-populated. The v2 optionality designation RE means "Required but MAY be Empty". A generator resolves each optional element one way and keeps resolving it that way for the whole run. The receiving system is then tested against exactly one of the two shapes it must accept.

Silence. Errors, timeouts and half-written records are states, and a generator emits well-formed output by construction. A negative corpus has to be authored deliberately, which is why the register above is a separate artefact.

Terminology drift. Chapter 22 of ICD-10-CM spans U00-U85 and holds three valid codes in FY2027: U070, U071 and U099. A generator that samples the declared range produces codes that do not exist, and a suite that accepts them is now asserting the wrong behaviour with every run.

Referential shape. An extract that flattens relationships and a generator that never creates them fail the same way. Authorisation checks walk from one record to another, and a corpus where every patient has exactly one encounter and one clinician has no boundary to cross. The consequences on a live estate are covered under PHI security testing.

Where does generated data stop?

At any question whose answer is a property of the production estate. Performance against real query patterns, audit records under 45 CFR 164.312(b), and the periodic evaluation the entity owes under 164.308(a)(8) all read the live system. A corpus can prepare those tests and cannot substitute for them.

For everything else, the exposure is avoidable and the position is the same on every engagement. We do not need production PHI to test. Environments run on synthetic and de-identified data. Where a corpus has to be governed across several teams and several environments at once, that is test data management for healthcare.

Sources for this page. The paragraphs quoted from 45 CFR parts 160, 162, 164 and 170 are read in the eCFR compilation as it stood on 31 August 2026. The HL7 v2, FHIR R4 and ICD-10-CM quotations are read in those standards, at the chapter, page or release file each one names. That the Security Rule mentions no test data anywhere is a recorded absence, established by reading 164.302 through 164.318 end to end.