This readme provides technical documentation on the MDM plugin for SanteDB. Overall architecture documentation can be found on the SanteDB WIKI.
The use case for the Master Data Management plugin is to allow robust linking of records from multiple source systems to a single MASTER (aka golden, enterprise, etc.) record. The goals of the MDM record plugin are:
- Allow source systems complete autonomy to edit/change their own source data feeds without impacting data feeds from other systems
- Ensure that all data from sensitive data sources (like HIV clinics) are not disclosed within the golden record
- Provide a transparent interface for callers of FHIR or HL7v2 to feed data to the MDM layer without knowing the specifics (i.e. it is transparent to the caller)
- Provide system administrators which a clear understanding of where data within their CDR is sourced
- Provide a mechanism for generated data (like identifiers, and tokens, etc.) to be segregated from source systems.
The MDM layer registers the following relationship types.
Mnemonic | Use |
---|---|
MDM-Master | Links a SOURCE/LOCAL entity to a MASTER entity |
MDM-OriginalMaster | Whenever the MDM layer changes the MASTER automatically |
MDM-RecordOfTruth | Links a MDM MASTER to a record which is promoted to RECORD OF TRUTH (this record trumps all source system) |
MDM-Duplicate | Identifies a SOURCE was detected as a duplicate or a candidate of a MASTER |
MDM-IgnoreCandidateLocalRecord | Indicates that the SOURCE and MASTER have been flagged as NOT the same and any future attempt to flag the record should be ignored. |
Note: The MDM layer provides its own identiifer based IRecordMatchingService
which wraps the configured IRecordMatchingService
whcih allows the MDM layer
to match records on identifiers, even when no other matching solution is configured.
This test case ensures that the basic test case is fulfilled in that a new record is created and a master is established for that record.
Pre-Conditions:
- None
Test Steps:
SOURCE_A
with minimal demographic data is registered (external id:MDM-01
)
Outcomes:
- MDM layer establishes
MASTER_A
- MDM layer links
SOURCE_A
withMASTER_A
with relationshipMDM-Master
and classificationAUTO
- Querying for
MDM-01
results inMASTER_A
being returned MASTER_A
contains the properties fromSOURCE_A
This test case ensures that when a new record matches an existing record (as defined in the matcher) the new source record is linked. This matching can be on identifier (like an foreign identifier).
Pre-Conditions:
SOURCE_A
with minimal demographic data is registered (external id:MDM-02A
)MASTER_A
has been established forSOURCE_A
Test Steps:
SOURCE_B
with matching demographics toSOURCE_A
is registered (external id:MDM-02B
)
Outcomes:
- MDM layer links
SOURCE_B
withMASTER_A
with relationshipMDM-Master
and classificationAUTO
- Querying for
MDM-02A
orMDM-02B
results inMASTER_A
being returned MASTER_A
contains both identifiersMDM-02A
andMDM-02B
MASTER_A
contains only one name (as they are identical)
This test case ensures that when a new source record is registered which has sufficient matching properties to be a Probable
classification, that the
MDM layer establishes appropriate linkages.
Pre-Conditions:
SOURCE_A
with minimal demographic data is registered (external id:MDM-03A
)MASTER_A
has been established forSOURCE_A
Test Steps:
SOURCE_B
with demographics which matchSOURCE_A
(with the exception ofMutlipleBirthIndicator
being different) is registered (external id:MDM-03B
)
Outcomes:
- MDM layer establishes
MASTER_B
for recordSOURCE_B
SOURCE_B
is linked toMASTER_B
with relationship typeMDM-Master
and classAUTO
SOURCE_B
is linked toMASTER_A
with relationship typeMDM-Duplicate
and classAUTO
This test case ensures that a master record with only one source record maintains its link with its source even when the source is updated to wildly different inforation.
Pre-Conditions:
SOURCE_A
with minimal demographic data is registered (external id:MDM-04
)MASTER_A
has been established forSOURCE_A
Test Steps:
- Chagne all the properties of
SOURCE_A
to new values - Save
SOURCE_A
Outcomes:
MASTER_A
remains established, no new master is createdSOURCE_A
remains pointed asMASTER_A
- All synthesized data in
MASTER_A
matches the updates toSOURCE_A
This test case ensures that a source record is flagged as a candidate record. At a later time, when an update to the candidate is received and the candidate now matches the master it was a candidate for, the relationship is changed such that the candidate is now a linked master.
Pre-Conditions:
SOURCE_A
with minimal demographic data is registered (external id:MDM-05A
)MASTER_A
has been established forSOURCE_A
SOURCE_B
with demographic data similar toSOURCE_A
is registered (external id:MDM-05B
)MASTER_B
has been established forSOURCE_B
SOURCE_B
has a relationship withMASTER_A
of typeMDM-Duplicate
and classAUTO
Test Steps:
- Update the properties in
SOURCE_B
so that they matchSOURCE_A
(keep the identifiers the same) - Save
SOURCE_B
Outcomes:
SOURCE_B
points toMASTER_A
with relationshipMDM-Master
and classAUTO
SOURCE_A
remains pointed toMASTER_A
with relationshipMDM-Master
and classAUTO
SOURCE_B
no longer has aMDM-Duplicate
relationship withMASTER_A
SOURCE_B
is linked toMASTER_B
with relationshipMDM-OriginalMaster
and classAUTO
MASTER_B
is obsoleted and does not appear in queriesMASTER_A
has a relationship withMASTER_B
of typeREPLACES
- Searching for
MDM-04A
orMDM-04B
returnsMASTER_A
In this test case, a master record contains two sources which had previously been automatically established. The source system updates one of the source records such that the matching engine no longer deems the source a match with the master. The source which was updated should be detached from the master.
Pre-Conditions:
SOURCE_A
is registered with minimal demographics (external id:MDM-06A
)SOURCE_A
points toMASTER_A
with relationshipMDM-Master
and classAUTO
SOURCE_B
is registered with same demographics asSOURCE_A
(external id:MDM-06B
)SOURCE_B
points toMASTER_A
with relationshipMDM-Master
and classAUTO
Test Steps:
SOURCE_B
properties are changed to vary wildly fromSOURCE_A
SOURCE_B
is saved
Outcomes:
SOURCE_A
remains pointed atMASTER_A
with typeMDM-Master
and classAUTO
SOURCE_B
points asMASTER_A
with typeMDM-OriginalMaster
and classAUTO
MASTER_B
is established in the databaseSOURCE_B
points toMASTER_B
with typeMDM-Master
and classAUTO
- Searching for
MDM-05A
returnsMASTER_A
- Searching for
MDM-05B
returnsMASTER_B
This test case ensures that when a candidate record is manually reconciled to be a match, that appropriate relationship steps occur.
Pre-Conditions:
SOURCE_A
with minimal demographic data is registered (external id:MDM-07A
)MASTER_A
has been established forSOURCE_A
SOURCE_B
with demographic data similar toSOURCE_A
is registered (external id:MDM-07B
)MASTER_B
has been established forSOURCE_B
SOURCE_B
has a relationship withMASTER_A
of typeMDM-Duplicate
and classAUTO
Test Steps:
- The
IRecordMergeService
instructs thatSOURCE_B
should be merged intoMASTER_A
Outcomes:
SOURCE_A
points toMASTER_A
with relationshipMDM-Master
and classAUTO
SOURCE_B
points toMASTER_A
with relationshipMDM-Master
and classVERIFIED
SOURCE_B
relationship toMASTER_A
with relationshipMDM-Duplicate
is removedSOURCE_B
relationship toMASTER_B
is removedMASTER_B
is obsolete (no longer appears in searches)MASTER_A
points toMASTER_B
with relationshipREPLACES
- Querying for
MDM-07A
orMDM-07B
returnsMASTER_A
In this test case, we perform the same test steps as Case #5, however becase the link between SOURCE_B
and MASTER_A
has a link type of VERIFIED
the relationship
does not change.
Pre-Steps:
SOURCE_A
with minimal demographic data is registered (external id:MDM-08A
)MASTER_A
has been established forSOURCE_A
with relationshipMDM-Master
and classAUTO
SOURCE_B
with demographic data similar toSOURCE_A
is registered (external id:MDM-08B
)SOURCE_B
has a relationship withMASTER_A
of typeMDM-Master
and classVERIFIED
Test Steps:
SOURCE_B
is updated such that the data is wildly different than previously estalbished
Outcomes:
SOURCE_B
remains linked toMASTER_A
withMDM-Master
and classVERIFIED
SOURCE_A
id detached fromMASTER_A
and a new masterMASTER_B
is establishedSOURCE_A
is linked toMASTER_B
with link typeMDM-Master
and classAUTO
SOURCE_A
is related toMASTER_A
with link typeMDM-OriginalMaster
and classAUTO
- Searching for
MDM-08B
returnsMASTER_A
- Searching for
MDM-08A
returnsMASTER_B
In this test case, we want to flag an identified candidate link as an ignore condition. Upon subsequent updates, the source should never be considered for matching or merging with another master, even if they meet criteria for candidate linking.
Pre-Steps:
SOURCE_A
with minimal demographic data is registered (external id:MDM-09A
)MASTER_A
has been established forSOURCE_A
with relationshipMDM-Master
and classAUTO
SOURCE_B
with demographic data similar toSOURCE_A
is registered (external id:MDM-09B
)MASTER_B
has been established forSOURCE_B
with relationshipMDM-Master
and classAUTO
SOURCE_B
has a link toMASTER_A
with relationshipMDM-Duplicate
and classAUTO
Test Steps A:
- The
IRecordMergeService
is called to ignore the relationship betweenSOURCE_B
andMASTER_A
Outcome A:
SOURCE_B
is related toMASTER_A
with relationshipMDM-IgnoreCandidateLocal
and classVERIFIED
Test Steps B:
- Update
SOURCE_B
such that the demographics information matchesSOURCE_A
- Save
SOURCE_B
Outcome B:
- No changes in
SOURCE_B
's relationship withMASTER_A
orMASTER_B
In this test case, we ensure that when a source is detached/unmerged from a master, that the detached source is never re-attached to the original master, even if it matches according to the normal rewrite rules.
Pre-Steps:
SOURCE_A
with minimal demographic data is registered (external id:MDM-10A
)MASTER_A
has been established forSOURCE_A
with relationshipMDM-Master
and classAUTO
SOURCE_B
with demographic data identical toSOURCE_A
is registered (external id:MDM-10B
)SOURCE_B
has a link toMASTER_A
with relationshipMDM-Master
and classAUTO
Test Steps A:
- Use the
IRecordMergingService
to UnmergeSOURCE_B
fromMASTER_A
Outcome A:
MASTER_B
should be established forSOURCE_B
with relationshipMDM-Master
and classVERIFIED
SOURCE_B
has a relationship withMASTER_A
with relationshipMDM-OriginalMaster
and classVERIFIED
- Querying for
MDM-09A
returnsMASTER_A
andMDM-09B
returnsMASTER_B
Test Steps B:
- Update
SOURCE_B
so that the dmeographics exactly matchSOURCE_A
- Save
SOURCE_B
Outcome B:
SOURCE_B
remains linked toMASTER_B
withMDM-Master
and classVERIFIED
In this test case we establish a known record of truth. Records of truth are special records which contain the most accurate information about a MASTER record.
In this test case we ensure that the Record Of Truth is not moved or merged in any fashion (similar to other locals). We also ensure that the synthesization of results take information from the ROT.
In this test case we setup two different policy levels on our source information.
SOURCE_A
-> No policies appliedSOURCE_B
-> TABOO policy applied
We then link SOURCE_A
and SOURCE_B
to a single MASTER_A
. Upon querying for data from MASTER_A
with a principal which has no access to TABOO
we note that only the data from SOURCE_A
is included in the result set. We then change our prinicipal to one where TABOO
is permitted and re-query. We should note that information from SOURCE_B
is updated.
In SanteDB no system is permitted to operate on a master record without appropriate policies in place. This test case will test a condition where a client mistakenly attempts to direclty update the master record from the API. The master record should realize this condition and should redirect the updates to the LOCAL record which the caller created in a previous step.
This is a special case of Test Case #13 whereby a new system has "downloaded" the master record and is attempting to re-submit the master. Here the MDM layer should establish a new SOURCE record for the update, and segregate (protect) the master/golden record.
This test case is a requirement for PMIR, we will attempt as a foreign credential to instruct the MDM layer to MERGE two master records together. The foreign credential which has no permission to MDM Write Master should result in an error/policy violation. When re-authenticating as a credential which has appropriate permission to administer MDM Masters, the two masters and their source records should be merged according to the logic specified.
This test case a LOCAL source for records SOURCE_A
and SOURCE_B
wishes to merge SOURCE_A
into SOURCE_B
. The serouce has control of the two records and therefore the merge is committed.
Pre-Steps:
SOURCE_A
is registered with master link toMASTER_A
(id:MDM-17A
)SOURCE_B
is registered with master link toMASTER_B
(id:MDM-17B
)
Test Steps:
- The
IRecordMergeService
is called to indicateSOURCE_A
replacesSOURCE_B
The MDM layer should perform the appropriate merging logic.
Outcomes:
SOURCE_A
is related toSOURCE_B
with relationshipREPLACES
SOURCE_B
is marked asOBSOLETE
SOURCE_A
contains bothMDM-17A
andMDM-17B
links
This test case ensures that a non-owner source of a LOCAL record cannot merge two source records with different ownership.
Pre-Steps:
- Authenticated as MDM-17A identity
SOURCE_A
is registered withMASTER_A
(id:MDM-18A
)SOURCE_B
is registered withMASTER_B
(id:MDM-18B
)
Test Steps:
- Authenticate as MDM-17B
- Use
IRecordMergeService
in an attempt to mergeSOURCE_A
intoSOURCE_B
Outcomes:
- An error is thrown indicating lack of ownership
SOURCE_A
andSOURCE_B
remain unchanged, linked toMASTER_A
andMASTER_B
respectively
In this test case, an API caller attempts to modify the MDM relationships directly and receives various behaviors related to the process:
Pre-Steps:
SOURCE_A
is registered withMASTER_A
(id:MDM-19A
)SOURCE_B
is registered withMASTER_A
(same demographics, idMDM-19B
)SOURCE_C
is registered withMASTER_B
(id:MDM-19C
)SOURCE_D
is registered withMASTER_C
and candidate link toMASTER_B
(id:MDM-19D
)
Test Steps:
- Directly call update to
EntityRelationship
in attempt toDELETE
link betweenSOURCE_A
andMASTER_A
- Directly call update in attempt to UPDATE link between `SOURCE_A
This test case ensures that a principal with access to TABOO Data is properly disclosed data in records marked as TABOO.
This test case ensures that the MDM layer processing bundles preserves the existing bundle transaction components. The test will ensure that a bundle with a Patient and Organization are properly stored and retrieved from the underlying MDM persistence layer.