Merge records DataModel.fetch_parquet
Closed this issue · 1 comments
rafarui commented
Need to review DataModel.fetch_parquet
When processing golden CDS data we are only looking at the mdmId to do the merges.
We should also consider the survivorship rules.
The way it is done today only works when I have a single staging feeding the DM.
rafarui commented
Closing. We will not implement this in pycarol side. Too many rules that need to be performed. We would need to replicate all merge and survivorship rules that Carol has.
one needs to consolidate the data using the API to guaranty the merges.