PEDSnet/pedsnetcdm_to_pcornetcdm

Update Chief Complaint Mapping

Closed this issue · 1 comments

Hi @ChavanShweta, please update the following join to cast both fields as lower().

inner join pcornet_maps.cheif_complaint_map comp on obs.observation_source_value = comp.observation_source_value

lower(obs.observation_source_value)=lower(comp.observation_source_value)

This change will pick up > 1 million chief complaint mapped observations.
Number of mapped observations without lower = 3,195,744
Number of mapped observations with lower = 4,445,083

Queries for comparison:

select count(distinct obs.observation_id)
from dcc_pedsnet.observation obs
inner join pcornet_maps.cheif_complaint_map mapped_cc
on obs.observation_source_value=mapped_cc.observation_source_Value
where observation_concept_id=42894222;

versus

select count(distinct obs.observation_id)
from dcc_pedsnet.observation obs
inner join pcornet_maps.cheif_complaint_map mapped_cc
on lower(obs.observation_source_value)=lower(mapped_cc.observation_source_Value)
where observation_concept_id=42894222;

There are still outstanding mappings. I'll update this issue with additional suggestions.

@ChavanShweta Please use this link for another 300 mappings that should pick up approximately 2.6 million observation_ids. Let's see how this improves the transformation and we can fold in the remaining observations when we get a chance. Thanks!!