Test of nhanesFromURL() on all data files listed in manifest
Closed this issue · 1 comments
deepayan commented
This is mostly for reference: Assuming that we want to populate the database using files listed in the NHANES table manifest using the new nhanesFromURL() function, I tried to go through all of them and make sure that there are no errors. Code here:
https://github.com/ccb-hms/nhanes-exploration/blob/main/scripts/test-nhanesFromURL.R
This does produce warnings flagging unusual behaviour. Edited output pasted below, in case it's helpful to debug problems later. Most of these are in 'special' food / diet / drug tables, which we may decide to exclude anyway. This also excludes two large (>7 GB) files, PAXMIN_G and PAXMIN_H.
----------
/Nchs/Nhanes/2011-2012/ALB_CR_G.XPT
Missing codebook table, skipping translation for variable: URXUMA
Missing codebook table, skipping translation for variable: URXUMS
Missing codebook table, skipping translation for variable: URXUCR
Missing codebook table, skipping translation for variable: URXCRS
Missing codebook table, skipping translation for variable: URDACT
----------
/Nchs/Nhanes/2003-2004/BPX_C.XPT
Variable not found in codebook, skipping translation for variable: BPXSAR
Variable not found in codebook, skipping translation for variable: BPXDAR
----------
/Nchs/Nhanes/1999-2000/DRXFMT.XPT
Skipping translation for character variable with missing codebook: FMTNAME
Variable not found in codebook, skipping translation for variable: START
Skipping translation for character variable with missing codebook: LABEL
----------
/Nchs/Nhanes/2001-2002/DRXFMT_B.XPT
Skipping translation for character variable with missing codebook: FMTNAME
Variable not found in codebook, skipping translation for variable: START
Skipping translation for character variable with missing codebook: LABEL
----------
/Nchs/Nhanes/2015-2016/DRXFCD_I.XPT
Variable not found in codebook, skipping translation for variable: DRXFDCD
Skipping translation for character variable with missing codebook: DRXFCSD
Skipping translation for character variable with missing codebook: DRXFCLD
----------
/Nchs/Nhanes/2017-2018/DRXFCD_J.XPT
Variable not found in codebook, skipping translation for variable: DRXFDCD
Skipping translation for character variable with missing codebook: DRXFCSD
Skipping translation for character variable with missing codebook: DRXFCLD
----------
/Nchs/Nhanes/2017-2018/P_DRXFCD.XPT
Variable not found in codebook, skipping translation for variable: DRXFDCD
Skipping translation for character variable with missing codebook: DRXFCSD
Skipping translation for character variable with missing codebook: DRXFCLD
Skipping translation for character variable with missing codebook: DRXFFCSD
Skipping translation for character variable with missing codebook: DRXFFDLD
----------
/Nchs/Nhanes/2011-2012/DS1IDS_G.XPT
Warning in code2numeric(x, cb, cleanse = cleanse_numeric) :
non-numeric descriptions found in apparently numeric variable: 1, 2, 3
----------
/Nchs/Nhanes/2011-2012/DS1TOT_G.XPT
Warning in code2numeric(x, cb, cleanse = cleanse_numeric) :
non-numeric descriptions found in apparently numeric variable: 1, 2, 3
----------
/Nchs/Nhanes/2011-2012/DS2TOT_G.XPT
Warning in code2numeric(x, cb, cleanse = cleanse_numeric) :
non-numeric descriptions found in apparently numeric variable: 1, 2, 3
----------
/Nchs/Nhanes/2003-2004/FOODLK_C.XPT
Variable not found in codebook, skipping translation for variable: FFQ_FOOD
Skipping translation for character variable with missing codebook: VALUE
----------
/Nchs/Nhanes/2005-2006/FOODLK_D.xpt
Variable not found in codebook, skipping translation for variable: FFQ_FOOD
Skipping translation for character variable with missing codebook: VALUE
----------
/Nchs/Nhanes/2003-2004/VARLK_C.XPT
Variable not found in codebook, skipping translation for variable: FFQ_VAR
Skipping translation for character variable with missing codebook: VALUE
----------
/Nchs/Nhanes/2005-2006/VARLK_D.xpt
Variable not found in codebook, skipping translation for variable: FFQ_VAR
Skipping translation for character variable with missing codebook: VALUE
----------
/Nchs/Nhanes/2009-2010/HPVSWR_F.XPT
Variable not found in codebook, skipping translation for variable: LBDRPI
----------
/Nchs/Nhanes/2001-2002/OHXPRL_B.XPT
Variable not found in codebook, skipping translation for variable: OHASCST5
----------
/Nchs/Nhanes/2001-2002/OHXPRU_B.XPT
Variable not found in codebook, skipping translation for variable: OHASCST5
----------
/Nchs/Nhanes/2001-2002/OSQ_B.XPT
Variable not found in codebook, skipping translation for variable: OSD030AA
Variable not found in codebook, skipping translation for variable: OSD030AB
Variable not found in codebook, skipping translation for variable: OSD030AC
Variable not found in codebook, skipping translation for variable: OSD030BA
Variable not found in codebook, skipping translation for variable: OSD030BB
Variable not found in codebook, skipping translation for variable: OSD030BC
Variable not found in codebook, skipping translation for variable: OSD030BD
Variable not found in codebook, skipping translation for variable: OSD030BE
Variable not found in codebook, skipping translation for variable: OSD030CA
Variable not found in codebook, skipping translation for variable: OSD030CB
Variable not found in codebook, skipping translation for variable: OSD030CC
Variable not found in codebook, skipping translation for variable: OSD030CD
Variable not found in codebook, skipping translation for variable: OSD030CE
Variable not found in codebook, skipping translation for variable: OSD030CF
----------
/Nchs/Nhanes/2005-2006/POOLTF_D.XPT
Variable not found in codebook, skipping translation for variable: WTSC2YRA
----------
/Nchs/Nhanes/2007-2008/POOLTF_E.XPT
Variable not found in codebook, skipping translation for variable: WTSA2YRA
----------
/Nchs/Nhanes/1999-2000/RXQ_DRUG.xpt
Skipping translation for character variable with missing codebook: RXDDRGID
Skipping translation for character variable with missing codebook: RXDDRUG
Variable not found in codebook, skipping translation for variable: RXDINGFL
Variable not found in codebook, skipping translation for variable: RXDDCI1A
Variable not found in codebook, skipping translation for variable: RXDDCI1B
Variable not found in codebook, skipping translation for variable: RXDDCI1C
Variable not found in codebook, skipping translation for variable: RXDDCI2A
Variable not found in codebook, skipping translation for variable: RXDDCI2B
Variable not found in codebook, skipping translation for variable: RXDDCI2C
Variable not found in codebook, skipping translation for variable: RXDDCI3A
Variable not found in codebook, skipping translation for variable: RXDDCI3B
Variable not found in codebook, skipping translation for variable: RXDDCI3C
Variable not found in codebook, skipping translation for variable: RXDDCI4A
Variable not found in codebook, skipping translation for variable: RXDDCI4B
Variable not found in codebook, skipping translation for variable: RXDDCI4C
Variable not found in codebook, skipping translation for variable: RXDICI1A
Variable not found in codebook, skipping translation for variable: RXDICI1B
Variable not found in codebook, skipping translation for variable: RXDICI1C
Variable not found in codebook, skipping translation for variable: RXDICI2A
Variable not found in codebook, skipping translation for variable: RXDICI2B
Variable not found in codebook, skipping translation for variable: RXDICI2C
Variable not found in codebook, skipping translation for variable: RXDICI3A
Variable not found in codebook, skipping translation for variable: RXDICI3B
Variable not found in codebook, skipping translation for variable: RXDICI3C
Variable not found in codebook, skipping translation for variable: RXDICI4A
Variable not found in codebook, skipping translation for variable: RXDICI4B
Variable not found in codebook, skipping translation for variable: RXDICI4C
Variable not found in codebook, skipping translation for variable: RXDICI5A
Variable not found in codebook, skipping translation for variable: RXDICI5B
Variable not found in codebook, skipping translation for variable: RXDICI5C
Variable not found in codebook, skipping translation for variable: RXDICI6A
Variable not found in codebook, skipping translation for variable: RXDICI6B
Variable not found in codebook, skipping translation for variable: RXDICI6C
Skipping translation for character variable with missing codebook: RXDDCN1A
Skipping translation for character variable with missing codebook: RXDDCN1B
Skipping translation for character variable with missing codebook: RXDDCN1C
Skipping translation for character variable with missing codebook: RXDDCN2A
Skipping translation for character variable with missing codebook: RXDDCN2B
Skipping translation for character variable with missing codebook: RXDDCN2C
Skipping translation for character variable with missing codebook: RXDDCN3A
Skipping translation for character variable with missing codebook: RXDDCN3B
Skipping translation for character variable with missing codebook: RXDDCN3C
Skipping translation for character variable with missing codebook: RXDDCN4A
Skipping translation for character variable with missing codebook: RXDDCN4B
Skipping translation for character variable with missing codebook: RXDDCN4C
Skipping translation for character variable with missing codebook: RXDICN1A
Skipping translation for character variable with missing codebook: RXDICN1B
Skipping translation for character variable with missing codebook: RXDICN1C
Skipping translation for character variable with missing codebook: RXDICN2A
Skipping translation for character variable with missing codebook: RXDICN2B
Skipping translation for character variable with missing codebook: RXDICN2C
Skipping translation for character variable with missing codebook: RXDICN3A
Skipping translation for character variable with missing codebook: RXDICN3B
Skipping translation for character variable with missing codebook: RXDICN3C
Skipping translation for character variable with missing codebook: RXDICN4A
Skipping translation for character variable with missing codebook: RXDICN4B
Skipping translation for character variable with missing codebook: RXDICN4C
Skipping translation for character variable with missing codebook: RXDICN5A
Skipping translation for character variable with missing codebook: RXDICN5B
Skipping translation for character variable with missing codebook: RXDICN5C
Skipping translation for character variable with missing codebook: RXDICN6A
Skipping translation for character variable with missing codebook: RXDICN6B
Skipping translation for character variable with missing codebook: RXDICN6C
sam-pullman commented
noted for reference, nhanesFromURL() is working on all 'public' nhanes files as tested with staging-v0.3.0