ncss-tech/SoilKnowledgeBase

parse errors due to non-standard typical pedon section header

dylanbeaudette opened this issue · 1 comments

c("ARIEL", "COLEMANTOWN", "DOCENA", "DUCKBOTTOM", "FIDDLETOWN", 
"GAPPMAYER", "GORING", "GREENWICH", "GUALALA", "HARKERS", "HATU", 
"LEIPSIC", "LUCKETTS", "MAJURO", "MANOKIN", "MIMMS", "MISPILLION", 
"MOYERS", "OLANTA", "PACKSADDLE", "PINICON", "QUEPONCO", "QUINDOCQUA", 
"ROMIA", "SABANA", "SCAPO", "SHOULDERBONE", "SIELO", "TENT", 
"TIZZARD", "TOLLGATE", "TRUSSUM", "TUSKAHOMA", "VELOW", "QUAKERBRIDGE", 
"LAKEHURST")
OSD JSON
ARIEL ARIEL.json
COLEMANTOWN COLEMANTOWN.json
DOCENA DOCENA.json
DUCKBOTTOM DUCKBOTTOM.json
FIDDLETOWN FIDDLETOWN.json
GAPPMAYER GAPPMAYER.json
GORING GORING.json
GREENWICH GREENWICH.json
GUALALA GUALALA.json
HARKERS HARKERS.json
HATU HATU.json
LEIPSIC LEIPSIC.json
LUCKETTS LUCKETTS.json
MAJURO MAJURO.json
MANOKIN MANOKIN.json
MIMMS MIMMS.json
MISPILLION MISPILLION.json
MOYERS MOYERS.json
OLANTA OLANTA.json
PACKSADDLE PACKSADDLE.json
PINICON PINICON.json
QUEPONCO QUEPONCO.json
QUINDOCQUA QUINDOCQUA.json
ROMIA ROMIA.json
SABANA SABANA.json
SCAPO SCAPO.json
SHOULDERBONE SHOULDERBONE.json
SIELO SIELO.json
TENT TENT.json
TIZZARD TIZZARD.json
TOLLGATE TOLLGATE.json
TRUSSUM TRUSSUM.json
TUSKAHOMA TUSKAHOMA.json
VELOW VELOW.json
QUAKERBRIDGE QUAKERBRIDGE.json
LAKEHURST LAKEHURST.json

I think a good addition to the OSD error reporting would be to parse the log files generated by each SKB refresh. This is something I always planned on implementing, but never integrated it with the newer post-processing scripts. SKB supports logging output for any of the create_* functions

Notably this would catch any OSDs using the "duplicated header" trick to put a bunch of extra front matter in, or any OSDs that are outright duplicated (it happens)

The .log files are generated on each run in a standard location (inst/extdata/OSD/OSD.log) but are not tracked in git. They have a standard format and the errors/names identified could be added to the types of errors we report on.

For example, OSD.log:

Processing OSD data snapshot from OSDRegistry...
Downloading snapshot...
 - OSD/A/AABAB.txt
 - OSD/A/AABERG.txt
 - OSD/A/AAGARD.txt
 - OSD/A/AARON.txt
 - OSD/A/AARUP.txt
 - OSD/A/AASTAD.txt
 - OSD/A/AAZDAHL.txt
 - OSD/A/ABAC.txt
 - OSD/A/ABAFT.txt
 - OSD/A/ABAJO.txt
 - OSD/A/ABALAN.txt
 - OSD/A/ABALOBADIAH.txt
 - OSD/A/ABARCA.txt
CHECK DUPLICATION OF HEADERS: OSD/A/ABARCA.txt
 - OSD/A/ABBAYE.txt
 - OSD/A/ABBEYLAKE.txt
 - OSD/A/ABBIE.txt
 - OSD/A/ABBOTT.txt
 - OSD/A/ABBOTTSTOWN.txt
 - OSD/A/ABBYVILLE.txt
CHECK DUPLICATION OF HEADERS: OSD/A/ABBYVILLE.txt
...