monarch-initiative/koza

CSV Reader: Handle varying line lengths

glass-ships opened this issue · 2 comments

Currently varying line lengths will raise an exception and hard fail.

We should probably make these warnings and report out which lines vary.

Could also create a custom exception and allow the client code to determine what to do here

@kevinschaper do you have any additional insight into this issue?

Does this include cases in which the transform parses multiple input files and column headings vary between the two (but they're all subsets of what the config specifies)?
Or is that a different issue?

to be honest I'm not too sure, this was basically copy pasted from a TODO comment in the code. I'm hoping Kevin has a better idea of what was meant there