Ignore lines with errors
pdilyard opened this issue · 4 comments
Is there a way I could ignore lines that have invalid encoding? Or at the very least, know what this invalid encoding is? All I currently get is:
** (CSV.Lexer.EncodingError) Invalid encoding on line 3642
lib/csv/decoder.ex:161: CSV.Decoder.handle_error_for_result!/1
(elixir) lib/stream.ex:454: anonymous fn/4 in Stream.map/2
(elixir) lib/enum.ex:2744: Enumerable.List.reduce/3
(elixir) lib/stream.ex:732: Stream.do_list_transform/9
(elixir) lib/stream.ex:1247: Enumerable.Stream.do_each/4
(elixir) lib/enum.ex:1477: Enum.reduce/3
(elixir) lib/enum.ex:609: Enum.each/2
I'm working with a very large file, so it's pretty hard to pick out a line that might be encoded incorrectly.
On latest master there is decode and decode! - decode now returns a tuple in the form of either {:ok, row} or {:err, "some message"}, which should allow to have different handling for errors rather than raising them. This will be released in 2.0.
Appreciate the feedback around the error message. It is a bit tricky to show an error message with context since the encoding is not valid. I'll put some thought into it.
The {:ok, row} tuple is just what I'm looking for! Thank you
It would be great to have a way to just ignore broken lines. I'm currently struggling with this
Closing this as 2.0 allows for this