tilo/smarter_csv

Bug with extra separators

ClaytonPassmore opened this issue · 4 comments

ClaytonPassmore commented

Hey there 👋

Some behaviour in Smarter CSV seems to have changed somewhere after v1.3.0.

Test case

One,Two
First,Second,

Previous behaviour (v.1.3.0):

SmarterCSV.process(csv) # => [{:one=>"First", :two=>"Second"}]

New behaviour (v1.4.2, but I believe even as early as v1.4.0):

SmarterCSV.process(csv) # => [{:one=>"First", :two=>"Second,"}]

Observations

Notice that there's a dangling separator now in the output.

Versions

  • Ruby: v2.7.5
  • Rails: v6.1.4.6
  • Smarter CSV: v1.4.2 (previously v1.3.0)

Questions

  • Was this an intentional divergence from the previous behaviour?
  • Are there plans to fix this or is there an existing workaround?

Thanks 🙂

tilo commented

it is a result of fixing an issue.

previously the CSV line was just using split(col_sep), but that caused issues in some cases.
SmarterCSV is now using split(col_sep, count), where count reflects the number of header elements.

I'll see if I can fix this issue in the next release

9mm commented

Im experiencing this too:

:source=>"SV",
:partner_id=>",,,,,,,,,,,,,,,,,"},

9mm commented

Hey just checking in on this again, this seems like a very serious bug for a CSV library to have (not parsing column separators properly). I have reverted to an older version for now but I'm not sure what other bugs exist in that older version or which cases those bugs appear

  • 👀1
tilo commented

@9mm @ClaytonPassmore this will be fixed in version 1.5.0

  • 👍2
  • 🎉1