Chunk not processed in 1.7.0

The following code works fine in 1.6.1, but doesn't process a single chunk in 1.7.0.
The block provided does not execute at all and thus chunk_count remains at 0.

Am I missing something?

  def self.csv_import
    options = { chunk_size: 100 }
    chunk_count = 0
    SmarterCSV.process(Style.csv_path, options) do |chunk|
      sleep(1)
      GenericProcessChunkJob.perform_later('Style', chunk)
      chunk_count += 1
    end
    chunk_count
  end

@ernestoe thank you for posting - can you contribute a test that demonstrates it?

@ernestoe I can not reproduce this.

Can you please have a look at this RSpec test

I'd appreciate if you could write a test which reproduces the issue

if your GenericProcessChunkJob.perform_later raises an exception, the chunk does not get counted..
What happens if chunk_count += 1 is placed right after the sleep statement?

@ernestoe were you able to reproduce this, and create a test? Or should I close this?

if remove_empty_values: true (by default) and this is the Rails app, nothing after line 100 executes including yield on line 140.

I think ternary operator fits on the line 100 the best:
hash.delete_if{|_k, v| has_rails ? v.blank? : blank?(v)}

smarter_csv/lib/smarter_csv.rb

Line 100 in 9826bfd

return hash.delete_if{|_k, v| v.blank?} if has_rails

@tilo, sorry for the delay, let me try to put together the test... and your comment on chunk_count

If I include my rails_helper in the provided RSpec test

require 'rails_helper'

the test fails. And it's mostly standard configuration.

It looks like version 1.6.0 does not have return:
343e10f#diff-4b823c212caa561f9ac49a3a0c4f8a04494869fd234e4475f71241dd1b8ff183L96-R100

@tilo is this on the right track? Do you consider it properly identified?

You should not return while you're in a loop :

smarter_csv/lib/smarter_csv.rb

Line 59 in 8c35567

    
           until fh.eof? # we can't use fh.readlines() here, because this would read the whole file into memory at once, and eof => true

smarter_csv/lib/smarter_csv.rb

Line 100 in 8c35567

return hash.delete_if{|_k, v| v.blank?} if has_rails

The previous version of this code was better :

smarter_csv/lib/smarter_csv/smarter_csv.rb

Lines 94 to 100 in 59f792c

    
           if options[:remove_empty_values] == true 
        
             if has_rails 
        
               hash.delete_if{|k,v| v.blank?} 
        
             else 
        
               hash.delete_if{|k,v| blank?(v)} 
        
             end 
        
           end

or the proposed one :

 hash.delete_if { |_k, v| has_rails ? v.blank? : blank?(v) }

The stranger thing is : it doesn't seem to be covered by the tests :/

@n-rodriguez thank you for pointing this out!

Will release 1.7.1 today

@tilo thank you!

@n-rodriguez @ernestoe @KXEinc Thank you for your input on this!

Sorry for introducing this in 1.7.0

What can I say?
I've been indoctrinated by Rubocop with having to use guard clauses - that I put one in place where it didn't belong 🤦‍♂️

It was harder to reproduce because the issue only showed up when smarter_csv was used in a Rails project 🤦‍♂️

1.7.1 was just released - keeping this issue open until you verify it is fixed

Bugfix release 1.7.1 was just published

Please re-evaluate and update this issue if whether fixes the problem or not

@tilo it works! thank you!

Thank you all for your patience!

I'll work on adding a Rails related test

	if options[:remove_empty_values] == true
	if has_rails
	hash.delete_if{\|k,v\| v.blank?}
	else
	hash.delete_if{\|k,v\| blank?(v)}
	end
	end