mysql_adapter.rb:29:in `+': String can't be coerced into Integer (TypeError)

Hi folks,

I'm getting the above error when trying my first activerecord-import test. I've checked the MySQL log and development.log, which have no other information. I'd appreciate any help in trying to resolve it.

I'm using:

ruby 2.4.10p364 (2020-03-31 revision 67879) [x86_64-linux]
Rails 4.2.11.3
mysql Ver 14.14 Distrib 5.7.33, for Linux (x86_64) using EditLine wrapper

My table looks like this:

CREATE TABLE `fastpass_intraday_times` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `entity_id` int(11) DEFAULT NULL,
  `CurrentDayTimeMSM` int(11) DEFAULT NULL,
  `CurrentFPReturnTimeMSM` int(11) DEFAULT NULL,
  `FutureTimeOfDayMSM` int(11) DEFAULT NULL,
  `FutureFPReturnTimeMSM` int(11) DEFAULT NULL,
  `FutureFPReturnTimeFrequency` int(11) DEFAULT NULL,
  `created_at` datetime NOT NULL,
  `updated_at` datetime NOT NULL,
  PRIMARY KEY (`id`),
  KEY `index_fastpass_intraday_times_on_entity_id` (`entity_id`),
  KEY `index_fastpass_intraday_times_on_CurrentDayTimeMSM` (`CurrentDayTimeMSM`),
  KEY `index_fastpass_intraday_times_on_CurrentFPReturnTimeMSM` (`CurrentFPReturnTimeMSM`),
  KEY `index_fastpass_intraday_times_on_FutureTimeOfDayMSM` (`FutureTimeOfDayMSM`),
  CONSTRAINT `fk_rails_e0c60007ab` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=277512 DEFAULT CHARSET=latin1;

The simplest sample code that creates records and generates this error, looks like this:

############################################
# This is a test of the bulk import utility
############################################

bulk_import_array = []

10.times do |i|
    # insert a new record into the table
    new_fpr = FastpassIntradayTime.new
    new_fpr.entity_id = i.to_i
    new_fpr.CurrentDayTimeMSM = i.to_i
    new_fpr.CurrentFPReturnTimeMSM = i.to_i
    new_fpr.FutureTimeOfDayMSM = i.to_i
    new_fpr.FutureFPReturnTimeMSM = i.to_i
    new_fpr.FutureFPReturnTimeFrequency = i.to_i

    # add to the bulk import array
    bulk_import_array << new_fpr
end

puts(bulk_import_array[0].to_json)

FastpassIntradayTime.import bulk_import_array

One of those sample records, when converted to JSON, looks like this:

{"id":null,"entity_id":0,"CurrentDayTimeMSM":0,"CurrentFPReturnTimeMSM":0,"FutureTimeOfDayMSM":0,"FutureFPReturnTimeMSM":0,"FutureFPReturnTimeFrequency":0,"created_at":null,"updated_at":null}

The complete error message is this:

/home/len/.rvm/gems/ruby-2.4.10/gems/activerecord-import-1.4.0/lib/activerecord-import/adapters/mysql_adapter.rb:29:in +': String can't be coerced into Integer (TypeError) from /home/len/.rvm/gems/ruby-2.4.10/gems/activerecord-import-1.4.0/lib/activerecord-import/adapters/mysql_adapter.rb:29:in insert_many'
from /home/len/.rvm/gems/ruby-2.4.10/gems/activerecord-import-1.4.0/lib/activerecord-import/import.rb:824:in block in import_without_validations_or_callbacks' from /home/len/.rvm/gems/ruby-2.4.10/gems/activerecord-import-1.4.0/lib/activerecord-import/import.rb:820:in each'
from /home/len/.rvm/gems/ruby-2.4.10/gems/activerecord-import-1.4.0/lib/activerecord-import/import.rb:820:in each_slice' from /home/len/.rvm/gems/ruby-2.4.10/gems/activerecord-import-1.4.0/lib/activerecord-import/import.rb:820:in import_without_validations_or_callbacks'
from /home/len/.rvm/gems/ruby-2.4.10/gems/activerecord-import-1.4.0/lib/activerecord-import/import.rb:761:in import_with_validations' from /home/len/.rvm/gems/ruby-2.4.10/gems/activerecord-import-1.4.0/lib/activerecord-import/import.rb:696:in import_helper'
from /home/len/.rvm/gems/ruby-2.4.10/gems/activerecord-import-1.4.0/lib/activerecord-import/import.rb:530:in bulk_import' from /home/len/Source/metatable/tmp/bulk_import_test.rb:24:in <top (required)>'
from /home/len/.rvm/gems/ruby-2.4.10/gems/railties-4.2.11.3/lib/rails/commands/runner.rb:60:in load' from /home/len/.rvm/gems/ruby-2.4.10/gems/railties-4.2.11.3/lib/rails/commands/runner.rb:60:in <top (required)>'
from /home/len/.rvm/gems/ruby-2.4.10/gems/railties-4.2.11.3/lib/rails/commands/commands_tasks.rb:123:in require' from /home/len/.rvm/gems/ruby-2.4.10/gems/railties-4.2.11.3/lib/rails/commands/commands_tasks.rb:123:in require_command!'
from /home/len/.rvm/gems/ruby-2.4.10/gems/railties-4.2.11.3/lib/rails/commands/commands_tasks.rb:90:in runner' from /home/len/.rvm/gems/ruby-2.4.10/gems/railties-4.2.11.3/lib/rails/commands/commands_tasks.rb:39:in run_command!'
from /home/len/.rvm/gems/ruby-2.4.10/gems/railties-4.2.11.3/lib/rails/commands.rb:17:in <top (required)>' from bin/rails:4:in require'
from bin/rails:4:in `

'

Thanks for any help.

Len

I should add that regular inserts, not using activerecord-import, work fine.

Can you identify why one of these values here is a string value when executing your code?

activerecord-import/lib/activerecord-import/adapters/mysql_adapter.rb

Line 29 in ea304fd

total_bytes = sql_size + values_in_bytes + comma_separated_bytes

Thanks, Jordan!

It looks like the issue is with values_in_bytes, which is a string, not a number.

Here's the output from a debugging section I added just before line 29:

value of sql_size=[218], type=Integer

value of values_in_bytes=[(NULL,0,0,0,0,0,0,'2022-07-05 13:15:58','2022-07-05 13:15:58')(NULL,1,1,1,1,1,1,'2022-07-05 13:15:58','2022-07-05 13:15:58')(NULL,2,2,2,2,2,2,'2022-07-05 13:15:58','2022-07-05 13:15:58')(NULL,3,3,3,3,3,3,'2022-07-05 13:15:58','2022-07-05 13:15:58')(NULL,4,4,4,4,4,4,'2022-07-05 13:15:58','2022-07-05 13:15:58')(NULL,5,5,5,5,5,5,'2022-07-05 13:15:58','2022-07-05 13:15:58')(NULL,6,6,6,6,6,6,'2022-07-05 13:15:58','2022-07-05 13:15:58')(NULL,7,7,7,7,7,7,'2022-07-05 13:15:58','2022-07-05 13:15:58')(NULL,8,8,8,8,8,8,'2022-07-05 13:15:58','2022-07-05 13:15:58')(NULL,9,9,9,9,9,9,'2022-07-05 13:15:58','2022-07-05 13:15:58')], type=String

value of comma_separated_bytes=[9], type=Integer

I appreciate your help in looking at this. If you have any suggestions for next steps, I'd appreciate it.

Len

Hm that is interesting, values_in_bytes should be the sum of the values bytesize so that should be an integer as well.

    # the number of bytes the requested insert statement values will take up
    values_in_bytes = values.sum(&:bytesize)

Can you see if Array#sum or String#bytesize is monkey patched in your project or possibly by another gem dependency?

For what it's worth, if the value of values_in_bytes is computed this way, everything appears to work in my demo:

    # the number of bytes the requested insert statement values will take up
    values_in_bytes = values.sum(&:bytesize)
    # Debug by len
    values_in_bytes = 0
    values.each do |v|
       values_in_bytes = values_in_bytes + v.length
    end

@jkowens - I'll check. Rails isn't my strength, so this might take a bit to figure out.

@jkowens Here's the output of that check:

array#sum location is ["/home/len/.rvm/gems/ruby-2.4.10/gems/statsample-2.1.0/lib/statsample.rb", 57]
string#bytesize location is []

Here's the code that produces it:

    tmp_arr =[]
    puts("array#sum location is #{tmp_arr.method(:sum).source_location}")
    str = String.new("abc")
    puts("string#bytesize location is #{str.method(:bytesize).source_location}")

Am I correct in assuming that array#sum is monkey patched by the statsample gem? And if so, how do I fix that?

Thanks again for the help here.

It does look like statsample monkey patches Array, but I don't see where it specifically overrides sum. It seems like you may have to monkeypatch the insert_many method in ActiveRecord::Import::MysqlAdapter. Just copy the whole method and change the code that calculates total_bytes to something that works for you.

Thanks, @jkowens. I'll try that.