Cannot serialize an object of class Mysql::Time into BSON. (BSON::InvalidDocument)

Question

Cannot serialize an object of class Mysql::Time into BSON. (BSON::InvalidDocument)

Closed this issue 9 years ago · 13 comments

I am getting this error when I am processing.

➜  Mongify  mongify process database.config test.rb
Copying betlings (1/1):                 (1304/1304) 100% |oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo| Time: 00:00:00
Copying bets (1/1):                       (341/341) 100% |oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo| Time: 00:00:00
/Library/Ruby/Gems/2.0.0/gems/bson-1.10.2/lib/bson/bson_c.rb:20:in `serialize': Cannot serialize an object of class Mysql::Time into BSON. (BSON::InvalidDocument)
    from /Library/Ruby/Gems/2.0.0/gems/bson-1.10.2/lib/bson/bson_c.rb:20:in `serialize'
    from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/collection_writer.rb:65:in `block in batch_write_incremental'
    from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/collection_writer.rb:57:in `catch'
    from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/collection_writer.rb:57:in `batch_write_incremental'
    from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/collection.rb:1156:in `batch_write'
    from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/collection.rb:411:in `insert'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.3/lib/mongify/database/no_sql_connection.rb:101:in `insert_into'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.3/lib/mongify/translation/process.rb:36:in `block (2 levels) in copy_data'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.3/lib/mongify/database/sql_connection.rb:86:in `block in select_rows'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.3/lib/mongify/database/sql_connection.rb:84:in `each'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.3/lib/mongify/database/sql_connection.rb:84:in `select_rows'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.3/lib/mongify/translation/process.rb:29:in `block in copy_data'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.3/lib/mongify/translation/process.rb:28:in `each'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.3/lib/mongify/translation/process.rb:28:in `copy_data'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.3/lib/mongify/translation/process.rb:14:in `process'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.3/lib/mongify/cli/command/worker.rb:68:in `execute'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.3/lib/mongify/cli/application.rb:28:in `execute!'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.3/bin/mongify:15:in `<top (required)>'
    from /usr/local/bin/mongify:23:in `load'
    from /usr/local/bin/mongify:23:in `<main>'

It has something to do with BSON.

I have BSON version 1.10.2 and mongify version: 1.2.4

Please help!

Answer 1 · 2016-01-24T16:06:13.000Z

Here is my gem list

Answer 2 · 2016-01-24T17:32:11.000Z

table "events" do
    column "id", :key, :as => :integer
    column "date_added", :timestamps
    column "eventDate", :date
    column "cust_id", :string
    column "cust_name", :string
    column "comment", :text
    column "special", :string
    column "employee_id", :integer
    column "userID", :integer
    column "start_time", :decimal
    column "clockStart", :string
    column "end_time", :decimal
    column "event_type_id", :integer
    column "event_type_name", :string
    column "event_type_class", :integer
    column "timespan", :string
    column "sms_status", :integer
end

This works when I remove column "eventDate", :date and change timestamp to timestamps

Any idea how I can use this with date?

Answer 3 · 2016-01-25T20:05:28.000Z

Hey, I think you might have found a bug in the time object. A quick work around is to do

table "events" do
    column "id", :key, :as => :integer
    column "date_added", :timestamps
    column "eventDate", :date
    column "cust_id", :string
    column "cust_name", :string
    column "comment", :text
    column "special", :string
    column "employee_id", :integer
    column "userID", :integer
    column "start_time", :decimal
    column "clockStart", :string
    column "end_time", :decimal
    column "event_type_id", :integer
    column "event_type_name", :string
    column "event_type_class", :integer
    column "timespan", :string
    column "sms_status", :integer
    before_save do |row|
      row. eventDate = row. eventDate.to_date
    end
end

Let me know how that works out for you

Answer 4 · 2016-01-26T02:12:24.000Z

Hey, thanks for the answer!

I get this now?

/Library/Ruby/Gems/2.0.0/gems/activerecord-3.2.0/lib/active_record/connection_adapters/mysql_adapter.rb:13:in `new': invalid date (ArgumentError)                                          | ETA:  --:--:--
    from /Library/Ruby/Gems/2.0.0/gems/activerecord-3.2.0/lib/active_record/connection_adapters/mysql_adapter.rb:13:in `to_date'
    from (eval):20:in `block (2 levels) in parse'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/table.rb:189:in `call'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/table.rb:189:in `run_before_save'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/table.rb:135:in `translate'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:33:in `block (3 levels) in copy_data'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:32:in `each'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:32:in `block (2 levels) in copy_data'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/sql_connection.rb:86:in `block in select_rows'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/sql_connection.rb:84:in `each'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/sql_connection.rb:84:in `select_rows'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:29:in `block in copy_data'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:28:in `each'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:28:in `copy_data'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:14:in `process'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/cli/command/worker.rb:68:in `execute'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/cli/application.rb:28:in `execute!'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/bin/mongify:15:in `<top (required)>'
    from /usr/local/bin/mongify:23:in `load'
    from /usr/local/bin/mongify:23:in `<main>'

Answer 5 · 2016-01-27T21:04:06.000Z

can you do a rescue for that and print out what the date was? So run this:

table "events" do
    column "id", :key, :as => :integer
    column "date_added", :timestamps
    column "eventDate", :date
    column "cust_id", :string
    column "cust_name", :string
    column "comment", :text
    column "special", :string
    column "employee_id", :integer
    column "userID", :integer
    column "start_time", :decimal
    column "clockStart", :string
    column "end_time", :decimal
    column "event_type_id", :integer
    column "event_type_name", :string
    column "event_type_class", :integer
    column "timespan", :string
    column "sms_status", :integer
    before_save do |row|
      row.eventDate = row.eventDate.to_date
    rescue => e
      puts "Errored eventDate was: #{eventDate}"
      raise e
    end
end

Hopefully we can figure out what's wrong with the eventDate.

Answer 6 · 2016-01-29T13:32:42.000Z

"unexpected keyword_rescue, expecting keyword_end"

➜  mongify2  mongify process database.config translation.rb
/Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation.rb:62:in `instance_eval': (eval):21: syntax error, unexpected keyword_rescue, expecting keyword_end (SyntaxError)
    rescue => e
          ^
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation.rb:62:in `parse'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/cli/command/worker.rb:55:in `execute'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/cli/application.rb:28:in `execute!'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/bin/mongify:15:in `<top (required)>'
    from /usr/local/bin/mongify:23:in `load'
    from /usr/local/bin/mongify:23:in `<main>'

Answer 7 · 2016-01-29T17:27:30.000Z

Hmm, sorry I don't have a way to test this on my side, but lets try replacing the before_save section with this:

    before_save do |row|
      begin
        row.eventDate = row.eventDate.to_date
      rescue => e
        puts "Errored eventDate was: #{eventDate}"
        raise e
      end
    end

Answer 8 · 2016-01-31T15:17:08.000Z

I can send you my config file so you can test this out. Thank you so much for the help!

Where can I send you the config file?

I am getting this btw..

(eval):23:in `rescue in block (2 levels) in parse': undefined local variable or method `eventDate' for #<Mongify::Database::Table:0x007fb67453e300> (NameError)
    from (eval):20:in `block (2 levels) in parse'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/table.rb:189:in `call'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/table.rb:189:in `run_before_save'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/table.rb:135:in `translate'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:33:in `block (3 levels) in copy_data'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:32:in `each'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:32:in `block (2 levels) in copy_data'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/sql_connection.rb:86:in `block in select_rows'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/sql_connection.rb:84:in `each'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/sql_connection.rb:84:in `select_rows'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:29:in `block in copy_data'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:28:in `each'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:28:in `copy_data'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:14:in `process'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/cli/command/worker.rb:68:in `execute'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/cli/application.rb:28:in `execute!'
    from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/bin/mongify:15:in `<top (required)>'
    from /usr/local/bin/mongify:23:in `load'
    from /usr/local/bin/mongify:23:in `<main>'

Answer 9 · 2016-01-31T22:19:39.000Z

Hey @kristjaningi,
I just followed up with you via email but just so it's down for others to read, my previous message had an extra space that caused you an issue. I've edited my previous comment and fixed it.
Let me know if you still are getting any issues.

Good luck!

Andrew

Answer 10 · 2016-02-22T15:41:09.000Z

I fixed this issue for me by changing all dates and timestamps to string and then I used moment.js afterwards to convert the string back to mongodb date object :)

Answer 11 · 2016-02-22T19:12:01.000Z

@kristjaningi awesome to hear! Too bad we didn't get it fixed via Ruby.
Good luck with the rest of your project!

Answer 12 · 2016-02-25T15:16:24.000Z

Thanks @anlek !
And again thanks for all your help :)

Answer 13 · 2016-02-25T17:06:48.000Z

Glad it worked out! :)