anlek/mongify

_id field as a string

Closed this issue · 10 comments

Some platform, something like Meteor.js, is using _id field as a string for some reason,
so is there any way not to set _id field as a ObjectId but String?

Thanks.

anlek commented

Hey @nicejwjin,
You could use the before_save feature to change it.

table "some_table" do 
  before_save do |row|
    row._id = row._id.to_s
  end
end  

Thanks @anlek -

But I got this error message when I set the translation.rb something like this you aswered

table "admin_user" do
    before_save do |row|
        row._id = row._id.to_s
    end
    column "id", :key, :as => :string
    column "pass", :string
    column "auth", :string
    column "name", :string
    column "business_id", :integer, :references => "businesses"
end

And this is the error message I got
Did I do something wrong?

jwjin@test$mongify process database.config translation.rb
/Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/db.rb:576:in command': Database command 'insert' failed: (ok: '1'; n: '1'; lastOp: 'seconds: 1439285223, increment: 16'; electionId: '55c9b38266446bd54ac7d8f9'; writeErrors: '[{"index"=>1, "code"=>11000, "errmsg"=>"insertDocument :: caused by :: 11000 E11000 duplicate key error index: campungFromMysql.admin_user.$_id_ dup key: { : \"\" }"}]'). (Mongo::OperationFailure) from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/collection_writer.rb:353:inblock in batch_message_send'
from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/functional/logging.rb:55:in block in instrument' from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/functional/logging.rb:20:ininstrument'
from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/functional/logging.rb:54:in instrument' from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/collection_writer.rb:352:inbatch_message_send'
from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/collection_writer.rb:82:in block in batch_write_incremental' from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/collection_writer.rb:57:incatch'
from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/collection_writer.rb:57:in batch_write_incremental' from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/collection.rb:1156:inbatch_write'
from /Library/Ruby/Gems/2.0.0/gems/mongo-1.10.2/lib/mongo/collection.rb:411:in insert' from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/no_sql_connection.rb:101:ininsert_into'
from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:36:in block (2 levels) in copy_data' from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/sql_connection.rb:86:inblock in select_rows'
from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/sql_connection.rb:84:in each' from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/database/sql_connection.rb:84:inselect_rows'
from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:29:in block in copy_data' from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:28:ineach'
from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:28:in copy_data' from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/translation/process.rb:14:inprocess'
from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/cli/command/worker.rb:68:in execute' from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/lib/mongify/cli/application.rb:28:inexecute!'
from /Library/Ruby/Gems/2.0.0/gems/mongify-1.2.4/bin/mongify:15:in <top (required)>' from /usr/bin/mongify:23:inload'
from /usr/bin/mongify:23:in `

'
jwjin@test$

I realise the code tried to use parent's (mysql) _id field to set a string one,
but actually there's no _id field on the parent.
How could I generate new _id field randomly on the mongify?

This is a solution that I found.

Fix translate.rb something like this.

require 'securerandom'

table "admin_user" do
    before_save do |row|
        row._id = SecureRandom.uuid
    end
    column "id", :key, :as => :string
    column "pass", :string
    column "auth", :string
    column "name", :string
    column "business_id", :integer, :references => "businesses"
end

anlek commented

Hey @nicejwjin,
Glad you were able to figure it out! :)

Just in case anyone wants to match their mongified ids with their meteor ids, here's what I figured out to make it happen.

$ npm install meteor-random
$ gem install posix-spawn

before_save do |row|
  require 'posix/spawn'
  pid, stdin, stdout, stderr = POSIX::Spawn::popen4("node -e \"Random = require('meteor-random'); console.log(Random.id());\"")
  row._id = stdout.read.strip
  ::Process.waitpid(pid)
end

It's slow, but it works.

anlek commented

@babenzele, are you trying to get your original IDs back? (before you used mongify to move it?)

No, I converted a MySQL db to mongoDB for use in a Meteor app and wanted _ids to match the style of ID that Meteor creates.

Since mongify calls mongoDB to create IDs, this results in uuid style ObjectIds. Meteor uses it's own ID generation library, which results in a sort of character limited hex string.

anlek commented

I see. Glad you got it working!

Me too! Thanks