Need examples for returning .find() as JSON

[asabaylus]

I'm struggling with BSON to JSON conversion when invoking .toArray() and running JSON.stringify on the returned data.
It's not clear to me from the existing examples how data is to be retrieved as JSON. Specifically the ObjectID

Please see, my question at Stackoverflow:
http://stackoverflow.com/questions/8811235/mongolian-deadbeef-toarray-returns-id-in-unexpected-format/8811832#8811832

Thanks!

[marcello3d]

The issue here is ObjectId is a special type in BSON, there's no direct mapping to a JSON type (and you don't necessarily want to expose ObjectIds to clients).

One hacky thing you could do (I don't necessarily recommend it), is add a toJSON method to the ObjectId prototype:

var ObjectId = require('mongolian').ObjectId
ObjectId.prototype.toJSON = function toJSON() { return this.toString() }

In theory this will make any ObjectId serialize to JSON as a regular string.

This will help with conversion from ObjectIds to strings, but there's no way to do the reverse conversion. There's no way to know which Strings in your JSON are supposed to be ObjectIds (plus it's highly unlikely that you want to use user-submitted JSON objects as-is).

If you wanted something more advanced, you could build a pseudo type system on top of JSON where you map ObjectIds to something like {$id:"..."}. Completely untested:

ObjectId.prototype.toJSON = function toJSON() { return { $id:this.toString() } }

JSON.parse(x, function jsonReviver(key, value) {
    if (value && value.$id && /^[a-f0-9]{24}$/i.test(value.$id)) {
        return new ObjectId(value.$id)
    }
    return value
}

Try it out and tell me what you think. Feel free to write some examples and pull request!

[asabaylus]

I was able to work around my issue by simply dropping the ObjectID from my query. Not really a solution. ex:

mycollection.find({}, { id:0 }).toArray(function(err, data){
   res.write(JSON.stringify(data));
});

As you point out it may not be a good idea to expose the ObjectID to clients. With that in mind, is there a better way to handle primary keys? Should I instead use a timestamp or some our unique data to populate a primary key? Is it better to ignore ObjectId which Mongo creates?

I cant envision any scenarios where I'd be able to use the converted ObjectID as returned by .toArray()
{ ...some data... , _id: { bytes: <Buffer 4f 0b 61 5a 00 00 00 7e 6e 00 00 06> } }
Should .toArray() then return the ObjectID by default?

Thanks @marcello3d for you patience, this is all very new to me, and I vastly prefer your driver to the node-mongodb-native.

[marcello3d]

You should definitely use ObjectIds internally—especially if you're relating data in different collections.

They are constructed with server-specific information and are (sort of) incremental, exposing information you might not want to. You could use a second field for user-exposed keys, such as a ObjectId hash or crypto random byte string, though they each have their own concerns.

It doesn't really make sense to automatically hide _id because it'd be fairly unexpected for people familiar with mongo.

[runvnc]

When I call toString() like you suggest above on the ID I get a nice hex string that I can send to my browser. My browser program DOES need to identify the object by Id when it sends a command back to my Node JS server. I took the hex string and tried to say new ObjectId(thehexstringfromObjectId_toString()) and it said 'Unrecognized type'.

I tried leaving it as a byte array in the data I sent to the browser and then sending the byte array back with NowJS, then calling new ObjectId with that byte array, didn't work.

Maybe I'm missing something, but I am having a lot of trouble accessing these objects by Id from my browser code, and I really need to do that.

[marcello3d]

@ithkuil: Did it just say Unrecognized type? Or did it say something like "Unrecognized type: X"? Can you paste the full stack trace? Perhaps console.log(thehexstringfromObjectId_toString()) or console.log(typeof thehexstringfromObjectId_toString()) before the error occurs?

[tegansnyder]

Is there any way to actually get the object id converted to a string.

locations.findOne({ _id: new ObjectId(req.params.id) }, function(err, post) {
        console.log(post._id.bytes.toString());
});

Console.log returns a bunch of gibberish for the id:

O��_ )�N

[tegansnyder]

Ok this works for me:

locations.findOne({ _id: new ObjectId(req.params.id) }, function(err, post) {

        var _id = post._id.toString();

        delete post._id.bytes;
        post._id = _id;

        res.send(JSON.stringify(post));

    });

[tegansnyder]

this would be for a find multiple

app.get('/addresses', function(req, res){

    var locations = db.collection("locations");

    locations.find().toArray(function (err, data) {

        var output = [];

        for(var i=0;i<data.length;i++){

            var _id = data[i]._id.toString();

            delete data[i]._id.bytes;
            data[i]._id = _id;

            output.push(data[i]);
        }

        res.send(JSON.stringify(output));

    });

});

[marcello3d]

You could simply write data[i]._id = data[i]._id.toString(). Alternatively, consider specifying a toJSON method:

ObjectId.prototype.toJSON = ObjectId.prototype.toString

That said, for security/privacy reasons you may not want to be exposing machine generated object ids like this.

[asabaylus]

Thanks for taking the time to explain. I'll post a link back here from the stackoverflow question.

[seanhess]

Can you have it serialize to a string by default? Right now it dumps out a crazy long nested thing. (The buffer has a parent?). This is at least more useful than what it does now, and is inline with what other libraries do, as well as what I expected to happen.

ObjectId.prototype.toJSON = ObjectId.prototype.toString