/mongodb-shell-extensions

Useful MongoDB shell extensions

Primary LanguageJavaScriptMIT LicenseMIT

MongoDB Shell Extensions Build Status

Collection of utilities to make the life inside of the MongoDB shell a little bit easier

Works for MongoDB: 2.4.X, 2.6.X and 3.0.X

Quick Examples

You have a collection visits like that

> db.visits.findOne()
{
  "_id" : "a0039342e1cda7446cbb55aac2108491-20140306",
  "at" : ISODate("2014-03-06T11:04:59.524Z"),
  "digest" : "a0039342e1cda7446cbb55aac2108491",
  "duration" : 150,
  "hits" : 5,
  "url" : "http://roob.biz/pearline"
}

You need to find how many visits there have been in the last 10 day... You know that dealing with dates is a mess, unless you have loaded the mighty MongoDB Shell Extensions in that case your life would be much, much easier

> moment.last(10).days().forEach('day', function(m) {
>   print(m.format('YYYYDDMM') + ': ' + db.visits.count({at: moment.$inDay(m)}))
> })

20140224: 153
20140225: 228
20140226: 228
20140227: 209
20140228: 246
20140301: 247
20140302: 243
20140303: 240
20140304: 208
20140305: 139
20140306: 204

You will have helpful output

> moment.last(10)
10 of what?
> moment.last(10).days()
"2014-02-24T11:36:50.509Z/2014-03-06T11:36:50.509Z"

You will have various helpful methods to reduce query verbosity

// Suppose we have a day d
> d
ISODate("2014-03-06T11:49:12.383Z")
> startOfDay = ISODate(d.toISOString())
> startOfDay.setUTCHours(0)
> startOfDay.setUTCMinutes(0)
> startOfDay.setUTCSeconds(0)
> startOfDay.setUTCMicroseconds(0)
> endOfDay = ISODate(d.toISOString())
> endOfDay.setUTCHours(23)
> endOfDay.setUTCMinutes(59)
> endOfDay.setUTCSeconds(59)
> endOfDay.setUTCMilliseconds(999)
> db.visits.count({at: {$gte: startOfDay, $lte: endOfDay}})
204

// YUCK! Can we do better?
// Yes, using dates manipulation functions
> db.visits.count({at: {
>   $gte: moment(d).startOf('day').toDate(),
>   $lte: moment(d).endOf('day').toDate()}
> })
204

// Can we do better?
// Yes, using moment.$between to generate $gte and $lte range
> db.visits.count({
>   at: moment.$between(
>     moment(d).startOf('day'),
>     moment(d).endOf('day'))
>   }
> )
204

// Can we do better?
// Yes, using moment.$inDay to use moment.$between and call startOf('day') and endOf('day')
> db.visits.count({at: moment.$inDay(d)})
204

// WOW! That's what I call an improvement!

Be mind, we only have scratched the surface of what we can do

Supported MongoDB Versions

  • 2.2.X
  • 2.4.X
  • 2.6.X
  • 3.0.X

How to Install

Download mongorc.js from the latest release and copy it into your home directory as .mongorc.js

curl -sL https://raw.github.com/gabrielelana/mongodb-shell-extensions/master/released/mongorc.js > ~/.mongorc.js

Or if you want you can install it using npm (N.B. This is going to install a bunch of dependencies, if you care about your disk space then prefer the first option)

npm install --global mongodb-shell-extensions

Now you have a .mongorc file in your home directory that contains all the extensions. This file will be loaded automatically in the next MongoDB shell session

The next time you'll start a MongoDB shell you should see a message like this (the message will not be displayed if the shell is in quiet mode mongo --quiet)

$ mongo
MongoDB shell version: 2.4.8
connecting to: test
+ MongoDB Shell Extensions by Gabriele Lana <gabriele.lana@gmail.com>
>

How to Temporary Disable

If you want to temporary disable the extensions you can start the MongoDB shell with the --norc flag

$ mongo --norc
MongoDB shell version: 2.4.8
connecting to: test
>

How to Uninstall

Remove .mongorc from your home directory

$ rm ~/.mongorc.js

Thanks To

This is really a bunch of wonderful open source projects put together with a little glue, so, many thanks to:

Documentation

Sorry, this is a work in progress, in the meantime, if you don't find what you are looking for "look at the source Luke" or drop me an email 😉

### `command pretty` Switch shell to pretty printing mode. Everything that could be pretty printed it will be automatically without asking for it ``` > db.users.first() { "_id" : ObjectId("53e0f55eca4f6f6589000001"), "name" : "Mervin", "surname" : "Witting", "job" : "Journalist" } > db.users.first().pretty() { "_id" : ObjectId("53e0f55eca4f6f6589000001"), "name" : "Mervin", "surname" : "Witting", "job" : "Journalist" } > pretty pretty printing: enabled > db.users.first() { "_id" : ObjectId("53e0f55eca4f6f6589000001"), "name" : "Mervin", "surname" : "Witting", "job" : "Journalist" } ``` ### `command ugly` Switch shell off from pretty printing mode ``` > db.users.first() { "_id" : ObjectId("53e0f55eca4f6f6589000001"), "name" : "Mervin", "surname" : "Witting", "job" : "Journalist" } > ugly pretty printing: disabled > db.users.first() { "_id" : ObjectId("53e0f55eca4f6f6589000001"), "name" : "Mervin", "surname" : "Witting", "job" : "Journalist" } ``` ### `command d|dbs|databases` List all databases and storage data. The format is NUMBER_OF_COLLECTIONS/SIZE_ON_DISK ``` > d recruiter 4/208MB playground 2/208MB waitress-test 3/208MB mongoose-trackable-test 2/208MB mongoose-eventful-test 6/80MB hangman 2/208MB ``` ### `command d|dbs|databases` List all collections and storage data. The format is NUMBER_OF_DOCUMENTS/SIZE_ON_DISK ``` > c archived 1/32KB roster 1/16KB scheduled 0/32KB system.indexes 3/4KB ``` ### `command so` Alias for `rs.slaveOk()` nothing fancy, I was just tired of typing it ### `DB#getCollections()` Returns an array of all collection instances ``` > db.getCollections().map(function(c) {return c.count()}) [ 5793, 4, 1003, 4373 ] ``` ### `Collection#distinctAndCount(field, query)` For each distinct value of `field` counts the occurrences in documents optionally filtered by `query` ``` > db.users.distinctAndCount('name', {name: /^a/i}) { "Abagail": 1, "Abbey": 3, "Abbie": 1, "Abdiel": 2, "Abdullah": 1, "Adah": 1, "Adalberto": 5, "Adela": 1, ... } ``` The `field` parameter could be an array of fields ``` > db.users.distinctAndCount(['name','job'], {name: /^a/i}) { "Austin,Educator" : 1, "Aurelia,Educator" : 1, "Augustine,Carpenter" : 1, "Augusta,Carpenter" : 2, "Audreanne,Zoologist" : 1, "Audreanne,Farmer" : 1, "Aubree,Lawyer" : 1, ... } ``` ### `Collection#first(n)` Returns the first `n` (ordered by `_id`) elements inserted in the collection ``` > db.users.first().length() 1 > db.users.first(3).length() 3 ``` ### `Collection#last(n)` Returns the last `n` (ordered by `_id`) elements inserted in the collection ``` > db.users.save({name: "Gabriele", surname: "Lana", job: "Software Craftsman"}) > db.users.last().pretty() { "_id" : ObjectId("531879529c812de54e6711e1"), "name" : "Gabriele", "surname" : "Lana", "job" : "Software Craftsman" } ``` ### `Query#first()` Same as [`Collection#first()`](#Collection-first) ### `Query#last()` Same as [`Collection#last()`](#Collection-last) ### `Query#reverse()` Reverse the order of the cursor ``` > db.users.first()._id === db.users.find().reverse().last()._id true ``` ### `Query#tojson()` ## `CSV` ### `tocsv(x)` Returns a `CSV` instance which is a collections of lines. The first line is the CSV header with the union of all the fields found in all the documents. The other lines are the CSV representation of the documents, one document per line. `CSV` inherits most of the collection methods implemented in `LoDash` ``` > tocsv(db.users.find({name: /^a/i})) _id,name,surname,job "5318565aca4f6f419b00001e","Abagail","Crona","Zoologist" "5318565aca4f6f419b000007","Abbey","Tromp","Writer" "5318565aca4f6f419b0000da","Abbie","Wilkinson","Carpenter" "5318565aca4f6f419b00007a","Abdiel","Schuster","Educator" "5318565aca4f6f419b0002d1","Abdiel","Schneider","Librarian" "5318565aca4f6f419b00030d","Abdullah","Baumbach","Librarian" "5318565aca4f6f419b0000dc","Adah","Lind","Dancer" "5318565aca4f6f419b0002ed","Adalberto","Reynolds","Librarian" "5318565aca4f6f419b000066","Adela","Keebler","Educator" "5318565aca4f6f419b0002d2","Adolf","Boyer","Farmer" ... ``` This is the same result of `printcsv` but don't be fooled, the shell calls `shellPrint()` method on every object that needs to be displayed by the shell itself, `shellPrint()` will print the `CSV` instance exactly as `printjson` would but you can do other things beside printing it ``` > tocsv(db.users.find({name: /^a/i})).head() _id,name,surname,job // sample will return a random line > tocsv(db.users.find({name: /^a/i})).sample() "5318565aca4f6f419b000098","Arlo","Huels","Lawyer" ``` It will work with everything has a `map` method ``` > tocsv([{name: "Gabriele", surname: "Lana"}]) name,surname "Gabriele","Lana" ``` ### `printcsv(x)` It will print each line of the CSV returned by `tocsv()`. This is useful when you want to export redirecting the output. Unfortunately `--eval` option will be evaluated **before** any script so it cannot be used to execute something defined in your `~/.mongorc.js` ``` $ cat > exportUsersToCSV.js <<SCRIPT heredoc> printcsv(db.users.find()) heredoc> SCRIPT $ mongo db-with-users --quiet ~/.mongorc.js ./exportUsersToCSV.js | tail -n +3 > users.csv ``` ### `Query#tocsv()` Same as `tocsv()` but called on a query ``` > db.users.find().tocsv() // It's the same as > tocsv(db.users.find()) ``` ### `Query#printcsv()` Same as `printcsv()` but called on a query ``` > db.users.find().printcsv() // It's the same as > printcsv(db.users.find()) ``` ## `JSONPath Support` ### `Query#select(x)` Applies one of more `JSONPath` expression to the current result set, useful when you have nested documents and you are interested only on some nested fields. ``` > db.nested.find() { "_id" : ObjectId("556f0715fa0cf10f7dff35aa"), "a" : [ 1, 2, 3 ], "b" : { "c" : 4 } } ``` `x` could be a single `JSONPath` expression or a list of `JSONPath` expressions. Only the fields that matches the expressions are kept in the result set. The selected fields are named after the `JSONPath` expression. ``` > db.nested.find().select('a') { "a" : [ 1, 2, 3 ] } > db.nested.find().select('a[0]') { "a[0]" : 1 } > db.nested.find().select(['a[0]', 'b.c']) { "a[0]" : 1, "b.c" : 4 } ``` `x` could be an hash where the values are `JSONPath` expressions and the related keys are the names that will be used to name the selected fields. ``` > db.nested.find().select({'a': 'a[0]', 'b': 'b.c'}) { "a" : 1, "b" : 4 } ``` ### `jsonpath(o, x)` Applies a `JSONPath` expression `x` on an arbitrary object `o`, useful to test a `JSONPath` expression before using it in `Query#select(x)` ``` > o = {a: [1, 2, 3], b: {c: 4}} > jsonpath(o, 'a[0]') 1 ``` ## `Temporary Collections` ## `Storable Cursors` ## `LoDash Integration` ## `MomentJS Integration`