mongodb tutorial https://docs.mongodb.com/manual/introduction/

Connect to Atlas dbs: mongo "mongodb://cluster0-shard-00-00-jxeqq.mongodb.net:27017,cluster0-shard-00-00-jxeqq.mongodb.net:27017,cluster0-shard-00-00-jxeqq.mongodb.net:27017/data?replicaSet=Cluster0-shard-0" --ssl --authenticationDatabase admin --username m001-student --password m001-mongodb-basics

get MongoDB Compass (or robomongo)

MongoDB Compass is a GUI that allows you to interact with your databases. Once you open the GUI, use your database credentials to connect to your DBs.

Download here

Robomongo https://robomongo.org/

Robomongo is a GUI that allows us to interact with our mongdb database directly.

After you start your mongodb (mongod) you can open the GUI and you should be able to see the dbs hosted on your system.

Mongo Atlas https://www.mongodb.com/cloud

Cloud based mongo service.

Create your account
Create a free or paid cluster (cluster of servers)
After cluster is created:

• select the "security" tab and add an admin SCRAM user

• whitelist the IPs that can connect to your cluster (you have an option to whitelist all IPs)

Whitelist

Under "overview" tab, click on "connect" to ge the code that connects with your cluster via mongo shell, via your app (ruby, node, java, python), or via mongo Compass.

Connect panel

Connect app

Connect shell

Create DB and insert data (check mongo methods on "loadMovieDetailsDataset.js")

cd into the directory where you have the data you want to insert on the DB
Make sure your .js file has the data and the right methods to insert that data where you need it.
Connect to your cluster or standalone DB -- you can connect to a local db if you want, just make sure you create it first via robomongo or via mongo shell (use <dbName> creates db and let's you interact with it).

e.g. (cluster connection)

#this command works for mongo shell 3.6 or later--this command prompts you to enter the password
mongo "mongodb+srv://sandbox-hj5gk.mongodb.net/test" --username m001-student

#for older versions, this is the command
mongo "mongodb://sandbox-shard-00-00-hj5gk.mongodb.net:27017,sandbox-shard-00-01-hj5gk.mongodb.net:27017,sandbox-shard-00-02-hj5gk.mongodb.net:27017/test?replicaSet=Sandbox-shard-0" --ssl --authenticationDatabase admin --username m001-student --password <PASSWORD>

run load(<dbFileName.js>)

This will create the new db as specified on the .js file you load (in this example, "video"). e.g.

Load data

After db is created, you can insert collections/documents using scripts as well. For instance, the command below runs the script "loadReviewsDataset.js" against the recently created "video" DB

#bash -v 3.6 or later
#notice you need to add the db name (video) on the db URL
mongo "mongodb+srv://sandbox-hj5gk.mongodb.net/video" --username m001-student loadReviewsDataset.js

# bash less than 3.6
mongo "mongodb://sandbox-shard-00-00-hj5gk.mongodb.net:27017,sandbox-shard-00-01-hj5gk.mongodb.net:27017,sandbox-shard-00-02-hj5gk.mongodb.net:27017/video?replicaSet=Sandbox-shard-0" --ssl --authenticationDatabase admin --username m001-student --password <PASSWORD> loadReviewsDataset.js

Common Shell Commands https://docs.mongodb.com/manual/reference/mongo-shell/

show dbs // shows all dbs
use <dbName> // use db
show collections // shows all collections in the db
db.collectionName.find() // shows collection content

# database is created on the fly when we switch to a new database and insert into a collection
# i.e.
use fruits
db.redFruits.insertOne({ name: "Strawberry" }) #"db" refers to "fruits"
#fruits is now created

#the .help() method can be called on a db or collection

# The "find" method does not return an array but the first 20 documents by default 
# and a cursor that allows us to cycle through the remanining documents. So in the mongo shell, 
# if you have a collection with more than 20 documents and issue "db.<collection>.find()" 
# the shell will display the first 20 docs and pop a message that says "Type 'it' for more". 
# You can chain the ".toArray()" method if you want the shell to cycle through 
# all documents for you, or the ".forEach()" as such 
# "db.<collection>.find().forEach(item => (printjson(item)))" where "printjson" is a shell function.

Common Methods https://docs.mongodb.com/manual/reference/mongo-shell/

CREATE

collection.insertOne()

  use <dbName> // use this db

  #insert one document 
  # db refers to the db we selected in the step above
  db.collectionName.insertOne({someKey: "some val", someKey: "some val"})

collection.insertMany()

  use <dbName>

  #insert many documents 
  #this method inserts documents in order by default (if there is an error, documents stop being 
  #inserted at that point--previously inserted documents are unaffected)
  #To ignore order and prevent errors stop the query before going through all inserts we can 
  #pass a second argument {ordered: false} after the documents to be inserted
  db.collectionName.insertMany([
    {someKey: "some val", someKey: "some val"},
    {someKey: "some val", someKey: "some val"}
  ])

  db.collectionName.insertMany([
    {someKey: "some val", someKey: "some val"},
    {someKey: "some val", someKey: "some val"}
  ], {
    ordered: false
  });

READ

db.collection.find()

#find all
db.collectionName.find()

When querying the DB we can use the pretty() method to display a formated json document e.g. db.collectionName.find().pretty()

db.collection.find({someKey: "someValue"}) with filter

#find specific by key
db.collectionName.find({someKey: "someValue"}) (e.g. find({age: 14}))

db.collection.find({"parentKey.childKey.grandchildKey": "someValue"}) with nested object filter

#find specific in nested obj
e.g

documents:

{
  id: ObjectId("11823081237"),
  name: "JD"
  profession: {
    title: "web engineer",
    experience: 12,
    currentCompany: {
      name: "Full Sail",
      field: "education"
    },
  },
  {
  id: ObjectId("11823081237"),
  name: "MD",
  profession: {
    title: "db engineer",
    experience: 3,
    currentCompany: {
      name: "UCF",
      field: "education"
    }
  }
}

#this query will return the two documents above
#need to put the key in quotes when it uses dot notation (in mongo shell)
db.collectionName.find({"profession.currentCompany.field": "education"})

db.collection.find() with operators

$and works like a regular query with two filters

db.collection.find({$and: [{name: "MD"}, {profession.experience: 3}]})

$or fetches where any filter matches

db.collection.find({$or: [{name: "MD"}, {profession.experience: 5}]})

$nor the opposite of $or (where no filter matches)

db.collection.find({$nor: [{name: "MD"}, {profession.experience: 5}]})

$regex

db.collection.find({name: {$regex: \MD\}})

$type

db.collection.find({name: {$type: "string"}})

$exists finds matches if property exists (even if value is null)

db.collection.find({name: {$exists: true}})

# if value is needed this operator can be combined with $ne (not equal)
db.collection.find({name: {$exists: true, $ne: null}})

$expr evaluates an expression and fetches matches that satisfy the result

db.collection.find({$expr: {$gt: ["$experience", 5]}) # fetches where $experience (notice the $ to represent the field as opposed to the string itself) is greater than 5

$sort

#ascending
db.collection.find().sort({name: 1})

#descending
db.collection.find().sort({name: -1})

#by multiple fields in order
db.collection.find().sort({name: 1, "profession.experience": -1})

db.collection.find() arrays

#find in arrays
e.g.

#Person collection

 {
  id: ObjectId("11823081237"),
  name: "JD",
  salary: null,
  profession: {
    title: "web engineer",
    experience: 12,
    currentCompany: {
      name: "Full Sail",
      field: "education"
    },
    pastCompanies: [
      "Amazon",
      "Google",
      "IBM"
    ]
  },
  {
  id: ObjectId("11823081237"),
  name: "MD",
  salary: null,
  profession: {
    title: "db engineer",
    experience: 3,
    currentCompany: {
      name: "UCF",
      field: "education"
    },
    pastCompanies: [
      "Google",
      "Amazon",
      "Reddit"
    ],
    positions: [
      {
        company: "Google"
        title: "Engineer"
      },
      {
        company: "Amazon"
        title: "Sr. Engineer"
      },
       {
        company: "Reddit"
        title: "Director"
      }
    ]
  }
}


#We can search all users who have worked on a certain company:
db.collectionName.find({"profession.pastCompanies": "Google"})

#Who have worked on a multiple companies
db.collectionName.find({"profession.pastCompanies": ["Google", "Amazon"]}) #returns arrays where "Google" comes before "Amazon" so the first document above is not a match

#Who have worked on a certain company as long as it is in a specific position in the array (useful when array is ordered by some kind of hierarchy)

db.collectionName.find({"profession.pastCompanies.0": "Google"}) #returns documents where "Google" is index 0 of "pastCompanies" array

db.collection.find() arrays with operators

dot notation in array

# we can use dot notation to access an object property within an array and filter by that property value
db.collectionName.find({"positions.company": "Google"}) #finds all that have "Google"

inclusive vs. exclusive

db.collectionName.find({"positions.company": "Google"}) #finds all that have "Google"
db.collectionName.find({"positions.company": ["Google"]}) #finds all that have ONLY "Google"

$all

# finds in array where all specified items exist regardless of order (without it mongodb only finds matches if document's array items are in the same order as items in query array)

db.collectionName.find({"profession.pastCompanies": {$all: ["Google", "Amazon"]}})

$elemMatch

# finds in array where documents match all conditions ($and in an array doesn't fulfill this criteria because it would find where document matches either condition)

db.collectionName.find({"profession.positions": {$elemMatch: {title: "Engineer", company: "Google"} }) #finds where array has an object whose title is "Engineer" AND company is "Google",

$size

#finds matches by array length

db.collectionName.find({"profession.positions": {$size: 2 })

utility methods

db.collection.find().count()

db.collection.find({someKey: "someValue"}).count()

db.collection.find().pretty()

#returns the matched documents formatted
db.collection.find({someKey: "someValue"}).pretty()

db.collection.find().explain() https://docs.mongodb.com/manual/reference/method/cursor.explain/

#explains how the query is made--like a debugger
db.collection.find({someKey: "someValue"}).explain()

#gives details about the query execution
db.collection.find({someKey: "someValue"}).explain("executionStats")

db.stats()

Shows stats about the database (name, collections, and other stats)

Projections

Projections allow us to limit the fields we want our query to return.

Projections are specified as a second argument to the query. If the projection has a key with the value "1", that field is included in the query; if the value is 0, that field is excluded from the query

e.g.

#consider the Person collection above
#this query returns only the name field (and _id by default) of people whose experience value is 3

db.collection.find({experience: 3}, {name: 1})

#this query excludes the id from the results
db.collection.find({experience: 3}, {name: 1, _id: 0})


#this query returns all fields but name
db.collection.find({experience: 3}, {name: 0})

simple index on collection for fast document lookups

Without an index, a query will look up all documents in a collection until it finds the one(s) that matches. If we create an index on a collection the lookups will be much faster as mongo will examine only documents that match the query.

# "someField" is the property we are indexing so queries performed on that
# field will be faster
db.collection.createIndex({<someField>: 1})

#e.g.
db.collection.createIndex({profession: 1})
db.collection.find({profession: "Software Engineer"})

#NOTE: the property being indexed is not between quotes

multi key (compound) index on collection for fast document lookups

compound indexes

# "someField" is the property we are indexing so queries performed on that
# field will be faster
db.collection.createIndex({<someField>: 1, <someOtherField>: 1})

#e.g.
db.collection.createIndex({item: 1, stock: 1 })
db.collection.find({item: "Software Engineer", stock: "John"})

#NOTE: compound indexes are not just a fast way to create multiple indexes at once
#In the example above, both keys "item" and "stock" are indexed. So whether you use
#them combined or individually the lookup will be fast. However, "The order of the fields listed
#in a compound index is important. The index will contain references to documents sorted first by
#the values of the 'item' field and, within each value of the 'item' field, sorted by values of
#the 'stock' field. See Sort Order for more information."

As an exercise, perform a query on a collection before indexing it and after indexing it. In both cases, use the .explain("executionStats") on the query and check the property totalDocsExamined. The indexed collection should return the count of documents that match the query while the count for the unindexed collection will be the total number documents in that collection.

UPDATE

collection.update()

update operators

#this update operation replaces the whole document of the matched query ("name" is replaced with "age")
collection.update({
  name: "JD"
},
{
  age: 23
})

#this update operation updates the first matched entry (adds new property to the document)
# works similarly to updateOne
collection.update({
  name: "JD"
},
{ $set: {
    age: 23
  }
})

collection.update() $addToSet

This operator adds items to a set (array) that already exists

#this update operation replaces the whole document of the matched query
collection.updateOne({
  name: "JD"
},
{
  $addToSet: {
    "profession.pastCompanies": "Microsoft"
  }
})

collection.updateOne(): $set

Finds a document by specified identifier and updates it. If there are multiple documents that match the identifier (say, {name: "Joe"}) mongo will only update the first document found.

We can update an existing field or add a new field

e.g.

#this query finds a person whose name is "JD" and adds an "age" field to the document
collection.updateOne({
  name: "JD"
},
{
  $set: {
    age: 34
  }
})

#this query finds a person whose name is "JD" and updates his name to "John Doe"
collection.updateOne({
  name: "JD"
},
{
  $set: {
    name: "John Doe"
  }
})

#this query finds a person whose name is "JD" and sets the prizes array to the document
collection.updateOne({
  name: "JD"
},
{
  $set: {
    prizes: [
      {
        name: "Best developer",
        amount: 1000
      },
      {
        name: "Best App",
        amount: 500
      },

    ]
  }
})

collection.updateOne(): $inc

#experience will be incremented by 3
collection.updateOne({
  name: "JD"
},
{
  $inc: {
    experience: 3, #also takes negative numbers to decrement
  }
})

collection.updateOne() with positional operator: $

To update the first element that matches the query in an array, use the positional $ operator if you do not know the position of the element in the array:

The first part of the query needs to have the unique identifier, the key pointing to the array, and the value that you want to update in that array.

#the first instance of "IBM" within "profession.pastCompanies" array will be updated to "LinkedIn"
collection.updateOne({
  name: "JD", "profession.pastCompanies": "IBM"
},
{
  $set: { 
    "profession.pastCompanies.$" : "LinkedIn" 
  } 
})

_collection.updateOne() with push:$push

Creates a new array with item passed into it (single or object)

#this query creates a new key (family) whose value is an array whose first element is the "family" object below

collections.updateOne(
   { name: "JD" },
   {
     $push: {
       family: {
        mother: "Jen",
        father: "Trevor",
        wife: "Margaret",
        firstSon: "Jonathan",
        firstDaugher: "Maria"
       }
     }
   }
)


#so

{
  id: ObjectId("11823081237"),
  name: "JD"
  profession: {
    title: "web engineer",
    experience: 12,
    currentCompany: {
      name: "Full Sail",
      field: "education"
    },
    pastCompanies: [
      "Amazon",
      "Google",
      "IBM"
    ]
  },
  family: [
    {
      mother: "Jen",
      father: "Trevor",
      wife: "Margaret",
      firstSon: "Jonathan",
      firstDaugher: "Maria"
    }
  ]
},

_collection.updateOne() with push with $each:$push $each

To push several individual objects at once we need to use the $each operator, otherwise all the objects would be inserted as the first element of the array

collections.updateOne(
 { name: "JD" },
 {
  $push: {
    friends: {
      $each: [
        {
          firstName: "Mark",
          lastName: "Smith",
        },
        {
          firstName: "Jeff",
          lastName: "Malone",
        },        
        {
          firstName: "Tara",
          lastName: "Sandbanks",
        }
      ]
    }      
  }
})

collection.updateOne() upsert

The upsert:true key/pair can be passed as the third argument to the update query. On an update operation, if a query matches a document in the collection, that document (or the part specified) will be replaced with the one passed to $set. If the query doesn't match any document in the collection but the query has upsert:true, the document will be inserted.

let Mark = {
  firstName: "Mark",
  lastName: "Twain"
};


collection.updateOne({
    name: "Mark" #doesn't exist in our Person collection
  },
  {
    $set: Mark
  },  
  {
    upsert: true
  },
})

collection.updateMany() updateMany

updateMany() works very similarly to updateOne(), except that it updates all documents that are matched in the query.

Below we $unset (remove) the field "salary" from all documents where "salary" is null

collection.updateMany({
  salary: null
  }, {
  $unset: {
    salary: "" #this value doesn't really matter 
  }
})

collection.replaceOne() replaceOne

Replaces the entire document matched on the query

let JD = {
  firstName: "Johnny",
  lastName: "Damon"
};

collection.replaceOne({
  name: "JD"
},
  JD
)

DELETE

collection.deleteOne() delete

Specify document to be deleted by unique identifier

#deletes this particular entry from db
collection.deleteOne({_id: Object("123455123412346")})

collection.deleteMany() Specify identifier that is shared among several documents

#deletes all entries in this collection
collection.deleteMany({}) # can specify filter 

#OR 

db.users.remove({}) # can specify filter 

#OR

db.users.drop()


#NOTE: For the most part these methods are interchegeable but there are differences in 
#performance and use cases for each. Look at the docs to choose the one that best applies for 
#what you are trying to accomplish.


#deletes all entries in this collection whose "userBlogId" is 12345 (meaning all blogs written 
#by this user)
collection.deleteMany({userBlogId: 12345})

#OR 

db.users.remove({userBlogId: 12345})

QUERY OPERATORS https://docs.mongodb.com/manual/reference/operator/query/

Below are just some of the existing query operators. Check the link above for more.

$gt, $gte, $lt, $lte

Greater than, greater than or equal to, less than, less than or equal to

Query operators are used as the first argument to the query

#the query below returns salaries that are greater than or equal to 3000 and less than or equal to 4000
collections.find({salary: {$gte: 3000, $lte: 4000}}) 


#this query to our video db returns the titles, actors, year, and imdb rating of movies whose imdb rating is greater than 7
#connect to atlas cluster, then:
use video
db.movieDetails.find({"imdb.rating": {$gte: 7}}, {title: 1, actors: 1, year:1, _id: 0, "imdb.rating": 1}).pretty()

$in

Matches any of the values specified in an array.

#this query to our video db returns the titles, actors, year, and imdb rating of movies whose imdb rating is either 7, 7.5, or 8
#connect to atlas cluster, then:
use video
db.movieDetails.find({"imdb.rating": {$in: [7, 7.5, 8]}}, {title: 1, actors: 1, year:1, _id: 0, "imdb.rating": 1}).pretty()

#this finds all movies whose cast includes "Jack Nicholson" or "John Huston", viewerRating greater than 7, and mpaRating === "R"
db.movies.find({cast: {$in: ["Jack Nicholson", "John Huston"]}, viewerRating: {$gt: 7}, mpaaRating: "R"}).pretty()

$nin

Does the opposite of $in (finds documents that do not include items in array).

elements operators https://docs.mongodb.com/manual/reference/operator/query-element/

#$exists: matches documents that have the specified field.
#$type: Selects documents if a field is of the specified type.

#check "m001_element_operators.js" file

db.moviesDetails.find({mpaaRating: {$exists: true}})

db.moviesDetails.find({mpaaRating: {$exists: false}})

#matches documents whose "mpaaRating" is set to "null" and those that don't have 
# the "mpaaRating" field at all
db.movieDetails.find({mpaaRating: null})

db.movieDetails.find({"imdb.rating": {$type: "int"}}).pretty()

logical operators https://docs.mongodb.com/manual/reference/operator/query-logical/index.html

#check "m001_logical_operators.js" file

array operators https://docs.mongodb.com/manual/reference/operator/query-array/

$all

#used to filter documents whose arrays contain all listed items regardless of the order they are listed
db.movieDetails.find({genres: {$all: ["Comedy", "Crime", "Drama"]}},{_id: 0, title: 1, genres: 1}).pretty()

#NOTE
#in this search the order matters
db.movieDetails.find({genres:["Comedy", "Crime", "Drama"]},{_id: 0, title: 1, genres: 1}).pretty()

$size

#used to filter documents whose arrays size matches the one provided in the query
db.movieDetails.find({genres: {$size: 3}},{_id: 0, title: 1, genres: 1}).pretty()

$elemMatch

#The $elemMatch operator matches documents that contain an array field with at least one element 
#that matches all the specified query criteria.

db.movieDetails.find({boxOffice: {$elemMatch: {"country": "Germany", "revenue": {$gt: 16}}}})


#NOTE
#in a query such as the one below, the two elements specified in the
#filter DO NOT need to match in the same array element:
#one filter could match document x and the other could match in document y
#so the query below would match our boxOffice array even though there is no 
#single document where the country is Germany AND the revenue is greater than 17

db.movieDetails.find({"boxOffice.country": "Germany", "boxOffice.revenue": {$gt: 17}}).pretty()

$regex

#the regex matches all documents whose "awards.text" field starts with "Won" and is followed by a space and any other character repeated any number of times

db.movieDetails.find({"awards.text": {$regex: /^Won .* /}}, {_id: 0, title: 1, "awards.text": 1}).pretty()

import documents on the shell import

Should be avoided on production

All import operations have to happen outside of the shell

Use mongoimport --help to see possible flags to be passed to this command

e.g.

#cd into the directory that has the file containing the data you want to import and
#launch mongo shell from that directory then:

# for a simple document
$ mongoimport --db <dbName> --collection <collectionName> --file contacts.json

# for an array of documents
$ mongoimport --db <dbName> --collection <collectionName> --jsonArray --file <fileName>.json

rayhankinan/demo-mongodb-PAT

mongodb tutorial https://docs.mongodb.com/manual/introduction/

get MongoDB Compass (or robomongo)

Robomongo https://robomongo.org/

Mongo Atlas https://www.mongodb.com/cloud

Create DB and insert data (check mongo methods on "loadMovieDetailsDataset.js")

Common Shell Commands https://docs.mongodb.com/manual/reference/mongo-shell/

Common Methods https://docs.mongodb.com/manual/reference/mongo-shell/

CREATE

READ

UPDATE

DELETE

QUERY OPERATORS https://docs.mongodb.com/manual/reference/operator/query/

$all

$size

$elemMatch

$regex