© Subhashini Chellappan and Dharanitharan Ganesan 2020
S. Chellappan, D. GanesanMongoDB Recipeshttps://doi.org/10.1007/978-1-4842-4891-1_7

7. MongoDB Monitoring and Backup

Subhashini Chellappan1  and Dharanitharan Ganesan2
(1)
Bangalore, India
(2)
Krishnagiri, Tamil Nadu, India
 
In Chapter 6, we discussed transactions in MongoDB. In this chapter, we are going to discuss the following topics:

  • MongoDB monitoring tools.

  • Backup and restore with MongoDB.

MongoDB Monitoring

MongoDB provides various tools to monitor the database. These MongoDB monitoring tools help us to understand what is happening with a database at various levels.

Log File

The first basic tool is the log file.

Recipe 7-1. Working with MongoDB Log Files

In this recipe, we are going to discuss MongoDB log files.

Problem

You want to see the MongoDB log file and set the log level.

Solution

The MongoDB log file is present in the following installation path:
c:Program FilesMongoDBServer4.0log>

Use the db.setLogLevel function to set the log level.

How It Works

Let’s follow the steps in this section to see the log file in mongo shell.

Step 1: To Display the Log File Content and Set the Log Level

Type the following command in the mongo shell .
show logs
Here is the output,
> show logs
global
startupWarnings
You can see the log entries by typing the next command.
show log global

We cannot filter the log messages in the console.

MongoDB has several logging levels, which can be set using the db.setLogLevel function.

The syntax for db.setLogLevel is
db.setLogLevel(<level>, <component>)
Here, level indicates the log verbosity level, which ranges from 0 to 5. The default log level is 0, which includes informational messages. Levels 1 to 5 increase the verbosity level to include debug messages. The second argument, component, is optional. This refers to the name of the component for which you want to set the verbosity level. The component name corresponds to the <name> from the corresponding systemLog.component.<name>.verbosity setting:
accessControl
command
control
ftdc
geo
index
network
query
replication
recovery
sharding
storage
storage.journal
transaction
write
To set the log level to 2 on the topic query, issue the following command.
db.setLogLevel(2,"query")
Here is the output,
> db.setLogLevel(2,"query")
{
        "was" : {
                "verbosity" : 0,
                "accessControl" : {
                        "verbosity" : -1
                },
                "command" : {
                        "verbosity" : -1
                },
                "control" : {
                        "verbosity" : -1
                },
                "executor" : {
                        "verbosity" : -1
                },
                "geo" : {
                        "verbosity" : -1
                },
                "index" : {
                        "verbosity" : -1
                },
                "network" : {
                        "verbosity" : -1,
                        "asio" : {
                                "verbosity" : -1
                        },
                        "bridge" : {
                                "verbosity" : -1
                        }
                },
                "query" : {
                        "verbosity" : -1
                },
                "replication" : {
                        "verbosity" : -1,
                        "heartbeats" : {
                                "verbosity" : -1
                        },
                        "rollback" : {
                                "verbosity" : -1
                        }
                },
                "sharding" : {
                        "verbosity" : -1
                },
                "storage" : {
                        "verbosity" : -1,
                        "recovery" : {
                                "verbosity" : -1
                        },
                        "journal" : {
                                "verbosity" : -1
                        }
                },
                "write" : {
                        "verbosity" : -1
                },
                "ftdc" : {
                        "verbosity" : -1
                },
                "tracking" : {
                        "verbosity" : -1
                },
                "transaction" : {
                        "verbosity" : -1
                }
        },
        "ok" : 1
}

MongoDB echoes back the previous configuration before the new setting was applied. At the top, you can see the default level, which indicates that any components that are not set will log at this level. Here, it will override only the query component. The -1 verbosity indicates that it inherits the default level from its parent.

To see the current level, issue the following command.
db. getLogComponents
Here is the output,
> db.getLogComponents()
{
        "verbosity" : 0,
        "accessControl" : {
                "verbosity" : -1
        },
        "command" : {
                "verbosity" : -1
        },
        "control" : {
                "verbosity" : -1
        },
        "executor" : {
                "verbosity" : -1
        },
        "geo" : {
                "verbosity" : -1
        },
        "index" : {
                "verbosity" : -1
        },
        "network" : {
                "verbosity" : -1,
                "asio" : {
                        "verbosity" : -1
                },
                "bridge" : {
                        "verbosity" : -1
                }
        },
        "query" : {
                "verbosity" : 2
        },
        "replication" : {
                "verbosity" : -1,
                "heartbeats" : {
                        "verbosity" : -1
                },
                "rollback" : {
                        "verbosity" : -1
                }
        },
        "sharding" : {
                "verbosity" : -1
        },
        "storage" : {
                "verbosity" : -1,
                "recovery" : {
                        "verbosity" : -1
                },
                "journal" : {
                        "verbosity" : -1
                }
        },
        "write" : {
                "verbosity" : -1
        },
        "ftdc" : {
                "verbosity" : -1
        },
        "tracking" : {
                "verbosity" : -1
        },
        "transaction" : {
                "verbosity" : -1
        }
}
>

Here, the query level is set to 2.

Now, issue the following query, which we’ll use to add an entry to the log file:
> db.demos.find()
>

This query will not return any results because there is no collection named demos in the test database.

Now, issue this command to see the log file.
> show log global
Here are the last few lines of log file output,
2019-02-17T16:08:43.681+0530 D QUERY    [conn1] Collection test.demos does not exist. Using EOF plan: query: {} sort: {} projection: {}
2019-02-17T16:09:03.805+0530 D QUERY    [conn1] Collection test.demos does not exist. Using EOF plan: query: {} sort: {} projection: {}
2019-02-17T16:09:31.633+0530 D QUERY    [LogicalSessionCacheRefresh] Using idhack: { _id: { id: UUID("9d7def64-8b49-4c85-9392-01f7ab88e019"), uid: BinData(0, E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855) } }

You can see some additional information about the query.

Now set the query log level as -1:
> db.setLogLevel(-1,"query")
Then, issue the following query and observe the details in the log file as explained further later.
> db.demos.find({y:1})
Now check the log file by using the following command.
> show log global

Now check the end of the log file: We cannot see the extra logging information for this find query.

Use db.setLogLevel(0) to turn off the logging level.

If we do not specify the option as query or write or anything, it will be applied for all options.

The log level 5 is very detailed; levels 1 to 3 are more useful options and these are recommended unless you need all of the detailed information provided by the highest log level of 5.

Also, setting the appropriate log level helps us to improve the performance of queries by identifying the query plans logged in the log file and also by identifying the slow operations by checking the time taken to execute a query.

Note

Refer to Recipe 7-10 to learn more about MongoDB query plans.

MongoDB Performance

It is mandatory to analyze the performance of the database when we develop new applications with MongoDB. Performance degradation could happen due to hardware availability issues, number of open database connections, and database access strategies.

Performance issues indicate the database is operating at full capacity and abnormal traffic load due to an increase in the number of connections.

The database profiler can help us to understand the various operations performed on the database that cause performance degradation.

Database Profiler

The database profiler is used to collect information about the commands that are executed against a running mongod instance. The database profiler writes all the collected data to the system.profile capped collection in the admin database.

MongoDB also has a Performance Advisor tool that automatically monitors slow queries and suggests new indexes to improve query performance. A query is considered to be slow if it takes more than 100 milliseconds to execute.

Note

The Performance Advisor tool is available in the MongoDB Atlas cluster. MongoDB Atlas is the global cloud database service for modern applications.

The profiler is turned off by default. It can be enabled on a per-database or per-instance basis at any one of the profiling levels mentioned in Table 7-1.
Table 7-1

Profiling Levels

Level

Description

0

The profiler is off and does not collect any data. This is the default profiler level.

1

The profiler collects data for operations that take longer than the value of the slowms parameter.

2

The profiler collects data for all operations.

Recipe 7-2. Working with Database Profiler

In this recipe, we are going to discuss how to enable database profiler and how to set the profiling level.

Problem

You want to enable database profiling with a profile level.

Solution

Use the db.setProfilingLevel() helper in the mongo shell.

How It Works

Let’s follow the steps in this section to enable the database profiler.

Step 1: Enable and Configure Database Profiler

To enable the database profiler to collect all database information use the following command.
db.setProfilingLevel(2)
Here is the output,
> db.setProfilingLevel(2)
{ "was" : 0, "slowms" : 100, "sampleRate" : 1, "ok" : 1 }

was is the key/value pair indicating the previous level of profiling.

You can specify the threshold for slow operations using the next command.
db.setProfilingLevel(1, { slowms: 30 })
Here is the output,
> db.setProfilingLevel(1, { slowms: 30 })
{ "was" : 2, "slowms" : 100, "sampleRate" : 1, "ok" : 1 }

Here, the slow operation threshold is 30 milliseconds.

To profile a random sample of slow operations, use the following command.
db.setProfilingLevel(1, { sampleRate: 0.53 })

That code sets the profiler to sample 53% of all slow operations.

Note

The default threshold value for the slow operation is 100 milliseconds.

You can get the profiling level by issuing the following command.
db.getProfilingStatus()
Here is the output,
> db.getProfilingStatus()
{ "was" : 1, "slowms" : 30, "sampleRate" : 1 }

The was field indicates the current profiling level.

To disable the database profiler, use this command.
db.setProfilingLevel(0)
Here is the output,
> db.setProfilingLevel(0)
{ "was" : 1, "slowms" : 30, "sampleRate" : 1, "ok" : 1 }
To enable profiling for an entire mongod instance, pass the following option to the mongod instance at the time of starting it.
mongod --profile 1 --slowms 20 --slowOpSampleRate 0.5

This command sets the profiling level to 1, the slow operation threshold to 20 milliseconds, and it profiles only 50% of the slow operations.

Recipe 7-3. View Database Profiler

In this recipe, we are going to discuss how to view database profiler.

Problem

You want to view database profiler information.

Solution

Use db.system.profile.find() in the mongo shell.

How It Works

Let’s follow the steps in this section to view database profiler information.

Step 1: Enable and Configure Database Profiler

The database profiler logs information in the system.profile collection. You need to query system.collection to view profiling information.

Create an employee collection in the example database as shown here.
> use example;
switched to db example
> db.setProfilingLevel(2)
{ "was" : 0, "slowms" : 100, "sampleRate" : 1, "ok" : 1 }
> show collections
> db.employee.insert({_id:1001,name:"Subhashini"})
WriteResult({ "nInserted" : 1 })
> db.employee.insert({_id:1001,name:"Shobana"})
WriteResult({
        "nInserted" : 0,
        "writeError" : {
                "code" : 11000,
                "errmsg" : "E11000 duplicate key error collection: example.employee index: _id_ dup key: { : 1001.0 }"
        }
})
> db.employee.find()
{ "_id" : 1001, "name" : "Subhashini" }
The profiler stores details in their respective keys as follows:

  • op: Component type (i.e., query or command).

  • ts: Timestamp.

  • ns: Collection details where the query is executed.

These options can be used to filter or sort as in the queries that follow.

To return operations performed on the employee collection, issue the following command.
> db.system.profile.find( { ns : 'example.employee' } ).pretty()
Here is the output,
{
        "op" : "insert",
        "ns" : "example.employee",
        "command" : {
                "insert" : "employee",
                "ordered" : true,
                "lsid" : {
                        "id" : UUID("93b33e8d-f561-46a1-8310-0f62d31442f1")
                },
                "$db" : "example"
        },
        "ninserted" : 1,
        "keysInserted" : 1,
        "numYield" : 0,
        "locks" : {
                "Global" : {
                        "acquireCount" : {
                                "r" : NumberLong(3),
                                "w" : NumberLong(3)
                        }
                },
                "Database" : {
                        "acquireCount" : {
                                "w" : NumberLong(2),
                                "W" : NumberLong(1)
                        }
                },
                "Collection" : {
                        "acquireCount" : {
                                "w" : NumberLong(2)
                        }
                }
        },
        "responseLength" : 45,
        "protocol" : "op_msg",
        "millis" : 21,
        "ts" : ISODate("2019-01-14T06:08:38.838Z"),
        "client" : "127.0.0.1",
        "appName" : "MongoDB Shell",
        "allUsers" : [ ],
        "user" : ""
}
{
        "op" : "insert",
        "ns" : "example.employee",
        "command" : {
                "insert" : "employee",
                "ordered" : true,
                "lsid" : {
                        "id" : UUID("93b33e8d-f561-46a1-8310-0f62d31442f1")
                },
                "$db" : "example"
        },
        "ninserted" : 0,
        "keysInserted" : 0,
        "numYield" : 0,
        "locks" : {
                "Global" : {
                        "acquireCount" : {
                                "r" : NumberLong(1),
                                "w" : NumberLong(1)
                        }
                },
                "Database" : {
                        "acquireCount" : {
                                "w" : NumberLong(1)
                        }
                },
                "Collection" : {
                        "acquireCount" : {
                                "w" : NumberLong(1)
                        }
                }
        },
        "responseLength" : 194,
        "protocol" : "op_msg",
        "millis" : 2,
        "ts" : ISODate("2019-01-14T06:08:50.297Z"),
        "client" : "127.0.0.1",
        "appName" : "MongoDB Shell",
        "allUsers" : [ ],
        "user" : ""
}
{
        "op" : "query",
        "ns" : "example.employee",
        "command" : {
                "find" : "employee",
                "filter" : {
                },
                "lsid" : {
                        "id" : UUID("93b33e8d-f561-46a1-8310-0f62d31442f1")
                },
                "$db" : "example"
        },
        "keysExamined" : 0,
        "docsExamined" : 1,
        "cursorExhausted" : true,
        "numYield" : 0,
        "nreturned" : 1,
        "locks" : {
                "Global" : {
                        "acquireCount" : {
                                "r" : NumberLong(1)
                        }
                },
                "Database" : {
                        "acquireCount" : {
                                "r" : NumberLong(1)
                        }
                },
                "Collection" : {
                        "acquireCount" : {
                                "r" : NumberLong(1)
                        }
                }
        },
        "responseLength" : 147,
        "protocol" : "op_msg",
        "millis" : 0,
        "planSummary" : "COLLSCAN",
        "execStats" : {
                "stage" : "COLLSCAN",
                "nReturned" : 1,
                "executionTimeMillisEstimate" : 0,
                "works" : 3,
                "advanced" : 1,
                "needTime" : 1,
                "needYield" : 0,
                "saveState" : 0,
                "restoreState" : 0,
                "isEOF" : 1,
                "invalidates" : 0,
                "direction" : "forward",
                "docsExamined" : 1
        },
        "ts" : ISODate("2019-01-14T06:22:07.873Z"),
        "client" : "127.0.0.1",
        "appName" : "MongoDB Shell",
        "allUsers" : [ ],
        "user" : ""
}
>
To return the most recent five log entries, issue the following command.
> db.system.profile.find().limit(5).sort( { ts : -1 } ).pretty()

Note

ts specifies the timestamp and the value -1 specifies to sort in descending order. We can specify 1 for ascending or -1 for descending order.

To return operations slower than 5 milliseconds, use this syntax.
> db.system.profile.find( { millis : { $gt : 5 } } ).pretty()

mongostat

mongostat is a monitoring utility that provides information about a mongod instance if you are working on a single mongod instance. If you are working on a shared cluster, it shows information about a mongos instance.

Recipe 7-4. Working with mongostat

In this recipe, we are going to discuss how to use mongostat to monitor the MongoDB server.

Problem

You want to see the details of a MongoDB server.

Solution

Use the mongostat tool, which is available in the installation directory. Usually the installation directory is C:Program FilesMongoDBServer4.0in or the custom path specified during your installation process.

Avoid navigating to the directory each time to use the mongostat tool and add the directory path to your PATH environment variable.

How It Works

Let’s follow the steps in this section to view server information using mongostat.

Step 1: mongostat Command

Issue the mongostat command by specifying the hostname and port of MongoDB server.

Here is the output,
c:Program FilesMongoDBServer4.0in>mongostat --host localhost:27017
insert query update delete getmore command dirty used flushes vsize   res qrw arw net_in net_out conn                time
    *0    *0     *0     *0       0   146|0  0.0% 0.0%       0 4.97G 28.0M 0|0 1|0  11.6k   4.49m    2 Feb 17 16:59:19.131
    *0    *0     *0     *0       0     2|0  0.0% 0.0%       0 4.97G 28.0M 0|0 1|0   163b   63.6k    2 Feb 17 16:59:20.095
    *0    *0     *0     *0       0     2|0  0.0% 0.0%       0 4.97G 28.0M 0|0 1|0   158b   61.7k    2 Feb 17 16:59:21.090
    *0    *0     *0     *0       0     1|0  0.0% 0.0%       0 4.97G 28.0M 0|0 1|0   157b   61.3k    2 Feb 17 16:59:22.090
    *0    *0     *0     *0       0     1|0  0.0% 0.0%       0 4.97G 28.0M 0|0 1|0   157b   61.2k    2 Feb 17 16:59:23.093
    *0    *0     *0     *0       0     2|0  0.0% 0.0%       0 4.97G 28.0M 0|0 1|0   158b   61.4k    2 Feb 17 16:59:24.092
    *0    *0     *0     *0       0     2|0  0.0% 0.0%       0 4.97G 28.0M 0|0 1|0   159b   62.1k    2 Feb 17 16:59:25.080
    *0    *0     *0     *0       0     1|0  0.0% 0.0%       0 4.97G 28.0M 0|0 1|0   157b   61.2k    2 Feb 17 16:59:26.082
    *0    *0     *0     *0       0     1|0  0.0% 0.0%       0 4.97G 28.0M 0|0 1|0   157b   61.0k    2 Feb 17 16:59:27.088
    *0    *0     *0     *0       0     2|0  0.0% 0.0%       0 4.97G 28.0M 0|0 1|0   158b   61.6k    2 Feb 17 16:59:28.083

Note

27017 is the default port. We can simply use mongostat without any options if we want to connect and get statistics of localhost and default port 27017. To connect a different or remote instance, specify the option -host as shown.

You can see that there are multiple columns in the output.

  • insert/query/update/delete: These columns show the number of insert, query, update, and delete operations per second.

  • getmore: This column shows the number of times the getmore operation is executed in one second.

  • command: This column shows the number of commands executed on the server in one second.

  • flushes: This column shows the number of times data was flushed to disk in one second.

  • mapped: This column shows the amount of memory used by the mongo process against a database. It is the same as the size of the database.

  • vsize (virtual size): This column represents virtual memory allocated to the entire mongod process.

  • res (resident memory): This column represents the physical memory used by MongoDB.

  • faults: This column shows the number of Linux page faults per second.

  • qr|qw: This column shows queued-up reads and writes that are waiting for the chance to be executed.

  • ar|aw: This column shows number of active clients.

  • netIn and netOut: These columns show the network traffic in and out of the MongoDB server within a given time frame.

  • conn: This column shows the number of open connections.

  • time: This column shows the time frame in which operations are performed.

mongotop

mongotop provides a method to track the amount of time spent by the mongod process during reads and writes. mongotop provides statistics on a per-collection level.

Recipe 7-5. Working with mongotop

In this recipe, we are going to discuss how to use mongotop to track the amount of time spent by the mongod process during reads and writes.

Problem

You want to see the amount of time spent by the mongod process during reads and writes.

Solution

Issue the mongotop command in the installation path of MongoDB.

How It Works

Let’s follow the steps in this section to view the amount of time spent during reads and writes.

Step 1: mongotop Command

Issue the following command.
c:Program FilesMongoDBServer4.0in>mongotop
2019-02-17T21:51:25.975+0530    connected to: 127.0.0.1
                      ns    total    read    write    2019-02-17T21:51:26+05:30
      admin.system.roles      0ms     0ms      0ms
    admin.system.version      0ms     0ms      0ms
  config.system.sessions      0ms     0ms      0ms
            emp.employee      0ms     0ms      0ms
         employee.author      0ms     0ms      0ms
        employee.authors      0ms     0ms      0ms
     employee.categories      0ms     0ms      0ms
       employee.employee      0ms     0ms      0ms
employee.employeedetails      0ms     0ms      0ms
          employee.names      0ms     0ms      0ms

db.stats()

db.stats() provides the statistics of a single database.

Recipe 7-6. Working with db.stats()

In this recipe, we are going to discuss how to get the statistics for a single database.

Problem

You want to see the disk and memory usage estimates for a database.

Solution

Use the db.stats command to display the statistics for a database.

How It Works

Let’s follow the steps in this section to view the statistics for the database.

Step 1: db.stats() Command

Issue the following commands to display the statistics for a database named employee.
> use employee
switched to db employee
> db.stats()
{
        "db" : "employee",
        "collections" : 12,
        "views" : 0,
        "objects" : 40,
        "avgObjSize" : 90.875,
        "dataSize" : 3635,
        "storageSize" : 401408,
        "numExtents" : 0,
        "indexes" : 12,
        "indexSize" : 212992,
        "fsUsedSize" : 208018800640,
        "fsTotalSize" : 509929914368,
        "ok" : 1
}
>
The details such as data size and storage size are given in bytes. To see the information in megabytes, issue the following command.
> db.stats(1000000)
{
        "db" : "employee",
        "collections" : 12,
        "views" : 0,
        "objects" : 40,
        "avgObjSize" : 90.875,
        "dataSize" : 0.003635,
        "storageSize" : 0.401408,
        "numExtents" : 0,
        "indexes" : 12,
        "indexSize" : 0.212992,
        "fsUsedSize" : 208020.402176,
        "fsTotalSize" : 509929.914368,
        "ok" : 1
}
>

db.serverStatus()

db.serverStatus() returns a document that provides statistics for the state of a database process.

Recipe 7-7. Working with db.serverStatus()

In this recipe, we are going to discuss the db.serverStatus() command .

Problem

You want to see the memory status of a database.

Solution

Use the db.serverStatus() command .

How It Works

Let’s follow the steps in this section to view the statistics for the memory status of a database.

Step 1: db.serverStatus() Command

Issue following command to view the memory statistics for a database.
> db.serverStatus().mem
{
        "bits" : 64,
        "resident" : 85,
        "virtual" : 5089,
        "supported" : true,
        "mapped" : 0,
        "mappedWithJournal" : 0
}

Backup and Restore with MongoDB Tools

MongoDB provides mongodump and mongorestore utilities to work with BSON data dumps. These utilities are useful for creating backups for small deployments. To create resilient and nondisruptive backups for large deployments, we can use file system or block-level disk snapshot methods.

We should use a system-level tool to create a copy of the device file system that holds MongoDB files. The file system snaphots or the disk-level snapshot backup method require additional system configuration outside of MongoDB, so we do not cover this in depth in this book.

When deploying MongoDB in production, we should have a backup and failover strategy for capturing and restoring backups in the case of data loss events.

Recipe 7-8. Working with mongodump

In this recipe, we are going to discuss how to back up data using mongodump.

Problem

You want to back up data using mongodump.

Solution

Use the mongodump command.

How It Works

Let’s follow the steps in this section to back up data.

The mongodump utility makes a backup by connecting to a running mongod or mongos instance.

When you run the mongodump command without specifying any arguments, it connects to the mongodb running on localhost on port 27017 and creates a database backup named dump in the current directory.

Let us discuss how to issue the mongodump command to create a backup.
  1. 1.

    Ensure mongod is running.

     
  2. 2.

    Open a command prompt with administrator privileges, navigate to the mongodb installation folder, and type mongodump as shown here.

    c:Program FilesMongoDBServer4.0in>mongodump
     
  3. 3.

    You can see the dump folder in the current directory as shown in Figure 7-1.

     
../images/475461_1_En_7_Chapter/475461_1_En_7_Fig1_HTML.jpg
Figure 7-1

MongoDB installation folder

You can specify the output directory by issuing the following command.
mkdir "c:Program FilesMongoDBServer4.0dump"
mongodump --out "c:Program FilesMongoDBServer"4.0dump
To create a backup of a collection, issue the following command.
mongodump --collection employee --db example --out "c:Program FilesMongoDBServer"4.0ackup
Here is the output,
c:Program FilesMongoDBServer4.0in>mongodump --collection employee --db example --out "c:Program FilesMongoDBServer"4.0ackup
2019-01-14T18:36:28.460+0530    writing example.employee to
2019-01-14T18:36:28.465+0530    done dumping example.employee (1 document)
You can make a backup from a remote host by specifying host and port arguments as shown here.
mongodump --host sample.net --port 5017 --username user --password "pass"

Automatic and Regular Backup Scheduling Using mongodump

It is good practice to perform a regular backup of all databases in case of any hard failover. Follow these steps to perform a regular backup at a specific interval.
  1. 1.

    We can create scripts to create the backup directory for the current date and time and export the database to the same directory.

    For Windows, create a bat (batch script) file with the commands shown in Figure 7-2.

     
../images/475461_1_En_7_Chapter/475461_1_En_7_Fig2_HTML.png
Figure 7-2

Creating a batch script file in Windows

For Unix/Linux machines, create a shell script file with the commands shown in Figure 7-3.
../images/475461_1_En_7_Chapter/475461_1_En_7_Fig3_HTML.png
Figure 7-3

Creating a shell script file in Unix/Linux

  1. 2.

    Schedule the script file to execute at a certain interval. Use task scheduler or any other scheduler for Windows, and use the CRON schedule for Unix/Linux.

     

Recipe 7-9. Working with mongorestore

In this recipe, we are going to discuss how to restore data using mongorestore.

Problem

You want to restore data using mongorestore .

Solution

Use the mongorestore command.

How It Works

Let’s follow the steps in this section to restore data.

Step 1: Restore Data Using mongorestore

To restore data, open a command prompt and navigate to the mongodb installation folder, then issue the following command.
c:Program FilesMongoDBServer4.0in>mongorestore dump/
Here, mongorestore imports the data present in the dump directory to the running mongod instance. By default, mongorestore looks for a database backup in the dump/ directory . We can connect to a different port of the active mongod instance and a different backup path by using this command.
mongorestore --port <port number> <path to the backup>

Step 2: Restore Data Using mongorestore to Remote Instances

By default, mongorestore connects to running instance on localhost and default port number 27017. To restore to a different host and different port, we can specify the same using the --host and --port options as shown here.
mongorestore --host <example.net> --port <myportNumber> --username myuser --password 'mypass' /backup/mongodumpfile

Specify the username and password using options --user and --password only if the remote mongod instance requires authentication.

Recipe 7-10. Working with mongodb Query Plans

In this recipe, we are going to discuss query plans to understand the MongoDB query execution details with query planner and query optimizer.

Problem

You want to understand the query execution plan.

Solution

Use the cursor.explain() command.

How It Works

Let’s follow the steps in this section to understand the query plan.

Step 1: Using explain() to Understand the Query Plan

To see the query plan during the MongoDB command execution, use cursor.explain(). Let’s assume the employee collection is available in the authors database with the data shown here.
MongoDB Enterprise > use authors
switched to db authors
MongoDB Enterprise > db.employee.find()
{ "_id" : ObjectId("5d50ed688dcf280c50fde439"), "empId" : 1, "empName" : "John", "state" : "KA", "country" : "India" }
{ "_id" : ObjectId("5d50ed688dcf280c50fde43a"), "empId" : 2, "empName" : "Smith", "state" : "CA", "country" : "US" }
{ "_id" : ObjectId("5d50ed688dcf280c50fde43b"), "empId" : 3, "empName" : "James", "state" : "FL", "country" : "US" }
{ "_id" : ObjectId("5d50ed688dcf280c50fde43c"), "empId" : 4, "empName" : "Josh", "state" : "TN", "country" : "India" }
{ "_id" : ObjectId("5d50ed688dcf280c50fde43d"), "empId" : 5, "empName" : "Joshi", "state" : "HYD", "country" : "India" }
To get the query plan while executing the command db.employee.find(), use .explain() as shown here.
MongoDB Enterprise > db.employee.find().explain()
Here is the output,
MongoDB Enterprise > db.employee.find().explain()
{
        "queryPlanner" : {
                "plannerVersion" : 1,
                "namespace" : "authors.employee",
                "indexFilterSet" : false,
                "parsedQuery" : {
                },
                "winningPlan" : {
                        "stage" : "COLLSCAN",
                        "direction" : "forward"
                },
                "rejectedPlans" : [ ]
        },
        "serverInfo" : {
                "host" : "DESKTOP-MEISTBV",
                "port" : 27017,
                "version" : "4.0.11",
                "gitVersion" : "417d1a712e9f040d54beca8e4943edce218e9a8c"
        },
        "ok" : 1
}

Observe in this output that there is no parsedQuery in the query planner, and the winning plan says that it involves the complete collection scan.

Now, let’s try to add a filter to the same query and observe the query plan.
MongoDB Enterprise > db.employee.find({country:"India"})
{ "_id" : ObjectId("5d50ed688dcf280c50fde439"), "empId" : 1, "empName" : "John", "state" : "KA", "country" : "India" }
{ "_id" : ObjectId("5d50ed688dcf280c50fde43c"), "empId" : 4, "empName" : "Josh", "state" : "TN", "country" : "India" }
{ "_id" : ObjectId("5d50ed688dcf280c50fde43d"), "empId" : 5, "empName" : "Joshi", "state" : "HYD", "country" : "India" }
We have added the filter condition to retrieve only country:India. Now check the query plan for the same query with the filter condition.
MongoDB Enterprise> db.employee.find({country:"India"}).explain()
Here is the output,
MongoDB Enterprise > db.employee.find({country:"India"}).explain()
{
        "queryPlanner" : {
                "plannerVersion" : 1,
                "namespace" : "authors.employee",
                "indexFilterSet" : false,
                "parsedQuery" : {
                        "country" : {
                                "$eq" : "India"
                        }
                },
                "winningPlan" : {
                        "stage" : "COLLSCAN",
                        "filter" : {
                                "country" : {
                                        "$eq" : "India"
                                }
                        },
                        "direction" : "forward"
                },
                "rejectedPlans" : [ ]
        },
        "serverInfo" : {
                "host" : "DESKTOP-MEISTBV",
                "port" : 27017,
                "version" : "4.0.11",
                "gitVersion" : "417d1a712e9f040d54beca8e4943edce218e9a8c"
        },
        "ok" : 1
}
Observe in the preceding output that there is a parsed query added by the query planner as
"parsedQuery" : { "country" : {"$eq" : "India" } }

Step 2: Understanding Different Modes in explain()

The explain() method accepts queryPlanner, executionStats, or allPlansExecution as the operating mode. The default mode is queryPlanner if we don’t specify otherwise.

Use the following command to use explain() with any of these parameters as a mode.
db.employee.find().explain("queryPlanner")
Here is the output,
MongoDB Enterprise > db.employee.find().explain("queryPlanner")
{
        "queryPlanner" : {
                "plannerVersion" : 1,
                "namespace" : "authors.employee",
                "indexFilterSet" : false,
                "parsedQuery" : {
                },
                "winningPlan" : {
                        "stage" : "COLLSCAN",
                        "direction" : "forward"
                },
                "rejectedPlans" : [ ]
        },
        "serverInfo" : {
                "host" : "DESKTOP-MEISTBV",
                "port" : 27017,
                "version" : "4.0.11",
                "gitVersion" : "417d1a712e9f040d54beca8e4943edce218e9a8c"
        },
        "ok" : 1
}
Now use executionStats as a operation mode to the explain() method as shown here.
db.employee.find().explain("executionStats")
Here is the output,
MongoDB Enterprise > db.employee.find().explain("executionStats")
{
        "queryPlanner" : {
                "plannerVersion" : 1,
                "namespace" : "authors.employee",
                "indexFilterSet" : false,
                "parsedQuery" : {
                },
                "winningPlan" : {
                        "stage" : "COLLSCAN",
                        "direction" : "forward"
                },
                "rejectedPlans" : [ ]
        },
        "executionStats" : {
                "executionSuccess" : true,
                "nReturned" : 5,
                "executionTimeMillis" : 0,
                "totalKeysExamined" : 0,
                "totalDocsExamined" : 5,
                "executionStages" : {
                        "stage" : "COLLSCAN",
                        "nReturned" : 5,
                        "executionTimeMillisEstimate" : 0,
                        "works" : 7,
                        "advanced" : 5,
                        "needTime" : 1,
                        "needYield" : 0,
                        "saveState" : 0,
                        "restoreState" : 0,
                        "isEOF" : 1,
                        "invalidates" : 0,
                        "direction" : "forward",
                        "docsExamined" : 5
                }
        },
        "serverInfo" : {
                "host" : "DESKTOP-MEISTBV",
                "port" : 27017,
                "version" : "4.0.11",
                "gitVersion" : "417d1a712e9f040d54beca8e4943edce218e9a8c"
        },
        "ok" : 1
}
MongoDB Enterprise >
In this output, observe the following:

  • "executionSuccess" : true: This shows whether the query is successfully executed or not.

  • "nReturned" : 5: This is the number of rows returned.

  • "executionTimeMillis" : 0: This is the execution time in milliseconds.

If the executionTimeMillis value is more, then it could be considered a slow query and we must rewrite the query to optimize the executiuon time by adding the required filters to retrieve only the required data. We could also consider using an index on keys for better performance.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset