Limitations

The aggregation pipeline can output results in the following three distinct ways:

  • Inline as a document containing the result set
  • In a collection
  • Returning a cursor to the result set

Inline results are subject to BSON maximum document size of 16 MB, meaning that we should use this only if our final result is of a fixed size. An example of this would be outputting ObjectId of the top five most ordered items from an e-commerce site.

A contrary example to this would be outputting the top 1,000 most ordered items, along with the product information, including the description and other fields of variable size.

Outputting results into a collection is the preferred solution if we want to perform further processing of the data. We can either output into a new collection or replace the contents of an existing collection. The aggregation output results will only be visible once the aggregation command succeeds; otherwise, it will not be visible at all.

The output collection cannot be a sharded or capped collection (as of v3.4). If the aggregation output violates indexes (including the built-in index on the unique ObjectId per document) or document validation rules, the aggregation will fail.

Each pipeline stage can have documents exceeding the 16 MB limit as these are handled by MongoDB internally. Each pipeline stage, however, can only use up to 100 MB of memory. If we expect more data in our stages, we should set allowDiskUse: to true to allow excess data to overflow to disk at the expense of performance.

The $graphLookup operator does not support datasets over 100 MB and will ignore any setting on allowDiskUse.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset