Now that we have a basic understanding of Drupal's capabilities of doing multi-request processing, let's switch gears and look at the Batch API.

In order to demonstrate how this works, we are going to rebuild the way our product JsonImporter plugin processes the product data it retrieves. Currently, we simply load all the products into an array of objects and loop through each, saving them to the database. So, if there are 100,000 products in the JSON response, we might get into trouble with this approach. To be fair, if the remote provider has so many products, it usually provides a paginated way of requesting them by passing an offset and a limit. This keeps the payloads smaller (which is good for both communicating servers) and makes it easier on the processing. On our side, we can treat it as we would treat a database. But for now, we'll go with the assumption that the number of returned products is large, but not too large as to pose problems with the communication or with the ability of PHP to store them in memory.

Moreover, while illustrating the Batch API, we will also perform an operation we "forgot" in Chapter 7, Your Own Custom Entity and Plugin Types. During the import, we also want to delete any products that have been previously imported but that are no longer in the JSON response. It is a kind of synchronization between the two data sources, if you will. So, let's get to it.