If in the previous chapter we kept things a bit more theoretical with me throwing "rules" at you, in this chapter I am going to make up for it and we are going to have some fun. This means we are going to write some code that demonstrates concepts related to data processing, especially larger amounts of it. And in doing so, we are going to cover a few topics.

First, we are going to look back at the hook_update_N() hook we saw in Chapter 8, The Database API. More specifically, we are going to see how the &$sandbox parameter can be used in order to handle updates that need to process some data that may take a bit longer and should be split across multiple requests. Next up, we are going to look at standalone batches (which basically use the same system) to process data in batches across multiple requests. And what better example to illustrate this technique than with our Importer that needs to process an undefined number of products?

We will take a look at a related subsystem that allows us to queue things for later processing (either in batches, during cron, or in simple requests). Since we are talking about cron, we will also go a bit into detail and see how this system works in Drupal. Finally, we will finish this chapter by taking a look at the Lock API in Drupal 8, an API that allows us to ensure multiple requests don't run a process at the same time.

By the end of this chapter, you will be a lean, mean, data-processing machine. So, let's get to it.