16

Working with Files and Images

In this chapter, we will look at how we can work with files and images in Drupal, supported by the core features. Although the Media module allows developers to provide new source plugins to expose media entities to all sorts of types of media, we won’t be going into this quite an advanced topic. Instead, we’ll focus on lower-level tools that can be used for working with files. And we will see some examples along the way. So, what are we going to discuss?

First, we are going to get an understanding of the Drupal filesystems. Related to this, we’re going to talk about stream wrappers and how Drupal handles native PHP file operations. We will even create our own custom stream wrapper a bit later in the chapter.

Then, we will talk a bit about the different ways to handle files in Drupal, namely, managed (tracked) and unmanaged files. While working with managed files, we will add an image field to our Product entity type and have images imported from a fictional remote environment. We will also create a brand-new CSV-based importer by which the product data is imported from a CSV file that we read. In this process, we will note the Entity CRUD hooks, a very important extension point in Drupal, and see how we can use them in our example context.

We will end the chapter by seeing how we can work with various APIs that deal specifically with images, especially for manipulating them via image toolkits and working with image styles. So, let’s get to it.

The main topics we will cover in this chapter are as follows:

• Understanding the different types of files
• Stream wrappers
• Working with managed files
• The Private file system
• Images in the Drupal world

The filesystem

Drupal defines four main types of file storage for any given site: the public, the private, the temporary, and the translation filesystems. When installing Drupal, the folders that map to these filesystems are created automatically. In case that fails—most likely due to permission issues—we have to create them ourselves and give them the correct permissions. Drupal takes care of the rest (for example, adds the relevant .htaccess files for security reasons). Make sure you check out the documentation on Drupal.org to see how to successfully install Drupal if you are unsure how this works.

Public files are available to the world at large for viewing or downloading. This is where things such as image content, logos, and anything that can be downloaded are stored. Your public file directory must exist somewhere under Drupal’s root, and it must be readable and writeable by whatever user your web server is running under. Public files have no access restrictions. Anyone, at any time, can navigate directly to a public file and view or download it. This also means that accessing these files does not require Drupal to bootstrap.

We can configure the path to the public filesystem in our settings.php file:

$settings['file_public_path'] = 'sites/default/files';

Private files, on the other hand, are not available to the world for general download. Therefore, the private files directory must not be accessible via the web. However, it still has to be writeable by the web server user. Isolating private files in this way allows developers to control who can and can’t access them. For instance, we could write a module that only allows users who have a specific role to access PDFs in the private filesystem.

We can configure the path to the private filesystem in our settings.php file:

$settings['file_private_path'] = 'sites/default/private';

Temporary file storage is typically only used by Drupal for internal operations. When files are first saved by Drupal, they are initially written into the temporary filesystem so they can be checked for security issues. After they have been deemed safe, they are written to their final location.

We can configure the path to the temporary filesystem in our settings.php file:

$settings['file_temp_path'] = '/tmp';

Finally, the translation file storage is used by Drupal for storing the .po files that contain string translation values that can be imported into the system in bulk. We can configure the location of translation files through the UI:

Figure 16.1: Filesystem configuration UI

Stream wrappers

If you’ve been writing PHP for a long time, you may have needed to work with local or remote files at some point. The following PHP code is a common way to read a file into a variable that you can do something with:

$contents = '';
$handle = fopen("/local/path/to/file/image.jpg", "rb");
while (!feof($handle)) {
 $contents .= fread($handle, 8192);
}
fclose($handle);

This is pretty straightforward. We get a handle to a local file using fopen() and read 8 KB chunks of the file using fread() until feof() indicates that we’ve reached the end of the file. At that point, we use fclose() to close the handle. The contents of the file are now in the $contents variable.

In addition to local files, we can also access remote ones through fopen() in the exact same way but by specifying the actual remote path instead of the local one we saw before (starting with http(s)://).

Data that we can access this way is streamable, meaning we can open it, close it, or seek to a specific place in it.

Stream wrappers are an abstraction layer on top of these streams that tell PHP how to handle specific types of data. When using a stream wrapper, we refer to the file just like a traditional URL—scheme://target. As a matter of fact, the previous example uses one of PHP’s built-in stream wrappers: the file:// wrapper for accessing files on local storage. It is actually the default scheme when none is specified, so that is why we got away with omitting it and just adding the file path. Had the file been in a remote location, we would have used something like http://example.com/file/path/image.jpg. That is another PHP built-in stream wrapper: http:// (for the HTTP protocol).

If that’s not enough, PHP also allows us to define our own wrappers for schemes that PHP does not handle out of the box; the Drupal File API was built to take advantage of this. This is where we link back to the different types of file storage we talked about earlier, as they all have their own stream wrappers defined by Drupal.

The public filesystem uses the rather known public:// stream wrapper, the private one uses private://, the temporary one uses temporary://, and the translation one uses translations://. These map to the local file paths that we defined in settings.php (or the UI). Later in the chapter, we will see how we can define our own stream wrapper and what some of the things that go into it are. First, though, let’s talk a bit about the different ways we can manage files in Drupal.

Managed versus unmanaged files

The Drupal File API allows us to handle files in two different ways. Files essentially boil down to two categories: they are either managed or unmanaged. The difference between the two lies in the way the files are used.

Managed files work hand-in-hand with the Entity system and are, in fact, tied to File entities. So, whenever we create a managed file, an entity gets created for it as well, which we can use in all sorts of ways. And the table where these records are stored is called file_managed. Moreover, a key aspect of managed files is the fact that their usage is tracked. This means that if we reference them on an entity or even manually indicate that we use them, this usage is tracked in a secondary table called file_usage. This way, we can see where each file is used and how many times, and Drupal even provides a way to delete “orphaned” files after a specific time if they are no longer needed.

A notable example of using managed files is the simple Image field type that we can add to an entity type. Using these fields, we can upload a file and attach it to the respective entity. This attachment is nothing more than a special (tracked) entity reference between the two entities.

By understanding how managed files are used, it’s not difficult to anticipate what unmanaged files are. The latter are the files we upload to make use of for various reasons but that, of course, do not need to be attached to any entity or have their usage tracked.

Using the File and Image fields

In order to demonstrate how to work with managed files, we will go back to our product entity importer and bring in some images for each product. However, in order to store them, we need to create a field on the Product entity. This will be an Image field.

Instead of creating this field through the UI and attaching it to a bundle, let’s do it the programmatic way and make it a base field (available on all bundles). We won’t need to do anything complex; for now, we are only interested in a basic field that we can use to store the images we bring in from the remote API. It can look something like this:

$fields['image'] = BaseFieldDefinition::create('image')
  ->setLabel(t('Image'))
  ->setDescription(t('The product image.'))
  ->setDisplayOptions('form', array(
    'type' => 'image_image',
    'weight' => 5,
  ));

If you remember from Chapter 6, Data Modeling and Storage, and Chapter 7, Your Own Custom Entity and Plugin Types, we are creating a base field definition that, in this case, is of the type image. This is the FieldType plugin ID of the ImageItem field type. That is where we need to look and see what kind of field and storage options we may have. For example, we can have a file extension limitation (which by default contains png, gif, jpg, and jpeg) and things like alt and title attributes, as well as image dimension configuration. Do check out ImageItem to get an idea of the possible storage and field settings. However, we are fine with the defaults in this case, so we don’t even have any field settings.

Another interesting thing to notice is that ImageItem extends the FileItem field type, which is a standalone FieldType plugin that we can use. However, it is more generic and lends itself for use with any kind of file upload situation. Since we are dealing with images, we might as well take advantage of the specific field type.

For the moment, we do not configure our image field to have any kind of display. We’ll look into that a bit later. However, we do specify the widget it should use on the entity form, namely the FieldWidget plugin with the ID of image_image. This maps to the default ImageWidget field widget. But again, we are fine with the setting defaults, so we don’t specify anything extra.

With this, our field definition is done. To have Drupal create the necessary database tables, we need to run the Devel entity updates contrib module’s Drush command (or use EntityDefinitionUpdateManager if we need to deploy this change to production):

drush entity-update

Now let’s create the interface methods for easily accessing and setting the images:

/**
 * Gets the Product image.
 *
 * @return DrupalfileFileInterface|null
 */
public function getImage();
/**
 * Sets the Product image.
 *
 * @param int $image
 *
 * @return DrupalproductsEntityProductInterface
 *   The called Product entity.
 */
public function setImage($image);

The getter method is supposed to return a FileInterface object (which is the actual File entity) or NULL, while the setter is supposed to receive the ID (fid) of the File entity to save. As for the implementations, it should not be anything new to us:

/**
 * {@inheritdoc}
 */
public function getImage() {
  return $this->get('image')->entity;
}
/**
 * {@inheritdoc}
 */
public function setImage($image) {
  $this->set('image', $image);
  return $this;
}

With this, we are ready to proceed with the import of images from the remote API.

To take advantage of the media management power in Drupal, instead of Image or File fields, you’d create entity reference fields to Media entities, and you’d create these fields on the latter. As such, Media entities basically wrap the File entities to provide some additional functionality and expose them to all the goodies of media management. For now, we will work directly with these field types to learn about low-level file handling without the overhead of Media entities.

Working with managed files

In this section, we will look at two examples of working with managed files. First, we will see how we can import product images from our fictional remote JSON-based API. Second, we will see how to create a custom form element that allows us to upload a file and use it in a brand-new CSV-based importer.

Attaching managed files to entities

Now that we have our product image field in place and we can store images, let’s revisit our JSON response that contains the product data and assume it looks something like this now:

{
  "products" : [
    {
      "id" : 1,
      "name": "TV",
      "number": 341,
      "image": "tv.jpg"
    },
    {
      "id" : 2,
      "name": "VCR",
      "number": 123,
      "image": "vcr.jpg"
    }
  ]
}

What’s new is the addition of the image key for each product, which simply references a filename for the image that goes with the respective product. The actual location of the images is at some other path we need to include in the code.

Going back to our JsonImporter::persistProduct() method, let’s delegate the handling of the image import to a helper method called handleProductImage(). We need to call this method both if we are creating a new Product entity and if we are updating an existing one (right before saving):

$this->handleProductImage($data, $product);

And this is what the actual method looks like:

protected function handleProductImage($data,
  ProductInterface $product) {
  $name = $data->image;
  // This needs to be hardcoded for the moment.
  $image_path = '';
  $image = file_get_contents($image_path . '/' . $name);
  if (!$image) {
    // Perhaps log something.
    return;
  }
  /** @var DrupalfileFileInterface $file */
  $file = $this->fileRepository->writeData($image,
    'public://product_images/' . $name,
      FileSystemInterface::EXISTS_REPLACE);
  if (!$file) {
    // Something went wrong, perhaps log it.
    return;
  }
  $product->setImage($file->id());
}

And the new use statement at the top:

use DrupalproductsEntityProductInterface;
use DrupalCoreFileFileSystemInterface;

First, we get the name of the image. Then we construct the path to where the product images are stored. In this example, it’s left blank, but if the example were to work, we’d have to add a real path there. I leave that up to you for now. If you want to test it out, create a local folder with some images and reference that.

Using the native file_get_contents() function, we load the data of the image from the remote environment into a string. We then pass this string to the FileRepository::writeData() method (which I will let you inject into our plugin), which saves a new managed file to the public filesystem. This method takes three parameters: the data to be saved, the URI of the destination, and a flag indicating what to do if a file with the same name already exists. You’ll notice that we used the Drupal public:// stream wrapper to build the URI, and we already know which folder this maps to.

As for the third parameter, we chose to replace the file in case one already exists. The alternative would have been to either use the EXISTS_RENAME or EXISTS_ERROR constants of the same interface. The first would have created a new file whose name would have gotten a number appended until the name became unique. The second would have simply not done anything and returned FALSE.

If all goes well, this function returns a File entity (that implements FileInterface) whose ID we can use in the Product image setter method. With that in place, we can also synchronize the individual product images.

If you run into issues after this, make sure you create the destination folder and have all the permissions in order in the public filesystem to allow the copy to take place properly. In the next section, you’ll learn about some helper functions you can use to better prepare the destination folder.

Moreover, in our database, a record is created in the file_usage table to indicate that this file is being used on the respective Product entity.

Helpful functions for dealing with managed files

Apart from the staple FileRepositoryInterface::writeData() method, we have a few other ones that can come in handy if we are dealing with managed files. Here are a few of them.

If we want to copy a file from one place to another while making sure a new database record is created, we can use copy() method on the same service. It takes three parameters:

• The FileInterface entity that needs to be copied
• The destination URI of where it should go
• The flag indicating what to do if a file with the same name exists

The parameters are the same as for writeData().

Apart from the actual copying, this function also invokes hook_file_copy(), which allows modules to respond to files being copied.

Very similar to copy(), we also have move(), which takes the same set of parameters but instead performs a file move. The database entry of the File entity gets updated to reflect the new file path. And hook_file_move() is invoked to allow modules to respond to this action.

Not strictly related to managed files, but rather useful in all cases, we also have the DrupalCoreFileFileSystem service (accessible via the file_system service name), which contains all sorts of useful methods for dealing with files. We’ll see some of them when we talk about unmanaged files. But one that is useful also for managed files is ::prepareDirectory(), which we can use to ensure the file destination is correct. It takes two arguments: the directory (a string representation of the path or stream URI) and a flag indicating what to do about the folder (constants on the interface):

• FileSystemInterface::CREATE_DIRECTORY will create the directory if it doesn’t already exist
• FileSystemInterface::MODIFY_PERMISSION will make the directory writable if it is found to be read-only

This function returns TRUE if the folder is good to go as a destination or FALSE if something went wrong or the folder doesn’t exist.

Note

If you are upgrading from Drupal 9, please be aware that the FileRepository service with the three methods we mentioned (writeData(), copy(), and move()) are actually replacements for the old file_save_data(), file_copy(), and file_move() respectively, which have been deprecated in Drupal 9.3 and completely removed in Drupal 10.

Managed file uploads

Next, we are going to look at how we can work with managed files using a custom form element. And to demonstrate this, we are finally going to create another Product importer plugin. This time, instead of a remote JSON resource, we will allow users to upload a CSV file that contains product data, and imports that into Product entities. This is what the example CSV data looks like:

id,name,number
1,Car,45345
2,Motorbike,54534

It basically has the same kind of data as the JSON resource we’ve been looking at so far but without the image reference. So, let’s get going with our new plugin class.

Here is our starting point:

namespace DrupalproductsPluginImporter;
use DrupalCoreStringTranslationStringTranslationTrait;
use DrupalproductsPluginImporterBase;
/**
 * Product importer from a CSV format.
 *
 * @Importer(
 *   id = "csv",
 *   label = @Translation("CSV Importer")
 * )
 */
class CsvImporter extends ImporterBase {
  use StringTranslationTrait;
  /**
   * {@inheritdoc}
   */
  public function import() {
    $products = $this->getData();
    if (!$products) {
      return FALSE;
    }
    foreach ($products as $product) {
      $this->persistProduct($product);
    }
    return TRUE;
  }
}

We start by extending from the ImporterBase class and implementing the obligatory import() method. Like before, we delegate to getData() to retrieve the product information, but in this case, we simply loop over the resulting records and use the persistProduct() method to save the Product entities. So no batch operations. Apart from no longer saving images, this latter method looks exactly like the one from the JsonImporter, so I won’t be copying it over again. But it makes for a good homework assignment to try to move it to the base class and abstract away the dynamic portions.

Managed file form element

Now, we need to handle the form and we have a few methods for this.

First, we have to define the default configurations of this plugin:

/**
 * {@inheritdoc}
 */
public function defaultConfiguration() {
  return [
    'file' => [],
  ];
}

Like we did with the JsonImporter but this time we keep track of a file reference. We default to an empty array because we will store an array of file IDs (with one value).

Second, we have the form definition for uploading the file:

/**
 * {@inheritdoc}
 */
public function buildConfigurationForm(array $form,
  FormStateInterface $form_state) {
  $form['file'] = [
    '#type' => 'managed_file',
    '#default_value' => $this->configuration['file'],
    '#title' => $this->t('File'),
    '#description' => $this->t('The CSV file containing the
      product records.'),
    '#required' => TRUE,
  ];
  return $form;
}

And the use statement at the top:

use DrupalCoreFormFormStateInterface;

The form element type is called managed_file (implemented by the ManagedFile form element class). The rest of the definition is straightforward. However, there are a couple of problems.

First, by default, using this form element, files are uploaded to the temporary:// filesystem of Drupal. Since we don’t want that, we need to specify an upload location:

'#upload_location' => 'public://'

The root of our public files folder will suffice for this example as we assume the file does not contain any sensitive information. If so, we could upload it to the private:// one and control who gets access. We'll talk about how that works later in the chapter.

Second, by default, using this form element, the allowed file extensions for upload are limited to jpg, jpeg, gif, png, txt, doc, xls, pdf, ppt, pps, odt, ods, and odp. So if we want to allow CSV files, we need to specify the extension in a list of allowed upload extensions. And we do this by overriding the default upload validators:

'#upload_validators' => [
  'file_validate_extensions' => ['csv'],
],

This is an array of validator callbacks we want Drupal to run when the file is uploaded. And allowing only CSV files is enough for our purposes. But another handy validator we could use is file_validate_size(). Moreover, we can implement hook_file_validate() ourselves and perform any custom validation of the files being uploaded. So that’s also something to keep in mind when dealing with validating files that don’t belong to your modules.

With this, our plugin configuration form is in place; it looks something like this:

Figure 16.2: Plugin configuration form

But let’s not forget about the form submit handler that stores the uploaded file ID into the plugin configuration:

/**
 * {@inheritdoc}
 */
public function submitConfigurationForm(array &$form,
  FormStateInterface $form_state) {
  $this->configuration['file'] = $form_state->getValue
    ('file');
}

However, there is still something we need to do in order for the uploaded file to be managed properly. When using this form element, the file gets correctly uploaded and a record is added to the file_managed table. So we get our File entity. However, its status is not permanent because it doesn’t have any usages. There are no records for it in the file_usage table. How could there be? So what we need to do is handle that ourselves and basically tell Drupal that the file uploaded in this form is used by the respective Importer configuration entity. And to do this, we need to know when the file is saved onto the entity, changed, and deleted.

With this, we can also learn about something very important that we skipped in Chapter 6, Data Modeling and Storage, and Chapter 7, Your Own Custom Entity and Plugin Types: entity CRUD hooks. But right before we jump into that, let’s not forget about the configuration schema of this new configuration item—the file key of the plugin configuration:

products.importer.plugin.csv:
  type: mapping
  label: Plugin configuration for the CSV importer plugin
  mapping:
    file:
      type: sequence
      label: File IDs
      sequence:
        type: integer
        label: CSV File ID

We are doing the same as we did for the url key of the JSON importer but, in this case, we need to account for the fact that file is actually an array. So we define it as a sequence whose individual items are integers. Feel free to check Chapter 6, Data Modeling and Storage, for more information on configuration schemas whenever you need a reminder.

Entity CRUD hooks

Whenever entities are created, updated, or deleted, a set of hooks are fired that allow us to act on this information. We can use these hooks simply to perform some actions whenever this happens or even make changes to the entity being saved. So let’s see what we have.

A very useful one is hook_entity_presave(), which gets fired during the saving process of an entity (both content and configuration). This applies to both when the entity is first created, as well as when it is being updated. Moreover, it allows us to inspect the original entity and detect changes made to it. And finally, since the entity has not yet been persisted, it allows us to make changes to it ourselves. So, very powerful stuff.

Since Drupal is very flexible, we also have the hook_ENTITY_TYPE_presave() version, which allows us to specifically target any entity type we want. We’ve already discussed the benefit of using more specific hooks to keep our code more organized as well as a little bit more performant. And this applies to all the entity CRUD hooks we are going to talk about next.

Next, we have hook_entity_insert() and hook_entity_update(), which get fired after an entity is created for the first time and after an entity is updated, respectively. We cannot make changes to the entity itself as it has already been saved, but they can come in handy at other times. The latter also gives us access to the original entity if we want to compare any changes. And similarly, we have hook_entity_delete(), which gets fired when an entity is deleted.

Finally, we also have hook_entity_load(), which allows us to perform actions whenever an entity is loaded. For example, we can tack on additional information if we want. So keep in mind these hooks, as they are going to be a very important tool in your module developer arsenal.

Now that we have an idea of the available entity CRUD hooks, we can implement three of them to handle our managed file problem. Because, if you remember, managed files are actually represented by the File entity type, so the Entity CRUD hooks get fired for these as well.

Managed file usage service

To mark a file as being used by something, we can use the DatabaseFileUsageBackend service (file.usage), which is an implementation of the FileUsageInterface. This has a few handy methods that allow us to add a usage or delete it. That is actually what we are going to do next.

What we want to do first is add a file usage whenever a new Importer entity gets created (and a file uploaded with it):

/**
 * Implements hook_ENTITY_TYPE_insert() for the Importer
   config entity type.
 */
function products_importer_insert(DrupalCoreEntity
  EntityInterface $entity) {
  if ($entity->getPluginId() !== 'csv') {
    return;
  }
  // Mark the current File as being used.
  $fid = _products_importer_get_fid_from_entity($entity);
  $file = Drupal::entityTypeManager()->getStorage('file')
    ->load($fid);
  Drupal::service('file.usage')->add($file, 'products',
    'config:importer', $entity->id());
}

We are implementing the specific version of hook_entity_insert() for our own entity type, and the first thing we are checking is whether we are looking at one using the CSV plugin. We’re not interested in any importers that don’t have a CSV file upload. If we are, we get the File entity ID from the importer using a private helper function:

function _products_importer_get_fid_from_entity(
  DrupalCoreEntityEntityInterface $entity) {
  $fids = $entity->getPluginConfiguration()['file'];
  $fid = reset($fids);
  return $fid;
}

You’ll notice that the file key in our plugin configuration array is an array of File IDs, even if we only uploaded one single file. That is just something we need to account for here (we also did so in our configuration schema earlier on).

Then, we load the File entity based on this ID and use the file.usage service to add a usage to it. The first parameter of the add() method is the File entity itself, the second is the module name that marks this usage, the third is the type of thing the file is used by, while the fourth is the ID of this thing. The latter two depend on the use case; we choose to go with our own notation (config:importer) to make it clear that we are talking about a configuration entity of the type importer. Of course, we used the ID of the entity.

With this, a new record will get created in the file_usage table whenever we save such an Importer entity for the first time. Now let’s handle the case in which we delete this entity—we don’t want this file usage lingering around, do we?

/**
 * Implements hook_ENTITY_TYPE_delete() for the Importer
   config entity type.
 */
function products_importer_delete(DrupalCoreEntity
  EntityInterface $entity) {
  if ($entity->getPluginId() !== 'csv') {
    return;
  }
  $fid = _products_importer_get_fid_from_entity($entity);
  $file = Drupal::entityTypeManager()->getStorage
    ('file')->load($fid);
  Drupal::service('file.usage')->delete($file, 'products',
    'config:importer', $entity->id());
}

Most of what we are doing in this specific version of hook_entity_delete() is the same as before. However, we are using the delete() method of the file.usage service but passing the same arguments. These $type and $id parameters are actually optional, so we can “un-use” multiple files at once. Moreover, we have an optional fifth parameter (the count) whereby we can specifically choose to remove more than one usage from this file. By default, this is 1, and that makes sense for us.

Finally, we also want to account for the cases in which the user edits the importer entity and changes the CSV file. We want to make sure the old one is no longer marked as used for this Importer. And we can do this with hook_entity_update():

/**
 * Implements hook_ENTITY_TYPE_update() for the Importer
   config entity type.
 */
function products_importer_update(DrupalCoreEntity
  EntityInterface $entity) {
  if ($entity->getPluginId() !== 'csv') {
    return;
  }
  /** @var DrupalproductsEntityImporterInterface
    $original */
  $original = $entity->original;
  $original_fid = _products_importer_get_fid_from_entity
    ($original);
  $new_fid = _products_importer_get_fid_from_entity
    ($entity);
  if ($original_fid !== $new_fid) {
    $original_file = Drupal::entityTypeManager()->
      getStorage('file')->load($original_fid);
    Drupal::service('file.usage')->delete($original_file,
      'products', 'config:importer', $entity->id());
    $file = Drupal::entityTypeManager()->getStorage('file')
      ->load($new_fid);
    Drupal::service('file.usage')->add($file, 'products',
      'config:importer', $entity->id());
  }
}

We are using the specific variant of this hook that only gets fired for Importer entities. Just like we’ve been doing so far. And as I mentioned, we can access the original entity (before the changes have been made to it) like so:

$original = $entity->original;

And if the File ID that was on the original entity is not the same as the one we are currently saving with it (meaning the file was changed), we can delete the usage of that old File ID. Of course, we also track the usage of the newly uploaded file instead.

Processing the CSV file

Now that our plugin configuration works—and uploaded files are properly managed and marked as used—it’s time to implement the getData() method by which we process the CSV file of the Importer entity. The result needs to be an array of product information, as expected by the import() method we saw earlier. So, we can have something like this:

/**
 * Loads the product data from the CSV file.
 *
 * @return array
 */
protected function getData() {
  $fids = $this->configuration['file'];
  if (!$fids) {
    return [];
  }
  $fid = reset($fids);
  /** @var DrupalfileFileInterface $file */
  $file = $this->entityTypeManager->getStorage('file')->
    load($fid);
  $wrapper = $this->streamWrapperManager->getViaUri($file->
    getFileUri());
  if (!$wrapper) {
    return [];
  }
  $url = $wrapper->realpath();
  $spl = new SplFileObject($url, 'r');
  $data = [];
  while (!$spl->eof()) {
    $data[] = $spl->fgetcsv();
  }
  $products = [];
  $header = [];
  foreach ($data as $key => $row) {
    if ($key == 0) {
      $header = $row;
      continue;
    }
    if ($row[0] == "") {
      continue;
    }
    $product = new stdClass();
    foreach ($header as $header_key => $label) {
      $product->{$label} = $row[$header_key];
    }
    $products[] = $product;
  }
  return $products;
}

First, quite expectedly, we check for the existence of the File ID in the configuration and load the corresponding File entity based on that. To do this, we use the entity manager we injected into the plugin base class. But then comes something new.

Once we have the File entity, we can ask it for its URI, which will return something like this: public://products.csv. This is what is stored in the database. But in order to turn that into something useful, we need to use the stream wrapper that defines this filesystem. And to get that, we use the StreamWrapperManager service (stream_wrapper_manager), which has a handy method of returning the stream wrapper instance responsible for a given URI—getViaUri(). And once we have our StreamWrapperInterface, we can use its realpath() method to get the local path of the resource. We will come back to stream wrappers a bit later in this chapter, and it will make more sense. But for the moment, it’s enough to understand that we are translating a URI in the scheme://target format into a useful path that we can use to create a new PHP-native SplFileObject instance, which, in turn, we can use to process the CSV file easily.

When creating the SplFileObject, we used the external URL of the file. This worked just fine, and we were able to also demonstrate how we can get our hands on the external URL if we ever need to. But, as we will see in the next chapter, it will also work directly with the stream URI, and we will switch to this approach instead.

With three lines of code, we are basically done getting all the rows from the CSV into the $data array. However, we also want to make this data look a bit more like what the JSON resource looked like—a map where the keys are the field names and the values are the respective product data. And we also want this map to contain PHP standard objects instead of arrays. Therefore, we loop through the data, establish the CSV header values, and use those as the keys in each row of a new $products array of objects. Our end result will look exactly like the product information coming from the decoded JSON response.

And with this, we are done. Well, not quite. We still need to inject the StreamWrapperManager service (stream_wrapper_manager) into our plugin. But you should know how to do that by now, so I will let you do it on your own.

Nothing we don’t yet know how to do. However, there is one thing I’d like to point out here. In Chapter 7, Your Own Custom Entity and Plugin Types, I mentioned how, at the time, I believed the Guzzle HTTP client is a service that would be useful to all Importer plugins. Well, I was clearly wrong, as the CSV-based one we just created now doesn’t need it, so there is no reason why it should be injected into it. What we need to do here is remove this dependency from the base plugin class and only use it in the JSON importer. However, I leave this up to you as homework.

Our CSV Importer plugin is now complete. If we did everything correctly, we can now create a new Importer entity that uses it, upload a correct CSV file, and import some Product entities via our Drush command. How neat.

Our own stream wrapper

At the beginning of this chapter, we briefly talked about stream wrappers and what they are used for. We saw that Drupal comes with four stream wrappers that map to the various types of file storage it needs. Now it’s time to see how we can create our own. And the main reason why we would want to implement one is to expose resources at a specific location to PHP’s native filesystem functions.

In this example, we will create a very simple stream wrapper that can basically only read the data from the resource. Just to keep things simple. And the data resource will be the product images hosted remotely (the ones we are importing via the JSON Importer). So there will be some rework there to use the new stream wrapper instead of the absolute URLs. Moreover, we will also learn how to use the site-wide settings service by which we can have environment-specific configurations set in the settings.php file and then read by our code.

The native way of registering a stream wrapper in PHP is by using the stream_wrapper_register() function. However, in Drupal, we have an abstraction layer on top of that in the form of services. So, a stream wrapper is a simple tagged service, albeit with many potential methods. Let’s see its definition, which we add to the products.services.yml file:

products.images_stream_wrapper:
  class: DrupalproductsStreamWrapper
    ProductsStreamWrapper
  tags:
    - { name: stream_wrapper, scheme: products }

Nothing too complicated. The service is tagged with stream_wrapper, and we use the scheme key to indicate the scheme of the wrapper. So the URIs will be in this format:

products://target

One important thing to note about stream wrapper services is that we cannot pass dependencies to them. The reason is that they are not instantiated in the normal way (by the container) but arbitrarily by PHP whenever some of its methods need to be called. So if we need to use some services, we’ll have to use the static way of loading them.

The stream wrapper service class needs to implement StreamWrapperInterface, which comes with a lot of methods. There are many possible filesystem interactions that PHP can do, and these methods need to account for them all. However, we will only be focusing on a few specific ones that have to do with reading data. After all, our resources are remote and we don't even have a clue how to make changes to them over there. So for the rest of the methods, we will be returning FALSE to indicate that the operation cannot be performed.

Let’s see this big class then:

namespace DrupalproductsStreamWrapper;
use DrupalComponentUtilityUrlHelper;
use DrupalCoreStreamWrapperStreamWrapperInterface;
use DrupalCoreStringTranslationStringTranslationTrait;
/**
 * Stream wrapper for the remote product image paths used
   by the JSON Importer.
 */
class ProductsStreamWrapper implements
  StreamWrapperInterface {
  use StringTranslationTrait;
  /**
   * The Stream URI
   *
   * @var string
   */
  protected $uri;
  /**
   * @var DrupalCoreSiteSettings
   */
  protected $settings;
  /**
   * Resource handle
   *
   * @var resource
   */
  protected $handle;
  /**
   * ProductsStreamWrapper constructor.
   */
  public function __construct() {
    // Dependency injection does not work with stream
       wrappers.
    $this->settings = Drupal::service('settings');
  }
  /**
   * {@inheritdoc}
   */
  public function getName() {
    return $this->t('Product images stream wrapper');
  }
  /**
   * {@inheritdoc}
   */
  public function getDescription() {
    return $this->t('Stream wrapper for the remote location
      where product images can be found by the JSON
        Importer.');
  }
  /**
   * {@inheritdoc}
   */
  public static function getType() {
    return StreamWrapperInterface::HIDDEN;
  }
  /**
   * {@inheritdoc}
   */
  public function setUri($uri) {
    $this->uri = $uri;
  }
  /**
   * {@inheritdoc}
   */
  public function getUri() {
    return $this->uri;
  }
  /**
   * Helper method that returns the local writable target
     of the resource within the stream.
   *
   * @param null $uri
   *
   * @return string
   */
  public function getTarget($uri = NULL) {
    if (!isset($uri)) {
      $uri = $this->uri;
    }
    [$scheme, $target] = explode('://', $uri, 2);
    return trim($target, '/');
  }
  /**
   * {@inheritdoc}
   */
  public function getExternalUrl() {
    $path = str_replace('', '/', $this->getTarget());
    return $this->settings->get('product_images_path') .
      '/' . UrlHelper::encodePath($path);
  }
  /**
   * {@inheritdoc}
   */
  public function realpath() {
    return $this->getTarget();
  }
  /**
   * {@inheritdoc}
   */
  public function stream_open($path, $mode, $options,
    &$opened_path) {
    $allowed_modes = array('r', 'rb');
    if (!in_array($mode, $allowed_modes)) {
      return FALSE;
    }
    $this->uri = $path;
    $url = $this->getExternalUrl();
    $this->handle = ($options && STREAM_REPORT_ERRORS) ?
      fopen($url, $mode) : @fopen($url, $mode);
    return (bool) $this->handle;
  }
  /**
   * {@inheritdoc}
   */
  public function dir_closedir() {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function dir_opendir($path, $options) {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function dir_readdir() {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function dir_rewinddir() {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function mkdir($path, $mode, $options) {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function rename($path_from, $path_to) {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function rmdir($path, $options) {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function stream_cast($cast_as) {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function stream_close() {
    return fclose($this->handle);
  }
  /**
   * {@inheritdoc}
   */
  public function stream_eof() {
    return feof($this->handle);
  }
  /**
   * {@inheritdoc}
   */
  public function stream_flush() {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function stream_lock($operation) {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function stream_metadata($path, $option, $value) {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function stream_read($count) {
    return fread($this->handle, $count);
  }
  /**
   * {@inheritdoc}
   */
  public function stream_seek($offset, $whence = SEEK_SET)
    {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function stream_set_option($option, $arg1, $arg2)
    {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function stream_stat() {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function stream_tell() {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function stream_truncate($new_size) {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function stream_write($data) {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function unlink($path) {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function url_stat($path, $flags) {
    return FALSE;
  }
  /**
   * {@inheritdoc}
   */
  public function dirname($uri = NULL) {
    return FALSE;
  }
}

The first thing to look at is the constructor in which we statically load the Settings service and store it as a class property. And speaking of which, we also define a $uri property to hold the actual URI this wrapper wraps and a $handle property to hold a generic PHP resource handle.

The getName() and getDescription() methods are pretty straightforward and are used for identifying the stream wrapper, while the getType() method returns the type of stream. We'll go with the hidden type because we don't want it visible in the UI. It's strictly for programmatic use so that we can read our product images. Do check out the available types and their meanings by looking at the StreamWrapperInterface constants.

Then, we have a getter and setter for the $uri property by which the Drupal StreamWrapperManager can create an instance of our wrapper based on a given URI. The getTarget() method is actually not in the interface but is a helper to extract a clean target from the URI (the target being the second part of the URI that comes after scheme://). And we use this method in getExternalUrl(), which is quite an important method responsible for returning an absolute URL to the resource in question. But here we also use our Settings service to get the product_images_path key. If you remember at the beginning of the chapter, we saw that the path to the public filesystem is defined in the settings.php file like so:

$settings['file_public_path'] = 'sites/default/files';

That $settings variable is the data array that is wrapped by the Settings service. So we want to do the same when defining our own remote path to the product images:

$settings['product_images_path'] = 'http://path/to/the
  /remote/product/images';

This way we are not committing to Git the actual remote URL and we can also change it later if we want. And this is the URL we are reading inside the getExternalUrl() method.

The other pillar of our read-only stream wrapper is the ability to open a file handle to the resource and allow us to read the data from it. And the stream_open() method does this as it gets called when we run either file_get_contents() or fopen() on our URI. Using the $mode parameter, we ensure that the operation is read-only and return FALSE otherwise—we do not support write or other flags.

Any mode can have b appended to it to indicate that the file should be opened in binary mode. So, where r indicates read-only, rb indicates read-only in binary mode.

The third argument is a bitmask of options defined by PHP. The one we’re dealing with here is STREAM_REPORT_ERRORS, which indicates whether or not PHP errors should be suppressed (for instance, if a file is not found). The second is STREAM_USE_PATH, which indicates whether PHP’s include path should be checked if a file is not found. This is not relevant to us, so we ignore it. If a file is found on the include path, then the fourth argument, ($opened_url), should be set with the file’s real path.

What we do then is translate the URI into the absolute URL of the external resource so that we can open a file handle on it. And in doing so, we make use of the STREAM_REPORT_ERRORS option to either prepend the @ to the fopen() function or not (doing so suppresses errors). Finally, we store the reference to the resource handle and return a Boolean based on it to indicate whether the operation succeeded.

Finally, we also implement the stream_read(), stream_eof(), and stream_close() methods so that we can actually also stream the resources if we want to. As for the rest of the methods, as already mentioned, we return FALSE.

All we have to do now is clear the cache and make use of our stream. As long as we have a valid URL declared in the settings.php file, our stream should work fine. And here are the kinds of things we could do with a URI like this:

$uri = 'products://tv.jpg';

To get the entire file content into a string, we can do this:

$contents = file_get_contents($uri);

Or we can use the example from the beginning of the chapter and stream the file bit by bit:

$handle = fopen($uri, 'r');
$contents = '';
while (!feof($handle)) {
  $contents .= fread($handle, 8192);
}
fclose($handle);

All these file operations, such as opening, reading, checking the end of a file, and closing, are possible due to our stream_*() method implementations from the wrapper.

And finally, maybe now it’s also a bit clearer what we did when writing the CSV Importer and using the StreamWrapperManager to identify the stream wrapper responsible for a given URI, and based on that, the real path of the URI.

To end the section on stream wrappers, let’s do some clean-up work by refactoring our JsonImporter::handleProductImage() method a bit. Our logic there involved hardcoding the URL to the remote API, which is really not a good idea. Instead, now that we have our stream wrapper, we can go ahead and use it. We can replace this:

// This needs to be hardcoded for the moment.
$image_path = '';
$image = file_get_contents($image_path . '/' . $name);

With this:

if (!file_exists('products://' . $name)) {
  return;
}
$image = file_get_contents('products://' . $name);

It’s that simple. And now we can control the remote URL from outside the Git repository and, if it changes, we don’t even have to alter our code. Granted, solely for this purpose, implementing a stream wrapper seems a bit excessive. After all, you can simply inject the Settings service and use the URL in the Importer plugin itself allowing for the same kind of flexibility. But we used the opportunity to learn about stream wrappers and how to create our own. And we even managed to find a small use case in the process.

Working with unmanaged files

Working with unmanaged files is actually pretty similar to doing so with managed files, except that they are not tracked in the database using the File entity type. There is a set of helper functions similar to what we’ve seen for managed files that can be accessed through the FileSystem service I mentioned earlier. Let’s see some examples.

To save a new file, we do almost like we did before with managed files:

$image = file_get_contents('products://tv.jpg');
// Load the service statically for quick demonstration.
$file_system = Drupal::service('file_system');
$path = $file_system->saveData($image, 'public://tv.jpg',
  FileSystemInterface::EXISTS_REPLACE);

We load the file data from wherever and use the saveData() method on the service the same way as we did with FileRepositoryInterface::writeData(). The difference is that the file is going to be saved but no database record is created. So the only way to use it is to rely on the path it is saved at and either try to access it from the browser or use it for whatever purpose we need. This method returns the URI of where the file is now saved or FALSE if there was a problem with the operation. So if all went well with the previous example, $path would now be public://tv.jpg.

And just like with the managed files, we also have a few other helpful methods in that service, such as move(), copy(), and delete(). I recommend you inspect that service to get more details on how these methods work.

Private filesystem

The private filesystem is used whenever we want to control access to the files being downloaded. Using the default public storage, users can get to the files simply by pointing to them in the browser, thereby bypassing Drupal completely. However, .htaccess rules prevent users from directly accessing any files in the private storage, making it necessary to create a route that delivers the requested file. It goes without saying that the latter is a hell of a lot less performant, as Drupal needs to be loaded for each file. Therefore, it's important to only use it when files should be restricted based on certain criteria.

Drupal already comes with a route and Controller ready to download private files, but we can create one as well if we really need to. For example, the Image module does so in order to control the creation and download of image styles—ImageStyleDownloadController.

The route definition for the default Drupal path looks like this:

system.files:
  path: '/system/files/{scheme}'
  defaults:
    _controller: 'DrupalsystemFileDownloadController::
      download'
    scheme: private
  requirements:
    _access: 'TRUE'

This is a bit of an odd route definition. We have a {scheme} parameter, which will be the actual file path requested for download. The URI scheme itself defaults to private, as illustrated by the signature of FileDownloadController::download(). Moreover, access is allowed at all times as Drupal delegates this check to other modules—as we will see in a minute.

If we look inside FileDownloadController::download(), we can see that it isn’t actually much that it is doing itself. However, we also note that in the first line, it looks for the query parameter called file in order to get the URI of the requested file:

$target = $request->query->get('file');

But based on the route definition, we don’t even have this parameter. This is where Path Processors come into play, more specifically, implementations of InboundPathProcessorInterface. These are tagged services that get invoked by the routing system when building up the routes by the requested path. And essentially, they allow the alteration of a given path as it comes in.

The core System module implements its own path processor for the purpose of handling the download of private files:

path_processor.files:
  class: DrupalsystemPathProcessorPathProcessorFiles
  tags:
    - { name: path_processor_inbound, priority: 200 }

It’s a simple tagged service definition whose class needs to implement the correct interface that has one method. In the case of PathProcessorFiles, it looks like this:

/**
 * {@inheritdoc}
 */
public function processInbound($path, Request $request) {
  if (strpos($path, '/system/files/') === 0 && !$request->
    query->has('file')) {
    $file_path = preg_replace('|^/system/files/|', '',
      $path);
    $request->query->set('file', $file_path);
    return '/system/files';
  }
  return $path;
}

The goal of this method is to return a path that can be the same as the one requested or changed for whatever reason. And what Drupal does here is check whether the path is the one defined earlier (that starts with /system/files/) and extracts the requested file path that comes as the first argument after that. It takes that and adds it to the current request parameter keyed by file. Finally, it returns a cleaner path called simply /system/files. So this is why the FileDownloadController::download() method looks there for the file path.

Turning back to the Controller, we see that it essentially checks for the file and, if it is not found, throws a 404 (NotFoundHttpException). Otherwise, it invokes hook_file_download(), which allows all modules to control access to the file. And these can do so in two ways: either by returning -1, which denies access or by returning an array of headers to control the download for that specific file. By default, files in the private filesystem cannot be downloaded unless a specific module allows this to happen.

So what does this mean? If we have a file in the private filesystem, we need to implement hook_file_download() and control access to it. Let's see an example of how this might work by assuming we have a folder called /pdfs, whose files we want to make accessible to users that have the administer site configuration permission:

/**
 * Implements hook_file_download().
 */
function module_name_file_download($uri) {
  $file_system = Drupal::service('file_system');
  $dir = $file_system->dirname($uri);
  if ($dir !== 'private://pdfs') {
    return NULL;
  }
  if (!Drupal::currentUser()->hasPermission('administer
    site configuration')) {
    return -1;
  }
  return [
    'Content-type' => 'application/pdf',
  ];
}

This hook receives as an argument the URI of the file being requested. And based on that, we try to get the name of the folder it’s in. To do this, we use the file_system service again.

If the file is not in the private filesystem inside the /pdfs folder, we simply return NULL to signify that we don’t control access to this file. Other modules may do so (and if none do, access is denied). If it is our file, we check for the permission we want and return -1 if the user doesn’t have it. This will deny access. Finally, if access is allowed, we return an array of headers we want to use in the file delivery. In our case, we simply use the PDF-specific headers that facilitate the display of the PDF file in the browser. If we wanted to trigger a file download, we could do something like this instead:

$name = $file_system->basename($uri);
return [
  'Content-Disposition' => "attachment;filename='$name'"
];

We use the filesystem service to determine the file name being requested and adjust our headers accordingly to treat it like an attachment that has to be downloaded.

And that is all there is to it. If we want more control (or a different path to download the files), we can implement our own route and follow the same approach. Without, of course, the need to invoke a hook, but simply handling the download inside the controller method. For example, this is what FileDownloadController::download() does to handle the actual response:

return new BinaryFileResponse($uri, 200, $headers, $scheme
  !== 'private');

This type of response is used when we want to deliver files to the browser, and it comes straight from Symfony.

Images

In this section, we are going a bit deeper into the world of images in Drupal while keeping the focus on module developers.

Image toolkits

The Drupal Image toolkits provide an abstraction layer over the most common operations used for manipulating images. By default, Drupal uses the GD image management library that is included with PHP. However, it also offers the ability to switch to a different library if needed by using the ImageToolkit plugins:

Figure 16.3: Image toolkit configuration UI

For instance, a contributed module could implement the ImageMagick library for developers who need support for additional image types, such as TIFF, which GD does not support. However, only one library can be used at a time as it needs to be configured site-wide.

Programmatically manipulating images using a toolkit involves instantiating an ImageInterface object that wraps an image file. This interface (implemented by the Image class) contains all the needed methods for applying the common manipulations to images, as well as saving the resulting image to the filesystem. And to get our hands on such an object, we use the ImageFactory service:

$factory = Drupal::service('image.factory');

The role of this factory is to create instances of Image using a given toolkit. And it works like this:

$image = $factory->get($uri);

The second parameter to this method is the ImageToolkit plugin ID we want the Image object to work with. By default, it uses the default toolkit configured for the entire application.

And now we can use the manipulation methods on the ImageInterface to change the file:

$image->scale(50, 50);
$image->save('public://thumbnail.jpg');

In this example, we scale the image to 50 x 50 and save it to a new path. Omitting the destination in the save() method would mean overwriting the original file with the changed version. If you need to perform such manipulations manually, I encourage you to explore the ImageInterface for all the available options.

Image styles

Even though, as we’ve seen, we can handle image manipulations programmatically ourselves, typically this is done as part of Image Styles, which can be created and configured via the UI. These involve the application of several possible Image Effects in order to create image variations used in different places:

Figure 16.4: Default image styles in Drupal

The image styles themselves are configuration entities that store configuration specific to the ImageEffect plugins they work with. Once they are created in the UI, we can make use of them in various ways. The most typical way is to use the image style in the display configuration of an entity field or even in Views when rendering an image field.

If you remember, at the beginning of the chapter we created the image field on the product entity but we did not configure a display. So for the moment, the imported images do not show up on the main product page. But we can add some display configuration to our base field definition so that images are shown with a specific image style:

->setDisplayOptions('view', array(
  'type' => 'image',
  'weight' => 10,
  'settings' => [
    'image_style' => 'large'
  ]
))

In this example, we are using the default image field formatter plugin, which can be configured to use an image style. So, under the settings key, we reference the large image style configuration entity that comes with Drupal core. Omitting this would simply just render the original image. Make sure you check back to Chapter 7, Your Own Custom Entity and Plugin Types, and Chapter 9, Custom Fields, if you are a bit fuzzy on the base field definitions.

Rendering images

In Chapter 4, Theming, we talked about theme hooks and how we use them in render arrays to build output. And we also saw a few examples of theme hooks that come with Drupal core and that can be used for common things (such as links or tables). But images are also something we’ll often end up rendering, and there are two ways we can do so (both using theme hooks defined by Drupal core).

First, we can use the image theme hook to simply render an image. And it's pretty simple to use it:

return [
  '#theme' => 'image',
  '#uri' => 'public://image.jpg',
];

And this will render the image as is. We can also pass some more options, such as the alt, title, width, or height, all of which are applied to the image tag as attributes, as well as an array of any other kinds of attributes we may want. Check out template_preprocess_image() for more information on how this works.

Alternatively, the Image module defines the image_style theme hook, which we can use to render the image using a given image style:

return [
  '#theme' => 'image_style',
  '#uri' => 'public://image.jpg',
  '#style_name' => 'large',
];

This theme hook works pretty much the same way, except that it has an extra parameter for the ID of the ImageStyle entity we want to use. And the rest of the parameters we find on the image theme hook can also be found here. In fact, image_style delegates to the image theme hook under the hood.

Finally, we may also find ourselves in a situation in which we need to get our hands on the URL of an image using a given image style. We need to work with the ImageStyle configuration entity for this:

$style = Drupal::entityTypeManager()->getStorage
  ('image_style')->load('thumbnail');
$url = $style->buildUrl('public://image.jpg');

Once we load the image style we want, we simply call its buildUrl() method, to which we pass the URI of the file for which we want the URL. The first time this URL is accessed, the image variation gets created and stored to disk. Future requests will load it directly from there for improved performance.

Summary

We are closing this chapter after covering a lot of different topics that have to do with working with files in Drupal.

We started with a couple of introductory sections in which we outlined some general concepts, such as the various filesystems (storages) that Drupal uses, as well as how stream wrappers come into play for working with them. We also introduced the different ways to work with files: managed versus unmanaged.

Next, we dove into working with managed files and created an image field on our Product entity type so that we could import images into it. The other example of working with managed files had us create a new Product importer based on a CSV file of data, and we also saw how to upload, read, and process such a file, as well as manually track its usage. As a parenthesis, we introduced a very powerful feature of Drupal that allows us to hook into the entity CRUD operations and perform actions whenever these are fired. This is a majorly important technique module developers typically use in Drupal.

We then switched gears and implemented our own stream wrapper to serve our imaginary remote API that stored the product images. Moreover, we talked about working with unmanaged files and some of the functions we can use for this—things similar to managed files except the method names are different and there are no File entities or usage tracking them.

We then continued with the private filesystem and talked about what this serves and how we can work with it to control access to our own files. This, as opposed to allowing users to bypass Drupal and download files from the public filesystem.

Finally, we finished the chapter with a look at the APIs surrounding images and how we can use toolkits to process images, both manually and as part of image styles. And even more useful, we saw how we can render images in all sorts of ways and get our hands on the image style URLs.

In the next chapter, we will look at automated testing and how we can ensure that our code works and that we don’t introduce regressions along the way.