Using the concurrent.futures Python module

The concurrent.futures module, which is part of the standard Python library, provides a level of abstraction on threads by modelling them as asynchronous functions.

This module is built by two main classes:

  • concurrent.futures.Executor: This is an abstract class that provides methods to execute calls asynchronously.
  • concurrent.futures.Future: This encapsulates the asynchronous execution of a callable. Future objects are instantiated by submitting tasks (functions with optional parameters) to Executors.

Here are some of the main methods of the module:

  • submit(function,argument): This schedules the execution of the callable function on the arguments.
  • map(function,argument): This executes the functions of arguments in asynchronous mode.
  • shutdown(Wait=True): This signals the executor to free any resource.

The executors are accessed through their subclasses: ThreadPoolExecutor or ProcessPoolExecutor. Because the instantiation of threads and processes is a resource-demanding task, it is better to pool these resources and use them as repeatable launchers or executors (hence the Executors concept) for parallel or concurrent tasks.

The approach we are taking here involves using a pool executor. We will submit the assets to the pool (thread and process) and get the futures, which are the results that will be available to us in the future. Of course, we can wait for all futures to become real results.

A thread or process pool (also called pooling) indicates a management software that is being used to optimize and simplify the use of threads and/or processes within a program. Through pooling, you can submit the task (or tasks) in order to execute them to the pooler.

The pool is equipped with an internal queue of tasks pending and several threads or processes that execute them. A recurring concept in pooling is reusing: a thread (or process) is used several times for different tasks during its life cycle. This decreases the overhead of creating new threads or processes and increases the performance of the program.

Reuse is not a rule, but it is one of the main reasons that lead a coder to use pooling in their application.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset