In the previous chapters, we looked at reinforcement learning algorithms, ranging from value-based to policy-based methods and from model-free to model-based methods. In this chapter, we'll provide another solution for solving sequential tasks, that is, with a class of black-box algorithms evolutionary algorithms (EA). EAs are driven by evolutionary mechanisms and are sometimes preferred to reinforcement learning (RL) as they don't require backpropagation. They also offer other complementary benefits to RL. We'll start this chapter by giving you a brief recap of RL algorithms so that you'll better understand how EA fits into these sets of problems. Then, you'll learn about the basic building blocks of EA and how those algorithms work. We'll also take advantage of this introduction and look at one of the most well-known EAs, namely evolution strategies (ES), in more depth.

A recent algorithm that was developed by OpenAI caused a great boost in the adoption of ES for solving sequential tasks. They showed how ES algorithms can be massively parallelized and scaled linearly on a number of CPUs while achieving high performance. After an explanation of evolution strategies, we'll take a deeper look at this algorithm and develop it in TensorFlow so that you'll be able to apply it to the tasks you care about.

The following topics will be covered in this chapter:

• Beyond RL
• The core of EAs
• Scalable evolution strategies
• Scalable ES applied to LunarLander