It's easy to see how ES can be scaled across multiple CPUs: each worker is assigned to a separate candidate solution of the population. The evaluation can be done in complete autonomy, and as described in the paper, optimization can be done in parallel on each worker, with only a few scalars shared between each CPU unit.

Specifically, the only information that's shared between workers is the scalar return, , of an episode and the random seed that has been used to sample . The amount of data can be further shrunk by sending only the return, but in this case, the random seed of each worker has to be synchronized with all the others. We decided to adopt the first technique, while the paper used the second one. In our simple implementation, the difference is negligible and both techniques require extremely low bandwidth.