With our single-agent experiences under our belt, we can move on to the more complex but equally entertaining world of working in multi-agent environments, training multiple agents to work in the same environment in a co-operative or competitive fashion. This also opens up several new opportunities for training agents with adversarial self-play, cooperative self-play, competitive self-play, and more. The possibilities become endless here, and this may be the true holy grail of AI.

In this chapter, we are going to cover several aspects of multi-agent training environments and the main section topics are highlighted here:

• Adversarial and cooperative self-play
• Competitive self-play
• Multi-brain play
• Adding individuality with intrinsic rewards
• Extrinsic rewards for individuality

This chapter assumes you have covered the three previous chapters and completed some exercises in each. In the next section, we begin to cover the various self-play scenarios.