Unity has embraced machine learning, and deep reinforcement learning in particular, with determination and vigor with the aim of producing a working seep reinforcement learning (DRL) SDK for game and simulation developers. Fortunately, the team at Unity, led by Danny Lange, has succeeded in developing a robust cutting-edge DRL engine capable of impressive results. This engine is the top of the line and outclasses the DQN model we introduced earlier in many ways. Unity uses a proximal policy optimization (PPO) model as the basis for its DRL engine. This model is significantly more complex and may differ in some ways, but, fortunately, this is at the start of many more chapters, and we will have plenty of time to introduce the concepts as we go—this is a hands-on book, after all.
In this chapter, we introduce the Unity ML-Agents tools and SDK for building DRL agents to play games and simulations. While this tool is both powerful and cutting-edge, it is also easy to use and provides a few tools to help us learn concepts as we go. In this chapter, we will cover the following topics:
- Installing ML-Agents
- Training an agent
- What's in a brain?
- Monitoring training with TensorBoard
- Running an agent
- Danny Lange (https://arxiv.org/search/cs?searchtype=author&query=Lange%2C+D)
- Arthur Juliani (https://arxiv.org/search/cs?searchtype=author&query=Juliani%2C+A)
- Vincent-Pierre Berges (https://arxiv.org/search/cs?searchtype=author&query=Berges%2C+V)
- Esh Vckay (https://arxiv.org/search/cs?searchtype=author&query=Vckay%2C+E)
- Yuan Gao (https://arxiv.org/search/cs?searchtype=author&query=Gao%2C+Y)
- Hunter Henry (https://arxiv.org/search/cs?searchtype=author&query=Henry%2C+H)
- Marwan Mattar (https://arxiv.org/search/cs?searchtype=author&query=Mattar%2C+M)
- Adam Crespi (https://arxiv.org/search/cs?searchtype=author&query=Crespi%2C+A)
- Jonathan Harper (https://arxiv.org/search/cs?searchtype=author&query=Harper%2C+J)
Be sure you have Unity installed as per the section in Chapter 4, Building a Deep Learning Gaming Chatbot, before proceeding with this chapter.