Offline training

Offline training is where a recorded gameplay file is generated from a player or agent playing a game or performing a task, and is then fed back as training observations to help an agent learn later on. While online learning certainly is more fun, and in some ways more applicable to the Tennis scene or other multiplayer games, it is less practical. After all, you generally need to play an agent in real time for several hours before an agent will become good. Likewise, in online training scenarios, you are typically limited to single agent training, whereas in offline training a demo playback can be fed to multiple agents for better overall learning. This also allows us to perform interesting training scenarios, similar to AlphaStar training, where we can teach an agent so that it can teach other agents. 

We will learn more about multi-agent gameplay in Chapter 11Building Multi-Agent Environments.

For this next exercise, we are going to revisit our old friend the Hallway/VisualHallway example. Again, we are doing this so we can compare our results to the previous sample exercises we ran with this environment. Follow this exercise to set up a new offline training session:

  1. Clone and download the ML-Agents code to a new folder, perhaps choosing ml-agents_bml-agents_c, or some other name. The reason we do this is to make sure that we run these new exercises with a clean environment. Also, it can sometimes help to go back to old environments and recall settings or configuration that you may forget to update. 
  2. Launch Unity and open the UnitySDK project and the Hallway or VisualHallway scene, your choice.
  1. The scene should be set to run in Player mode. Just confirm this. If you need to change it, then do so.
  2. Disable any additional agent training environments in the scene if others are active.
  3. Select HallwayArea | Agent in the Hierarchy window.
  4. Click the Add Component button at the bottom of the Inspector window, type demo, and select the Demonstration Recorder component as shown in the following screenshot:

Adding a Demonstration Recorder
  1. Click Record on the new Demonstration Recorder component, as shown in the preceding screenshot, check throughout. Also, fill in the Demonstration Name property of the recording, which is also shown.
  2. Save the scene and project.
  3. Press Play and play the scene for a fair amount of time, more than a few minutes but perhaps less than hours. Of course, how well you play will also determine how well the agent learns. If you play poorly, so will the agent.
  4. After you think enough time has passed, and you have played as well as you could, stop the game.

After playing the game, you should see a new folder called Demonstrations created in the Assets root folder in your Project window. Inside the folder will be your demonstration recording. This is the recording we will feed the agent in the next section.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset