Training multiple agents with one demonstration

Now, with the recording of us playing tennis, we can use this to feed into the training of multiple agents all feeding back into one policy. Open Unity to the tennis scene, the one with the multiple environments, and follow the next exercise:

  1. Type agent into the Filter bar at the top of the Hierarchy window as shown in the following screenshot:

Searching for all the agents in the scene
  1. Select all the agent objects in the scene and bulk change their Brain to use TennisLearning and not TennisPlayer.
  2. Select Academy and make sure to enable it to control the brains.
  3. Open the config/offline_bc_config.yaml file.
  1. Add the following new section for the TennisLearning brain at the bottom:
TennisLearning:
trainer: offline_bc
max_steps: 5.0e5
num_epoch: 5
batch_size: 64
batches_per_epoch: 5
num_layers: 2
hidden_units: 128
sequence_length: 16
use_recurrent: true
memory_size: 256
sequence_length: 32
demo_path: ./UnitySDK/Assets/Demonstrations/TennisAgent.demo
  1. Save the scene and the project.
  2. Open the Python/Anaconda window and run training with the following code:
mlagents-learn config/offline_bc_config.yaml --run-id=tennis_ma --train
  1. You may want to add the --slow switch in order to watch the training, but it should not be required.
  2. Let the agents train for some time and notice its improved progress. Even with a short observation recording input, the agent becomes a capable player rather quickly.

There are multiple ways to perform this type of IL andtransfer learningchaining that will allow your agent some flexibility in training. You could even use the trained model's checkpoint without IL and run the agents with transfer learning as we did earlier. The possibilities are limitless, and it remains to be seen what will emerge as best practices.

In the next section, we'll provide some exercises that you can use for your own personal learning.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset