Multi-brain play

One of the truly great things about the ML-Agents kit is the ability to add multiple agents powered by multiple brains quickly. This in turns gives us the ability to build more complex game environments or scenarios with fun agents/AI to play both with and against. Let's see how easy it is to convert our soccer example to let the agents all use individual brains:

  1. Open up the editor to the SoccerTwos scene we looked at earlier.
  2. Locate the Brains folder for the example at Assets | ML-Agents | Examples | Soccer | Brains.
  3. Click the Create menu in the upper right corner of the window and from the Context menu, and select ML-Agents | Learning Brain:

Creating a new learning brain
  1. Name the new brain RedStrikerLearning.  Create three more new brains named RedGoalieLearning, BlueGoalieLearning, and BlueStrikerLearning in the same folder.
  1. Select RedStrikerLearning. Then select and drag the StrikerLearning brain and drop it into the Copy Brain Parameters from slot:

Copying brain parameters from another brain
  1. Do this for BlueStrikerLearning, copying parameters from StrikerLearning. Then do the same for the RedGoalieLearning and BlueGoalieLearning, copying parameters from GoalieLearning.
  2. Select the RedAgent in the Hierarchy window and set the Agent Soccer | Brain to RedStrikerLearning. Do this for each of the other agents, matching the color with a position.  BlueGoalie -> BlueGoalieLearning.
  3. Select Academy and remove all the current Brains from the Soccer Academy | Brains list. Then add all the new brains we just created back into the list using the Add New button and set them to Control:

Adding the new brains to Academy
  1. Save the scene and the project. Now, we just swapped the example from using two concurrent brains in self-play mode to be individual agents on teams.  
  2. Open a Python/Anaconda window set up for training and launch with it the following:
mlagents-learn config/trainer_config.yaml --run-id=soccer_mb --train
  1. Let the training run and note how the agents start off playing just as well as they did previously. Take a look at the console output as well. You will see it now reports for four agents, but the agents are still somewhat symbiotic, as the red striker is opposite the blue goalie. However, they now train much more slowly, due in part to each brain seeing only half the observations now. Remember that we had both striker agents feeding to a single brain previously, and, as we learned, this additional input of state can expedite training substantially.

At this point, we have four agents with four individual brains playing a game of soccer. Of course, since the agents are still training symbiotically by sharing a reward function, we can't really describe them as individuals. Except, as we know, individuals who play on teams are often influenced by their own internal or intrinsic reward system. We will look at how the application of intrinsic rewards can make this last exercise more interesting in the next section.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset