Transfer learning

Imitation Learning, by definition, falls into a category of Transfer Learning (TL). We can define Transfer Learning as the process by which an agent or DL network is trained by transference of experiences from one to the other. This could be as simple as the observation training we just performed, or as complex as swapping layers/layer weights in an agent's brain, or just training an agent on a similar task. 

Intransfer learningwe need to make sure the experiences or previous weights we use are generalized. Through the foundational chapters in this book (chapters 1-3), we learned the value of generalization using techniques such as dropout and batch normalization. We learned that these techniques are important for more general training; the form of training that allows the agent/network better inference on test data. This is no different than if we were to use an agent trained on one task to learn on another task. A more general agent will, in effect, be able to transfer knowledge more readily than a specialist agent could, if at all.

We can demonstrate this in a quick example starting with training the following simple exercise:

  1. Open up the VisualHallway scene in the Unity editor.
  2. Disable any additional training areas.
  3. Confirm that Academy is in Control of the Brain.
  4. Select the VisualHallwayLearning brain from the Hallway/Brains folder and set Vector Action | Branches Size | Branch 0 Size to 7, as shown in the following screenshot:

Increasing the vector action space of the agent
  1. We increase the action space for the brain so that it is compatible with the required action space for our transfer learning environment, which we will get to later.
  2. Save the scene and project.
  3. Open a Python/Anaconda window that is prepared for training.
  4. Launch a training session with the following code:
mlagents-learn config/trainer_config.yaml --run-id=vishall --train  --save-freq=10000
  1. Here, we have introduced a new parameter that controls the frequency at which model checkpoints are created. The default is currently set to 50,000, but we just don't want to wait that long.
  2. Run the agent in training in the editor for at least one model checkpoint save, as shown in the following screen excerpt:

The ML-Agents trainer creating a checkpoint
  1. Checkpoints are a way of taking snapshots of a brain and saving them for later. This allows you to go back and continue training where you left off.
  2. Let the agent train to a checkpoint and then terminate training by pressing Ctrl or command C on Mac in the Python/Anaconda window.

When you have terminated training, it is time to try this saved brain on another learning environment in the next section.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset