Exercises 

Attempt one or two of the following exercises on your own:

  1. Run the CrawlerStaticTarget example scene and compare its performance to the dynamic sample.
  1. Double the time_horizon, batch_size, and buffer_size brain hyperparameters in one of the other control examples:
time_horizon: 2000
batch_size: 4048
buffer_size: 40480
  1. Perform the same modification of time_horizon, batch_size, and buffer_size on another control sample and observe the combined effect.
  2. Modify the num_layers and hidden_units brain hyperparameters to values we used in a control sample and apply them to a discrete action example, such as the Hallway example, as shown in the following code. How did it affect training?
num_layers: 3
hidden_units: 512
  1. Alter the num_layers and hidden_units hyperparameters on another continuous or discrete action example and combine it with other parameter modifications.
  2. Modify the lambda lambd brain hyperparameter in a discrete action example to a value of .99. Remember that this will have the effect of strengthening the rewards:
lambd: .99
  1. Create your own control creature with joints and limbs. A good place to start is using the Crawler example and modifying that.
  2. Modify one of the control samples by adding new limbs or joints. 
  3. Modify the Walker control example to give the agent a weapon and a target. You will have to combine elements of the Walker and Reacher examples.
  4. Run the VisualHallwayLearning sample scene with altered num_epoch and batch_size parameters. Are the results what you expected?

As we progress through the book, these exercises may become more and more tedious, especially if you run them on an older and slower system. However, it is important to understand how these parameters can alter an agent's training. 

When speaking to deep learning and RL practitioners, they will often compare the subtlely of training to the difference between being a good or great cook. A good cook may make things taste good and serve a completely acceptable meal, but it takes a great cook, and their attention to detail, to make you an exceptional meal that you will remember.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset