Fortunately, setting up the agent to train it visually is quite straightforward, especially if you worked through the exercises in the last chapter. Open the Unity editor to the VisualHallway scene, have a Python command or Anaconda window ready, and let's begin:

1. In Unity, change Agent | Hallway Agent | Brain to VisualHallwayLearning, as shown in the following screenshot:

2. Click on the VisualHallwayLearning brain to locate it in the Project window.
3. Click on the VisualHallwayLearning brain to view its properties in the Inspector window, and as shown in the following screen excerpt:

4. Make sure that the Brain parameters are set to accept a single Visual Observation at a resolution of 84 x 84 pixels, and are not using Gray scale. Gray is simply the removal of the color channels, which makes the input one channel instead of three. Recall our discussion of CNN layers in Chapter 2, Convolutional and Recurrent Networks. Also, be sure that the Vector Observation | Space Size is 0, as shown in the preceding screenshot.
5. From the Menu, select File | Save and File | Save Project to save all your changes.
6. Switch to your Python window or Anaconda prompt, make sure you are in the ML-Agents/ml-agents directory, and run the following command:

mlagents-learn config/trainer_config.yaml --run-id=visualhallway --train

7. After the command runs, wait for the prompt to start the editor. Then, run the editor when prompted and let the sample run to completion, or however long you have the patience for.
8. After you run the sample to completion, you should see something like the following:

Assuming you trained your agent to the end of the run that is, for 500 K iterations, then you can confirm that the agent does, in fact, learn nothing. So, why would Unity put an example like that in their samples? Well, you could argue that it was an intentional challenge, or perhaps just an oversight on their part. Either way, we will take it as a challenge to better understand reinforcement learning.

Before we tackle this challenge, let's take a step back and reaffirm our understanding of this environment by looking at the easier to train Hallway example in the next section.