RL has become so popular that there is now a race to just build tools that help build RL algorithms. The two major competitors in this area right now are OpenAI Gym and Unity. Unity has quickly become the RL racing machine we will explore extensively later. For now, we will put our training wheels on and run OpenAI Gym to explore the fundamentals of RL further.

We need to install the OpenAI Gym toolkit before we can continue, and installation may vary greatly depending on your operating system. As such, we will focus on the Windows installation instructions here, as it is likely other OS users will have less difficulty. Follow the next steps to install OpenAI Gym on Windows:

1. Install a C++ compiler; if you have Visual Studio 2017 installed, you may already have a recommended one. You can find other supported compilers here: https://wiki.python.org/moin/WindowsCompilers.
2. Be sure to have Anaconda installed, and open an Anaconda command prompt and run the following commands:

conda create -n gym
conda activate gym
conda install python=3.5  # reverts Python, for use with TensorFlow later
pip install tensorflow
pip install keras
pip install gym

3. For our purposes, in the short term, we don't need to install any other Gym modules. Gym has plenty of example environments, Atari games and MuJoCo (robotics simulator) being some of the most fun to work with. We will take a look at the Atari games module later in this chapter.

That should install the Gym environment for your system. Most of what we need will work with minimal setup. If you decide to do more with Gym, then you will likely want to install other modules; there are several. In the next section, we are going to test this new environment as we learn about Q-Learning.