The concept of generalization refers to two aspects that are different, but somehow related. In general terms, the concept of generalization in reinforcement learning refers to the capability of an algorithm to obtain good performance in a related environment. For example, if an agent has been trained to walk on dirty roads, we might expect that the same agent will perform well on paved roads. The better the generalization capabilities, the better the agent will perform in different environments. The second and lesser-used means of generalization refers to the property of the algorithm to achieve good performance in an environment where only limited data can be gathered.

In RL, the agent can choose the states to visit by itself and do so for as long as it wants so that it can also overfit on a certain problem space. However, if good generalization capabilities are required, a trade-off has to be found. This is only partially true if the agent is allowed to gather potentially infinite data for the environment as it will act as a sort of self-regularization method.

Nonetheless, to help with generalization across other environments, an agent must be capable of abstract reasoning to discern from the mere state-action mapping and interpret the task using multiple factors. Examples of abstract reasoning can be found in model-based reinforcement learning, transfer learning, and in the use of auxiliary tasks. We'll cover the latter topic later, but in brief, it is a technique that's used to improve generalization and sample efficiency by augmenting an RL agent with auxiliary tasks that were learned jointly with the main task.