In the absence of a model, model-free (MF) algorithms run trajectories within a given policy to gain experience and to improve the agent. MF algorithms are made up of three main steps that are repeated until a good policy is created:

1. The generation of new samples by running the policy in the environment. The trajectories are run until a final state is reached or for a fixed number of steps. 
2. The estimation of the return function.
3. The improvement of the policy using the samples collected, and the estimation done in step 2. 

These three components are at the heart of this type of algorithm, but based on how each step is performed, they generate different algorithms. Value-based algorithms and policy gradient algorithms are two such examples. They seem to be very different, but they are based on similar principles and both use the three-step approach.