Stacking is the second ensemble learning technique that we will study. Together with voting, it belongs to the non-generative methods class, as they both use individually trained classifiers as base learners.

In this chapter, we will present the main ideas behind stacking, its strengths and weaknesses, and how to select base learners. Furthermore, we will go through the processes of implementing stacking for both regression and classification problems with scikit-learn.

The main topics covered in this chapter are as follows:

• The methodology of stacking and using a meta-learner to combine predictions
• The motivation behind using stacking
• The strengths and weaknesses of stacking
• Selecting base learners for an ensemble
• Implementing stacking for regression and classification problems