8. Learning with Ensembles
by John Hearty, David Julian, Sebastian Raschka
Python: Deeper Insights into Machine Learning
- Python: Deeper Insights into Machine Learning
- Table of Contents
- Python: Deeper Insights into Machine Learning
- Python: Deeper Insights into Machine Learning
- Credits
- Preface
- 1. Module 1
- 1. Giving Computers the Ability to Learn from Data
- Building intelligent machines to transform data into knowledge
- The three different types of machine learning
- An introduction to the basic terminology and notations
- A roadmap for building machine learning systems
- Using Python for machine learning
- Summary
- 2. Training Machine Learning Algorithms for Classification
- 3. A Tour of Machine Learning Classifiers Using Scikit-learn
- Choosing a classification algorithm
- First steps with scikit-learn
- Modeling class probabilities via logistic regression
- Maximum margin classification with support vector machines
- Solving nonlinear problems using a kernel SVM
- Decision tree learning
- K-nearest neighbors – a lazy learning algorithm
- Summary
- 4. Building Good Training Sets – Data Preprocessing
- 5. Compressing Data via Dimensionality Reduction
- 6. Learning Best Practices for Model Evaluation and Hyperparameter Tuning
- 7. Combining Different Models for Ensemble Learning
- 8. Applying Machine Learning to Sentiment Analysis
- 9. Embedding a Machine Learning Model into a Web Application
- 10. Predicting Continuous Target Variables with Regression Analysis
- Introducing a simple linear regression model
- Exploring the Housing Dataset
- Implementing an ordinary least squares linear regression model
- Fitting a robust regression model using RANSAC
- Evaluating the performance of linear regression models
- Using regularized methods for regression
- Turning a linear regression model into a curve – polynomial regression
- Summary
- 11. Working with Unlabeled Data – Clustering Analysis
- 12. Training Artificial Neural Networks for Image Recognition
- Modeling complex functions with artificial neural networks
- Classifying handwritten digits
- Training an artificial neural network
- Developing your intuition for backpropagation
- Debugging neural networks with gradient checking
- Convergence in neural networks
- Other neural network architectures
- A few last words about neural network implementation
- Summary
- 13. Parallelizing Neural Network Training with Theano
- 1. Giving Computers the Ability to Learn from Data
- 2. Module 2
- 1. Thinking in Machine Learning
- 2. Tools and Techniques
- 3. Turning Data into Information
- 4. Models – Learning from Information
- 5. Linear Models
- 6. Neural Networks
- 7. Features – How Algorithms See the World
- 8. Learning with Ensembles
- 9. Design Strategies and Case Studies
- 3. Module 3
- 1. Unsupervised Machine Learning
- 2. Deep Belief Networks
- 3. Stacked Denoising Autoencoders
- 4. Convolutional Neural Networks
- 5. Semi-Supervised Learning
- 6. Text Feature Engineering
- 7. Feature Engineering Part II
- 8. Ensemble Methods
- 9. Additional Python Machine Learning Tools
- 10. Chapter Code Requirements
- A. Biblography
- Index
The motivation for creating machine learning ensembles comes from clear intuitions and is grounded in a rich theoretical history. Diversity, in many natural and human-made systems, makes them more resilient to perturbations. Similarly, we have seen that averaging results from a number of measurements can often result in a more stable models that are less susceptible to random fluctuations, such as outliers or errors in data collection.
In this chapter, we will divide this rather large and diverse space into the following topics:
- Ensemble types
- Bagging
- Random forests
- Boosting
Ensemble techniques can be broadly divided into two types:
- Averaging method: This is the method in which several estimators are run independently and their predictions are averaged. This includes random forests and bagging methods.
- Boosting method: This is the method in which weak learners are built sequentially using weighted distributions of the data based on the error rates.
Ensemble methods use multiple models to obtain better performance than any single constituent model. The aim is to not only build diverse and robust models, but also work within limitations, such as processing speed and return times. When working with large datasets and quick response times, this can be a significant developmental bottleneck. Troubleshooting and diagnostics are an important aspect of working with all machine learning models, but especially when we are dealing with models that may take days to run.
The types of machine learning ensembles that can be created are as diverse as the models themselves, and the main considerations revolve around three things: how we divide our data, how we select the models, and the methods we use to combine their results. This simplistic statement actually encompasses a very large and diverse space.
-
No Comment