In previous chapters, we introduced many classification methods; each method has its own advantages and disadvantages. However, when it comes to the problem of how to choose the best fitted model, you need to compare all the performance measures generated from different prediction models. To make the comparison easy, the caret package allows us to generate and compare the performance of models. In this recipe, we will use the function provided by the caret package to compare different algorithm trained models on the same dataset.
Comparing an ROC curve using the caret package
by Chiu Yu-Wei, AshishSingh Bhatia
Machine Learning with R Cookbook - Second Edition
- Preface
- Practical Machine Learning with R
- Introduction
- Downloading and installing R
- Downloading and installing RStudio
- Installing and loading packages
- Understanding of basic data structures
- Basic commands for subsetting
- Reading and writing data
- Manipulating data
- Applying basic statistics
- Visualizing data
- Getting a dataset for machine learning
- Data Exploration with Air Quality Datasets
- Analyzing Time Series Data
- R and Statistics
- Introduction
- Understanding data sampling in R
- Operating a probability distribution in R
- Working with univariate descriptive statistics in R
- Performing correlations and multivariate analysis
- Conducting an exact binomial test
- Performing a student's t-test
- Performing the Kolmogorov-Smirnov test
- Understanding the Wilcoxon Rank Sum and Signed Rank test
- Working with Pearson's Chi-squared test
- Conducting a one-way ANOVA
- Performing a two-way ANOVA
- Understanding Regression Analysis
- Introduction
- Different types of regression
- Fitting a linear regression model with lm
- Summarizing linear model fits
- Using linear regression to predict unknown values
- Generating a diagnostic plot of a fitted model
- Fitting multiple regression
- Summarizing multiple regression
- Using multiple regression to predict unknown values
- Fitting a polynomial regression model with lm
- Fitting a robust linear regression model with rlm
- Studying a case of linear regression on SLID data
- Applying the Gaussian model for generalized linear regression
- Applying the Poisson model for generalized linear regression
- Applying the Binomial model for generalized linear regression
- Fitting a generalized additive model to data
- Visualizing a generalized additive model
- Diagnosing a generalized additive model
- Survival Analysis
- Classification 1 - Tree, Lazy, and Probabilistic
- Introduction
- Preparing the training and testing datasets
- Building a classification model with recursive partitioning trees
- Visualizing a recursive partitioning tree
- Measuring the prediction performance of a recursive partitioning tree
- Pruning a recursive partitioning tree
- Handling missing data and split and surrogate variables
- Building a classification model with a conditional inference tree
- Control parameters in conditional inference trees
- Visualizing a conditional inference tree
- Measuring the prediction performance of a conditional inference tree
- Classifying data with the k-nearest neighbor classifier
- Classifying data with logistic regression
- Classifying data with the Naïve Bayes classifier
- Classification 2 - Neural Network and SVM
- Introduction
- Classifying data with a support vector machine
- Choosing the cost of a support vector machine
- Visualizing an SVM fit
- Predicting labels based on a model trained by a support vector machine
- Tuning a support vector machine
- The basics of neural network
- Training a neural network with neuralnet
- Visualizing a neural network trained by neuralnet
- Predicting labels based on a model trained by neuralnet
- Training a neural network with nnet
- Predicting labels based on a model trained by nnet
- Model Evaluation
- Introduction
- Estimating model performance with k-fold cross-validation
- Estimating model performance with Leave One Out Cross Validation
- Performing cross-validation with the e1071 package
- Performing cross-validation with the caret package
- Ranking the variable importance with the caret package
- Ranking the variable importance with the rminer package
- Finding highly correlated features with the caret package
- Selecting features using the caret package
- Measuring the performance of the regression model
- Measuring prediction performance with a confusion matrix
- Measuring prediction performance using ROCR
- Comparing an ROC curve using the caret package
- Measuring performance differences between models with the caret package
- Ensemble Learning
- Introduction
- Using the Super Learner algorithm
- Using ensemble to train and test
- Classifying data with the bagging method
- Performing cross-validation with the bagging method
- Classifying data with the boosting method
- Performing cross-validation with the boosting method
- Classifying data with gradient boosting
- Calculating the margins of a classifier
- Calculating the error evolution of the ensemble method
- Classifying data with random forest
- Estimating the prediction errors of different classifiers
- Clustering
- Introduction
- Clustering data with hierarchical clustering
- Cutting trees into clusters
- Clustering data with the k-means method
- Drawing a bivariate cluster plot
- Comparing clustering methods
- Extracting silhouette information from clustering
- Obtaining the optimum number of clusters for k-means
- Clustering data with the density-based method
- Clustering data with the model-based method
- Visualizing a dissimilarity matrix
- Validating clusters externally
- Association Analysis and Sequence Mining
- Introduction
- Transforming data into transactions
- Displaying transactions and associations
- Mining associations with the Apriori rule
- Pruning redundant rules
- Visualizing association rules
- Mining frequent itemsets with Eclat
- Creating transactions with temporal information
- Mining frequent sequential patterns with cSPADE
- Using the TraMineR package for sequence analysis
- Visualizing sequence, Chronogram, and Traversal Statistics
- Dimension Reduction
- Introduction
- Why to reduce the dimension?
- Performing feature selection with FSelector
- Performing dimension reduction with PCA
- Determining the number of principal components using the scree test
- Determining the number of principal components using the Kaiser method
- Visualizing multivariate data using biplot
- Performing dimension reduction with MDS
- Reducing dimensions with SVD
- Compressing images with SVD
- Performing nonlinear dimension reduction with ISOMAP
- Performing nonlinear dimension reduction with Local Linear Embedding
- Big Data Analysis (R and Hadoop)
- Introduction
- Preparing the RHadoop environment
- Installing rmr2
- Installing rhdfs
- Operating HDFS with rhdfs
- Implementing a word count problem with RHadoop
- Comparing the performance between an R MapReduce program and a standard R program
- Testing and debugging the rmr2 program
- Installing plyrmr
- Manipulating data with plyrmr
- Conducting machine learning with RHadoop
- Configuring RHadoop clusters on Amazon EMR
Comparing an ROC curve using the caret package
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.