Whenever we are building a model, it needs to be tested and evaluated to ensure that it will not only work on trained data, but also on unseen data and can generate results with accuracy. A model should not generate a random result though some noise is permitted. If the model is not evaluated properly then the chances are that the result produced with unseen data is not accurate. Furthermore, model evaluation can help select the optimum model, which is more robust and can accurately predict responses for future subjects.
Why do models need to be evaluated?
by Chiu Yu-Wei, AshishSingh Bhatia
Machine Learning with R Cookbook - Second Edition
- Preface
- Practical Machine Learning with R
- Introduction
- Downloading and installing R
- Downloading and installing RStudio
- Installing and loading packages
- Understanding of basic data structures
- Basic commands for subsetting
- Reading and writing data
- Manipulating data
- Applying basic statistics
- Visualizing data
- Getting a dataset for machine learning
- Data Exploration with Air Quality Datasets
- Analyzing Time Series Data
- R and Statistics
- Introduction
- Understanding data sampling in R
- Operating a probability distribution in R
- Working with univariate descriptive statistics in R
- Performing correlations and multivariate analysis
- Conducting an exact binomial test
- Performing a student's t-test
- Performing the Kolmogorov-Smirnov test
- Understanding the Wilcoxon Rank Sum and Signed Rank test
- Working with Pearson's Chi-squared test
- Conducting a one-way ANOVA
- Performing a two-way ANOVA
- Understanding Regression Analysis
- Introduction
- Different types of regression
- Fitting a linear regression model with lm
- Summarizing linear model fits
- Using linear regression to predict unknown values
- Generating a diagnostic plot of a fitted model
- Fitting multiple regression
- Summarizing multiple regression
- Using multiple regression to predict unknown values
- Fitting a polynomial regression model with lm
- Fitting a robust linear regression model with rlm
- Studying a case of linear regression on SLID data
- Applying the Gaussian model for generalized linear regression
- Applying the Poisson model for generalized linear regression
- Applying the Binomial model for generalized linear regression
- Fitting a generalized additive model to data
- Visualizing a generalized additive model
- Diagnosing a generalized additive model
- Survival Analysis
- Classification 1 - Tree, Lazy, and Probabilistic
- Introduction
- Preparing the training and testing datasets
- Building a classification model with recursive partitioning trees
- Visualizing a recursive partitioning tree
- Measuring the prediction performance of a recursive partitioning tree
- Pruning a recursive partitioning tree
- Handling missing data and split and surrogate variables
- Building a classification model with a conditional inference tree
- Control parameters in conditional inference trees
- Visualizing a conditional inference tree
- Measuring the prediction performance of a conditional inference tree
- Classifying data with the k-nearest neighbor classifier
- Classifying data with logistic regression
- Classifying data with the Naïve Bayes classifier
- Classification 2 - Neural Network and SVM
- Introduction
- Classifying data with a support vector machine
- Choosing the cost of a support vector machine
- Visualizing an SVM fit
- Predicting labels based on a model trained by a support vector machine
- Tuning a support vector machine
- The basics of neural network
- Training a neural network with neuralnet
- Visualizing a neural network trained by neuralnet
- Predicting labels based on a model trained by neuralnet
- Training a neural network with nnet
- Predicting labels based on a model trained by nnet
- Model Evaluation
- Introduction
- Estimating model performance with k-fold cross-validation
- Estimating model performance with Leave One Out Cross Validation
- Performing cross-validation with the e1071 package
- Performing cross-validation with the caret package
- Ranking the variable importance with the caret package
- Ranking the variable importance with the rminer package
- Finding highly correlated features with the caret package
- Selecting features using the caret package
- Measuring the performance of the regression model
- Measuring prediction performance with a confusion matrix
- Measuring prediction performance using ROCR
- Comparing an ROC curve using the caret package
- Measuring performance differences between models with the caret package
- Ensemble Learning
- Introduction
- Using the Super Learner algorithm
- Using ensemble to train and test
- Classifying data with the bagging method
- Performing cross-validation with the bagging method
- Classifying data with the boosting method
- Performing cross-validation with the boosting method
- Classifying data with gradient boosting
- Calculating the margins of a classifier
- Calculating the error evolution of the ensemble method
- Classifying data with random forest
- Estimating the prediction errors of different classifiers
- Clustering
- Introduction
- Clustering data with hierarchical clustering
- Cutting trees into clusters
- Clustering data with the k-means method
- Drawing a bivariate cluster plot
- Comparing clustering methods
- Extracting silhouette information from clustering
- Obtaining the optimum number of clusters for k-means
- Clustering data with the density-based method
- Clustering data with the model-based method
- Visualizing a dissimilarity matrix
- Validating clusters externally
- Association Analysis and Sequence Mining
- Introduction
- Transforming data into transactions
- Displaying transactions and associations
- Mining associations with the Apriori rule
- Pruning redundant rules
- Visualizing association rules
- Mining frequent itemsets with Eclat
- Creating transactions with temporal information
- Mining frequent sequential patterns with cSPADE
- Using the TraMineR package for sequence analysis
- Visualizing sequence, Chronogram, and Traversal Statistics
- Dimension Reduction
- Introduction
- Why to reduce the dimension?
- Performing feature selection with FSelector
- Performing dimension reduction with PCA
- Determining the number of principal components using the scree test
- Determining the number of principal components using the Kaiser method
- Visualizing multivariate data using biplot
- Performing dimension reduction with MDS
- Reducing dimensions with SVD
- Compressing images with SVD
- Performing nonlinear dimension reduction with ISOMAP
- Performing nonlinear dimension reduction with Local Linear Embedding
- Big Data Analysis (R and Hadoop)
- Introduction
- Preparing the RHadoop environment
- Installing rmr2
- Installing rhdfs
- Operating HDFS with rhdfs
- Implementing a word count problem with RHadoop
- Comparing the performance between an R MapReduce program and a standard R program
- Testing and debugging the rmr2 program
- Installing plyrmr
- Manipulating data with plyrmr
- Conducting machine learning with RHadoop
- Configuring RHadoop clusters on Amazon EMR
Why do models need to be evaluated?
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.