Recall that in Chapter 9, Stock Prices Prediction with Regression Algorithms, SGD-based linear regression and SVR models require features to be standardized by removing the mean and rescaling to unit variance. So when is feature scaling needed and when is it not?

In general, naive Bayes and tree-based algorithms are not sensitive to features at different scales, as they look at each feature independently. Logistic or linear regression normally is not affected by the scales of input features, with one exception, when the weights are optimized with stochastic gradient descent.

In most cases, an algorithm that involves any form of distance (separation in spaces) of samples in learning factors requires scaled/standardized input features, such as SVC and SVR. Feature scaling is also a must for any algorithm using SGD for optimization. We have so far covered tips regarding data preprocessing and we will now discuss best practices of feature engineering as another major aspect of training sets generation. We will do so from two perspectives: