The SelectFromModel selector is able to handle any machine learning model that exposes a feature_importances_ or coef_ attribute post-fitting. Tree-based models expose the former, while linear models expose the latter. After fitting, linear models such as Linear Regression, Logistic Regression, Support Vector Machines, and others all place coefficients in front of features that represent the slope of that feature/how much it affects the response when that feature is changed. SelectFromModel can equate this to a feature importance and choose features based on the coefficients given to features while fitting.

Before we can use these models, however, we must introduce a concept called regularization, which will help us select truly only the most important features.