Features correlated with the outcome are desirable, but those that are also correlated among themselves can make the model unstable. In this recipe, we will show you how to calculate correlations between features.
- Title Page
- Copyright and Credits
- Packt Upsell
- Contributors
- Preface
- Installing and Configuring Spark
- Abstracting Data with RDDs
- Introduction
- Creating RDDs
- Reading data from files
- Overview of RDD transformations
- Getting ready
- How to do it...
- .map(...) transformation
- .filter(...) transformation
- .flatMap(...) transformation
- .distinct() transformation
- .sample(...) transformation
- .join(...) transformation
- .repartition(...) transformation
- .zipWithIndex() transformation
- .reduceByKey(...) transformation
- .sortByKey(...) transformation
- .union(...) transformation
- .mapPartitionsWithIndex(...) transformation
- How it works...
- Overview of RDD actions
- Pitfalls of using RDDs
- Abstracting Data with DataFrames
- Introduction
- Creating DataFrames
- Accessing underlying RDDs
- Performance optimizations
- Inferring the schema using reflection
- Specifying the schema programmatically
- Creating a temporary table
- Using SQL to interact with DataFrames
- Overview of DataFrame transformations
- Getting ready
- How to do it...
- The .select(...) transformation
- The .filter(...) transformation
- The .groupBy(...) transformation
- The .orderBy(...) transformation
- The .withColumn(...) transformation
- The .join(...) transformation
- The .unionAll(...) transformation
- The .distinct(...) transformation
- The .repartition(...) transformation
- The .fillna(...) transformation
- The .dropna(...) transformation
- The .dropDuplicates(...) transformation
- The .summary() and .describe() transformations
- The .freqItems(...) transformation
- See also
- Overview of DataFrame actions
- Preparing Data for Modeling
- Machine Learning with MLlib
- Machine Learning with the ML Module
- Introducing Transformers
- Introducing Estimators
- Introducing Pipelines
- Selecting the most predictable features
- Predicting forest coverage types
- Estimating forest elevation
- Clustering forest cover types
- Tuning hyperparameters
- Extracting features from text
- Discretizing continuous variables
- Standardizing continuous variables
- Topic mining
- Structured Streaming with PySpark
- GraphFrames – Graph Theory with PySpark
Computing correlations
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.