Often, it is hard to get our hands on data that is labeled. Also, sometimes you might want to find underlying patterns in your dataset. In this recipe, we will learn how to build the popular k-means clustering model in Spark.
- Title Page
- Copyright and Credits
- Packt Upsell
- Contributors
- Preface
- Installing and Configuring Spark
- Abstracting Data with RDDs
- Introduction
- Creating RDDs
- Reading data from files
- Overview of RDD transformations
- Getting ready
- How to do it...
- .map(...) transformation
- .filter(...) transformation
- .flatMap(...) transformation
- .distinct() transformation
- .sample(...) transformation
- .join(...) transformation
- .repartition(...) transformation
- .zipWithIndex() transformation
- .reduceByKey(...) transformation
- .sortByKey(...) transformation
- .union(...) transformation
- .mapPartitionsWithIndex(...) transformation
- How it works...
- Overview of RDD actions
- Pitfalls of using RDDs
- Abstracting Data with DataFrames
- Introduction
- Creating DataFrames
- Accessing underlying RDDs
- Performance optimizations
- Inferring the schema using reflection
- Specifying the schema programmatically
- Creating a temporary table
- Using SQL to interact with DataFrames
- Overview of DataFrame transformations
- Getting ready
- How to do it...
- The .select(...) transformation
- The .filter(...) transformation
- The .groupBy(...) transformation
- The .orderBy(...) transformation
- The .withColumn(...) transformation
- The .join(...) transformation
- The .unionAll(...) transformation
- The .distinct(...) transformation
- The .repartition(...) transformation
- The .fillna(...) transformation
- The .dropna(...) transformation
- The .dropDuplicates(...) transformation
- The .summary() and .describe() transformations
- The .freqItems(...) transformation
- See also
- Overview of DataFrame actions
- Preparing Data for Modeling
- Machine Learning with MLlib
- Machine Learning with the ML Module
- Introducing Transformers
- Introducing Estimators
- Introducing Pipelines
- Selecting the most predictable features
- Predicting forest coverage types
- Estimating forest elevation
- Clustering forest cover types
- Tuning hyperparameters
- Extracting features from text
- Discretizing continuous variables
- Standardizing continuous variables
- Topic mining
- Structured Streaming with PySpark
- GraphFrames – Graph Theory with PySpark
Building a clustering models
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.