- Python: Advanced Predictive Analytics
- Table of Contents
- Python: Advanced Predictive Analytics
- Credits
- Preface
- 1. Module 1
- 1. Getting Started with Predictive Modelling
- 2. Data Cleaning
- Reading the data – variations and examples
- Various methods of importing data in Python
- Basics – summary, dimensions, and structure
- Handling missing values
- Creating dummy variables
- Visualizing a dataset by basic plotting
- Summary
- 3. Data Wrangling
- Subsetting a dataset
- Generating random numbers and their usage
- Grouping the data – aggregation, filtering, and transformation
- Random sampling – splitting a dataset in training and testing datasets
- Concatenating and appending data
- Merging/joining datasets
- Summary
- 4. Statistical Concepts for Predictive Modelling
- 5. Linear Regression with Python
- 6. Logistic Regression with Python
- Linear regression versus logistic regression
- Understanding the math behind logistic regression
- Implementing logistic regression with Python
- Model validation and evaluation
- Model validation
- Summary
- 7. Clustering with Python
- 8. Trees and Random Forests with Python
- 9. Best Practices for Predictive Modelling
- A. A List of Links
- 2. Module 2
- 1. From Data to Decisions – Getting Started with Analytic Applications
- 2. Exploratory Data Analysis and Visualization in Python
- 3. Finding Patterns in the Noise – Clustering and Unsupervised Learning
- 4. Connecting the Dots with Models – Regression Methods
- 5. Putting Data in its Place – Classification Methods and Analysis
- 6. Words and Pixels – Working with Unstructured Data
- 7. Learning from the Bottom Up – Deep Networks and Unsupervised Features
- Learning patterns with neural networks
- A network of one – the perceptron
- Combining perceptrons – a single-layer neural network
- Parameter fitting with back-propagation
- Discriminative versus generative models
- Vanishing gradients and explaining away
- Pretraining belief networks
- Using dropout to regularize networks
- Convolutional networks and rectified units
- Compressing Data with autoencoder networks
- Optimizing the learning rate
- The TensorFlow library and digit recognition
- Summary
- Learning patterns with neural networks
- 8. Sharing Models with Prediction Services
- 9. Reporting and Testing – Iterating on Analytic Systems
- Bibliography
- Index
Creating dummy variables is a method to create separate variable for each category of a categorical variable., Although, the categorical variable contains plenty of information and might show a causal relationship with output variable, it can't be used in the predictive models like linear and logistic regression without any processing.
In our dataset, sex is a categorical variable with two categories that are male and female. We can create two dummy variables out of this, as follows:
dummy_sex=pd.get_dummies(data['sex'],prefix='sex')
The result of this statement is, as follows:
Fig. 2.17: Dummy variable for the sex variable in the Titanic dataset
This process is called dummifying, the variable creates two new variables that take either 1 or 0 value depending on what the sex of the passenger was. If the sex was female, sex_female would be 1 and sex_male would be 0. If the sex was male, sex_male would be 1 and sex_female would be 0. In general, all but one dummy variable in a row will have a 0 value. The variable derived from the value (for that row) in the original column will have a value of 1.
These two new variables can be joined to the source data frame, so that they can be used in the models. The method to that is illustrated, as follows:
column_name=data.columns.values.tolist()
column_name.remove('sex')
data[column_name].join(dummy_sex)The column names are converted to a list and the sex is removed from the list before joining these two dummy variables to the dataset, as it will not make sense to have a sex variable with these two dummy variables.
-
No Comment