Graphing Anscombe's quartet

Anscombe's quartet is a classic example that illustrates why visualizing data is important. The quartet consists of four datasets with similar statistical properties. Each dataset has a series of x values and dependent y values. We will tabulate these metrics in an IPython notebook. However, if you plot the datasets, they look surprisingly different compared to each other.

How to do it...

For this recipe, you need to perform the following steps:

  1. Start with the following imports:
    import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib as mpl
from dautil import report
from dautil import plotting
import numpy as np
from tabulate import tabulate
  2. Define the following function to compute the mean, variance, and correlation of x and y within a dataset, the slope, and the intercept of a linear fit for each of the datasets:
    df = sns.load_dataset("anscombe")

    agg = df.groupby('dataset')
             .agg([np.mean, np.var])
             .transpose()
    groups = df.groupby('dataset')

    corr = [g.corr()['x'][1] for _, g in groups]
    builder = report.DFBuilder(agg.columns)
    builder.row(corr)

    fits = [np.polyfit(g['x'], g['y'], 1) for _, g in groups]
    builder.row([f[0] for f in fits])
    builder.row([f[1] for f in fits])
    bottom = builder.build(['corr', 'slope', 'intercept'])

    return df, pd.concat((agg, bottom))
  3. The following function returns a string, which is partly Markdown, partly restructured text, and partly HTML, because core Markdown does not officially support tables:
    def generate(table):
    writer = report.RSTWriter()
    writer.h1('Anscombe Statistics')
    writer.add(tabulate(table, tablefmt='html', floatfmt='.3f'))
    
    return writer.rst
  4. Plot the data and corresponding linear fits with the Seaborn lmplot() function:
    def plot(df):
    sns.set(style="ticks")
    g = sns.lmplot(x="x", y="y", col="dataset", 
         hue="dataset", data=df,
         col_wrap=2, ci=None, palette="muted", size=4,
         scatter_kws={"s": 50, "alpha": 1})

    plotting.embellish(g.fig.axes)
  5. Display a table with statistics, as follows:
    df, table = aggregate()
from IPython.display import display_markdown
display_markdown(generate(table), raw=True)

    The following table shows practically identical statistics for each dataset (I modified the custom.css file in my IPython profile to get the colors):

    How to do it...
  6. The following lines plot the datasets:
    %matplotlib inline
plot(df)

Refer to the following plot for the end result:

How to do it...

A picture says more than a thousand words. The source code is in the anscombe.ipynb file in this book's code bundle.

See also

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset