Python 3.x notebooks about real-world data cleaning and visualization

A set of iPython notebooks on data wrangling and visualization for Stanford Computational Journalism, using the standard lib as well as pandas and matplotlib.

(in progress)

Practicum

These notebooks are walkthroughs of common matplotlib and pandas techniques.

Data extraction

Recipes for extracting (but not cleaning/wrangling) machine-readable data from raw sources.

The results can be found in the extracted subdirectories in data.

NASA plaintext data files
California Dept. of Education Excel spreadsheets - extracting SAT scores and student poverty data from workbooks of various design.
California Dept. of Education fixed-width fields - scraping a HTML table to get the field-boundaries for the CDE's legacy data format for school performance.
Texas Dept. of Justice; Death row inmates - HTML scraping with lxml
Texas Dept. of Justice; Executions - HTML crawling with Beautiful Soup (in progress)

Wrangling

Pivot Tables with Pandas - An exploration into San Francisco crime data.

Visualization

Visualizing the relationship between SAT performance and percentage of students eligible for free-or-reduced-price lunch.

dannguyen/python-notebooks-data-wrangling

Python 3.x notebooks about real-world data cleaning and visualization

Contents

Practicum

Data extraction

Wrangling

Visualization

Resources

Matplotlib

Pandas