Sundry notes and code bits for Stanford's Programming in Journalism class (COMM 177P/277P).
- Syllabus
- Assignments and Grading
- Bookshelf - recommended books, tutorials, cheatsheets, etc.
- Code solutions - A private repo containing solutions to class exercises.
- Datakit - overview and details on install and usage
- Getting Help - Resources and strategies for finding help.
- Glossary - technical terms used in class
- Python - overview, tutorials, etc.
- Technical setup and FAQ - recommended and required software (all free)
- Workflow advice - working on the command line, etc.
- Course overview
- Discuss the history of code and data analysis in journalism
- Software install party (w00t!!)
- The Owl, Problem Solving, and the Unix Workbench
- Bash intro
- Bash drill
- Software installs, part deux (config and Python install)
- Assignment 0 - Unix practice and failed banks script
No class.
- Bash questions?
- Python overview and coding contexts
- Python Syntax Crash Course
- Assignment 1 - Python reading/practice and code challenge
- Assignment 1 questions?
- Dictionaries crash course
- Reading and writing text files
- Assignment 2 - Python lists/dicts and DataKit overview
- Automating workflows
- DataKit overview
- Complete Tech Setup configuration and install DataKit
- Assignment 3 - Practical Python skills and Failed Banks in Python
- APIs and the News overview and presentation
- Working with APIs - As of Feb. 1, 2021, certain unauthenticated OpenCorporates API endpoints were disabled due to excessive bot usage, so not all portions of the tutorial are functional.
- Quakebot exercise - hands-on practice with a JSON feed
- Quakebot exercise - hands-on practice with JSON feed
- The Art of Writing Functions
- Assignment 4 - Study Python functions, sorting and web basics
- Assignment 5 - Senate compromisers Python script
- Web scraping for the news - a high-level overview of web scraping in a news context.
- Web scraping 101 - how to dissect a website, various obstacles to scraping and strategies for overcoming them
- Web scraping exercises - some websites to challenge your newfound scraping skills
- Web scraping resources - tutorials, etc.
- Assignment 6 - Web Scraping Ramp-up
- API Services
- Refinitiv Entity Extraction
- Data Pipelines with Modules
- Project: FDA Recall Entities - Scrape FDA recalls and extract entities using the OpenCalais/Refinitiv API
No class.
- Review and discuss solutions to recent assignments
- Strategies for gaining visibility into code while you work
- Assignment 7 - Data Journalism intro readings
- Data Journalism - A Very Brief Tour
- Data Journalism in the wild - Dilcia Mercedes and Justin Mayo
- Preview of Final Project - SF Data Analysis and begin working on story ideas and data sleuthing/vetting.
- Assignment 8 - Prepare and present final project story ideas
- Whirlwind Tour of the Data Journalism Process
- Data Analysis with Jupyter and pandas
- Assignment 9 - First Python Notebook
- Present story ideas for the final project as part of Assignment 8.
- Overview of requirements for final project
- One-on-one technical consults
- Guest lecture: SF police data analysis with guest lecturer Eric Sagara, of Big Local News
- One-on-one check-ins for the final project
- Students presentations for the final project
- Course Retrospective
- Provide feedback on how to improve the course