#pulitzer-scraper
Digging through the new pulitzer.org site, I found the API used to pull winners by year or category. Calls are based on term id (tid), a three digit number that corresponds to a specific category. I wrote a quick node-based scraper that pulls together lists of all the Pulitzer prize winners and finalists in separate CSV files, as well as a list of all the revisions to categories over the years.
NOTE: Running the app requires 205 separate API calls to the Pulitzer site. Use responsibly.
##To run:
Clone repo:
git clone https://github.com/jonseitz/pulitzer-scraper.git
cd pulitzer-scraper
Install dependencies:
npm install
run app:
node app
##Pulitzer API endpoints
GET winners in a given Category
http://www.pulitzer.org/cms/api/1/winners/cat/ [tid] /raw.json
The tid range for current categories goes from 204 ("Public Service") - 225 ("Music"):
- 204: Public Service
- 205: Breaking News Reporting
- 206: Investigative Reporting
- 207: Explanatory Reporting
- 208: Local Reporting
- 209: National Reporting
- 210: International Reporting
- 211: Feature Writing
- 212: Commentary
- 213: Criticism
- 214: Editorial Writing
- 215: Editorial Cartooning
- 216: Breaking News Photography
- 217: Feature Photography
- 218: Drama
- 219: Fiction
- 220: History
- 221: Blank
- 222: Biography or Autobiography
- 223: General Nonfiction
- 224: Poetry
- 225: Music
Deprecated/Special categories are in a separate range from 260 (Special Awards & Citations) - 278 (Beat Reportting):
- 260: Special Awards and Citations
- 261: Novel
- 262: Blank
- 263: Correspondence
- 264: Explanatory Journalism
- 265: General News Reporting
- 266: Local General or Spot News Reporting
- 267: Local Investigative Specialized Reporting
- 268: Local Reporting - Edition time
- 269: Local Reporting - No edition time
- 270: Newspaper History Award
- 271: Photography
- 272: Reporting
- 273: Specialized Reporting
- 274: Spot News Photography
- 275: Spot News Reporting
- 276: Telegraphic Reporting - International
- 277: Telegraphic Reporting - National
- 278: Beat Reporting
GET finalists in a given category
http://www.pulitzer.org/cms/api/1/finalist/ [tid] /all/raw.json
tid ranges mirror those above.
GET winners for a given year
http://www.pulitzer.org/cms/api/1/winners/year/ [tid] /raw.json
The tid range for years goes from 105 (2015) - 203 (1917).
GET finalists for a given year
http://www.pulitzer.org/cache/api/1/finalist/all/ [tid] /raw.json
tid ranges mirror those above.
Get term information
http://www.pulitzer.org/cms/api/1/term/ [tid] /raw.json
Enter any tid to translate the category or year id into a real category.
FYI: tid values above 278 appear to be categories or tags. The highest value I found is 549, which is "Alaska".