The Takeaway is a New York Public Radio (NYPR) show that aired on WNYC for many years. It often included guests. More than 300,000 of the show's production files are in a folder in the station's internal playout system, DAVID.
On the other hand, 17,000+ segments live on the station's web site.
This project aimed to match the published segments with the production files using the guest's name. We are particularly interested in full, raw interviews with the guests that appeared on the program.
Steps:
- Download all Takeaway segments using the station's API (due to memory issues, we created five xml documents)
- Clean out the html within the json results and extract only relevant information using regex
- Parse out guests within the descriptions ("body") of each segment. Guests are generally marked as strong or bold, or simply introduced with the string 'Guest: '
- Extract the guest's last name, and transliterate special characters. This will be our term
- Determine the "uniqueness" of the term in DAVID titles using previously-performed term frequency analysis. If the term is common, include a warning
- Find files in DAVID that were created within 10 days previous to publishing the segment
- Find titles from those files whose DAVID title includes the term (i.e. the guest's last name)
- Sort files in decreasing length
- Publish a list of files without warnings and longer than a specified length (e.g. 10 minutes)