puzzle-english.com offline material downloader
usage scenario
if you’re far up in the mountains and don’t have a stable connection but nevertheless really would like to have an edge over those cypriots by brushing up on your english, comrade.
- works through a hardcoded list of urls
- requires a premium account for the video links!
moving parts
- ext/
- chrome extension to automate the download of lesson text
- killprintwindow.sh
- kills the print window that keeps popping up when traversing the urls
- urls.txt
- initial list of urls to download
- ./writer.sh
- once scrape is done we can write everything to disk
- output/
- the folder where ./writer.sh spits stuff out final consumables
prerequisites
- linux or something alike
- postgresql
- postgrest
- wmctrl
- youtube-dl
running
initialize database schema & load list of urls
- sudo -u postgres psql template1 -c ‘create database puzzle_english’
- psql puzzle_english < schema.sql
- psql puzzle_english -c ‘© urls.txt into lessons…’
install chrome extension to automate
- cd ext && npm i && npm run dev chrome &&
- google-chrome chrome://extensions <– put it here
run postgrest & print dialog killer
- ./killprintwindow.sh
- postgrest postgrest.conf
sign into puzzle-english.com with a premium account
- enable the extension while with an open tab on the site and watch it go bzzt
- psql puzzle_english -c ‘select * from stats’ – to follow progress
export the downloaded contents into a local directory.
- ./writer.sh # this will also download the videos for local consumption