Filed

Modular scraping framework for retrieving statements online and saving to cloud storage provider (Google Drive).

Setup

Download dependencies using requirements.txt: pip install -r requirements.txt
Edit 'accounts.json' file with your accounts
Turn on Google Drive API as instructed here (and add created 'client_secret.json' to project root): https://developers.google.com/drive/v3/web/quickstart/python
Create 'logs', 'tmp', and '.credentials' directories
Start Selenium node server (for certain scrapers). See this great tutorial on Dockerized Selenium: https://github.com/SeleniumHQ/docker-selenium

python Filed.py --logging_level DEBUG --noauth_local_webserver

'--noauth_local_webserver' needs to only be run the first time for Google Drive setup

'--logging_level' supports multiple modes (INFO, WARN, etc.)

python Filed.py -s 12:01

Schedules Filed to run every day at 12:01 PM

Most scrapers do not have paging logic to pull full account history. Assumes that most accounts have recent statements in Google Drive (the ADP scraper currently does support full account history)
Google Drive folder id's can be found by right clicking on the folder and selecting 'Get Link' (copy from end of hyperlink)
See working scrapers list below.