Rough draft of a script for crawling the fritter network.
npx fritter-crawler
fritters
& not-fritters
) in the working directory and begin writing files there.
Writes all profile.json
files to fritters/${key}.json
.
If no profile.json
is found in a dat link, writes the 404 error to not-fritters/${key}.json
.
The goal is to eventually analyze and visualize the fritter network using the dataset produced by this crawler.
- retries
- better logging
- make starting key and concurrency modifiable via CLI args
- add option to pipe to STDOUT
ISC