/protein-monster-crawler

Script to crawl all URLs of Protein Monster ARG: https://protein.monster

Primary LanguagePython

Installation

Make sure you have python installed.
Then, create a venv, activate it and install requirements.
Now you're good to go!

This can be used with almost any ARG websites as soon as urls are public. Just replace start_url = "https://protein.monster" by whatever.

Launch the script with:
python main.py