Rips entire websites (or at least attempts to), or individual pages depending on configuration.
- Run
git clone https://github.com/sabihismail/Website-Ripper
and move into the directory - Run
virtualenv venv && source .env/bin/activate && pip install -r requirements.txt
- You will need a compatible ChromeDriver.exe available from here
- Place it in the
res/
folder
- Place it in the
- Copy
job.example.json
tojob.json
and configure variables below - Run
python main.py
and cross your fingers
- To-Do
There will probably be many bugs and inconsistencies since a website can deploy any number of measures to be inconsistent from other websites.
The code is provided as-is, and it should not be used maliciously. I do not take responsibility for the misuse of this tool. It's purely for learning purposes.