/POC-scrapper

Primary LanguagePython

POC scrapper

Requirement

python3
make

Build

make build

Note

Before you run the tool, you should activate the environment of this project.

source env/bin/activate

Run

python app/main.py --url=[] --save_path=[]

Parameter

--url : the website you want to scrap,Note the tool just crawls the sitemap.
--save_path : the path file will be responded to the data. Note, currently the tool just support format txt file.