Google Search Scraping Tool
Overview
scrape.py
- Script to scrape google search engine.
- Three parameters should be specified by following
--<options>:
- Keyword to Search:
--keyword
- Target content type:
--ctype∈ {"text", "image"}
- Number of contents :
--cnum ∈ ℕ
Usage
1. Use as a script.
1.1 Requirements:
- Python
3.x
bs4, lxml
pip install bs4, lxml to install the packages.
1.2 How to run
python scrape.py --keyword 'Junya Watanabe' --ctype image --cnum 100
2. Run the script on docker container
2.1 Requirements:
docker -v
Docker version 19.03.2, build 6a30dfc
2.2 How to run
- Clone this repository and move to working directory.
- Build the API server
docker build -t api/app .
- Run docker container
docker run -p 4000:3000 --rm --name app -d -t api/app
- Test the server
docker ps # to check wheather the server is working
curl http://0.0.0.0:3000/junya%E3%80%80watanabe/image/
