A simple WebCrawler that returns title, keywords, and exports data as JSON
*Args
- target: url of the site[s] you want to crawl. For crawling more than one site, targets must be seperated by ,'s.
- depth: this is the time complexity. i.e deeper the depth, more webpages the program crawls, and the greater time it takes. ( O(depth) ).
*Out
- All data is exported to WebCrawler.json
Resources:
- https://docs.python.org/3/library/json.html#
- https://www.youtube.com/watch?time_continue=101&v=0hJPngNmlvc&feature=emb_title&ab_channel=Trinitysoftwareacademy
Additional Comments
- I coded this from scratch after withnessing another individual create a Webcrawler in PHP. I am very pround of my work and hope to improve on it.