/crawl-engine

A Web crawler written in Python that finds and hits URL(s) in a page and goes on.

Primary LanguagePython

An Internet web-crawler that finds URL(s) in a given root or seed URL and continues to do the same in those spotted URL.

Usage

	python crawl.py