scripted web crawling on Linux
Primary LanguagePHP
Usage: ./crawl.sh url.txt will list all links embedded in the pages at URLs in url.txt