/wordlistScraper

Scrapes a website for all non-common words

Primary LanguagePython

wordlistScraper.py

Scrapes a website for all non-common words

> python wordlistScraper.py https://wtfman.com 1

getting links...
getting links from https://wtfman.com
getting links from http://www.wtfman.com/quotes.htm
getting links from http://www.wtfman.com/forums/
getting links from http://www.wtfman.com/flash/
getting links from http://www.wtfman.com/toronto.htm
getting links from http://www.wtfman.com/archives.htm
getting links from http://www.wtfman.com/jov/
getting links from http://www.wtfman.com/mcd/main.htm
getting links from http://www.wtfman.com/uoevil/
done getting links. found 68 links
starting word harvesters...
done gathering words.  total words: 19123