This is a modified version of James Mills' web crawler from http://code.activestate.com/recipes/576551-simple-web-crawler/ . Props go to him, complaints go to me.
This is a modified version of James Mills' web crawler from http://code.activestate.com/recipes/576551-simple-web-crawler/ . Props go to him, complaints go to me.