/juicyscraper

This simple script can scrape all the Disallowed(juicy) paths and make a file containing the disallowed URLs.

Primary LanguageShell

Stargazers