a curated list of the most common and most interesting robots.txt disallowed directories.
Primary LanguageShell