A java clone of Google's robotst.txt parser: https://github.com/google/robotstxt
Primary LanguageJavaApache License 2.0Apache-2.0
This repository is not active