robotparser-scala implements a parser for the robots.txt file format in Scala.
Primary LanguageScalaApache License 2.0Apache-2.0