vahidhashemi/RobotReader

Robot.txt reader (suitable for spider/crawler)

NOASSERTION