p0dalirius/robotstester
This Python script can enumerate all URLs present in robots.txt files, and test whether they can be accessed or not.
PythonGPL-3.0
Stargazers
- Am0rphousNorwegian University of Science and Technology
- andreyoshimura@dragon.networks
- Bbyhex
- beamzerNetherlands
- danielserbuRomania
- Double2Sky
- Edr4
- FelixCMD
- idfix007
- ivanisconfused
- jamiedevsandbox/dev/urandom
- JasertoDublin, Ireland
- jeffrysleddensRotterdam University of Applied Sciences
- kernoebBretagne
- lightoyou
- Madhor
- mashaalAl3nizy
- MilleniumSpark
- mwatkins-leviathan
- p0daliriusPodalirius Labs
- pwnwithlove
- R38orn
- ResistOrRegret
- seajaysecThe Walt Disney Company
- sirrushooUSA
- sn0wZero
- Sythenv
- TarunavBAIn your computer.
- Th3Tul1p3swiss
- TH3xACE@Team B3aT
- upgradepa
- Username2022a
- wlayzz/var/run/docker.sock
- yutarochan@IBMResearch
- Zab-an
- zhifana