this version is with some minor changes, the main change is on output files. V2 is ready to integrate with your recon process. unlike V1, which is prompting for inputs, this version will run on a single command, also i removed identification from output files, so you will now get pure urls as a output, which you can use for your further recon.
- Fastest web crawling
- works on python asyncio, which makes it way more faster than other crawlers
- zero duplicate links
- Link Finder
- JavaScript files Finder
- Brute force and parse sitemap.xml
- Parse robots.txt
- Find AWS-S3 from response source
- Get URLs from Wayback Machine
- Format output easy to Grep
- save output in txt
- Support Burp input
- Crawl multiple sites in parallel
- option to include Third party urls to your results
- Easy to use
git clone https://github.com/kartikhunt3r/Adrishya-Spider-V2.git
cd Adrishya-Spider-V2
chmod +x *
pip install -r requirements.txt
python3 adrishya.py example.com