Feature: Broken link checks
ehsandeep opened this issue · 4 comments
Hello @mhmdiaa,
As i'm aware of that this tool is suppose to find the same things what i am suggesting here, but as far i can see it's only checking for Amazon S3 and Wufoo, but it make more sense if any external sources have 404 status for possible takeover the contents after manual checks.
Why am saying this?
https://gist.github.com/EdOverflow/24e0bb929169eb948bb7f3d0a2d5528f
Just check the above link for better idea about it.
Why if there is existing tool to do same?
We knew very well that GO is the fastest language atm in terms of speed.
I was thinking about getting a list of providers from the user, and checking for them, the way tko-subs works.
The idea of checking for any 404 response seems neat, though. We can add both! Like, add a flag to check for any 404 page, also add the option to find any page that's hosted on a set of known providers whether it returns a 404 response or not. You know, some resources are interesting even if they are still used by the target, and function normally, like S3 buckets.
Let me think about it.
Thanks
Hello @mhmdiaa,
for sure, but the things tko-subs do is for checking for specific strings for given list of subdomains but second-order do crawl for given single target, and if am not wrong are you talking about checking the stings in externals resources while crawling ? if yes that would be good add for sure, and yeah adding flag for both thing is good indeed and make sure to log the result as well, like storing pages with 404 status code into file.
Thanks @MadaraTech, I've just pushed an update to add this and some other features (including the blacklist you suggested on #2). See the updated README for more details.
Impressive update, thank you for all your work :)