Inconsistent results
Closed this issue · 5 comments
Used xnl with default config on yahoo.com but results were different most of the times.
python3 xnLinkFinder.py -i yahoo.com
Number of links it extracted each time
- 1: 244
- 2: 5
- 3: 237
- 4: 242
- 5: 229
- 6: 237
- 7: 237
- 8: 5
- 9: 243
- 10: 235
Hi @Silentsoul04
It seems like they could be rate limiting you or something. Can you try running with -vv to see what errors are appearing, maybe the 5's are when you get 429?
You could also try changing the timeout with -t argument.
Thanks
Xnl
Yeah, you are right.. I did it with --vv and the 5s are for 403 status. Again did it with google.com (which is more static target) and results were between 17-18 each time. :)
That's annoying that some targets are blocking though. I'll have a look into that to see if there's anyway around it.
Also, when you do run for a real target, don't forget to use -sp, -sf and -d to get the most out of it. Good luck!
Hi @Silentsoul04
Just a quick update... I found that yahoo.com only retruns a 403 when the User-Agent "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:15.0) Gecko/20100101 Firefox/15.0.1" is used. I have no idea, but I'm going to remove that one from the list in the next update so we don't see that issue again.
Thanks
Xnl
That's interesting behaviour @xnl-h4ck3r . Glad it is solved here.