sjhgvr/oisd

Question about 'Domains that did not resolve'

Closed this issue · 8 comments

jarelllama commented

Hi! I'm the maintainer of https://github.com/jarelllama/Scam-Blocklist.

I was looking at the dead domains from my lists presented by oisd.nl, and noticed a lot of them are parked domains, ones my own clunky and slow parked domain detection missed. I'm curious to how you're detecting these parked domains with such accuracy?

sjhgvr commented

Afaik I'm not actively excluding parked domains. And since I'm indeed seeing domains on here;
https://local.oisd.nl/showme.php?show=dead&list=c7188f126bfb7a91afd03127a2c3dbe4 and
https://local.oisd.nl/showme.php?show=dead&list=2406a659e2a5bc8248525247fa49b270
that still resolve, makes me wonder if I need to investigate my batch dig script for proper functionality.
Currently a complete dig of all domains is done every 5 days. Last one is due for today I think, so maybe it's because the data is 5 days old?

I'll look into it and report back.

jarelllama commented

Looks like I mistook a bug for a feature haha.

To provide some insight, here are the number of dead domains and resurrected domains from my blocklist today:

Dead domains removed today: 559
Resurrected domains added today: 199

I suspect within the 5 days, some of the dead domains become resolving again. I find it quite a common case, that's why I check for resurrected domains daily alongside dead domains.

I guess just by luck, the resurrected domains I checked in oisd.nl happen to be mostly parked, so I assumed it was some kind of parked domains detection at play.

sjhgvr commented

That seems like a most logical explaination.
I've slightly altered my dig script, also to include more logging, so I can backcheck more easily next time.
I'll close this for now.
Thanks again.

  • 👍1
jarelllama commented

Also out of curiosity, I saw the number of unused/false positives went up in my list:

Jarelllama's Scam Blocklist [homepage] [license]
https://raw.githubusercontent.com/jarelllama/Scam-Blocklist/main/lists/adblock/scams.txt
586 | 1497 | 37455

Example domain:

18294927-coinbase.com
Why is 18294927-coinbase.com not included in the oisd blocklist?

No info on this domain, sorry.


18294927-coinbase.com is found listed in the following lists;

https://raw.githubusercontent.com/jarelllama/Scam-Blocklist/main/lists/adblock/scams.txt [included]

https://get.domainsblacklists.com/blacklist.txt

Could I ask how these domains are excluded?

sjhgvr commented

At my first glance.. that doesn't make sense. There should be logic behind it (or it's buggy).
Thanks for pointing this out to me. I'll look into it asap.

  • 👍1
sjhgvr commented

I think I know whats happening.
As I've manually cleared the "alive" and "dead" DNS info... The display of that info is off as well.
Now the blocklist isnt updating, however, that info is still being updated each few hours. (Shouldn't be)
It displays about 100 domains less each 3 to 4 hours. So it should eventually settle down and show correct info once the batch dig is complete.

jarelllama commented

Thanks for the fast answers! I'll let you know if any other questions pop up.

  • 👍1
jarelllama commented

Appears to be done:

Jarelllama's Scam Blocklist [homepage] [license]
https://raw.githubusercontent.com/jarelllama/Scam-Blocklist/main/lists/adblock/scams.txt
2185 | 2 | 37864

Thanks again!

  • 👍1