remusao/tldts

generate non-private subset

Closed this issue · 3 comments

(First off, your library is EXACTLY what we were looking for. Many folks have a PSL distribution on npm, but you've done awesome work here. Thank you!)

The PSL has all its public ICANN domains and then at the bottom a bunch of PRIVATE domains.

image

For our uses, we don't care about the private domains, as we just need the PSL for improved URL parsing/display for the publically accessible web. Also, we're very bundle size conscious. (So we appreciate the trie!) Publishing an icann-only subset (or whatever you want to name it) would be super helpful.

Here's an initial idea of the size difference:
image

cc @alexnj

Hi @paulirish,

Thanks for reaching out, and for the kind words. I am happy that tldts is useful to you.

I have created a PR with preliminary work to create a separate tldts-icann which only contains the ICANN section: #1888

I'll have another look tomorrow but it's looking pretty good:

  • minified: 72K
  • gzipped (--best): 28K
  • brotli (-q 11): 24K
  • zstd (--ultra -22): 26K

I hope that helps,

You can find the new package here: https://www.npmjs.com/package/tldts-icann