tb0hdan/domains

Issues

• Issue with your patreon

  #25 opened 3 months ago by noissu
  1

• Please specifiy your crawler User-agent for robots.txt

  #28 opened 3 months ago by DamonHD
  5

• Do not download large files such as FLAC and MP3

  #27 opened 3 months ago by DamonHD
  3

• Error while clone the project : repository is over its data quota

  #26 opened 3 months ago by punyapaste
  1

• I recently sponsored this project

  #24 opened 9 months ago by sonofescobar1337
  1

• 401 Authorization Required

  #23 opened a year ago by pompompurins
  1

• Dataset Source: crawler.ninja

  #22 opened a year ago by elliotwutingfeng
  0

• ICANN CZDS to seed domains

  #21 opened a year ago by senecaso
  2

• Description of data sources

  #20 opened 2 years ago by hss14
  0

• Domain dataset source: ICANN Centralized Zone Data Service (CZDS)

  #18 opened 2 years ago by elliotwutingfeng
  1

• Domain dataset source (ipsniper.info)

  #13 opened 2 years ago by elliotwutingfeng
  3

• French domain names from public orgs

  #14 opened 2 years ago by JulienPalard
  1

• can't be cloned due to error

  #17 opened 2 years ago by sevmardi
  1

• Slovak domains dataset (open access)

  #16 opened 2 years ago by elliotwutingfeng
  1

• Certificate Transparency Data?

  #9 opened 2 years ago by p0lr
  2

• Crawler causing SYN floods

  #15 opened 2 years ago by apienk
  6

• More Statistics

  #7 opened 2 years ago by cheznewa
  2

• Suggestion: Add amount of storage taken by data into STATS.md

  #12 opened 2 years ago by jerrygreen
  0

• Wget returns 403 forbidden

  #11 opened 2 years ago by trentwiles
  2

• Domain dataset source: CubDomain.com

  #10 opened 2 years ago by elliotwutingfeng
  0

• Missing domains

  #8 opened 3 years ago by carlesnospam
  0

• Error when downloading

  #6 opened 3 years ago by marcelolarrosa
  0

• Raw data page returns 403.

  #5 opened 3 years ago by trentwiles
  1

• If you are only searching domains, why is your crawler scraping pages???

  #2 opened 4 years ago by Jas0n99
  9

• Extra domain

  #4 opened 3 years ago by duncte123
  1

• Add example of what data looks like

  #1 opened 4 years ago by aubergene
  2