tballison/commoncrawl-fetcher-lite

Issues

• Redownloading cdx files

  #47 opened a year ago by gleporeNARA
  4

• 3. Allow a different extracted file naming scheme

  #29 opened 2 years ago by gleporeNARA
  5

• Won't download older crawls

  #30 opened 2 years ago by gleporeNARA
  12

• /tmp filling up

  #27 opened 2 years ago by gleporeNARA
  5

• Backoff messages

  #26 opened 2 years ago by gleporeNARA
  1

• NPE if no fetcher is specified

  #22 opened 2 years ago by tballison
  0

• Add mime to logged files

  #16 opened 2 years ago by tballison
  0

• Allow users to download truncated bytes

  #14 opened 2 years ago by tballison
  0

• Add checkstyle

  #1 opened 2 years ago by tballison
  1

• Add maven enforcer to prevent version collisions

  #2 opened 2 years ago by tballison
  1

• Configure release

  #6 opened 2 years ago by tballison
  1

• Add loggers for extracted files and for truncated

  #7 opened 2 years ago by tballison
  0

• Allow writing extracted files to s3

  #5 opened 2 years ago by tballison
  0

• Incorporate index urls into json file

  #3 opened 2 years ago by tballison
  0

• Shutdown fetching when throttling fails

  #4 opened 2 years ago by tballison
  0