oscar-project/ungoliant

Issues

• [BUG] download malfunctioning

  #132 opened 7 months ago by kargaranamir
  1

• [BUG] corrupt deflate stream

  #131 opened 7 months ago by kargaranamir
  0

• [BUG] UnexpectedEof While running Ungoliant Pipeline

  #130 opened 8 months ago by nattkorat
  3

• [Feature request] Add larger timeouts between 503 retries for CC download

  #123 opened a year ago by Uinelj
  0

• [Feature request] Secure against length extension attacks

  #108 opened a year ago by chris-ha458
  4

• [Feature request] Train a classifier to better classify languages

  #21 opened 3 years ago by Muhtasham
  2

• Option to keep documents that can't be identified

  #88 opened 2 years ago by Uinelj
  1

• [Feature request] Document how to set fasttext model

  #106 opened a year ago by chris-ha458
  2

• [BUG] Error when downloading full CC snapshot

  #94 opened 2 years ago by ngan-nt
  4

• [BUG] Deduplication with Ungoliant

  #93 opened 2 years ago by Hammamwa47
  1

• Automatically add binaries on releases

  #92 opened 2 years ago by Uinelj
  0

• Automate release and deployment to crates.io

  #60 opened 2 years ago by Uinelj
  1

• Blocklists checklist

  #79 opened 2 years ago by Uinelj
  0

• Avoid creating Blocklist for each shard

  #74 opened 2 years ago by Uinelj
  1

• [BUG] Cannot install via cargo

  #77 opened 2 years ago by new5558
  3

• Configuration file for `ungoliant pipeline`

  #75 opened 2 years ago by Uinelj
  0

• Bug in `MeanLength` filter

  #70 opened 2 years ago by sadra-barikbin
  1

• [BUG] No hard fail when blocklist path is invalid

  #64 opened 2 years ago by Uinelj
  0

• Add integration testing for rebuild files with oscardoc pipeline

  #61 opened 2 years ago by Uinelj
  1

• Handle dependabot vulnerabilities

  #58 opened 2 years ago by Uinelj
  0

• Cache dependencies

  #66 opened 2 years ago by Uinelj
  1

• Rename `master` branch to `main` and protect it

  #59 opened 2 years ago by Uinelj
  0

• Question about the -o <offset> option in download

  #63 opened 2 years ago by TristanThrush
  1

• [BUG] Chavacano marked as "cbr" rather than cbk

  #53 opened 2 years ago by Uinelj
  0

• Feature: Add retry option on downloader

  #33 opened 3 years ago by Uinelj
  0

• Revamp the error reporting

  #43 opened 3 years ago by Uinelj
  1

• [Feature request] Pipeline remove download file after process and extract single language

  #56 opened 2 years ago by acul3
  3

• Feature `std_rng` depends on `rand_hc` which is not an optional dependency

  #55 opened 2 years ago by DavidNemeskey
  2

• [Question] Different multilingual identification methods

  #50 opened 3 years ago by codedecde
  2

• [BUG] ungoliant::io::reader::corpus] [<lang>] no text/meta file.

  #48 opened 3 years ago by kirianguiller
  3

• [Feature request] Controling the number of thread being used

  #47 opened 3 years ago by kirianguiller
  3

• [BUG] Pipeline command not working

  #41 opened 3 years ago by kirianguiller
  5

• [BUG] Cargo install of Ungoliant not working

  #40 opened 3 years ago by kirianguiller
  2

• Feature: Failures handling

  #4 opened 3 years ago by Uinelj
  1

• Publish on crates.io

  #17 opened 3 years ago by Uinelj
  1

• Fix Cargo.toml errors for crates.io publishing

  #44 opened 3 years ago by Uinelj
  1

• Feature: Zipflike validation on documents at character-level

  #28 opened 3 years ago by Uinelj
  1

• Feature: Header/Footer annotation

  #25 opened 3 years ago by Uinelj
  0

• Feature: Noisy annotation on Documents with a high punctuation/letters ratio.

  #26 opened 3 years ago by Uinelj
  0

• Feature: Multilingual documents

  #27 opened 3 years ago by Uinelj
  0

• Improve operation order in pipeline

  #34 opened 3 years ago by Uinelj
  2

• Bug: Bad computation of Identification probability score

  #36 opened 3 years ago by Uinelj
  0

• Feature: Pipeline and Benchmarking

  #3 opened 3 years ago by Uinelj
  0