Helsinki-NLP/OpusFilter

Issues

• Opusfilter fails to compress data when it is downloaded via moses

  #75 opened 4 months ago by thfrkielikone
  3

• Cache behaviour

  #74 opened 4 months ago by thfrkielikone
  3

• Installing on fedora 40

  #72 opened 4 months ago by thfrkielikone
  5

• SentenceEmbeddingFilter chunksize clashes with general chunksize

  #70 opened 7 months ago by miau1
  1

• Issue with opus-fast-mosestokenizer dep for ARM-macs

  #69 opened 9 months ago by rggdmonk
  3

• LMclassify always score 1

  #68 opened 10 months ago by wuyangjian
  3

• Add support for fastspell for language identification

  #66 opened a year ago by marco-c
  0

• Issue during installation

  #61 opened a year ago by evramnarouz
  3

• eflomal crashes during filtering

  #63 opened 2 years ago by yvesscherrer
  1

• opusfilter : command not found

  #26 opened 3 years ago by Pkscode
  2

• Add pyyaml to requirements

  #60 opened 2 years ago by yvesscherrer
  1

• insufficient documentation

  #59 opened 2 years ago by jairosg
  1

• Better word alignment filter

  #52 opened 2 years ago by svirpioj
  1

• Integration with MTData

  #53 opened 2 years ago by svirpioj
  0

• Use multicore to accelerate score, filter and tokenize processes.

  #28 opened 2 years ago by BrightXiaoHan
  5

• How to choose threshold for WordAlignFilter?

  #24 opened 2 years ago by BrightXiaoHan
  0

• TypeError when processing ParaCrawl

  #11 opened 3 years ago by lefterav
  1

• Process Killed

  #46 opened 3 years ago by bayesrule
  2

• Installation using pip fails

  #21 opened 3 years ago by aarnetalman
  5

• Is it possible to generate score file during training alignment model?

  #34 opened 3 years ago by BrightXiaoHan
  6

• Debug the configuration by export filtered corpus.

  #41 opened 3 years ago by BrightXiaoHan
  2

• Specify different "unit" types in filters.

  #38 opened 3 years ago by BrightXiaoHan
  2

• Version 2.3.0 breaks train_classifier function

  #37 opened 3 years ago by wujameszj
  1

• Additional filter suggestion: remove lines with repeated content

  #8 opened 3 years ago by yvesscherrer
  3

• tokenizer ignored when creating align.priors

  #30 opened 3 years ago by tomsbergmanis
  2

• Possible bug in word_alignment accept function

  #31 opened 3 years ago by tomsbergmanis
  5

• pandas<1.0.0 not supported in opusfilter>=2.0.0

  #25 opened 3 years ago by svirpioj
  1

• Installation fails on Windows

  #22 opened 3 years ago by aarnetalman
  1

• Option to suppress download confirmation for "opus_read"

  #10 opened 3 years ago by lefterav
  2

• Tokenization behavior in WordAlignFilter

  #9 opened 4 years ago by yvesscherrer
  4

• Language id filter comparison

  #7 opened 4 years ago by yvesscherrer
  3

• Option to keep blank lines

  #4 opened 4 years ago by jbrry
  3

• LanguageIDFilter filter error

  #6 opened 4 years ago by virgulvirgul
  2

• standardize_dataframe_scores receives empty data frame in classifier.py on nlingual-rebase branch

  #3 opened 4 years ago by jbrry
  4

• LM paths do not use output_directory

  #2 opened 5 years ago by yvesscherrer
  2

• Infinite scores from word aligment

  #1 opened 5 years ago by svirpioj
  0