Is it possible to generate score file during training alignment model?

Question

Is it possible to generate score file during training alignment model?

BrightXiaoHan opened this issue 3 years ago · 6 comments

Thanks

Answer 1 · 2022-01-07T09:23:07.000Z

Not at the moment; the forward and backward scores files generated by eflomal are saved as temporary files are removed afterwards. But it should be easy to add an option to save them in a permanent file. But they would be in eflomal's original score format, not as OpusFilter score file. Which one are you looking for?

Answer 2 · 2022-01-08T08:57:32.000Z

"OpusFilter score file" is what I want.

Answer 3 · 2022-01-11T09:11:18.000Z

Implemented in #36

Answer 4 · 2022-02-13T17:47:49.000Z

I try it by the following config

  - type: train_alignment
    parameters:
      src_data: zh.rules
      tgt_data: en.rules
      parameters:
        model: 3
        src_tokenizer: [jieba, zh]
        tgt_tokenizer: [moses, en]
        scores: align_score.jsonl
      output: align.priors

but scores file align_score.jsonl was not generated.

Answer 5 · 2022-02-15T08:30:20.000Z

This confused me for a while before I noticed that it's a problem in the documentation: The scores file is a top-level option, not under inner parameters. Thanks for noticing! I fixed the README.

The issue also revealed that many OpusFilter methods do not warn about extra parameters... Something to be improved in the future.