facebookresearch/stopes

Runtime Error when running Demo

rumourscape opened this issue · 0 comments

I am trying to run Demo

 python -m stopes.pipelines.bitext.global_mining_pipeline src_lang=fuv tgt_lang=zul +preset=demo embed_text=laser3

But I am receiving the following error:

2024-03-11 11:48 INFO 1036353:stopes.moses - Preprocess fuv (en), 1 files
2024-03-11 11:48 INFO 1036353:stopes.moses - moses command: $/bin/bash -o pipefail -c 'zcat /speech/advait/align/stopes/demo/data/fuv.gz | perl /speech/advait/align/stopes/stopes/modules/preprocess/moses/scripts/tokenizer/remove-non-printing-char.perl | perl /speech/advait/align/stopes/stopes/modules/preprocess/moses/scripts/tokenizer/normalize-punctuation.perl -l en | perl /speech/advait/align/stopes/stopes/modules/preprocess/moses/scripts/tokenizer/deescape-special-chars.perl | perl /speech/advait/align/stopes/stopes/modules/preprocess/moses/scripts/tokenizer/lowercase.perl'
submitit ERROR (2024-03-11 11:48:15,483) - Submitted job triggered an exception
2024-03-11 11:48 ERROR 1036353:submitit - Submitted job triggered an exception
Traceback (most recent call last):
  File "/speech/advait/miniconda3/envs/stopes/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/speech/advait/miniconda3/envs/stopes/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
    submitit_main()
  File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
    process_job(args.folder)
  File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/submission.py", line 69, in process_job
    raise error
  File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
    result = delayed.result()
  File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/utils.py", line 133, in result
    self._result = self.function(*self.args, **self.kwargs)
  File "/speech/advait/align/stopes/stopes/core/stopes_module.py", line 97, in __call__
    res = self.run(iteration_value=iteration_value, iteration_index=iteration_index)
  File "/speech/advait/align/stopes/stopes/modules/preprocess/preprocess_encode_module.py", line 91, in run
    return encode_module.run(  # type:ignore[no-any-return]
  File "/speech/advait/align/stopes/stopes/modules/preprocess/line_processor.py", line 139, in run
    proc.process_lines(lines_with_numbers)
  File "/speech/advait/align/stopes/stopes/modules/preprocess/encode_to_npy.py", line 63, in process_lines
    array = self.encode_to_np(lines_with_number)
  File "/speech/advait/align/stopes/stopes/modules/preprocess/laser_sentence_encoder.py", line 400, in encode_to_np
    return self.encoder.encode_sentences(tokenized_sents)
  File "/speech/advait/align/stopes/stopes/modules/preprocess/laser_sentence_encoder.py", line 157, in encode_sentences
    results.append(self._process_batch(batch))
  File "/speech/advait/align/stopes/stopes/modules/preprocess/laser_sentence_encoder.py", line 95, in _process_batch
    sentemb = self.encoder(tokens, lengths)["sentemb"]
  File "/speech/advait/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/speech/advait/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/speech/advait/align/stopes/stopes/modules/preprocess/laser_sentence_encoder.py", line 201, in forward
    encoder_out = super().forward(src_tokens, src_lengths)
  File "/speech/advait/.local/lib/python3.10/site-packages/fairseq/models/transformer/transformer_encoder.py", line 165, in forward
    return self.forward_scriptable(
  File "/speech/advait/.local/lib/python3.10/site-packages/fairseq/models/transformer/transformer_encoder.py", line 294, in forward_scriptable
    lr = layer(x, encoder_padding_mask=encoder_padding_mask_out)
  File "/speech/advait/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/speech/advait/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/speech/advait/.local/lib/python3.10/site-packages/fairseq/modules/transformer_layer.py", line 319, in forward
    output = torch._transformer_encoder_layer_fwd(
RuntimeError: Mask Type should be defined

This is taken from the error logs.