Runtime Error when running Demo
rumourscape opened this issue · 0 comments
rumourscape commented
I am trying to run Demo
python -m stopes.pipelines.bitext.global_mining_pipeline src_lang=fuv tgt_lang=zul +preset=demo embed_text=laser3
But I am receiving the following error:
2024-03-11 11:48 INFO 1036353:stopes.moses - Preprocess fuv (en), 1 files
2024-03-11 11:48 INFO 1036353:stopes.moses - moses command: $/bin/bash -o pipefail -c 'zcat /speech/advait/align/stopes/demo/data/fuv.gz | perl /speech/advait/align/stopes/stopes/modules/preprocess/moses/scripts/tokenizer/remove-non-printing-char.perl | perl /speech/advait/align/stopes/stopes/modules/preprocess/moses/scripts/tokenizer/normalize-punctuation.perl -l en | perl /speech/advait/align/stopes/stopes/modules/preprocess/moses/scripts/tokenizer/deescape-special-chars.perl | perl /speech/advait/align/stopes/stopes/modules/preprocess/moses/scripts/tokenizer/lowercase.perl'
submitit ERROR (2024-03-11 11:48:15,483) - Submitted job triggered an exception
2024-03-11 11:48 ERROR 1036353:submitit - Submitted job triggered an exception
Traceback (most recent call last):
File "/speech/advait/miniconda3/envs/stopes/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/speech/advait/miniconda3/envs/stopes/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
submitit_main()
File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
process_job(args.folder)
File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/submission.py", line 69, in process_job
raise error
File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
result = delayed.result()
File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/utils.py", line 133, in result
self._result = self.function(*self.args, **self.kwargs)
File "/speech/advait/align/stopes/stopes/core/stopes_module.py", line 97, in __call__
res = self.run(iteration_value=iteration_value, iteration_index=iteration_index)
File "/speech/advait/align/stopes/stopes/modules/preprocess/preprocess_encode_module.py", line 91, in run
return encode_module.run( # type:ignore[no-any-return]
File "/speech/advait/align/stopes/stopes/modules/preprocess/line_processor.py", line 139, in run
proc.process_lines(lines_with_numbers)
File "/speech/advait/align/stopes/stopes/modules/preprocess/encode_to_npy.py", line 63, in process_lines
array = self.encode_to_np(lines_with_number)
File "/speech/advait/align/stopes/stopes/modules/preprocess/laser_sentence_encoder.py", line 400, in encode_to_np
return self.encoder.encode_sentences(tokenized_sents)
File "/speech/advait/align/stopes/stopes/modules/preprocess/laser_sentence_encoder.py", line 157, in encode_sentences
results.append(self._process_batch(batch))
File "/speech/advait/align/stopes/stopes/modules/preprocess/laser_sentence_encoder.py", line 95, in _process_batch
sentemb = self.encoder(tokens, lengths)["sentemb"]
File "/speech/advait/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/speech/advait/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/speech/advait/align/stopes/stopes/modules/preprocess/laser_sentence_encoder.py", line 201, in forward
encoder_out = super().forward(src_tokens, src_lengths)
File "/speech/advait/.local/lib/python3.10/site-packages/fairseq/models/transformer/transformer_encoder.py", line 165, in forward
return self.forward_scriptable(
File "/speech/advait/.local/lib/python3.10/site-packages/fairseq/models/transformer/transformer_encoder.py", line 294, in forward_scriptable
lr = layer(x, encoder_padding_mask=encoder_padding_mask_out)
File "/speech/advait/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/speech/advait/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/speech/advait/.local/lib/python3.10/site-packages/fairseq/modules/transformer_layer.py", line 319, in forward
output = torch._transformer_encoder_layer_fwd(
RuntimeError: Mask Type should be defined
This is taken from the error logs.