Nemo 1.21.0 module loading problem

[FredSRichardson]

I'm starting from a conda environment using Python 3.10.13.

I've run into a problem where I first install Nemo 1.21.0 using:

pip install Cython
python -m pip install 'git+https://github.com/NVIDIA/NeMo.git@r1.21.0#egg=nemo_toolkit[all]'

And then install riva-asrlib-decoder using:

git clone https://github.com/nvidia-riva/riva-asrlib-decoder.git
cd riva-asrlib-decoder
git checkout release/2.13.0
pip install -e .[testing]

The following code hangs on the second load statement - unfortunately I'm not at all sure how to debug this.

import nemo.collections.asr
from riva.asrlib.decoder.python_decoder import BatchedMappedDecoderCuda

If I only install riva-asr-lib using pip install -e .[testing] as above which also installs the PyPi version of nemo (r1.20.0) the above module imports work without any problems.

[FredSRichardson]

I believe I can close this issue - it is due to a second instance of OpenFST being installed by pynini. OpenFST sets a mutex lock on initialization and two instances of the library will end up in some kind of dead lock over this. As an aside, debugging this was pretty easy using GDB (once I had a clue on how to do it!):

gdb python
(gdb) run my-script.py

After that I just hit Ctrl-C and printed out that back trace. I'll close this issue. Some documentation would be great, but I don't know how you document these kinds of module interactions...