ModuleNotFoundError: No module named 'torch'
Closed this issue · 7 comments
Hi,
I am impressed by this amazing tool, and I followed the installation instructions as follows:
git clone https://github.com/WGLab/DeepMod2.git
conda env create -f DeepMod2/environment.yml
conda activate deepmod2
python DeepMod2/deepmod2 --help
After that, I encountered an error message when running python DeepMod2/deepmod2 --help:
Traceback (most recent call last):
File "/nfs/turbo/hsinlun/bin/DeepMod2/deepmod2", line 3, in <module>
import time, itertools, torch
ModuleNotFoundError: No module named 'torch'
Here is my conda list:
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_kmp_llvm conda-forge
bzip2 1.0.8 hd590300_5 conda-forge
c-ares 1.26.0 hd590300_0 conda-forge
ca-certificates 2024.2.2 hbcca054_0 conda-forge
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
filelock 3.13.1 pyhd8ed1ab_0 conda-forge
fsspec 2024.2.0 pyhca7485f_0 conda-forge
gmp 6.3.0 h59595ed_0 conda-forge
gmpy2 2.1.2 py310h3ec546c_1 conda-forge
h5py 3.10.0 nompi_py310h65828d5_101 conda-forge
hdf5 1.14.3 nompi_h4f84152_100 conda-forge
icu 73.2 h59595ed_0 conda-forge
iso8601 2.1.0 pypi_0 pypi
jinja2 3.1.3 pyhd8ed1ab_0 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
krb5 1.21.2 h659d440_0 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
lib-pod5 0.3.6 pypi_0 pypi
libabseil 20230802.1 cxx17_h59595ed_0 conda-forge
libaec 1.1.2 h59595ed_1 conda-forge
libblas 3.9.0 21_linux64_openblas conda-forge
libcblas 3.9.0 21_linux64_openblas conda-forge
libcurl 8.5.0 hca28451_0 conda-forge
libdeflate 1.18 h0b41bf4_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 hd590300_2 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 13.2.0 h807b86a_5 conda-forge
libgfortran-ng 13.2.0 h69a702a_5 conda-forge
libgfortran5 13.2.0 ha4646dd_5 conda-forge
libhwloc 2.9.3 default_h554bfaf_1009 conda-forge
libiconv 1.17 hd590300_2 conda-forge
mkl 2023.2.0 h84fe81f_50496 conda-forge
more-itertools 10.2.0 pypi_0 pypi
mpc 1.3.1 hfe3b2da_0 conda-forge
mpfr 4.2.1 h9458935_0 conda-forge
mpmath 1.3.0 pyhd8ed1ab_0 conda-forge
ncurses 6.4 h59595ed_2 conda-forge
networkx 3.2.1 pyhd8ed1ab_0 conda-forge
numba 0.59.0 py310h7dc5dd1_1 conda-forge
numpy 1.26.4 py310hb13e2d6_0 conda-forge
ont-fast5-api 4.1.2 pyhdfd78af_0 bioconda
openssl 3.2.1 hd590300_0 conda-forge
packaging 23.2 pyhd8ed1ab_0 conda-forge
pip 24.0 pyhd8ed1ab_0 conda-forge
pod5 0.3.6 pypi_0 pypi
polars 0.20.8 pypi_0 pypi
progressbar33 2.4 py_0 conda-forge
pyarrow 14.0.2 pypi_0 pypi
pysam 0.22.0 py310h41dec4a_0 bioconda
python 3.10.13 hd12c33a_1_cpython conda-forge
python_abi 3.10 4_cp310 conda-forge
pytorch 2.1.2 cpu_mkl_py310haa9fb50_101 conda-forge
pytz 2024.1 pypi_0 pypi
readline 8.2 h8228510_1 conda-forge
setuptools 69.0.3 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sleef 3.5.1 h9b69904_2 conda-forge
sympy 1.12 pypyh9d50eac_103 conda-forge
tar 1.34 hb2e2bae_1 conda-forge
tbb 2021.11.0 h00ab1b0_1 conda-forge
tk 8.6.13 noxft_h4845f30_101 conda-forge
tqdm 4.66.2 pyhd8ed1ab_0 conda-forge
typing_extensions 4.9.0 pyha770c72_0 conda-forge
tzdata 2024a h0c530f3_0 conda-forge
vbz-h5py-plugin 1.0.1 pypi_0 pypi
wheel 0.42.0 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
zlib 1.2.13 hd590300_5 conda-forge
zstd 1.5.5 hfc55251_0 conda-forge
I am wondering if you have any suggestions on this or could provide me with the version you used in your environment.
Thank you,
Hsin
Hi Hsin,
It seems like the above method should work because pytorch is listed under your environment packages:
pytorch 2.1.2 cpu_mkl_py310haa9fb50_101 conda-forge
Is it possible that you were not using the deepmod2 conda environment when you ran the help command? What happens if you just run the following in command line:
conda activate deepmod2
python -c 'import torch'
Best,
Umair
Hi Umair,
Thanks for your prompt response.
- I am attempting to run the following without the deepmod2 conda environment:
[hsinlun@gl-login1 bin]$ python DeepMod2/deepmod2 --help
Traceback (most recent call last):
File "/nfs/turbo/hsinlun/bin/DeepMod2/deepmod2", line 3, in <module>
import time, itertools, torch
ModuleNotFoundError: No module named 'torch'
- I am trying to run the following under the deepmod2 conda environment:
[hsinlun@gl-login1 bin]$ conda activate deepmod2
(deepmod2) [hsinlun@gl-login1 bin]$ python -c 'import torch'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'torch'
Thank you,
Hsin
It seems like your PATH variable may not be set correctly and you are not using the python from within the environment. Can you run the following commands and let me know what you get? They will help us figure out whether the you are using the correct python installation.
echo $CONDA_PREFIX
echo $PATH
which python
python -c 'import os;print(os.environ["PATH"])'
python -c "import sys; print(sys.executable)"
Sure. I activated the deepmod2 environment via conda activate deepmod2 and then ran these commands.
(deepmod2) [hsinlun@gl-login1 bin]$ echo $CONDA_PREFIX
/home/hsinlun/miniconda3/envs/deepmod2
(deepmod2) [hsinlun@gl-login1 bin]$ echo $PATH
/sw/spack/bio/pkgs/gcc-10.3.0/igv/2.8.9-mzrraluh/bin:/sw/pkgs/arc/openjdk/jdk-18.0.1.1/bin:/opt/singularity/3.11.1/bin:/sw/spack/bio/pkgs/gcc-10.3.0/samtools/1.13-fwwss5nm/bin:/sw/spack/bio/pkgs/gcc-10.3.0/htslib/1.13-dzhwvpsr/bin:/sw/spack/bio/pkgs/gcc-10.3.0/python/3.9.7-rv5ybzg3/bin:/home/hsinlun/bin/RepeatMasker:/home/hsinlun/bin/deepsignal/scripts:/home/hsinlun/bin/ncbi-blast-2.13.0+/bin:/home/hsinlun/bin/TRF/src:/home/hsinlun/bin/DeepSimulator:/home/hsinlun/bin/NanoSim/src:/nfs/turbo/Source/OSC/minimap2-2.24_x64-linux:/nfs/turbo/ying/ont-guppy/bin:/nfs/turbo/Source/OSC/nanopolish_v0.14.0/nanopolish:/nfs/turbo/Source/OSC/minia-2.0.3-Linux/bin:/home/hsinlun/miniconda3/envs/deepmod2/bin:/home/hsinlun/miniconda3/condabin:/home/hsinlun/.local/bin:/home/hsinlun/bin:/opt/TurboVNC/bin:/sw/pkgs/arc/usertools/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lpp/mmfs/bin:/opt/ddn/ime/bin
(deepmod2) [hsinlun@gl-login1 bin]$ which python
/sw/spack/bio/pkgs/gcc-10.3.0/python/3.9.7-rv5ybzg3/bin/python
(deepmod2) [hsinlun@gl-login1 bin]$ python -c 'import os;print(os.environ["PATH"])'
/sw/spack/bio/pkgs/gcc-10.3.0/igv/2.8.9-mzrraluh/bin:/sw/pkgs/arc/openjdk/jdk-18.0.1.1/bin:/opt/singularity/3.11.1/bin:/sw/spack/bio/pkgs/gcc-10.3.0/samtools/1.13-fwwss5nm/bin:/sw/spack/bio/pkgs/gcc-10.3.0/htslib/1.13-dzhwvpsr/bin:/sw/spack/bio/pkgs/gcc-10.3.0/python/3.9.7-rv5ybzg3/bin:/home/hsinlun/bin/RepeatMasker:/home/hsinlun/bin/deepsignal/scripts:/home/hsinlun/bin/ncbi-blast-2.13.0+/bin:/home/hsinlun/bin/TRF/src:/home/hsinlun/bin/DeepSimulator:/home/hsinlun/bin/NanoSim/src:/nfs/turbo/Source/OSC/minimap2-2.24_x64-linux:/nfs/turbo/ying/ont-guppy/bin:/nfs/turbo/Source/OSC/nanopolish_v0.14.0/nanopolish:/nfs/turbo/Source/OSC/minia-2.0.3-Linux/bin:/home/hsinlun/miniconda3/envs/deepmod2/bin:/home/hsinlun/miniconda3/condabin:/home/hsinlun/.local/bin:/home/hsinlun/bin:/opt/TurboVNC/bin:/sw/pkgs/arc/usertools/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lpp/mmfs/bin:/opt/ddn/ime/bin
(deepmod2) [hsinlun@gl-login1 bin]$ python -c "import sys; print(sys.executable)"
/sw/spack/bio/pkgs/gcc-10.3.0/python/3.9.7-rv5ybzg3/bin/python
Yes, that seems to be the problem. Your conda environment is /home/hsinlun/miniconda3/envs/deepmod2 and there should be a python installation within the bin subfolder: /home/hsinlun/miniconda3/envs/deepmod2/bin/python. This is the correct ptyhon installation that should be used. Normally when you activate a conda environment, the its path is added to the front of PATH to prioritize packages from conda environment. Therefore, if you run python command, it will find a python installation within conda environment and use that. In your case, it seems like you are loading several more software into the PATH, such as IGV and SAMtools etc. Normally this would not be a problem, however you are also loading a different version of python from the one installed in conda environment. This can be seen in your PATH variable as /sw/spack/bio/pkgs/gcc-10.3.0/python/3.9.7-rv5ybzg3/bin, and thus when you run which python, it shows that you are running a different python installation: /sw/spack/bio/pkgs/gcc-10.3.0/python/3.9.7-rv5ybzg3/bin/python.
Here are a few alternative ways to fix the problem:
1- Do not load another python into PATH in addition to activating conda environment.
2- Prioritize conda environment in your path: export PATH=/home/hsinlun/miniconda3/envs/deepmod2/bin:$PATH
3- Use /home/hsinlun/miniconda3/envs/deepmod2/bin/python deepmod2 --help to specify which python installation you want to use.
Hi Umair,
Thank you for your prompt and clear response. I apologize for not noticing that the conda environment path was accidentally removed. This error was fixed based on your suggestion.
Additionally, I attempted to execute your code on example data as outlined in the Examples section. I noticed a potential typo in the command:
python ${INPUT_DIR}/DeepMod2/deepmod2 detect-guppy --file_type fast5 --bam ${BASECALL_DIR}/fast5_data/merged.aligned.bam --input ${INPUT_DIR}/raw_fast5 --output ${OUTPUT_DIR}/mod_from_fast5_aligned_BAM --ref ${INPUT_DIR}/GRCh38.fa
I believe it should be detect rather than detect-guppy
After making this correction, I encountered the following error:
usage: deepmod2 detect [-h] [--prefix PREFIX] [--output OUTPUT] [--qscore_cutoff QSCORE_CUTOFF] [--length_cutoff LENGTH_CUTOFF] [--mod_t MOD_T] [--unmod_t UNMOD_T] [--include_non_cpg_ref]
[--threads THREADS] [--ref REF] --model MODEL --bam BAM --file_type {fast5,pod5} --input INPUT [--guppy_group GUPPY_GROUP] [--chrom [CHROM ...]] [--fast5_move]
[--skip_per_site] [--device DEVICE] [--disable_pruning] [--exclude_ref_features] [--batch_size BATCH_SIZE] [--bam_threads BAM_THREADS] [--skip_unmapped]
deepmod2 detect: error: the following arguments are required: --model
I have found the answer from issue #16.
However, a similar issue occurred again after executing the command: python ../deepmod2 detect --file_type fast5 --bam ${BASECALL_DIR}/fast5_data/merged.aligned.bam --input ${INPUT_DIR}/raw_fast5 --output ${OUTPUT_DIR}/mod_from_fast5_aligned_BAM --ref ${INPUT_DIR}/GRCh38.fa --model transformer_r10.4.1_4khz
The error message is:
2024-02-15 23:32:30.270085: Starting DeepMod2.
2024-02-15 23:32:30.278242:
Command: python ../deepmod2 detect --file_type fast5 --bam basecall/fast5_data/merged.aligned.bam --input data/raw_fast5 --output mod/mod_from_fast5_aligned_BAM --ref data/GRCh38.fa --model transformer_r10.4.1_4khz
Traceback (most recent call last):
File "/nfs/turbo/hsinlun/bin/DeepMod2/example/../deepmod2", line 146, in <module>
from src import detect
File "/nfs/turbo/hsinlun/bin/DeepMod2/src/detect.py", line 20, in <module>
import pod5 as p5
ModuleNotFoundError: No module named 'pod5'
I check my environment as follow:
(deepmod2) [hsinlun@gl-login1 example]$ echo $CONDA_PREFIX
/home/hsinlun/miniconda3/envs/deepmod2
(deepmod2) [hsinlun@gl-login1 example]$ echo $PATH
/home/hsinlun/miniconda3/envs/deepmod2/bin:/sw/spack/bio/pkgs/gcc-10.3.0/igv/2.8.9-mzrraluh/bin:/sw/pkgs/arc/openjdk/jdk-18.0.1.1/bin:/opt/singularity/3.11.1/bin:/sw/spack/bio/pkgs/gcc-10.3.0/samtools/1.13-fwwss5nm/bin:/sw/spack/bio/pkgs/gcc-10.3.0/htslib/1.13-dzhwvpsr/bin:/home/hsinlun/bin/RepeatMasker:/home/hsinlun/bin/deepsignal/scripts:/home/hsinlun/bin/ncbi-blast-2.13.0+/bin:/home/hsinlun/bin/TRF/src:/home/hsinlun/bin/DeepSimulator:/home/hsinlun/bin/NanoSim/src:/nfs/turbo/Source/OSC/minimap2-2.24_x64-linux:/nfs/turbo/ont-guppy/bin:/nfs/turbo/Source/OSC/nanopolish_v0.14.0/nanopolish:/nfs/turbo/Source/OSC/minia-2.0.3-Linux/bin:/home/hsinlun/miniconda3/envs/nanosim/bin:/home/hsinlun/miniconda3/condabin:/home/hsinlun/.local/bin:/home/hsinlun/bin:/opt/TurboVNC/bin:/sw/pkgs/arc/usertools/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lpp/mmfs/bin:/opt/ddn/ime/bin
(deepmod2) [hsinlun@gl-login1 example]$ which python
~/miniconda3/envs/deepmod2/bin/python
(deepmod2) [hsinlun@gl-login1 example]$ python -c 'import os;print(os.environ["PATH"])'
/home/hsinlun/miniconda3/envs/deepmod2/bin:/sw/spack/bio/pkgs/gcc-10.3.0/igv/2.8.9-mzrraluh/bin:/sw/pkgs/arc/openjdk/jdk-18.0.1.1/bin:/opt/singularity/3.11.1/bin:/sw/spack/bio/pkgs/gcc-10.3.0/samtools/1.13-fwwss5nm/bin:/sw/spack/bio/pkgs/gcc-10.3.0/htslib/1.13-dzhwvpsr/bin:/home/hsinlun/bin/RepeatMasker:/home/hsinlun/bin/deepsignal/scripts:/home/hsinlun/bin/ncbi-blast-2.13.0+/bin:/home/hsinlun/bin/TRF/src:/home/hsinlun/bin/DeepSimulator:/home/hsinlun/bin/NanoSim/src:/nfs/turbo/Source/OSC/minimap2-2.24_x64-linux:/nfs/turbo/ont-guppy/bin:/nfs/turbo/Source/OSC/nanopolish_v0.14.0/nanopolish:/nfs/turbo/Source/OSC/minia-2.0.3-Linux/bin:/home/hsinlun/miniconda3/envs/nanosim/bin:/home/hsinlun/miniconda3/condabin:/home/hsinlun/.local/bin:/home/hsinlun/bin:/opt/TurboVNC/bin:/sw/pkgs/arc/usertools/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lpp/mmfs/bin:/opt/ddn/ime/bin
(deepmod2) [hsinlun@gl-login1 example]$ python -c "import sys; print(sys.executable)"
/home/hsinlun/miniconda3/envs/deepmod2/bin/python
(deepmod2) [hsinlun@gl-login1 example]$ which pod5
~/miniconda3/envs/deepmod2/bin/pod5
I have some following up questions:
I am wondering if you provide a model for this example. Can we use our trained model? Are they any tips for choosing a model for R9.4.1 data, bilstm_r9.4.1 or transformer_r9.4.1?
Thank you for your assistance.
This is very odd, because your output shows that pod5 is installed under the environment. What happens if you run:
pip show pod5
or
python -c "import pod5"
Perhaps reinstalling pod5 separately may fix the problem. Please activate the environment and reinstall pod5 using pip install pod5.
With regards to model selection, we have provided several models in the github repo, you can check all available models using the command python deepmod2 --print_models. When you run detect module, just specify the name of the model. We would recommend bilstm_r9.4.1 model for r9.4.1, and you can check more details on evaluation here:
Ahsan, M.U., Gouru, A., Chan, J. et al. A signal processing and deep learning framework for methylation detection using Oxford Nanopore sequencing. Nat Commun 15, 1448 (2024). https://doi.org/10.1038/s41467-024-45778-y
You can also provide your own model, only if it is trained with DeepMod2. We have released the code for training the models here: https://github.com/WGLab/DeepMod2/tree/main/train. Once you have trained a model, provide it as --model MODELTYPE,PATH, where MODELTYPE is either "bilstm" or "transformer" and PATH is the path to saved model, e.g.
--model bilstm,/home/data/model.h5