kishwarshafin/helen

Issue on stitch.py

nozabey opened this issue · 7 comments

nozabey commented

Hello!

I have a problem with stitch.py. When I use the script with the following command A), I got the error B). Do you have any suggestions to resolve the issue. Thanks in advance!

A)
python3.6 /home/nozawa/Software/helen/stitch.py -t 16
-i /home/nozawa/Data/pal/MinION/MarginPolish_HELEN/consensus/HELEN_prediction.hdf
-o /home/nozawa/Data/pal/MinION/MarginPolish_HELEN/consensus

B)
Traceback (most recent call last):
File "/home/nozawa/Software/helen/stitch.py", line 5, in
from modules.python.Stitch import Stitch
File "/home/nozawa/Software/helen/modules/python/Stitch.py", line 10, in
from build import HELEN
ImportError: cannot import name 'HELEN'

kishwarshafin commented

Hi @nozabey

Looks like you didn't run ./build.sh from inside the helen directory. Please do this:

cd /home/nozawa/Software/helen/
./build.sh

This should solve your issue.

nozabey commented

Hi!

Thank you very much for your reply! I followed your suggestion. Then, the following message appeared.

Could you please give me any additional suggestions?

For just in case, I am attaching CMakeOutput.log.
CMakeOutput.log

Best regards,

--------------from here------------------
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /usr/local/bin/python3 (found suitable version "3.6.3", minimum required is "3")
-- Found PythonLibs: /usr/local/lib/libpython3.6m.a (found suitable version "3.6.3", minimum required is "3")
Python found
CMake Error at modules/pybind11.cmake:1 (include):
include could not find load file:

FetchContent

Call Stack (most recent call first):
CMakeLists.txt:25 (INCLUDE)

CMake Error at modules/pybind11.cmake:2 (FetchContent_Declare):
Unknown CMake command "FetchContent_Declare".
Call Stack (most recent call first):
CMakeLists.txt:25 (INCLUDE)

-- Configuring incomplete, errors occurred!
See also "/home/nozawa/Software/helen/build/CMakeFiles/CMakeOutput.log".
---------------up to here---------------

kishwarshafin commented

Hi @nozabey ,

HELEN requires cmake a newer cmake than you have in your system. You can check your cmake version by running:

cmake --version

To install cmake 3.14, please run this:

wget https://github.com/Kitware/CMake/releases/download/v3.14.4/cmake-3.14.4-Linux-x86_64.sh && sudo mkdir /opt/cmake && sudo sh cmake-3.14.4-Linux-x86_64.sh --prefix=/opt/cmake --skip-license && sudo ln -s /opt/cmake/bin/cmake /usr/local/bin/cmake
cmake --version

This should install cmake 3.14 and solve your issue.

nozabey commented

Hi!

Thank you for your additional suggestions. I now installed cmake 3.16 and successfully run ./build.sh from inside the helen directory as follows.

---------from here----------
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.5.2", minimum required is "3")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.5m.so (found suitable version "3.5.2", minimum required is "3")
Python found
-- Found PythonInterp: /usr/bin/python3 (found version "3.5.2")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.5m.so
-- pybind11 v2.2.4
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- LTO enabled
-- Configuring done
-- Generating done
-- Build files have been written to: /home/nozawa/Software/helen/build
Scanning dependencies of target HELEN
[ 50%] Building CXX object CMakeFiles/HELEN.dir/modules/src/pybind_api.cpp.o
[100%] Linking CXX shared module HELEN.cpython-35m-x86_64-linux-gnu.so
[100%] Built target HELEN
---------up to here----------

However, I still got the same message as follows.

---------from here----------
Traceback (most recent call last):
File "/home/nozawa/Software/helen/stitch.py", line 5, in
from modules.python.Stitch import Stitch
File "/home/nozawa/Software/helen/modules/python/Stitch.py", line 10, in
from build import HELEN
ImportError: cannot import name 'HELEN'
---------up to here----------

I am wondering the problem is that helen uses python 3.5.2.

So, I changed the command as follows.

---------from here----------
python3 /home/nozawa/Software/helen/stitch.py -t 16
-i /home/nozawa/Data/pal/MinION/MarginPolish_HELEN/consensus/HELEN_prediction.hdf
-o /home/nozawa/Data/pal/MinION/MarginPolish_HELEN/consensus
---------up to here----------

Then, I got the following error.

---------from here----------
Traceback (most recent call last):
File "/home/nozawa/Software/helen/stitch.py", line 1, in
import h5py
ImportError: No module named 'h5py'
---------up to here----------

I of course tried to install "h5py tqdm numpy torchnet pyyaml" as follows.

---------from here----------
python3 -m pip install h5py tqdm numpy torchnet pyyaml
---------up to here----------

However, they seem to have been installed on python 3.6.3.

---------from here----------
Requirement already satisfied: h5py in /usr/local/lib/python3.6/site-packages (2.10.0)
Requirement already satisfied: tqdm in /usr/local/lib/python3.6/site-packages (4.38.0)
Requirement already satisfied: numpy in /usr/local/lib/python3.6/site-packages (1.17.4)
Requirement already satisfied: torchnet in /usr/local/lib/python3.6/site-packages (0.0.4)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/site-packages (5.1.2)
Requirement already satisfied: six in /usr/local/lib/python3.6/site-packages (from h5py) (1.13.0)
Requirement already satisfied: torch in /usr/local/lib/python3.6/site-packages (from torchnet) (1.2.0+cpu)
Requirement already satisfied: visdom in /usr/local/lib/python3.6/site-packages (from torchnet) (0.1.8.9)
Requirement already satisfied: scipy in /usr/local/lib/python3.6/site-packages (from visdom->torchnet) (1.3.1)
Requirement already satisfied: requests in /usr/local/lib/python3.6/site-packages (from visdom->torchnet) (2.22.0)
Requirement already satisfied: tornado in /usr/local/lib/python3.6/site-packages (from visdom->torchnet) (6.0.3)
Requirement already satisfied: pyzmq in /usr/local/lib/python3.6/site-packages (from visdom->torchnet) (18.1.0)
Requirement already satisfied: jsonpatch in /usr/local/lib/python3.6/site-packages (from visdom->torchnet) (1.24)
Requirement already satisfied: torchfile in /usr/local/lib/python3.6/site-packages (from visdom->torchnet) (0.1.0)
Requirement already satisfied: websocket-client in /usr/local/lib/python3.6/site-packages (from visdom->torchnet) (0.56.0)
Requirement already satisfied: pillow in /usr/local/lib/python3.6/site-packages (from visdom->torchnet) (6.2.1)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/site-packages (from requests->visdom->torchnet) (2.8)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/site-packages (from requests->visdom->torchnet) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/site-packages (from requests->visdom->torchnet) (1.25.6)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/site-packages (from requests->visdom->torchnet) (2019.9.11)
Requirement already satisfied: jsonpointer>=1.9 in /usr/local/lib/python3.6/site-packages (from jsonpatch->visdom->torchnet) (2.0)
---------up to here----------

I deeply apologize for disturbing you many times, could you please let me know how to install these packages on python 3.5.2?

Best regards,

kishwarshafin commented

Hi @nozabey ,

It looks like you have two different versions of python installed in your system. The cmake finds and uses /usr/bin/python3 while you are using a different version. Can you please run this command and see what happens:

/usr/bin/python3 -m pip install h5py tqdm numpy torchnet pyyaml

/usr/bin/python3 /home/nozawa/Software/helen/stitch.py -t 16
-i /home/nozawa/Data/pal/MinION/MarginPolish_HELEN/consensus/HELEN_prediction.hdf
-o /home/nozawa/Data/pal/MinION/MarginPolish_HELEN/consensus

In your system, maybe python3 and python3.6 are the same? You can check by running which python3 and which python3.6.

Also, we highly suggest using docker if you are still facing this issue:
Install docker:

sudo apt-get update

sudo apt install docker.io
sudo systemctl start docker
sudo systemctl enable docker

sudo groupadd docker
sudo gpasswd -a $USER docker
# now log out and log in once or run:
newgrp docker

Run stitch.py

sudo docker pull kishwars/helen:0.0.1.cpu

docker run -it --rm --user=`id -u`:`id -g` --cpus="16" -v /home/nozawa/Data:/data kishwars/helen:0.0.1.cpu stitch.py \
-i pal/MinION/MarginPolish_HELEN/consensus/HELEN_prediction.hdf \
-o pal/MinION/MarginPolish_HELEN/consensus \
-p helen_polished_assembly \
-t 16

Please let me know if you have figured out the issue. Thanks.

nozabey commented

Hi,

Thank you very much for your detailed instructions. Certainly, I have installed different versions of python which apparently cause this complication. I now changed the priority of python version and consistently use python3.6. And finally, I now can use stitch.py without any errors!

Thank you again for your kind instructions. I may ask further questions in the near future.

Best regards,

kishwarshafin commented

Great! I'm closing this issue, please feel free to open another if you need any help.