higlass/clodius

error installation

egenomics opened this issue · 5 comments

egenomics commented

I am getting this error while trying to install clodius.

sudo pip install clodius

Collecting clodius
Downloading https://files.pythonhosted.org/packages/7f/f4/9409fa3e1917cd8de859943de7dd77286391873c7aa0f55bccfdfad86cb2/clodius-0.9.3.tar.gz (90kB)
100% |████████████████████████████████| 92kB 2.1MB/s
Requirement already satisfied: cython in /usr/local/lib64/python3.6/site-packages (from clodius) (0.28.5)
Requirement already satisfied: numpy in /usr/local/lib64/python3.6/site-packages (from clodius) (1.15.1)
Requirement already satisfied: negspy in /usr/local/lib/python3.6/site-packages (from clodius) (0.2.20)
Requirement already satisfied: pysam in /usr/local/lib64/python3.6/site-packages (from clodius) (0.15.0)
Requirement already satisfied: requests in /usr/lib/python3.6/site-packages (from clodius) (2.18.4)
Requirement already satisfied: h5py in /usr/local/lib64/python3.6/site-packages (from clodius) (2.8.0)
Requirement already satisfied: pandas in /usr/local/lib64/python3.6/site-packages (from clodius) (0.23.4)
Requirement already satisfied: slugid in /usr/local/lib/python3.6/site-packages (from clodius) (1.0.7)
Requirement already satisfied: sortedcontainers in /usr/local/lib/python3.6/site-packages (from clodius) (2.0.5)
Requirement already satisfied: nose in /usr/local/lib/python3.6/site-packages (from clodius) (1.3.7)
Requirement already satisfied: Click in /usr/local/lib64/python3.6/site-packages (from clodius) (6.7)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python3.6/site-packages (from requests->clodius) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/lib/python3.6/site-packages (from requests->clodius) (2.5)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/lib/python3.6/site-packages (from requests->clodius) (1.22)
Requirement already satisfied: six in /usr/lib/python3.6/site-packages (from h5py->clodius) (1.11.0)
Requirement already satisfied: pytz>=2011k in /usr/lib/python3.6/site-packages (from pandas->clodius) (2017.2)
Requirement already satisfied: python-dateutil>=2.5.0 in /usr/local/lib/python3.6/site-packages (from pandas->clodius) (2.7.3)
Installing collected packages: clodius
Running setup.py install for clodius ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-install-t95klw2h/clodius/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-vpzh88ol/install-record.txt --single-version-externally-managed --compile:
packages: ['clodius', 'scripts', 'test', 'clodius.cli']
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/clodius
copying clodius/describe_dataset.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/save_tiles.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/init.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/tiles.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/hdf_tiles.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/fpark.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/higlass_getter.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/chromosomes.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/db_tiles.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/multivec.py -> build/lib.linux-x86_64-3.6/clodius
creating build/lib.linux-x86_64-3.6/clodius/cli
copying clodius/cli/utils.py -> build/lib.linux-x86_64-3.6/clodius/cli
copying clodius/cli/init.py -> build/lib.linux-x86_64-3.6/clodius/cli
copying clodius/cli/convert.py -> build/lib.linux-x86_64-3.6/clodius/cli
copying clodius/cli/aggregate.py -> build/lib.linux-x86_64-3.6/clodius/cli
running build_ext
building 'clodius.fast' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/clodius
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -mcet -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -I/usr/local/lib64/python3.6/site-packages/numpy/core/include -I/usr/local/lib64/python3.6/site-packages/numpy/core/include -c clodius/fast.c -o build/temp.linux-x86_64-3.6/clodius/fast.o
clodius/fast.c:24:10: fatal error: Python.h: No such file or directory
#include "Python.h"
^~~~~~~~~~
compilation terminated.
error: command 'gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-install-t95klw2h/clodius/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-vpzh88ol/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-t95klw2h/clodius/

itsameerkat commented

I'm having very similar issues:

pip install clodius
Collecting clodius
Using cached https://files.pythonhosted.org/packages/7f/f4/9409fa3e1917cd8de859943de7dd77286391873c7aa0f55bccfdfad86cb2/clodius-0.9.3.tar.gz
Requirement already satisfied: cython in /home/sameer/miniconda3/lib/python3.6/site-packages (from clodius) (0.27.3)
Requirement already satisfied: numpy in /home/sameer/miniconda3/lib/python3.6/site-packages (from clodius) (1.15.1)
Requirement already satisfied: negspy in /home/sameer/miniconda3/lib/python3.6/site-packages (from clodius) (0.2.20)
Requirement already satisfied: pysam in /home/sameer/miniconda3/lib/python3.6/site-packages (from clodius) (0.11.2.2)
Requirement already satisfied: requests in /home/sameer/miniconda3/lib/python3.6/site-packages (from clodius) (2.18.4)
Requirement already satisfied: h5py in /home/sameer/miniconda3/lib/python3.6/site-packages (from clodius) (2.7.0)
Requirement already satisfied: pandas in /home/sameer/miniconda3/lib/python3.6/site-packages (from clodius) (0.23.4)
Requirement already satisfied: slugid in /home/sameer/miniconda3/lib/python3.6/site-packages (from clodius) (1.0.7)
Requirement already satisfied: sortedcontainers in /home/sameer/miniconda3/lib/python3.6/site-packages (from clodius) (1.5.9)
Requirement already satisfied: nose in /home/sameer/miniconda3/lib/python3.6/site-packages (from clodius) (1.3.7)
Requirement already satisfied: Click in /home/sameer/miniconda3/lib/python3.6/site-packages (from clodius) (6.7)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/sameer/miniconda3/lib/python3.6/site-packages (from requests->clodius) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /home/sameer/miniconda3/lib/python3.6/site-packages (from requests->clodius) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /home/sameer/miniconda3/lib/python3.6/site-packages (from requests->clodius) (1.22)
Requirement already satisfied: certifi>=2017.4.17 in /home/sameer/miniconda3/lib/python3.6/site-packages (from requests->clodius) (2018.8.24)
Requirement already satisfied: six in /home/sameer/miniconda3/lib/python3.6/site-packages (from h5py->clodius) (1.11.0)
Requirement already satisfied: python-dateutil>=2.5.0 in /home/sameer/miniconda3/lib/python3.6/site-packages (from pandas->clodius) (2.6.1)
Requirement already satisfied: pytz>=2011k in /home/sameer/miniconda3/lib/python3.6/site-packages (from pandas->clodius) (2018.3)
Building wheels for collected packages: clodius
Running setup.py bdist_wheel for clodius ... error
Complete output from command /home/sameer/miniconda3/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-install-b62abxmu/clodius/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/pip-wheel-_312vcb5 --python-tag cp36:
packages: ['clodius', 'scripts', 'test', 'clodius.cli']
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/clodius
copying clodius/fpark.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/tiles.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/multivec.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/describe_dataset.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/chromosomes.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/hdf_tiles.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/save_tiles.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/higlass_getter.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/init.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/db_tiles.py -> build/lib.linux-x86_64-3.6/clodius
creating build/lib.linux-x86_64-3.6/clodius/cli
copying clodius/cli/utils.py -> build/lib.linux-x86_64-3.6/clodius/cli
copying clodius/cli/aggregate.py -> build/lib.linux-x86_64-3.6/clodius/cli
copying clodius/cli/convert.py -> build/lib.linux-x86_64-3.6/clodius/cli
copying clodius/cli/init.py -> build/lib.linux-x86_64-3.6/clodius/cli
running build_ext
building 'clodius.fast' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/clodius
gcc -pthread -B /home/sameer/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/sameer/miniconda3/include/python3.6m -I/home/sameer/miniconda3/lib/python3.6/site-packages/numpy/core/include -I/home/sameer/miniconda3/lib/python3.6/site-packages/numpy/core/include -c clodius/fast.c -o build/temp.linux-x86_64-3.6/clodius/fast.o
In file included from /home/sameer/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1823:0,
from /home/sameer/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
from /home/sameer/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from clodius/fast.c:546:
/home/sameer/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it by "
^
gcc -pthread -shared -B /home/sameer/miniconda3/compiler_compat -L/home/sameer/miniconda3/lib -Wl,-rpath=/home/sameer/miniconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/clodius/fast.o -o build/lib.linux-x86_64-3.6/clodius/fast.cpython-36m-x86_64-linux-gnu.so
/home/sameer/miniconda3/compiler_compat/ld: cannot find -lpthread
/home/sameer/miniconda3/compiler_compat/ld: cannot find -lc
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

Failed building wheel for clodius
Running setup.py clean for clodius
Failed to build clodius
Installing collected packages: clodius
Running setup.py install for clodius ... error
Complete output from command /home/sameer/miniconda3/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-install-b62abxmu/clodius/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-a8pxzu1s/install-record.txt --single-version-externally-managed --compile:
packages: ['clodius', 'scripts', 'test', 'clodius.cli']
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/clodius
copying clodius/fpark.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/tiles.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/multivec.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/describe_dataset.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/chromosomes.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/hdf_tiles.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/save_tiles.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/higlass_getter.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/init.py -> build/lib.linux-x86_64-3.6/clodius
copying clodius/db_tiles.py -> build/lib.linux-x86_64-3.6/clodius
creating build/lib.linux-x86_64-3.6/clodius/cli
copying clodius/cli/utils.py -> build/lib.linux-x86_64-3.6/clodius/cli
copying clodius/cli/aggregate.py -> build/lib.linux-x86_64-3.6/clodius/cli
copying clodius/cli/convert.py -> build/lib.linux-x86_64-3.6/clodius/cli
copying clodius/cli/init.py -> build/lib.linux-x86_64-3.6/clodius/cli
running build_ext
building 'clodius.fast' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/clodius
gcc -pthread -B /home/sameer/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/sameer/miniconda3/include/python3.6m -I/home/sameer/miniconda3/lib/python3.6/site-packages/numpy/core/include -I/home/sameer/miniconda3/lib/python3.6/site-packages/numpy/core/include -c clodius/fast.c -o build/temp.linux-x86_64-3.6/clodius/fast.o
In file included from /home/sameer/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1823:0,
from /home/sameer/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
from /home/sameer/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from clodius/fast.c:546:
/home/sameer/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it by "
^
gcc -pthread -shared -B /home/sameer/miniconda3/compiler_compat -L/home/sameer/miniconda3/lib -Wl,-rpath=/home/sameer/miniconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/clodius/fast.o -o build/lib.linux-x86_64-3.6/clodius/fast.cpython-36m-x86_64-linux-gnu.so
/home/sameer/miniconda3/compiler_compat/ld: cannot find -lpthread
/home/sameer/miniconda3/compiler_compat/ld: cannot find -lc
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

Command "/home/sameer/miniconda3/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-install-b62abxmu/clodius/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-a8pxzu1s/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-b62abxmu/clodius/

pkerpedjiev commented

Sorry for the delayed response. With @nvictus's help, we removed the cython dependencies and created a new release. Could you try out 0.9.4 and see if you get the same errors?

LeilyR commented

no other version newer than 0.9.4 cannot actually be installed. I am having trouble with some package conflict (openssl, libpng) and installation fails. ( ERROR: Failed building wheel for pybbi)
I even tried direct installation from github but that failed too. on the other hand 0.9.4 has issue with loading fly genome and throw this error:
'uid': slugid.nice().decode('utf-8'),
AttributeError: 'str' object has no attribute 'decode'

I know from code that decode('utf-8') is not in the code anymore that is why I needed a more recent version

This is actually the error from openssl:

C_INCLUDE_PATH: /data/processing3/leily/miniconda3/envs/clodius/include/libpng16:/data/processing3/leily/miniconda3/envs/clodius/include/openssl:
cd src && make
make[1]: Entering directory '/tmp/pip-install-iv3r4vhp/pybbi/src'
gcc -g -pthread -fPIC -static -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DMACHTYPE_x86_64 -L/tmp/pip-install-iv3r4vhp/pybbi/src/x86_64 -L/usr/lib -lz -lc -lpthread -I/tmp/pip-install-iv3r4vhp/pybbi/include -I/tmp/pip-install-iv3r4vhp/pybbi/src -I/usr/include -o udc.o -c udc.c
udc.c:39:10: fatal error: openssl/sha.h: No such file or directory
#include <openssl/sha.h>
^~~~~~~~~~~~~~~

I really appreciate your help.

nvictus commented

It's more C header dependencies. Are you able to install them with sudo apt install libssl1.0-dev libpng-dev and try again?

For the future, I can look into distributing binaries of pybbi so this isn't necessary, or we can migrate to pyBigWig, but even that requires libcurl if not using conda.

LeilyR commented

thanks. I have already installed both openssl and libpng via conda at the same env but now I will install them with apt-get and see if that makes a difference.