Unable to import compress

[mildewey]

I'm trying to import bbi on ubuntu 18.04 in an anaconda environment. I get the following error on trying to import:

$ python
Python 3.6.7 |Anaconda, Inc.| (default, Oct 23 2018, 19:16:44) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import bbi
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mildewey/.local/lib/python3.6/site-packages/bbi/__init__.py", line 3, in <module>
    from .cbbi import (
ImportError: /home/mildewey/.local/lib/python3.6/site-packages/bbi/cbbi.cpython-36m-x86_64-linux-gnu.so: undefined symbol: compress

[nvictus]

Looks like a linking error to zlib. Does your Anaconda environment have its own zlib?

[mildewey]

I've tried to install zlib1g-dev but that doesn't resolve the issue. Is there a particular zlib I need to have installed?

[mildewey]

For the record, I double checked that I had installed all of build-essential, make, libssl-dev, libpng16-dev (libpng-dev on 18.04).

[nvictus]

I'm wondering if there's a linking conflict between your system's libraries and ones that are in your conda environment. Maybe the wrong one got used when pybbi compiled. Do you still get the error if you build on a barebones conda environment? e.g.

conda create -n test numpy cython
source activate test
pip install pybbi

[nvictus]

Another person was able to reproduce this. The issue seemed to be that some systems might have their C compiler set to use an optimization that doesn't link a library if it doesn't appear to be used by the extension code.

Could you try the following?

$ LDFLAGS='-Wl,--no-as-needed' DISTUTILS_DEBUG=1 pip install -v pybbi --log debug.log

If it doesn't work, I'd be curious to see the line in the log where the extension module (cbbi.cpython-*.so) file gets compiled

[mildewey]

Yes, these flags worked for me! Thanks for chasing that down.

[zzwch]

Another person was able to reproduce this. The issue seemed to be that some systems might have their C compiler set to use an optimization that doesn't link a library if it doesn't appear to be used by the extension code.

Could you try the following?

$ LDFLAGS='-Wl,--no-as-needed' DISTUTILS_DEBUG=1 pip install -v pybbi --log debug.log

If it doesn't work, I'd be curious to see the line in the log where the extension module (cbbi.cpython-*.so) file gets compiled

I tried this command, and installation is successful again.
But I still encountered the Import Error.
The curious thing is that there is no line matching with cbbi.cpython-*.so.

my OS is Centos 6.6

[nvictus]

@lizc07, you should try running with the verbose logging but without the LDFLAGS variable. I suspect you are missing some required development headers like those for zlib, openssl or libpng.

I got my hands on a Centos 6.6 instance. The following prerequisites worked for me:

sudo yum install make automake gcc gcc-c++ kernel-devel zlib-devel libpng-devel openssl-devel

Then I created a fresh conda environment with numpy and cython, and pip install pybbi.
Both build and import succeeded.

[zzwch]

Tried as you suggest, and everything is ok in the fresh Conda environment.

But same error is still there in the old conda environment. wonder what is the key to solve the problem...

whatever, I will use the fresh environment for my following analysis. Thanks for your help！

[zzwch]

I create a fresh Conda environment with python==3.6.8, and then the error appeared again....
However python==3.7 is OK!
But HiCExplorer Requires Python 3.6. There is a conflict.

So weird...

[nvictus]

pybbi is not a dependency of HiCExplorer. Are you confusing it with pyBigWig?

[zzwch]

sorry for delayed reply.

I am analyzing HiC data using both HiCExplorer and higlass. Higlass depends on pybbi.