stephan-hof/pyrocksdb

Install problem on centos 5

Closed this issue · 8 comments

dineshbvadhia commented

Below is the output - cannot see what the problem is. Thanks!

$ g++ --version
g++ (GCC) 4.8.1
$ gcc --version
gcc (GCC) 4.8.1

$ cd rocksdb
$ git checkout 2.8.fb
Note: checking out '2.8.fb'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b new_branch_name

HEAD is now at f01a04e... Update HISTORY.md

$ export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/home/scratch/software/rocksdb/include
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/scratch/software/rocksdb
$ export LIBRARY_PATH=$LIBRARY_PATH:/home/scratch/software/rocksdb

$ pip install git+git://github.com/stephan-hof/pyrocksdb.git@v0.2
Downloading/unpacking git+git://github.com/stephan-hof/pyrocksdb.git@v0.2
Cloning git://github.com/stephan-hof/pyrocksdb.git (to v0.2) to /tmp/pip-HZQ_kU-build
Running setup.py (path:/tmp/pip-HZQ_kU-build/setup.py) egg_info for package from git+git://github.com/stephan-hof/pyrocksdb.git@v0.2
missing cimport in module 'pair': /home/anaconda/lib/python2.7/site-packages/Cython/Includes/libcpp/deque.pxd
Compiling rocksdb/_rocksdb.pyx because it changed.
Cythonizing rocksdb/_rocksdb.pyx

Requirement already satisfied (use --upgrade to upgrade): setuptools in /home/anaconda/lib/python2.7/site-packages/setuptools-3.6-py2.7.egg (from pyrocksdb==0.2)
Requirement already satisfied (use --upgrade to upgrade): Cython>=0.20 in /home/anaconda/lib/python2.7/site-packages (from pyrocksdb==0.2)
Installing collected packages: pyrocksdb
Running setup.py install for pyrocksdb
missing cimport in module 'pair': /home/anaconda/lib/python2.7/site-packages/Cython/Includes/libcpp/deque.pxd

building 'rocksdb._rocksdb' extension
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/anaconda/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=gnu++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: error: unrecognized command line option "-std=gnu++11"
error: command 'gcc' failed with exit status 1
Complete output from command /home/anaconda/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-HZQ_kU-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-JAnhVo-record/install-record.txt --single-version-externally-managed --compile:
missing cimport in module 'pair': /home/anaconda/lib/python2.7/site-packages/Cython/Includes/libcpp/deque.pxd

running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/interfaces.py -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/errors.py -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/init.py -> build/lib.linux-x86_64-2.7/rocksdb

creating build/lib.linux-x86_64-2.7/rocksdb/tests

copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-2.7/rocksdb/tests

copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-2.7/rocksdb/tests

copying rocksdb/tests/init.py -> build/lib.linux-x86_64-2.7/rocksdb/tests

running egg_info

creating pyrocksdb.egg-info

writing requirements to pyrocksdb.egg-info/requires.txt

writing pyrocksdb.egg-info/PKG-INFO

writing top-level names to pyrocksdb.egg-info/top_level.txt

writing dependency_links to pyrocksdb.egg-info/dependency_links.txt

writing manifest file 'pyrocksdb.egg-info/SOURCES.txt'

warning: manifest_maker: standard file '-c' not found

reading manifest file 'pyrocksdb.egg-info/SOURCES.txt'

writing manifest file 'pyrocksdb.egg-info/SOURCES.txt'

copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-2.7/rocksdb

running build_ext

building 'rocksdb._rocksdb' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/rocksdb

gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/anaconda/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=gnu++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing

cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++

cc1plus: error: unrecognized command line option "-std=gnu++11"

error: command 'gcc' failed with exit status 1

Cleaning up...
Command /home/anaconda/bin/python -c "import setuptools, tokenize;file='/tmp/pip-HZQ_kU-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-JAnhVo-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip-HZQ_kU-build
Storing debug log for failure in /home/.pip/pip.log

stephan-hof commented

Strange. According to the error message the "-std=gnu++11" switch is not recognized, but I have no evidence that this switch is removed in recent gcc versions. See here:

xxx@xxx-VirtualBox:~$ g++ --version
g++ (Ubuntu 4.8.2-19ubuntu1) 4.8.2
xxx@xxx-VirtualBox:~$ cat a.cpp 
int main(){}
xxx@xxx-VirtualBox:~$ g++ -std=gnu++11 a.cpp 
xxx@xxx-VirtualBox:~$

However I used this switch to compile with gcc-4.6 and gcc-4.7, because this was the only option to force c++11 in both versions. In gcc-4.6 they had -std=c++0x which was dropped in gcc-4.7 in favor of -std=c++11.

In gcc 4.7 and greater -std=c++11 seems the flag to use, but then it is failing with gcc-4.6. However rocksdb is also using -std=c++11 so I will migrate to that too. A commit will come within the next day.

dineshbvadhia commented

Sorry, below is the correct error output ie. it cannot find "lsnappy" though I'm sure the paths have been set correctly.

The bashrc file includes:
export PATH="/home/scratch/software/bin":$PATH
export LD_LIBRARY_PATH="/home/scratch/software/lib64:/home/scratch/software/lib":$LD_LIBRARY_PATH

The /home/scratch/software/lib folder shows:
libsnappy.so, libsnappy.la, libsnappy.so.1, libsnappy.so.1.2.0

$ export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/home/scratch/software/rocksdb/include
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/scratch/software/rocksdb
$ export LIBRARY_PATH=$LIBRARY_PATH:/home/scratch/software/rocksdb

$ pip install git+git://github.com/stephan-hof/pyrocksdb.git@v0.2
Downloading/unpacking git+git://github.com/stephan-hof/pyrocksdb.git@v0.2
Cloning git://github.com/stephan-hof/pyrocksdb.git (to v0.2) to /tmp/pip-bfFdWv-build
Running setup.py (path:/tmp/pip-bfFdWv-build/setup.py) egg_info for package from git+git://github.com/stephan-hof/pyrocksdb.git@v0.2
missing cimport in module 'pair': /home/anaconda/lib/python2.7/site-packages/Cython/Includes/libcpp/deque.pxd
Compiling rocksdb/_rocksdb.pyx because it changed.
Cythonizing rocksdb/_rocksdb.pyx

Requirement already satisfied (use --upgrade to upgrade): setuptools in /home/anaconda/lib/python2.7/site-packages/setuptools-3.6-py2.7.egg (from pyrocksdb==0.2)
Requirement already satisfied (use --upgrade to upgrade): Cython>=0.20 in /home/anaconda/lib/python2.7/site-packages (from pyrocksdb==0.2)
Installing collected packages: pyrocksdb
Running setup.py install for pyrocksdb
missing cimport in module 'pair': /home/anaconda/lib/python2.7/site-packages/Cython/Includes/libcpp/deque.pxd

building 'rocksdb._rocksdb' extension
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/anaconda/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=gnu++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing
cc1plus: warning: command line option \u2018-Wstrict-prototypes\u2019 is valid for C/ObjC but not for C++ [enabled by default]
rocksdb/_rocksdb.cpp: In function \u2018PyObject* __pyx_pf_7rocksdb_8_rocksdb_19PyBloomFilterPolicy_6create_filter(__pyx_obj_7rocksdb_8_rocksdb_PyBloomFilterPolicy*, PyObject*)\u2019:
rocksdb/_rocksdb.cpp:5594:99: warning: conversion to \u2018int\u2019 from \u2018std::vector<rocksdb::Slice>::size_type {aka long unsigned int}\u2019 may alter its value [-Wconversion]
     __pyx_v_self->policy->CreateFilter(py_rocks::vector_data(__pyx_v_c_keys), __pyx_v_c_keys.size(), (&__pyx_v_dst));
                                                                                                   ^
rocksdb/_rocksdb.cpp: In function \u2018int __Pyx_PyUnicode_Equals(PyObject*, PyObject*, int)\u2019:
rocksdb/_rocksdb.cpp:160:50: warning: conversion to \u2018int\u2019 from \u2018long unsigned int\u2019 may alter its value [-Wconversion]
   #define __Pyx_PyUnicode_KIND(u)         (sizeof(Py_UNICODE))
                                                  ^
rocksdb/_rocksdb.cpp:35007:16: note: in expansion of macro \u2018__Pyx_PyUnicode_KIND\u2019
         kind = __Pyx_PyUnicode_KIND(s1);
                ^
g++ -pthread -shared build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -L/home/anaconda/lib -lrocksdb -lsnappy -lbz2 -lz -lpython2.7 -o build/lib.linux-x86_64-2.7/rocksdb/_rocksdb.so
/usr/bin/ld: cannot find -lsnappy
collect2: error: ld returned 1 exit status
error: command 'g++' failed with exit status 1
Complete output from command /home/anaconda/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-bfFdWv-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-hjWu9g-record/install-record.txt --single-version-externally-managed --compile:
missing cimport in module 'pair': /home/anaconda/lib/python2.7/site-packages/Cython/Includes/libcpp/deque.pxd

running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/interfaces.py -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/errors.py -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/init.py -> build/lib.linux-x86_64-2.7/rocksdb

creating build/lib.linux-x86_64-2.7/rocksdb/tests

copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-2.7/rocksdb/tests

copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-2.7/rocksdb/tests

copying rocksdb/tests/init.py -> build/lib.linux-x86_64-2.7/rocksdb/tests

running egg_info

creating pyrocksdb.egg-info

writing requirements to pyrocksdb.egg-info/requires.txt

writing pyrocksdb.egg-info/PKG-INFO

writing top-level names to pyrocksdb.egg-info/top_level.txt

writing dependency_links to pyrocksdb.egg-info/dependency_links.txt

writing manifest file 'pyrocksdb.egg-info/SOURCES.txt'

warning: manifest_maker: standard file '-c' not found

reading manifest file 'pyrocksdb.egg-info/SOURCES.txt'

writing manifest file 'pyrocksdb.egg-info/SOURCES.txt'

copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-2.7/rocksdb

running build_ext

building 'rocksdb._rocksdb' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/rocksdb

gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/anaconda/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=gnu++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing

cc1plus: warning: command line option \u2018-Wstrict-prototypes\u2019 is valid for C/ObjC but not for C++ [enabled by default]

rocksdb/_rocksdb.cpp: In function \u2018PyObject* __pyx_pf_7rocksdb_8_rocksdb_19PyBloomFilterPolicy_6create_filter(pyx_obj_7rocksdb_8_rocksdb_PyBloomFilterPolicy, PyObject)\u2019:

rocksdb/_rocksdb.cpp:5594:99: warning: conversion to \u2018int\u2019 from \u2018std::vectorrocksdb::Slice::size_type {aka long unsigned int}\u2019 may alter its value [-Wconversion]

 __pyx_v_self->policy->CreateFilter(py_rocks::vector_data(__pyx_v_c_keys), __pyx_v_c_keys.size(), (&__pyx_v_dst));

                                                                                               ^

rocksdb/_rocksdb.cpp: In function \u2018int Pyx_PyUnicode_Equals(PyObject, PyObject, int)\u2019:

rocksdb/_rocksdb.cpp:160:50: warning: conversion to \u2018int\u2019 from \u2018long unsigned int\u2019 may alter its value [-Wconversion]

#define __Pyx_PyUnicode_KIND(u) (sizeof(Py_UNICODE))

                                              ^

rocksdb/_rocksdb.cpp:35007:16: note: in expansion of macro \u2018__Pyx_PyUnicode_KIND\u2019

     kind = __Pyx_PyUnicode_KIND(s1);

            ^

g++ -pthread -shared build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -L/home/anaconda/lib -lrocksdb -lsnappy -lbz2 -lz -lpython2.7 -o build/lib.linux-x86_64-2.7/rocksdb/_rocksdb.so

/usr/bin/ld: cannot find -lsnappy

collect2: error: ld returned 1 exit status

error: command 'g++' failed with exit status 1

dineshbvadhia commented

Re-compiled gflags, snappy and rocksdb and now get a "rocksdb/slice.h" cannot be found error. Both the "pip install ..." and "python setup.py install" options generate the same error.

Installing rocksdb/pyrocksdb on a cluster, and do not have sudo/su access, hence the creation of a specific folder (ie. /scratch/software) to install into. Thanks!

$ export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/scratch/software/rocksdb/include
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/scratch/software/rocksdb
$ export LIBRARY_PATH=$LIBRARY_PATH:/scratch/software/rocksdb

$ python setup.py install
missing cimport in module 'pair': /home/anaconda/lib/python2.7/site-packages/Cython/Includes/libcpp/deque.pxd
running install
running bdist_egg
running egg_info
writing requirements to pyrocksdb.egg-info/requires.txt
writing pyrocksdb.egg-info/PKG-INFO
writing top-level names to pyrocksdb.egg-info/top_level.txt
writing dependency_links to pyrocksdb.egg-info/dependency_links.txt
reading manifest file 'pyrocksdb.egg-info/SOURCES.txt'
writing manifest file 'pyrocksdb.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
running build_ext
building 'rocksdb._rocksdb' extension
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/anaconda/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=gnu++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing
cc1plus: warning: command line option \u2018-Wstrict-prototypes\u2019 is valid for C/ObjC but not for C++ [enabled by default]
rocksdb/_rocksdb.cpp:360:27: fatal error: rocksdb/slice.h: No such file or directory
#include "rocksdb/slice.h"
^
compilation terminated.
error: command 'gcc' failed with exit status 1

stephan-hof commented

Hmm

 rocksdb/_rocksdb.cpp:360:27: fatal error: rocksdb/slice.h: No such file or directory

seems like gcc cannot find the rocksdb folder containing the include files. Did you execute this command in the same shell/console as the pip install ?

export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/scratch/software/rocksdb/include

this environment variable should tell gcc where it can search additionally for include files.

dineshbvadhia commented

The export CPLUS_INCLUDE_PATH statement was executed in the same terminal. Tried both "pip install" and "python setup.py install" methods with same result. Will repeat install steps again later.

dineshbvadhia commented

Below is the ouput from the "pip install" method and shows that it cannot find "lsnappy".

The snappy installation ends with:
Libraries have been installed in:
/home/scratch/software/lib
and this path is added to LD_LIBRARY_PATH (see below).

Do you see where I'm going wrong?

$ cd scratch/software/pyrocksdb-0.2.1
$ module load gcc/4.8.1
$ module load cmake/2.8.8

$ export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/home/scratch/software/rocksdb/include
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:home/scratch/software/rocksdb:home/scratch/software/lib
$ export LIBRARY_PATH=$LIBRARY_PATH:/home/scratch/software/rocksdb

$ pip install git+git://github.com/stephan-hof/pyrocksdb.git@v0.2
Downloading/unpacking git+git://github.com/stephan-hof/pyrocksdb.git@v0.2
  Cloning git://github.com/stephan-hof/pyrocksdb.git (to v0.2) to /tmp/pip-Xn3pqU-build
  Running setup.py (path:/tmp/pip-Xn3pqU-build/setup.py) egg_info for package from git+git://github.com/stephan-hof/pyrocksdb.git@v0.2
    missing cimport in module 'pair': /home/anaconda/lib/python2.7/site-packages/Cython/Includes/libcpp/deque.pxd
    Compiling rocksdb/_rocksdb.pyx because it changed.
    Cythonizing rocksdb/_rocksdb.pyx

Requirement already satisfied (use --upgrade to upgrade): setuptools in /home/anaconda/lib/python2.7/site-packages/setuptools-3.6-py2.7.egg (from pyrocksdb==0.2)
Requirement already satisfied (use --upgrade to upgrade): Cython>=0.20 in /home/anaconda/lib/python2.7/site-packages (from pyrocksdb==0.2)
Installing collected packages: pyrocksdb
  Running setup.py install for pyrocksdb
    missing cimport in module 'pair': /home/anaconda/lib/python2.7/site-packages/Cython/Includes/libcpp/deque.pxd

    building 'rocksdb._rocksdb' extension
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/anaconda/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=gnu++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing
    cc1plus: warning: command line option \u2018-Wstrict-prototypes\u2019 is valid for C/ObjC but not for C++ [enabled by default]
    rocksdb/_rocksdb.cpp: In function \u2018PyObject* __pyx_pf_7rocksdb_8_rocksdb_19PyBloomFilterPolicy_6create_filter(__pyx_obj_7rocksdb_8_rocksdb_PyBloomFilterPolicy*, PyObject*)\u2019:
    rocksdb/_rocksdb.cpp:5594:99: warning: conversion to \u2018int\u2019 from \u2018std::vector<rocksdb::Slice>::size_type {aka long unsigned int}\u2019 may alter its value [-Wconversion]
         __pyx_v_self->policy->CreateFilter(py_rocks::vector_data(__pyx_v_c_keys), __pyx_v_c_keys.size(), (&__pyx_v_dst));
                                                                                                       ^
    rocksdb/_rocksdb.cpp: In function \u2018int __Pyx_PyUnicode_Equals(PyObject*, PyObject*, int)\u2019:
    rocksdb/_rocksdb.cpp:160:50: warning: conversion to \u2018int\u2019 from \u2018long unsigned int\u2019 may alter its value [-Wconversion]
       #define __Pyx_PyUnicode_KIND(u)         (sizeof(Py_UNICODE))
                                                      ^
    rocksdb/_rocksdb.cpp:35007:16: note: in expansion of macro \u2018__Pyx_PyUnicode_KIND\u2019
             kind = __Pyx_PyUnicode_KIND(s1);
                    ^
    g++ -pthread -shared build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -L/home/anaconda/lib -lrocksdb -lsnappy -lbz2 -lz -lpython2.7 -o build/lib.linux-x86_64-2.7/rocksdb/_rocksdb.so
    /usr/bin/ld: cannot find -lsnappy
    collect2: error: ld returned 1 exit status
    error: command 'g++' failed with exit status 1
    Complete output from command /home/anaconda/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-Xn3pqU-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-41uRKQ-record/install-record.txt --single-version-externally-managed --compile:
    missing cimport in module 'pair': /home/anaconda/lib/python2.7/site-packages/Cython/Includes/libcpp/deque.pxd

running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/interfaces.py -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/errors.py -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/__init__.py -> build/lib.linux-x86_64-2.7/rocksdb

creating build/lib.linux-x86_64-2.7/rocksdb/tests

copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-2.7/rocksdb/tests

copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-2.7/rocksdb/tests

copying rocksdb/tests/__init__.py -> build/lib.linux-x86_64-2.7/rocksdb/tests

running egg_info

creating pyrocksdb.egg-info

writing requirements to pyrocksdb.egg-info/requires.txt

writing pyrocksdb.egg-info/PKG-INFO

writing top-level names to pyrocksdb.egg-info/top_level.txt

writing dependency_links to pyrocksdb.egg-info/dependency_links.txt

writing manifest file 'pyrocksdb.egg-info/SOURCES.txt'

warning: manifest_maker: standard file '-c' not found



reading manifest file 'pyrocksdb.egg-info/SOURCES.txt'

writing manifest file 'pyrocksdb.egg-info/SOURCES.txt'

copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-2.7/rocksdb

running build_ext

building 'rocksdb._rocksdb' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/rocksdb

gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/anaconda/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=gnu++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing

cc1plus: warning: command line option \u2018-Wstrict-prototypes\u2019 is valid for C/ObjC but not for C++ [enabled by default]

rocksdb/_rocksdb.cpp: In function \u2018PyObject* __pyx_pf_7rocksdb_8_rocksdb_19PyBloomFilterPolicy_6create_filter(__pyx_obj_7rocksdb_8_rocksdb_PyBloomFilterPolicy*, PyObject*)\u2019:

rocksdb/_rocksdb.cpp:5594:99: warning: conversion to \u2018int\u2019 from \u2018std::vector<rocksdb::Slice>::size_type {aka long unsigned int}\u2019 may alter its value [-Wconversion]

     __pyx_v_self->policy->CreateFilter(py_rocks::vector_data(__pyx_v_c_keys), __pyx_v_c_keys.size(), (&__pyx_v_dst));

                                                                                                   ^

rocksdb/_rocksdb.cpp: In function \u2018int __Pyx_PyUnicode_Equals(PyObject*, PyObject*, int)\u2019:

rocksdb/_rocksdb.cpp:160:50: warning: conversion to \u2018int\u2019 from \u2018long unsigned int\u2019 may alter its value [-Wconversion]

   #define __Pyx_PyUnicode_KIND(u)         (sizeof(Py_UNICODE))

                                                  ^

rocksdb/_rocksdb.cpp:35007:16: note: in expansion of macro \u2018__Pyx_PyUnicode_KIND\u2019

         kind = __Pyx_PyUnicode_KIND(s1);

                ^

g++ -pthread -shared build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -L/home/anaconda/lib -lrocksdb -lsnappy -lbz2 -lz -lpython2.7 -o build/lib.linux-x86_64-2.7/rocksdb/_rocksdb.so

/usr/bin/ld: cannot find -lsnappy

collect2: error: ld returned 1 exit status

error: command 'g++' failed with exit status 1

----------------------------------------
Cleaning up...
Command /home/anaconda/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-Xn3pqU-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-41uRKQ-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip-Xn3pqU-build
Traceback (most recent call last):
  File "/home/anaconda/bin/pip", line 6, in <module>
    sys.exit(main())
  File "/home/anaconda/lib/python2.7/site-packages/pip/__init__.py", line 198, in main
    return command.main(cmd_args)
  File "/home/anaconda/lib/python2.7/site-packages/pip/basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 42: ordinal not in range(128)
stephan-hof commented

Hi, unfortunately I have no idea what is wrong with your environment I tested the install procedure with debian/gentoo. There I could install snappy as a normal package (apt, emerge).

Could it be that you just missed a slash before home ?

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:home/scratch/software/rocksdb:home/scratch/software/lib
=> 
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/scratch/software/rocksdb:/home/scratch/software/lib

Try this first and if you have still problems with linking the shared libraries you can try a 'static' link, by changing pyrocksdbs setup.py like this (I guessed the absolute paths to the .a files, adjust/verify them if needed)

--- a/setup.py
+++ b/setup.py
@@ -12,9 +12,11 @@ extension_defaults = {
         '-fno-strict-aliasing'
     ],
     'language': 'c++',
+    'extra_link_args': [
+        '/home/scratch/software/rocksdb/librocksdb.a',
+        '/home/scratch/software/lib/libsnappy.a'
+    ],
     'libraries': [
-        'rocksdb',
-        'snappy',
         'bz2',
         'z'
     ]

For snappy the .a file is build automaticaly (as far as I know) and for rocksdb you have to type

make librocksdb.a

Btw: When you paste next time the console output here it github (which is nice of course, so all information is availalbe), can you enclose this output with two triple backticks ?
https://help.github.com/articles/markdown-basics#multiple-lines
I did it for you last comment, so you can compare the result.

stephan-hof commented

No response since a while so I would consider it as solved.
If not feel free to reopen again.