Build failure on raspberry pi running arch
mike-lawrence opened this issue · 6 comments
Log:
[user@scene_pi python-blosc]$ sudo python setup.py install
running install
running bdist_egg
running egg_info
writing dependency_links to blosc.egg-info/dependency_links.txt
writing top-level names to blosc.egg-info/top_level.txt
writing blosc.egg-info/PKG-INFO
reading manifest file 'blosc.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.txt'
warning: no files found matching '*.cpp' under directory 'c-blosc'
warning: no files found matching '*.hpp' under directory 'c-blosc'
writing manifest file 'blosc.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-armv7l/egg
running install_lib
running build_py
creating build
creating build/lib.linux-armv7l-3.4
creating build/lib.linux-armv7l-3.4/blosc
copying blosc/test.py -> build/lib.linux-armv7l-3.4/blosc
copying blosc/__init__.py -> build/lib.linux-armv7l-3.4/blosc
copying blosc/version.py -> build/lib.linux-armv7l-3.4/blosc
copying blosc/toplevel.py -> build/lib.linux-armv7l-3.4/blosc
running build_ext
building 'blosc.blosc_extension' extension
creating build/temp.linux-armv7l-3.4
creating build/temp.linux-armv7l-3.4/blosc
creating build/temp.linux-armv7l-3.4/c-blosc
creating build/temp.linux-armv7l-3.4/c-blosc/blosc
creating build/temp.linux-armv7l-3.4/c-blosc/internal-complibs
creating build/temp.linux-armv7l-3.4/c-blosc/internal-complibs/lz4-1.7.0
creating build/temp.linux-armv7l-3.4/c-blosc/internal-complibs/snappy-1.1.1
creating build/temp.linux-armv7l-3.4/c-blosc/internal-complibs/zlib-1.2.8
gcc -pthread -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -fPIC -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -Ic-blosc/blosc -Ic-blosc/internal-complibs/lz4-1.7.0 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/zlib-1.2.8 -I/usr/include/python3.4m -c blosc/blosc_extension.c -o build/temp.linux-armv7l-3.4/blosc/blosc_extension.o
gcc -pthread -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -fPIC -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -Ic-blosc/blosc -Ic-blosc/internal-complibs/lz4-1.7.0 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/zlib-1.2.8 -I/usr/include/python3.4m -c c-blosc/blosc/blosc.c -o build/temp.linux-armv7l-3.4/c-blosc/blosc/blosc.o
c-blosc/blosc/blosc.c: In function 'blosc_getitem':
c-blosc/blosc/blosc.c:1275:7: warning: unused variable 'tmp_init' [-Wunused-variable]
int tmp_init = 0;
^
gcc -pthread -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -fPIC -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -Ic-blosc/blosc -Ic-blosc/internal-complibs/lz4-1.7.0 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/zlib-1.2.8 -I/usr/include/python3.4m -c c-blosc/blosc/shuffle-sse2.c -o build/temp.linux-armv7l-3.4/c-blosc/blosc/shuffle-sse2.o
c-blosc/blosc/shuffle-sse2.c:14:4: error: #error SSE2 is not supported by the target architecture/platform and/or this compiler.
#error SSE2 is not supported by the target architecture/platform and/or this compiler.
^
c-blosc/blosc/shuffle-sse2.c:17:23: fatal error: emmintrin.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
Hi,
Yes, python-blosc is being developed in Intel and normally only checked with it. To overcome this problem we need to recognize the architecture prior to call the C compiler: this would allow to set compiler options accordingly. PR welcome!
Ah, I thought that because c-blosc is supposedly compatible with arm/RPi (at least, according to the cblosc 1.3.6 to 1.4.0 change notes), that python-blosc would be as well.
Any news on this? Perhaps a new version has fixed the issue? Or maybe it magically disappeared?
I had a surprising amount of luck with blosc on a recent ARM core. Eagerly looking forward to c-blosc2's NEON support, but figured I'd chime in and suggest that there no fundamental issues with c-blosc or python-blosc on ARM.
Some stats and comparisons below.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
python-blosc version: 1.4.4
Blosc version: 1.11.2 ($Date:: 2017-01-27 #$)
Compressors available: ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']
Compressor library versions:
BloscLZ: 1.0.5
LZ4: 1.7.5
Snappy: 1.1.1
Zlib: 1.2.8
Zstd: 1.1.2
Python version: 3.6.0 (default, Dec 31 2016, 21:20:16)
[GCC 4.9.2]
Platform: Linux-3.4.113-sun8i-armv7l (#50 SMP PREEMPT Mon Nov 14 08:41:55 CET 2016)
Linux dist: debian 9.0
Processor: not recognized
Byte-ordering: little
Detected cores: 4
Number of threads to use by default: 4
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
*** ctypes.memmove() *** Time for memcpy(): 0.015 s (93.57 MB/s)
Times for compressing/decompressing with clevel=5 and 4 threads
*** user input ***
*** blosclz , noshuffle *** 0.015 s (89.93 MB/s) / 0.010 s (138.32 MB/s) Compr. ratio: 2.7x
*** blosclz , shuffle *** 0.023 s (60.25 MB/s) / 0.012 s (112.71 MB/s) Compr. ratio: 2.3x
*** blosclz , bitshuffle *** 0.018 s (77.63 MB/s) / 0.021 s (66.76 MB/s) Compr. ratio: 7.3x
*** lz4 , noshuffle *** 0.008 s (177.14 MB/s) / 0.009 s (159.00 MB/s) Compr. ratio: 3.6x
*** lz4 , shuffle *** 0.010 s (131.29 MB/s) / 0.012 s (117.69 MB/s) Compr. ratio: 3.5x
*** lz4 , bitshuffle *** 0.015 s (89.97 MB/s) / 0.022 s (63.62 MB/s) Compr. ratio: 8.4x
*** lz4hc , noshuffle *** 0.071 s (19.30 MB/s) / 0.007 s (186.64 MB/s) Compr. ratio: 8.6x
*** lz4hc , shuffle *** 0.079 s (17.30 MB/s) / 0.014 s (95.99 MB/s) Compr. ratio: 6.2x
*** lz4hc , bitshuffle *** 0.062 s (22.23 MB/s) / 0.027 s (51.53 MB/s) Compr. ratio: 9.7x
*** snappy , noshuffle *** 0.008 s (173.87 MB/s) / 0.009 s (148.77 MB/s) Compr. ratio: 4.4x
*** snappy , shuffle *** 0.011 s (123.22 MB/s) / 0.016 s (85.16 MB/s) Compr. ratio: 4.4x
*** snappy , bitshuffle *** 0.015 s (89.02 MB/s) / 0.021 s (64.87 MB/s) Compr. ratio: 6.2x
*** zlib , noshuffle *** 0.047 s (29.26 MB/s) / 0.011 s (121.83 MB/s) Compr. ratio: 14.7x
*** zlib , shuffle *** 0.080 s (17.20 MB/s) / 0.022 s (63.61 MB/s) Compr. ratio: 9.4x
*** zlib , bitshuffle *** 0.059 s (23.50 MB/s) / 0.033 s (41.10 MB/s) Compr. ratio: 10.5x
*** zstd , noshuffle *** 0.113 s (12.21 MB/s) / 0.011 s (124.64 MB/s) Compr. ratio: 15.6x
*** zstd , shuffle *** 0.154 s (8.92 MB/s) / 0.026 s (52.56 MB/s) Compr. ratio: 9.9x
*** zstd , bitshuffle *** 0.116 s (11.86 MB/s) / 0.036 s (38.40 MB/s) Compr. ratio: 11.4x
Script is a slightly modified version of compress_ptr.py -
https://gist.github.com/itdaniher/544215e2ebd25495a95d9165633972c6
Tried a few different formats for my sampled analog signal compression - pysoundfile's ogg and flac encodings, python-lz4, and python's integrated zlib. Very happy with Blosc overall - stellar performance even without native SIMD (yet!)
@FrancescAlted I'd be glad to order your choice of single board computer to better enable you to test on ARM CPUs. Also have plenty of recommendations, the ability to offer native hosted ARM continuous integration, etc. Contact me @gmail.com if you're interested! Thanks for Blosc-and-friends!
@itdaniher Thanks for reporting. As you had no fundamental flaws with C-Blosc in ARM, I am going to close this one. I already contacted you about the ARM board and CI for ARM. Thanks again.