Bluepy-helper not being built
DavidLP opened this issue · 38 comments
Make is not called when using the PyPi package via pip install bluepy
on raspian jessy.
Do you get any errors? Here it compiles just fine..
No, installing does not give any errors. But as I said bluepy-helper is not being build with the setup.py
hosted in pypi.
When you pip install bluepy
you will have the following files in the site-package/bluepy
folder of your python distribution:
blescan.py bluepy-helper.c bluez-src.tgz btle.py get_services.py __init__.py Makefile __pycache__ sensortag.py uuids.json
You can see already that there is a bluepy-helper.c
file but no object. If I run make within the folder the objects are created and the folder contains:
blescan.py bluepy-helper.c bluez-src.tgz get_services.py Makefile sensortag.py bluepy-helper bluez-5.29 btle.py __init__.py __pycache__ uuids.json
That is a setup script bug.
Ah, good catch! Actually it seems to be fixed in git already:
tpr@maksalaatikko|~/temp
) virtualenv .
Using base prefix '/usr'
New python executable in /home/tpr/temp/bin/python3
Also creating executable in /home/tpr/temp/bin/python
Installing setuptools, pip, wheel...done.
tpr@maksalaatikko|~/temp
) . bin/activate
(temp) tpr@maksalaatikko|~/temp
) pip install git+https://github.com/IanHarvey/bluepy.git
Collecting git+https://github.com/IanHarvey/bluepy.git
Cloning https://github.com/IanHarvey/bluepy.git to /tmp/pip-_mal6z3y-build
Installing collected packages: bluepy
Running setup.py install for bluepy ... done
Successfully installed bluepy-1.0.5
(temp) tpr@maksalaatikko|~/temp
) find .|grep bluepy-helper
./lib/python3.5/site-packages/bluepy/bluepy-helper.c
./lib/python3.5/site-packages/bluepy/bluepy-helper
(temp) tpr@maksalaatikko|~/temp
No, it is not fixed in in git. pip3 in Debian jessie behaves differently for a install from pypi and git url.
For pip3 -v install bluepy
it runs setup.py bdist_wheel which runs 'build' before 'install'. Compiling bluepy-helper works as expected but it is not copied to the build directory and thus not installed.
See PR #162
progress please?
I am also seeing this on X64 arch under Ubuntu 16.04.2. For the 1.1.0 release the helper is not compiled. When downgrading to 1.0.5, same thing except the ARM binary for the helper was pre-packaged as described in #127.
Note, going the non-pip route and compiling from source worked fine.
Same problem. I had to compile by hand in order to make it works (bluepy.Helper was missing, on my debian, module installed with pip3, for python3.5)
I'm having the same problem here on Debian jesse. I'm not a programmer so I'm looking for help with some details on how fix this. How do i compile bluepy-helper.c without breaking anything.
Thanks
Hi guys,
You're going to have to give me some clues here. Please post a transcript of what you tried, and what the resulting messages were. If pip install
or pip3 install
methods don't work, can you try cloning the repo and doing python setup.py install
.
The helper itself can be built by going into the bluepy/bluepy
directory and just running make
.
I really can't help if I don't know which stage is failing. It works for me...
Cheers
Ian
It seems that pip builds a different setup.py target by default:
# pip3 install bluepy
Collecting bluepy
Downloading bluepy-1.1.1.tar.gz (210kB)
100% |████████████████████████████████| 215kB 2.0MB/s
Building wheels for collected packages: bluepy
Running setup.py bdist_wheel for bluepy ... done
Stored in directory: /root/.cache/pip/wheels/cc/d9/f8/c207bfb612ae77ac7d9b741db6c69819641a712486c03293b1
Successfully built bluepy
Installing collected packages: bluepy
Successfully installed bluepy-1.1.1
This does not work, the bluepy-helper is not getting installed.
However, if you disable the wheel handling like so:
# pip3 install --no-use-wheel bluepy
DEPRECATION: --no-use-wheel is deprecated and will be removed in the future. Please use --no-binary :all: instead.
Collecting bluepy
Downloading bluepy-1.1.1.tar.gz (210kB)
100% |████████████████████████████████| 215kB 2.2MB/s
Skipping bdist_wheel for bluepy, due to binaries being disabled for it.
Installing collected packages: bluepy
Running setup.py install for bluepy ... done
Successfully installed bluepy-1.1.1
then the bluepy helper is getting built and installed.
Note that setup.py gets passed different targets: in the non-working case it uses bdist_wheel, while in the working case "install" is used as the target.
So I'm still struggling to reproduce this. What platform is this on, please?
And what's the output from pip3 --version
?
Thanks
Ian
This is on debian unstable, the version is
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.5)
Thanks for looking into this.
As a temporary workaround, I've changed the installation instructions to use pip install --no-binary :all: bluepy
, which seems to force it to rebuild the helper on installation.
I'm not sure whether it's possible to build wheels which have multiple architectures (i.e. various ARMs and x86's) in them; the 'net seems rather quiet on the topic.
Hi Ian,
I've seen @bennuttall talk on the topic of piwheels and he says "wheels for packages implemented in C are architecture-specific".
There is more information at:
http://bennuttall.com/piwheels-building-a-faster-python-package-repository-for-raspberry-pi-users/
Which references:
https://www.python.org/dev/peps/pep-0425/
Hope that helps
It seems bluepy is a platform-agnostic wheel (the any
tag in bluepy-1.1.1-py2-none-any.whl
), so that doesn't make any difference. However, the lack of a Python 3 wheel means Python 3 users won't get the wheel and will have to build from source. You can upload a Python 3 wheel in addition.
Regarding piwheels, we built Python 3 wheels on a Raspberry Pi 3, which you can find here: https://www.piwheels.hostedpi.com/simple/bluepy/
Note that packages which require compilation get tagged with armv7l
(e.g. numpy)), and those that don't get tagged with any
.
In the setup.py, a compile of the C module is run so I suspect that any
is not correct value and should in fact contain the output of:
import distutils.util
distutils.util.get_platform()
As Bluepy is built on BlueZ it should indicate Linux as a minimum.
My current theory is that, because I'm not actually building a .so
extension module (I'm just happening to run make
), the system isn't figuring out that it's platform-dependent. I tried building a wheel on ARM and pushing it to PyPI, but then found that pip install
on an x86 box ended up installing the ARM executable, which it couldn't then run.
I'm guessing there's a magic config item I can override here.
I've now made a 1.1.2 release, and build some 2.7 and 3.4 wheels for it on my Pi.
It's possible I made earlier releases with an old version of setuptools
- the hack in 6af5c30 didn't appear to work until I updated them.
Anyway, if anyone reading could confirm or deny that 1.1.2 installs for them with modern pip
, on any platform, that would be useful.
Thanks
Ian
Interesting. What method did you use to upload arm wheels to pypi?
It's also been picked up by piwheels: https://www.piwheels.hostedpi.com/simple/bluepy/
Note that the wheels are for Python 2.7 and 3.4 only, so this will not work on Python 3 in Raspbian Stretch (as it's Python 3.5), or other Python 3 versions.
Are you going to upload a manylinux wheel too?
# rm -r .cache/pip/
# pip3 install bluepy
Collecting bluepy
Downloading bluepy-1.1.2.tar.gz (497kB)
100% |████████████████████████████████| 501kB 1.6MB/s
Building wheels for collected packages: bluepy
Running setup.py bdist_wheel for bluepy ... done
Stored in directory: /root/.cache/pip/wheels/0b/c0/8a/af9c1567bee8b7ef3ce1136bb917f75e45e2d3b89f35328111
Successfully built bluepy
Installing collected packages: bluepy
Successfully installed bluepy-1.1.2
# python3
Python 3.5.4 (default, Sep 5 2017, 18:32:10)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from bluepy.btle import Peripheral
>>> p = Peripheral()
>>> p._startHelper()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/dist-packages/bluepy/btle.py", line 246, in _startHelper
universal_newlines=True)
File "/usr/lib/python3.5/subprocess.py", line 676, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.5/subprocess.py", line 1289, in _execute_child
raise child_exception_type(errno_num, err_msg)
OSError: [Errno 8] Exec format error
# rm -r .cache/pip/
# pip3 --no-cache-dir install bluepy
Collecting bluepy
Downloading bluepy-1.1.2.tar.gz (497kB)
100% |████████████████████████████████| 501kB 5.3MB/s
Installing collected packages: bluepy
Running setup.py install for bluepy ... done
Successfully installed bluepy-1.1.2
# python3
Python 3.5.4 (default, Sep 5 2017, 18:32:10)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from bluepy.btle import Peripheral
>>> p = Peripheral()
>>> p._startHelper()
I don't quite understand what cache this refers to - since I've cleared the local pip cache in both cases... (Maybe it tries to build a local wheel in the 1st case, and building a wheel works differently from installing?)
But yeah, some improvement I'd say...
@bennuttall - I uploaded with twine
(1.9.1). setuptools
was upgraded to 36.4.0, and wheel
is 0.30.0. I'd like to get the not-building-from-source problems sorted before I start making manylinux wheels.
@simon-budig - hmm. It's possibly not picking up the wheel from PyPI because I built it for Python 3.4 (bluepy-1.1.2-cp34-cp34m-linux_armv7l.whl
) and you have 3.5. I don't know why the cache should make a difference, though, there shouldn't be any 1.1.2 wheels out there with the "pure python" flag set. Are you on ARM or x86, by the way?
This is all making a pure-Python rewrite look much more appealing...
This is on x86_64
I uploaded with twine
@IanHarvey but where did you upload it to? I believe PyPI doesn't allow uploading ARM wheels but warehouse (next-gen PyPI) does. See pypi/warehouse#2003
On my raspbian (stretch, latest version)
apt install python3-pip libglib2.0-dev
pip3 install bluepy
then the same test as @simon-budig works perfectly well
# python3
Python 3.5.4 (default, Sep 5 2017, 18:32:10)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from bluepy.btle import Peripheral
>>> p = Peripheral()
>>> p._startHelper()
Version 1.1.2 doesn't work for me yet:
$ sudo pip3 install --verbose bluepy
Converted retries value: Retry(total=5, connect=None, read=None, redirect=None) -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=None, read=None, redirect=None)
Converted retries value: Retry(total=5, connect=None, read=None, redirect=None) -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=None, read=None, redirect=None)
Collecting bluepy
1 location(s) to search for versions of bluepy:
* https://pypi.python.org/simple/bluepy/
Getting page https://pypi.python.org/simple/bluepy/
Looking up "https://pypi.python.org/simple/bluepy/" in the cache
Current age based on date: 236
Freshness lifetime from max-age: 600
Freshness lifetime from request max-age: 600
The response is "fresh", returning cached response
600 > 236
Analyzing links from page https://pypi.python.org/simple/bluepy/
Found link https://pypi.python.org/packages/13/4f/b2f99f8c5b14ea1d9b2ff311fe0d232affc5a60a2c204825daa51703a358/bluepy-1.1.1.tar.gz#md5=037215b3c60e2297b3037c9f6cabafe8 (from https://pypi.python.org/simple/bluepy/), version: 1.1.1
Found link https://pypi.python.org/packages/2f/5e/b0f10966bf23b98029164d034d3fc474f1f957885e660e48d61fa462a292/bluepy-1.0.5.tar.gz#md5=34c749540cf7b889c1c3d1b040afcdc6 (from https://pypi.python.org/simple/bluepy/), version: 1.0.5
Skipping link https://pypi.python.org/packages/33/bd/b4ee0fb003917e21b377ec7af53bfad977fcf89f8700868368b000fde13e/bluepy-1.1.2-cp27-cp27mu-linux_armv7l.whl#md5=68b3847da7d2212b1267a0f6ee13bd56 (from https://pypi.python.org/simple/bluepy/); it is not compatible with this Python
Skipping link https://pypi.python.org/packages/75/59/e26c0ce5d184e6172e323c1db882bc5b5e28ed20e34d08f2be90341f75b7/bluepy-1.1.2-cp34-cp34m-linux_armv7l.whl#md5=834c23d93af7554ff2c00bb4add29e52 (from https://pypi.python.org/simple/bluepy/); it is not compatible with this Python
Found link https://pypi.python.org/packages/a2/23/38f57b2f84f99d9aa56efd20fbd0189ceb202f7a48028f634644b8524356/bluepy-1.0.3.tar.gz#md5=59abc628b1d851e18db8d11dfd20d9d2 (from https://pypi.python.org/simple/bluepy/), version: 1.0.3
Found link https://pypi.python.org/packages/b7/6f/99900cbffd7714e75e433ad1ea594f6d0f9cdf9c8bbaffdfef3c5cbf8500/bluepy-1.0.4.tar.gz#md5=6c089943b2bd23c1e12262b28a872c71 (from https://pypi.python.org/simple/bluepy/), version: 1.0.4
Found link https://pypi.python.org/packages/ea/45/911b293581760e04212fa1c3d4412ded8faa4b6c4e8e39acc4e1f18ae286/bluepy-1.0.2.tar.gz#md5=be53d5848d697ede54dfcca271313695 (from https://pypi.python.org/simple/bluepy/), version: 1.0.2
Found link https://pypi.python.org/packages/f1/fa/4830ec2ab95baa0201151bf18d5c031d94c520bd05ffdf692573a5261fe3/bluepy-1.1.2.tar.gz#md5=65ef76a2b8538be6fb909e668e9354af (from https://pypi.python.org/simple/bluepy/), version: 1.1.2
Found link https://pypi.python.org/packages/fe/1c/73f960e8a1e378e390139d0a81f6d8388f70b7eea79bb711910841911d0e/bluepy-1.1.0.tar.gz#md5=c72f0adde853149f6c60b6106ae3a4f5 (from https://pypi.python.org/simple/bluepy/), version: 1.1.0
Using version 1.1.2 (newest of versions: 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2)
Using cached wheel link: file:///root/.cache/pip/wheels/0b/c0/8a/af9c1567bee8b7ef3ce1136bb917f75e45e2d3b89f35328111/bluepy-1.1.2-cp35-cp35m-linux_x86_64.whl
Installing collected packages: bluepy
changing mode of /usr/local/bin/blescan to 755
changing mode of /usr/local/bin/thingy52 to 755
changing mode of /usr/local/bin/sensortag to 755
Successfully installed bluepy-1.1.2
Cleaning up...
$ python3
Python 3.5.3 (default, Jan 19 2017, 14:11:04)
[GCC 6.3.0 20170118] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from bluepy.btle import Scanner
>>> Scanner().scan(10)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/dist-packages/bluepy/btle.py", line 679, in scan
self.start(passive=passive)
File "/usr/local/lib/python3.5/dist-packages/bluepy/btle.py", line 616, in start
self._startHelper(iface=self.iface)
File "/usr/local/lib/python3.5/dist-packages/bluepy/btle.py", line 246, in _startHelper
universal_newlines=True)
File "/usr/lib/python3.5/subprocess.py", line 676, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.5/subprocess.py", line 1282, in _execute_child
raise child_exception_type(errno_num, err_msg)
OSError: [Errno 8] Exec format error
>>>
$ file /usr/local/lib/python3.5/dist-packages/bluepy/bluepy-helper
/usr/local/lib/python3.5/dist-packages/bluepy/bluepy-helper: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=9efb6c5c9af3ae8e284ffbc77bf3820a355d7d97, not stripped
As this is an x86_64 machine (a regular laptop with Bluetooth), something goes very wrong as it installs an ARM executable.
OK. Let's try again.
There's now a version 1.1.4 up on PyPI (https://pypi.python.org/pypi?:action=display&name=bluepy&version=1.1.4) which has a revised setup.py
, which invokes make
in the build_py
phase instead of install
. This works for me for both setup.py install
installation and pip install
, on two flavours of ARM.
Please note, this is an sdist
-only version for now - please don't ask for a wheel upload as I simply don't have the time to debug it.
Thanks
Ian
Works on my Pi Zero W (aka the other flavour of ARM). sdist
is completely fine.
On to figuring out notifications.
piwheels isn't building at the moment, but when it kicks off again we'll pick this up and build for Pi (ARMv6 and ARMv7).
Hello guys, I have been desperately trying to install python-eq3bt on onion omega 2+ which has openWRT system. This package has bluepy as its dependency. But i've been unsuccessful at my attempts to install bluepy.
Upon research i found there is no 'make' on openWRT. And for that there is lengthy process. But I don't have that much of knowledge.
So it would be very, very helpful, if someone could help me out in this. All i want is to have an interface with the EQ3 bluetooth devices. Any help would be highly appreciated.
This package has bluepy as its dependency. But i've been unsuccessful at my attempts to install bluepy.
Here's an OpenWrt package:
https://pb.nanl.de/show.php?id=5166&hash=52065413
https://pb.nanl.de/show.php?id=5167&hash=65423715
https://pb.nanl.de/show.php?id=5168&hash=06542471
piwheels now has the latest bluepy wheels for Armv6/v7 and for py34 (jessie) and py35 (stretch): https://www.piwheels.org/simple/bluepy/
bluepy-1.1.4-cp34-cp34m-linux_armv7l.whl
bluepy-1.1.4-cp34-cp34m-linux_armv6l.whl
bluepy-1.1.4-cp35-cp35m-linux_armv7l.whl
bluepy-1.1.4-cp35-cp35m-linux_armv6l.whl
Here I'm trying to install bluepy and having the same problem.
Environment:
- Debian 9.4
- Python 3.6.3
- PIP 10.0.1
- CPU: aarch64
@lupa18 what hardware?