Expected gatttool formatting now out of date, breaking BLEHeartRateLogger?
Closed this issue · 9 comments
So back in November 2014 or so, I got a Zephyr HxM Smart Heart Rate Monitor so I could monitor heart-rate variability and effects of various substances. BLEHRL was the simplest application I could find which would reliably log the data to an SQL DB, so I set up BLEHRL and used it for a while without a problem, and then events caused me to set the belt aside for a while and then my old Debian Dell laptop died. I got a new Acer laptop, and to get GPU/deep learning support, I installed Ubuntu 14.04.3 LTS; unfortunately this did not support the WiFi+BT card and so I bought a SoundBot® SB340 Bluetooth 4.0 USB Adapter to enable BT until such time as I am able to safely upgrade Ubuntu to a kernel that does support BT. This also works fine - for example, I can use a pair of Bluetooth headphones. So the full Bluetooth stack on my install & laptop is working.
When I went back to use my heart-rate belt with BLEHRL HEAD, it failed. I would always get:
$ python BLEHeartRateLogger.py
2016-02-18 16:56:13,102 Trying to find a BLE device
2016-02-18 16:56:14,907 Establishing connection to 78:A5:04:81:B2:5D
2016-02-18 16:56:45,012 Establishing connection to 78:A5:04:81:B2:5D
...
After double-checking the Bluetooth was working and following the debugging instructions, what I see is that the heart-rate belt is connected and is working and even sending what I think is heart-rate data:
$ hcitool lescan
LE Scan ...
78:A5:04:81:B2:5D Zephyr HXM200008002
78:A5:04:81:B2:5D (unknown)
$ gatttool -b 78:A5:04:81:B2:5D --interactive
[ ][78:A5:04:81:B2:5D][LE]> connect
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4f f0 02 fc 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4f 00 03
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4f f0 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 50 d8 02 d8 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 50 dc 02
[CON][78:A5:04:81:B2:5D][LE]> characteristics
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 50 cc 02 d4 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 51 dc 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 50 f8 02
[CON][78:A5:04:81:B2:5D][LE]>
handle: 0x0002, char properties: 0x02, char value handle: 0x0003, uuid: 00002a00-0000-1000-8000-00805f9b34fb
handle: 0x0004, char properties: 0x02, char value handle: 0x0005, uuid: 00002a01-0000-1000-8000-00805f9b34fb
handle: 0x0006, char properties: 0x0a, char value handle: 0x0007, uuid: 00002a02-0000-1000-8000-00805f9b34fb
handle: 0x0008, char properties: 0x0a, char value handle: 0x0009, uuid: 00002a03-0000-1000-8000-00805f9b34fb
handle: 0x000a, char properties: 0x02, char value handle: 0x000b, uuid: 00002a04-0000-1000-8000-00805f9b34fb
handle: 0x000d, char properties: 0x20, char value handle: 0x000e, uuid: 00002a05-0000-1000-8000-00805f9b34fb
handle: 0x0011, char properties: 0x10, char value handle: 0x0012, uuid: 00002a37-0000-1000-8000-00805f9b34fb
handle: 0x0014, char properties: 0x02, char value handle: 0x0015, uuid: 00002a38-0000-1000-8000-00805f9b34fb
handle: 0x0016, char properties: 0x08, char value handle: 0x0017, uuid: 00002a39-0000-1000-8000-00805f9b34fb
handle: 0x0019, char properties: 0x02, char value handle: 0x001a, uuid: 00002a23-0000-1000-8000-00805f9b34fb
handle: 0x001b, char properties: 0x02, char value handle: 0x001c, uuid: 00002a24-0000-1000-8000-00805f9b34fb
handle: 0x001d, char properties: 0x02, char value handle: 0x001e, uuid: 00002a25-0000-1000-8000-00805f9b34fb
handle: 0x001f, char properties: 0x02, char value handle: 0x0020, uuid: 00002a26-0000-1000-8000-00805f9b34fb
handle: 0x0021, char properties: 0x02, char value handle: 0x0022, uuid: 00002a27-0000-1000-8000-00805f9b34fb
handle: 0x0023, char properties: 0x02, char value handle: 0x0024, uuid: 00002a28-0000-1000-8000-00805f9b34fb
handle: 0x0025, char properties: 0x02, char value handle: 0x0026, uuid: 00002a29-0000-1000-8000-00805f9b34fb
handle: 0x0027, char properties: 0x02, char value handle: 0x0028, uuid: 00002a2a-0000-1000-8000-00805f9b34fb
handle: 0x0029, char properties: 0x02, char value handle: 0x002a, uuid: 00002a50-0000-1000-8000-00805f9b34fb
handle: 0x002c, char properties: 0x12, char value handle: 0x002d, uuid: 00002a19-0000-1000-8000-00805f9b34fb
handle: 0x0030, char properties: 0x10, char value handle: 0x0031, uuid: befdff11-c979-11e1-9b21-0800200c9a66
handle: 0x0033, char properties: 0x10, char value handle: 0x0034, uuid: befdff12-c979-11e1-9b21-0800200c9a66
handle: 0x0037, char properties: 0x10, char value handle: 0x0038, uuid: befdffb1-c979-11e1-9b21-0800200c9a66
handle: 0x003a, char properties: 0x08, char value handle: 0x003b, uuid: befdffb2-c979-11e1-9b21-0800200c9a66
[CON][78:A5:04:81:B2:5D][LE]>
[CON][78:A5:04:81:B2:5D][LE]> exit
][78:A5:04:81:B2:5D][LE]>
But BLEHRL itself is refusing to work. I noticed that this gatttool output doesn't look exactly the same as I remembered it; I don't think it had the [CON] or [LE] parts, for example. I looked a bit through BLEHeartRateLogger.py and while I'm not a Python programmer, it looks like in the 'expect'-related calls, the program is hardwiring the formatting it expects/parses the gatttool data into. Could BLEHRL now be broken by upstream changes to how gatttool prints stuff?
As you pointed in your issue, the format of your version of gatttool isn't handled properly by BLEHRL. It seems like the version of bluez (which contains gatttool) shipped with Ubuntu 14.04 is actually an old one (bluez 4.101) compared to the one I'm using (bluez 5.36).
I pushed a change which should address your issue and handle both formats. Let me know if it is working for you now.
Thanks but looks like it's not quite there:
$ python BLEHeartRateLogger.py
2016-02-19 12:22:37,413 Trying to find a BLE device
2016-02-19 12:22:38,840 Establishing connection to 78:A5:04:81:B2:5D
2016-02-19 12:22:38,966 Connected to 78:A5:04:81:B2:5D
2016-02-19 12:23:09,067 Couldn't find the heart rate measurement handle?!
$
Looks like no:
$ python BLEHeartRateLogger.py -d
2016-02-28 12:35:16,350 Trying to find a BLE device
2016-02-28 12:35:18,123 Establishing connection to 78:A5:04:81:B2:5D
[ ][78:A5:04:81:B2:5D][LE]> connect
connect
[CON][78:A5:04:81:B2:5D][LE]> 2016-02-28 12:35:18,745 Connected to 78:A5:04:81:B2:5D
char-desc
char-desc
[ ][78:A5:04:81:B2:5D][LE]> 2016-02-28 12:35:23,851 Couldn't find the heart rate measurement handle?!
$ gatttool -b 78:A5:04:81:B2:5D --interactive
[ ][78:A5:04:81:B2:5D][LE]> connect
[CON][78:A5:04:81:B2:5D][LE]> char-desc
[CON][78:A5:04:81:B2:5D][LE]>
handle: 0x0001, uuid: 2800
handle: 0x0002, uuid: 2803
handle: 0x0003, uuid: 2a00
handle: 0x0004, uuid: 2803
handle: 0x0005, uuid: 2a01
[CON][78:A5:04:81:B2:5D][LE]> exit
I notice the 'char-desc' and also 'connect' and 'characteristics' seem to take a few seconds to return results, while BLEHeartRateLogger.py exits quickly. Could BLEHRL being impatient and not waiting long enough for results?
BLEHRL might indeed be impatient. Could you try modifying the value of timeout on line 248 in BLEHeartRateLogger.py?
If this is successful, I'll add a new parameter to the script in order to be able to set other values.
I tried changing it to 20/30/60, didn't help. Output still looks like this:
$ python BLEHeartRateLogger.py -d
2016-02-29 12:43:54,585 Trying to find a BLE device
2016-02-29 12:43:56,847 Establishing connection to 78:A5:04:81:B2:5D
[ ][78:A5:04:81:B2:5D][LE]> connect
connect
[CON][78:A5:04:81:B2:5D][LE]> 2016-02-29 12:43:57,474 Connected to 78:A5:04:81:B2:5D
char-desc
char-desc
[CON][78:A5:04:81:B2:5D][LE]>
handle: 0x0001, uuid: 2800
handle: 0x0002, uuid: 2803
handle: 0x0003, uuid: 2a00
handle: 0x0004, uuid: 2803
handle: 0x0005, uuid: 2a01
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 53 8c 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 55 7c 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 55 88 02 a4 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 56 bc 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 55 e0 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 54 00 03 18 03
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 53 ec 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 53 f8 02 e4 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 53 d0 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 53 c0 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 53 bc 02 b8 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 54 b0 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 54 e0 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 53 20 03 8c 03
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 51 4c 03
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 50 58 03
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4f 40 03
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4f f4 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4f ec 02 f4 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4f e4 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 50 c8 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 50 e0 02 00 03
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4f 0c 03
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4f f8 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4f 14 03 2c 03
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4e 04 03
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4e 04 03
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 4f ec 02 c4 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 50 c8 02
[CON][78:A5:04:81:B2:5D][LE]>
Notification handle = 0x0012 value: 16 50 cc 02
[CON][78:A5:04:81:B2:5D][LE]> 2016-02-29 12:44:27,575 Couldn't find the heart rate measurement handle?!
I pushed a new change which should enable you to manually set the HR handle. Please try again with the same command as you did earlier as it now sends characteristics in place of char-desc.
Otherwise, you can also force the script to skip this discovery phase:
$ python BLEHeartRateLogger.py -H 0x0012
Yes, that seems to be working!
$ BLEHeartRateLogger.py -v -o /home/gwern/selfexperiment/hrv.sql
2016-02-29 14:50:48,070 Trying to find a BLE device
2016-02-29 14:50:49,964 Establishing connection to 78:A5:04:81:B2:5D
2016-02-29 14:50:50,579 Connected to 78:A5:04:81:B2:5D
2016-02-29 14:51:05,682 {'rr': [776], 'rr_interval': True, 'hr': 81, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
2016-02-29 14:51:05,684 {'rr': [740, 760], 'rr_interval': True, 'hr': 81, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
2016-02-29 14:51:05,684 {'rr': [740], 'rr_interval': True, 'hr': 81, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
2016-02-29 14:51:05,685 {'rr': [704, 688], 'rr_interval': True, 'hr': 82, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
2016-02-29 14:51:05,685 {'rr': [700], 'rr_interval': True, 'hr': 82, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
2016-02-29 14:51:05,685 {'rr': [728], 'rr_interval': True, 'hr': 82, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
2016-02-29 14:51:05,697 sqlite commit time: 0.0125398635864
2016-02-29 14:51:05,698 {'rr': [688, 716], 'rr_interval': True, 'hr': 83, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
...
Thank you for your help figuring out this bug.