Работа термометров в Home Assistant c Bluetooth LE Long Range.
pvvx opened this issue · 89 comments
Предварительное описание решения.
Для примера можно взять:
Low-cost Realtek RTL8761 BT adapter
USB ID 0bda:8771 Realtek Semiconductor Corp. Bluetooth Radio
Сurrent version of Linux fw on RTL8761 supports BT5.1:
# hciconfig hci1 version
hci1: Type: Primary Bus: USB
BD Address: 8C:88:2B:00:E1:EA ACL MTU: 1021:6 SCO MTU: 255:12
HCI Version: 5.1 (0xa) Revision: 0x999
LMP Version: 5.1 (0xa) Subversion: 0x646b
Manufacturer: Realtek Semiconductor Corporation (93)
Для работы HA c LE Long Range в интеграции “Bluetooth” желательно установить последнюю версию bluez и добавить в файле /lib/systemd/system/bluetooth.service
ExecStart=/usr/local/libexec/bluetooth/bluetoothd --experimental
Из-за неверной инициализации адаптеров BT5.+ в Bluez, перед запуском Home Assistant необходимо переключить BT адаптер на работу в режим Bluetoch 5.0:
hcitool -i hci1 cmd 08 31 03 05 05
или во всех режимах PHY 1M/2M/Coded:
hcitool -i hci1 cmd 08 31 03 07 07
Номер hciN - свой.
Проверить переключение возможно в btmon (log btmon):
@ RAW Open: hcitool (privileged) version 2.22
< HCI Command: LE Set Default PHY (0x08|0x0031) plen 3
All PHYs preference: 0x03
No TX PHY preference
No RX PHY preference
TX PHYs preference: 0x07
LE 1M
LE 2M
LE Coded
RX PHYs preference: 0x07
LE 1M
LE 2M
LE Coded
> HCI Event: Command Complete (0x0e) plen 4
LE Set Default PHY (0x08|0x0031) ncmd 2
Status: Success (0x00)
В итоге, в интеграции "BTHome" получаем успешный прием термометров со стандартной рекламой и работающих в LE Long Range:
Если адаптер будет работать неустойчиво, тогда HA может его переинициализировать и настройка на работу в Coded PHY (Long Range) собьется. Ожидать официальной поддержки в интеграции “Bluetooth” нет никакой возможности. Выбранный автором API “Bleak” не позволяет управлять PHY адаптера и не предвидится никакой поддержки BLUETOOTH SPECIFICATION Version 5.0 от 2016 года.
BLUETOOTH SPECIFICATION Version 5.0 | Vol 1, Part C page 291
Core Specification Change History
9 CHANGES FROM v4.2 TO 5.0
9.1 NEW FEATURES
Several new features are introduced in the Bluetooth Core Specification 5.0
Release. The major areas of improvement are:
• Slot Availability Mask (SAM)
• 2 Msym/s PHY for LE
• LE Long Range
• High Duty Cycle Non-Connectable Advertising
• LE Advertising Extensions
• LE Channel Selection Algorithm #2
Проверить адаптер на совместимость возможно с помощью команд:
# btmgmt phy
Supported phys: BR1M1SLOT BR1M3SLOT BR1M5SLOT EDR2M1SLOT EDR2M3SLOT EDR2M5SLOT EDR3M1SLOT EDR3M3SLOT EDR3M5SLOT LE1MTX LE1MRX LE2MTX LE2MRX LECODEDTX LECODEDRX
Configurable phys: BR1M3SLOT BR1M5SLOT EDR2M1SLOT EDR2M3SLOT EDR2M5SLOT EDR3M1SLOT EDR3M3SLOT EDR3M5SLOT LE2MTX LE2MRX LECODEDTX LECODEDRX
Selected phys: BR1M1SLOT BR1M3SLOT BR1M5SLOT EDR2M1SLOT EDR2M3SLOT EDR2M5SLOT EDR3M1SLOT EDR3M3SLOT EDR3M5SLOT LECODEDTX LECODEDRX
# hciconfig -a
hci0: Type: Primary Bus: USB
BD Address: 8C:88:2B:20:8B:42 ACL MTU: 1021:6 SCO MTU: 255:12
UP RUNNING
RX bytes:2020 acl:0 sco:0 events:178 errors:0
TX bytes:23965 acl:0 sco:0 commands:178 errors:0
Features: 0xff 0xff 0xff 0xfe 0xdb 0xfd 0x7b 0x87
Packet type: DM1 DM3 DM5 DH1 DH3 DH5 HV1 HV2 HV3
Link policy: RSWITCH HOLD SNIFF PARK
Link mode: SLAVE ACCEPT
Name: 'nanopineoplus2'
Class: 0x000000
Service Classes: Unspecified
Device Class: Miscellaneous,
HCI Version: 5.1 (0xa) Revision: 0x999
LMP Version: 5.1 (0xa) Subversion: 0x646b
Manufacturer: Realtek Semiconductor Corporation (93)
Установка вариантов PHY в btmgmt работает не совсем корректно (в некоторых вариантах имеются ошибки). Но и с помощью этой программы возможно установить необходимые PHY.
Для этого необходимо:
- Вывести текущие установки PHY:
# btmgmt -i 1 phy
Supported phys: BR1M1SLOT BR1M3SLOT BR1M5SLOT EDR2M1SLOT EDR2M3SLOT EDR2M5SLOT EDR3M1SLOT EDR3M3SLOT EDR3M5SLOT LE1MTX LE1MRX LE2MTX LE2MRX LECODEDTX LECODEDRX
Configurable phys: BR1M3SLOT BR1M5SLOT EDR2M1SLOT EDR2M3SLOT EDR2M5SLOT EDR3M1SLOT EDR3M3SLOT EDR3M5SLOT LE2MTX LE2MRX LECODEDTX LECODEDRX
Selected phys: BR1M1SLOT BR1M3SLOT BR1M5SLOT EDR2M1SLOT EDR2M3SLOT EDR2M5SLOT EDR3M1SLOT EDR3M3SLOT EDR3M5SLOT LE1MTX LE1MRX
- К вариантам 'Selected phys' добавить опции ‘LECODEDTX’ и ‘LECODEDRX’:
# btmgmt -i 1 phy BR1M1SLOT BR1M3SLOT BR1M5SLOT EDR2M1SLOT EDR2M3SLOT EDR2M5SLOT EDR3M1SLOT EDR3M3SLOT EDR3M5SLOT LE1MTX LE1MRX LECODEDTX LECODEDRX
PHY Configuration successfully set
Для установки PHY через API в bluez смотреть /doc/mgmt-api.txt.
Set PHY Configuration Command
=============================
Command Code: 0x0045
Controller Index: <controller id>
Command Parameters: Selected_PHYs (4 Octet)
Return Parameters:
This command is used to set the default PHY to the controller.
This will be stored and used for all the subsequent scanning
and connection initiation.
The list of supported PHYs can be retrieved via the
Get PHY Configuration command. Selecting unsupported or
deselecting default PHYs will result in an Invalid Parameter
error.
This can be called at any point to change the Selected PHYs.
Refer Get PHY Configuration command for PHYs parameter.
This command generates a Command Complete event on success
or a Command Status event on failure.
Possible errors: Invalid Parameters
Invalid Index
Или через hci_send_cmd(). Пример для Python (неполный и не совсем корректный):
import os
import sys
import struct
import bluetooth._bluetooth as _bt
OGF_LE_CTL = 0x08
OCF_SET_DEFAULT_PHY = 0x0031
def set_hci_phy(hci_sock, phy):
# save current filter
old_filter = hci_sock.getsockopt(_bt.SOL_HCI, _bt.HCI_FILTER, 14)
# Setup socket filter to receive only events related to the
# write_inquiry_mode command
flt = _bt.hci_filter_new()
_bt.hci_filter_set_ptype(flt, _bt.HCI_EVENT_PKT)
_bt.hci_filter_set_event(flt, _bt.EVT_CMD_COMPLETE);
_bt.hci_filter_set_opcode(flt, _bt.cmd_opcode_pack(OGF_LE_CTL, OCF_SET_DEFAULT_PHY))
hci_sock.setsockopt( _bt.SOL_HCI, _bt.HCI_FILTER, flt )
#< HCI Command: ogf 0x08, ocf 0x0031, plen 3 [ 03 05 05 ]
_bt.hci_send_cmd(hci_sock, OGF_LE_CTL, OCF_SET_DEFAULT_PHY, phy)
#> HCI Event: 0x0e plen 4 02 31 20 00
pkt = hci_sock.recv(255)
status = struct.unpack("xxxxxxB", pkt)[0]
print(status)
# restore old filter
hci_sock.setsockopt( _bt.SOL_HCI, _bt.HCI_FILTER, old_filter )
return status
if __name__ == "__main__":
dev_id = 1
hci_sock = _bt.hci_open_dev(dev_id)
status = set_hci_phy(hci_sock, struct.pack("<BBB", 3, 5, 5))
_bt.hci_close_dev(dev_id)
НА в докере, не получилось заставить работать с LR по предложенной инструкции. Может быть нужно ещё что то предпринять?
На ходу, когда уже загружен HA, переключение адаптера не работает.
Требуется остановить HA.
Или выполнить:
hcitool -i hci1 cmd 08 31 03 04 04
hciconfig -a hci1 down
hciconfig -a hci1 up
В Linux, в Bluez, адаптер BT5.0 по старту системы инициализируется неправильно. От этого все проблемы в Linux.
Это аналогично тому, что адаптер с USB3.0 принудительно переключается в USB1.1, а API для переключения в USB3.0 нет. В итоге неясно, как Bluez получил какие либо сертификации. Остается одна надежда на ChatGPT и массовые увольнения “программистов”.
Интеграция "bluetooth" работает через прокладку "Bleak". "Bleak" в свою очередь работает через DBUS, Далее следующая "прокладка" - "Bluez", далее kernel. Это упрощенно, т.к. там десятки так называемых API прокладок.
Писатели "Bleak" вообще не в курсе что в 2016 году вышел стандарт BT5.0 и есть какие-то переключения PHY у адаптеров.
Просто недавно в bluez починили ошибки и в запрос сканирования теперь вставляется значение PHY по умолчанию, которые заданы на низком уровне адаптеру.
Но утилиты bluez ещё не умеют создавать connect с разными вариантами PHY описанными в BT5.0. Там ещё не исправлены ошибки.
WEB Bluetooth API нормально не работает в Linux. В Android оно может соединяться с любыми PHY, но меню сканирования у Chrome ещё не умеет работать с BT5.0. В Windows ситуация с WEB Bluetooth API почти аналогична с Linux, но там другие проблемы. Linux тормозят Bluez и Товальдс, подписывая кривые решения.
API Android работает без проблем с Coded PHY и всеми остальными форматами PHY. Примером является nRFConnect и многие другие приложения на Android.
Apple вообще никогда не придерживалось стандартов Bluetooth SIG. И там свои предрассудки.
По идее, в докере, всё должно быть аналогично, если произвести переключение в той части, которая управляет адаптером. Адаптеры BT5.0 умеют сканировать во всех вариантах PHY с 2016 года.
Бесшовно принимают рекламные пакеты от Bluetooth и всех вариантов BLE BT5.0, включая не стандартные на 2M PHY, если включить эти опции. Но работать с LE Long Range обычным пользователям не дают писатели софта и корпорации по неизвестным причинам. Возможно это подрывает коммерцию ZigBee...
Интеграция "ble_monitor" в HA включает сканирование только на 1M PHY. Автор ещё не умеет переключать адаптер на работу в BT5.0.
Спасибо за описание. Удалось таки перевести датчики lywsd03mmc на использование BLE LR в HA в докере. Пришлось перейти с ble_monitor на xiaomi_ble и заново их определить.
Через btmon проверил подключение действительно в LE Coded.
Посмотрю как это будет работать.
Посмотрю как это будет работать.
Потребление от батарейки сильно возрастает в режиме LE Long Range. Более чем в 3 раза из-за увеличения времени передачи в 8 раз на пониженной модуляции.
Желательно изменить период трансляции рекламных сообщений на более чем 5 секунд.
Включение LE Long Range актуально если термометр расположен на удалении и адаптер его плохо видит.
Тестовые проверки при включении LE Long Range в Xiaomi LYWSD03MMC показывают, что связь с ним сохраняется на расстояниях более 500 метров и до 1 км по прямой видимости. В основном это требуется на отдельных частных участках с большими размерами. В городских "человейниках" эта опция бесполезна, т.к. 2.4ГГц плохо распространяется через препятствия в виде железо-бетонных и каменных стен.
Для достижения той же дальности в режиме "Legacy", по сравнению с режимом "LE long Range", требуется увеличение отдачи RF TX с +7 дБм (без учета коррекции ошибок в режиме "LE long Range"). Это недоступно на CR2032 из-за увеличения тока во столько же раз, во сколько требуется увеличение мощности передатчика.
По сравнению с Bluetooth 4.0 требуется ещё увеличить мощность передатчика в несколько раз...
При переключении на "LE Long Range" время передачи возрастает в 8 раз, но ток от батареи остается прежним (6..7мА при +0 дБм).
Стабильность работы проверена на приеме на дешевый адаптер без доп. антенн (указанный в заголовке темы) от термометра в соседнем доме-бане, находящимся примерно в 100 метрах. При этом уровень приема (RSSI) колеблется на уровне -102..-108 дБм по показаниям данного адаптера. Смартфон при этом спокойно соединяется с термометром в nRFConnect...
Это температура в предбаннике... При протопке печка дает повышение, без печки температура удерживается электронагревателем (мелкая волнистость линии - авто включение/выключение обогревателя)...
# lsusb
Bus 007 Device 002: ID 0a12:0001 USB2.0-BT
# hciconfig -a
hci0: Type: Primary Bus: USB
BD Address: 04:7F:0E:4A:0C:C9 ACL MTU: 1021:9 SCO MTU: 255:4
UP RUNNING INQUIRY
RX bytes:6445 acl:0 sco:0 events:463 errors:0
TX bytes:3579 acl:0 sco:0 commands:347 errors:0
Features: 0xbf 0xee 0xcd 0xfe 0xdb 0xff 0x7b 0x87
Packet type: DM1 DM3 DM5 DH1 DH3 DH5 HV1 HV3
Link policy: RSWITCH SNIFF
Link mode: PERIPHERAL ACCEPT
Name: 'BlueZ 5.64'
Class: 0x000000
Service Classes: Unspecified
Device Class: Miscellaneous,
HCI Version: 5.1 (0xa) Revision: 0x89
LMP Version: 5.1 (0xa) Subversion: 0x89
Manufacturer: Barrot Technology Limited (2279)
hcitool -i hci0 cmd 08 31 03 05 05
Long Range - ok.
@pvvx
Добрый день! Xiaomi LYWSD03MMC прошит LYWSD03MMC Custom Firmware Version 4.2 на компе веб скрипты (сканер) видит устройство, дает прошить, иногда не может получить температуру влажность, иногда получает :)
19:48:54: Watching advertisements from "ATC_0B27B9"... 19:48:54: Argh! TypeError: device.watchAdvertisements is not a function
НО в HA со свистком Bluetooth 5.0 Realtek один раз добавился через Xiaomi BLE. Потом я недолго поигрался с параметрами через веб конфигуратор, и потом, чтобы я не делал, устройство более не добавляется (не видится)в HA (при этом CGPR1 продолжает нормально работать). Я и прошивал и на оригинальную и на модифицированную и батарею вынимал, и команду DD слал и 56, ничего не помогает.
Термометр в 3 метрах от Bluetooth свистка
на компе веб скрипты (сканер) видит устройство, дает прошить, иногда не может получить температуру влажность, иногда получает :)
19:48:54: Watching advertisements from "ATC_0B27B9"... 19:48:54: Argh! TypeError: device.watchAdvertisements is not a function
На Windows "Advertising.html" практически не работает. Это проблема в Windows c Web bluetooth API.
Работает только на Android.
НО в HA со свистком Bluetooth 5.0 Realtek один раз добавился через Xiaomi BLE.
В HA две интеграции для работы с термометрами - "Bluetooth" и "Bluetooth Low Energy Monitor - Passive BLE monitor".
"Bluetooth Low Energy Monitor" работает со всеми выбранными в TelinkMiFlasher.html форматами.
"Bluetooth" имеет расширения "BTHome" и "Xiaomi BLE". Для них необходимо выбрать в TelinkMiFlasher.html формат BTHome или Mijia. А так-же поставить галочку "AdFlag":
со свистком Bluetooth 5.0 Realtek
Рисунки и названия адаптеров от продавцов ни о чем не говорят.
В Windows надо смотреть свойства адаптера в "диспетчере устройств".
По USB VID/PID можно узнать какой чип.
По номерам HCI/LMP - какой версии bluetooth:
@pvvx
Может стоит в шапке веб конфигуратора указывать что нужно с мобильного заходить или я не нашел. с телефона конфигуратор заработал. Спасибо
Попробовал два формата BTHome, Mijia - не находит. Настройки же сразу применяются, батарею вытаскивать не нужно?
Настройки применяются сразу, но BLE реклама не работает во время соединения.
Подключаться для настроек в TelinkMiFlasher.html можно в любой ОС.
А вот прием BLE рекламы в Windows зависит многих факторов. И пока в Chrome и других это опция - экспериментальная и предупреждение описано.
Если не что-то там не работает - обращайтесь примерно туда:
https://github.com/WebBluetoothCG/web-bluetooth
https://github.com/WebBluetoothCG/web-bluetooth/blob/main/implementation-status.md
По USB VID/PID можно узнать какой чип.
цель зависти все это не на виндовс, а на RPI 4 c Home Assistant и с этим свистком
Google: USB\VID_0BDA&PID_8771 а это и есть 8761B
цель зависти все это не на виндовс, а на RPI 4 c Home Assistant и с этим свистком
А это уже ваши проблемы с версиями ОС на RPI 4 и Home Assistant.
Данный адаптер = тормоз. Ловит в 5..10 раз меньше реклам в единицу времени чем другие...
то есть просто ждать и выгадать верное время для соединения ?
Может отсоединяется по причине посаженной батарейки. Для соединения желательно чтобы было более 40% (+20С).
Далее будет работать c HA и до 0%.
батарея 99%. а подключение к chrome на телефоне может мешать подключению к RPI4?
Во время подключения BLE реклама, ту которую принимает HA, не передается.
При подключении потребление больше...
спасибо за уделенное время. пока безуспешно. на тлф в браузере все ок
на HA - тишина, хотя с таким же конфигом неделю назад термометр соединился, а теперь ни в какую
Отключите соединение с термометром в Android и в других. Закройте эксплорер.
В HA термометр появляется с задержкой... Лучше перезапустить всю систему с HA и подождать пару минут.
@pvvx еще маленький вопрос, значок Bluetooth на самом устройстве что обозначает и должен ли он отображаться в момент рекламы (или соединения)?
Значок Bluetooth активен только во время соединения и мигает если нажата впаянная кнопка или геркон с выбором в меню функции геркона/кнопки "соединение" (обычно для Long Range).
@pvvx вместо Xiaomi BLE, установил аддон Passive BLE Monitor
И он нашел оба BLE устройства ¯_(ツ)_/¯ (на том же железе)
hcitool -i hci0 cmd 08 31 03 05 05Long Range - ok.
Hello, I got this dongle and tried to enable long range on my thermometer. Since none of my phones can do 5.0/coded phy and nrfconnect don't see the device I tried this dongle in Ubuntu 22.04 and I have partial success :-)
hcitool switch works and then when using bluetoothctl for scanning like described here https://stackoverflow.com/questions/60318829/bluez-bluetoothctl-scan-vs-hcitool-scan I can see the thermometer - its name and mac address. Then I tried gattool to connect to the mac address but unfortunately it does not work when trying to enumerate services/characteristic and for everything it fails with
Command Failed: Disconnected
So most probably it does not even connect properly. Here is output from btmon
advertisement
> HCI Event: LE Meta Event (0x3e) plen 57 #247 [hci0] 1065.463599
LE Extended Advertising Report (0x0d)
Num reports: 1
Entry 0
Event type: 0x0001
Props: 0x0001
Connectable
Data status: Complete
Address type: Public (0x00)
Address: A4:C1:38:C9:52:1E (Telink Semiconductor (Taipei) Co. Ltd.)
Primary PHY: LE Coded
Secondary PHY: LE Coded
SID: 0x00
TX power: 127 dBm
RSSI: -96 dBm (0xa0)
Periodic advertising interval: 0.00 msec (0x0000)
Direct address type: Public (0x00)
Direct address: 00:00:00:00:00:00 (OUI 00-00-00)
Data length: 0x1f
12 16 1a 18 1e 52 c9 38 c1 a4 db 09 14 12 c1 0a .....R.8........
4f 0e 04 0b 09 4c 59 57 53 44 30 33 2d 31 45 O....LYWSD03-1E
Service Data (UUID 0x181a): 1e52c938c1a4db091412c10a4f0e04
Name (complete): LYWSD03-1E
and here is connection attempt
< HCI Command: LE Extended Create Connection (0x08|0x0043) plen 42 #250 [hci0] 1065.465694
Filter policy: Accept list is not used (0x00)
Own address type: Public (0x00)
Peer address type: Public (0x00)
Peer address: A4:C1:38:C9:52:1E (Telink Semiconductor (Taipei) Co. Ltd.)
Initiating PHYs: 0x05
Entry 0: LE 1M
Scan interval: 60.000 msec (0x0060)
Scan window: 60.000 msec (0x0060)
Min connection interval: 30.00 msec (0x0018)
Max connection interval: 50.00 msec (0x0028)
Connection latency: 0 (0x0000)
Supervision timeout: 420 msec (0x002a)
Min connection length: 0.000 msec (0x0000)
Max connection length: 0.000 msec (0x0000)
Entry 1: LE Coded
Scan interval: 60.000 msec (0x0060)
Scan window: 60.000 msec (0x0060)
Min connection interval: 30.00 msec (0x0018)
Max connection interval: 50.00 msec (0x0028)
Connection latency: 0 (0x0000)
Supervision timeout: 420 msec (0x002a)
Min connection length: 0.000 msec (0x0000)
Max connection length: 0.000 msec (0x0000)
> HCI Event: Command Status (0x0f) plen 4 #251 [hci0] 1065.466611
LE Extended Create Connection (0x08|0x0043) ncmd 1
Status: Success (0x00)
< HCI Command: LE Create Connection Cancel (0x08|0x000e) plen 0 #252 [hci0] 1069.513740
> HCI Event: Command Complete (0x0e) plen 4 #253 [hci0] 1069.515681
LE Create Connection Cancel (0x08|0x000e) ncmd 1
Status: Success (0x00)
> HCI Event: LE Meta Event (0x3e) plen 31 #254 [hci0] 1069.521684
LE Enhanced Connection Complete (0x0a)
Status: Unknown Connection Identifier (0x02)
Handle: 0
Role: Central (0x00)
Peer address type: Public (0x00)
Peer address: 00:00:00:00:00:00 (OUI 00-00-00)
Local resolvable private address: 00:00:00:00:00:00 (Non-Resolvable)
Peer resolvable private address: 00:00:00:00:00:00 (Non-Resolvable)
Connection interval: 0.00 msec (0x0000)
Connection latency: 0 (0x0000)
Supervision timeout: 0 msec (0x0000)
Central clock accuracy: 0x00
somehow it does not handle the connection - gives error " Unknown Connection Identifier (0x02)" what is interesting is that there are those two connection entries
Entry 0: LE 1M
Entry 1: LE Coded
Is this something that can be changed in the firmware? Maybe it does not make sense to offer LE 1M connection if the advertisement is only over Coded PHY? Maybe with only one "LE Coded" entry gattool would not be confused and would possibly connect?
but when thinking about it maybe those two entries are on PC side(?) so not related to firmware - maybe it says which phy versions the PC supports/can try.
Just tried hcitool -i hci0 cmd 08 31 03 04 04 and indeed, it enables only Coded PHY and there is only 1 entry when connecting and it does not work too, same error.
With this 04 setting bluetoothctl scan shows only thermometer and nothing else, which may be sometimes good.
Oh, is is a bug, and quite old one https://www.spinics.net/lists/linux-bluetooth/msg65444.html
The unknown connection identifier is actually the success case for LE Create Connection Cancel command.
And the cancel is sent becasue of the 4s connection timeout.
Linux does not support the Bluetooth 5.0 specification released back in 2016.
All utilities included in Bluez (hciattach hciconfig hcitool hcidump rfcomm sdptool ciptool gatttool in 2017) have long been declared obsolete and cannot be used to work with BLE devices.
All other functionality has violations or neglect of the Bluetooth SIG specification
There are only small pieces of Bluez and kernel v6.4+ code that work with some adapters.
USB-BT released a couple of years ago on ATS2851, like many others, does not work on Linux, even if all the latest patches to the kernel are applied.
Patches in the Kernel are built according to the scheme - we do not understand how the function works or what flags are returned - a patch is put on excluding the use of this function.
That is, the implementation of the functioning of BT5.0 and higher by such patches is simply excluded from support (the type of this HCI_xxx_BROKEN) for many BT adapters (selectively by functions). This leads to incomprehensible errors in old utilities as well.
Just tried hcitool -i hci0 cmd 08 31 03 04 04 and indeed, it enables only Coded PHY and there is only 1 entry when connecting and it does not work too, same error.
With this 04 setting bluetoothctl scan shows only thermometer and nothing else, which may be sometimes good.
Yes. Because the rest of the functionality is performed by the adapter itself, and the reception of advertising with the reinitialization of the primary and secondary PHY is not patched yet :)
And the cancel is sent becasue of the 4s connection timeout.
Originally, in ancient times, timeouts were set correctly. Then they were fixed so that Linux would not work with BLE, and then fixed by many patch variants. But they never fixed it.
Now the solution is as follows:
https://gist.github.com/mironovdm/cb7f47e8d898e9a3977fc888d990e8a9
PS: The only OS in which it is possible to work with BT5+ devices today is Android.
Now the solution is as follows: https://gist.github.com/mironovdm/cb7f47e8d898e9a3977fc888d990e8a9
Thanks! That worked and now it waits much longer however then after those 22 seconds it is cancelled anyway (maybe it does not get any data from thermometer?). Can this be related to some connection parameters like the scan window or max connection interval so that the dongle is not really listening when the data comes?
< HCI Command: LE Extended Create Connection (0x08|0x0043) plen 26 #1336 [hci0] 78319.407277
Filter policy: Accept list is not used (0x00)
Own address type: Public (0x00)
Peer address type: Public (0x00)
Peer address: A4:C1:38:C9:52:1E (Telink Semiconductor (Taipei) Co. Ltd.)
Initiating PHYs: 0x04
Entry 0: LE Coded
Scan interval: 60.000 msec (0x0060)
Scan window: 60.000 msec (0x0060)
Min connection interval: 30.00 msec (0x0018)
Max connection interval: 50.00 msec (0x0028)
Connection latency: 0 (0x0000)
Supervision timeout: 420 msec (0x002a)
Min connection length: 0.000 msec (0x0000)
Max connection length: 0.000 msec (0x0000)
> HCI Event: Command Status (0x0f) plen 4 #1337 [hci0] 78319.408178
LE Extended Create Connection (0x08|0x0043) ncmd 1
Status: Success (0x00)
< HCI Command: LE Create Connection Cancel (0x08|0x000e) plen 0 #1338 [hci0] 78341.829151
> HCI Event: Command Complete (0x0e) plen 4 #1339 [hci0] 78341.830377
LE Create Connection Cancel (0x08|0x000e) ncmd 1
Status: Success (0x00)
That worked and now it waits much longer however then after those 22 seconds it is cancelled anyway (maybe it does not get any data from thermometer?).
Linux does not support the Bluetooth 5.0 specification released back in 2016.
The connection in the "Long Range" mode works very well in Android in a variety of applications using the basic API, including nRFConnect.
Similarly, no problems are observed in many SDKs for SoC with BLE. But not on Linux.
I was unable to achieve a connection with BT5.0 devices in Linux on the available adapters. I also used kernel v6.4+ with the latest patches.
Due to the implementation curves in Linux and Windows, the "Web bluetooth API" cannot come to the implementation of BT5.0 scanning.
But if:
- Enable the "Legacy" type of advertising on the device (BT 4.2)
- Open the connection program in the Chrome browser
- Start scanning before the device appears in the list of BLE connections
- Switch the device to work in extended advertising with "LE Long Range"
- Click "connect" in Chrome.
In Android, the connection will be successful.
As a result, the browser does not yet know how to scan devices with extended and other advertising (BT5.0), but it can connect :)
Linux and Windows keep users from accessing new features...
For these reasons, for bad operating systems, thermometers have a button to temporarily switch to the outdated mode for connection.
now it waits much longer however then after those 22 seconds
This is the reason for the use of incorrect algorithms in BT Linux drivers.
Earlier it was announced that the implementation of scanning and connecting via the BLE protocol in Linux blocks the driver. This is unacceptable for implementation. It's as if the whole system or program is standing still until the driver figures it out.
To connect, it is enough to get one advertising event. But not in Linux - it requires at least two. And if the second one is lost, then a timeout will come out, and 22 seconds will not be enough. Although for other operating systems, 10 seconds is enough, without any blocking.
Ok, then maybe it is worth to try some other bluetooth stack like https://github.com/bluekitchen/btstack and forget about bluez. btstack is opensource and free for non-commercial usage and is supposed to work in linux directly with usb dongles over libusb. BTW, this is the stack which also used in Raspberry Pico W.
The scanning and connection functions available in the Linux API for BLE do not have a primary and secondary PHY installation. This is required by the BLE specification starting from version 5.
The current Bluez has a separate function for installing primary PHY, but most programs don't know about it.
hbldh/bleak#1225 (comment)
Most likely, the connection request is transmitted when PHY is incorrectly installed for Cоded PHY mode (Long Range)....
All this limits the possibilities.
Thermometers can already provide "Periodic advertising with Responses (PAwR)", "Encrypted advertising data", significantly reducing consumption. But external software is not ready to support this functionality, unlike hardware. It remains only to wait, for years...
Perhaps, rather than relying on BLE USB sticks with dubious O.S. support of the BLE 5.x version, an alternative method could be exploiting an I2C-BLE 5 module, like TB-03F (or similar), to acquire Coded PHY advertisements sent by BLE thermometers in range, so that they are saved to RAM and cataloged by each MAC with related storage timestamp. Then, an I2C slave interface implemented in the device can read the last values saved in RAM each time an I2C master (host) polls them. But this requires developing this gateway from scratch.
@Ircama - The main purpose of the project is precisely to raise the level of external software related to BLE. Only in this way is it possible to ensure the availability of these technologies for the majority of users.
I am not a programmer, but a developer, and I can only provide such an option - the gradual inclusion of new standards in the device so that users increase the requirements for external software.
As time has shown, other variants of the approach have not yielded any results. The latest developments in this area are exclusively with Google (Android, Chrome, ...), but they do not release devices with BLE and the development of this direction is stuck.
To support the latest BLE standard, version 5.4 requires a chip that supports version 5.0 (released back in 2016) in hardware. Changes are exclusively in the software part, except for AoA and AoD technology.
Many USB-BT adapters use downloadable software and Bluetooth version 5.4 support does not require chip replacement (except in cases of marketing :) ).
The latest innovations in the BLE standard completely abolish proprietary protocols such as ZigBee. And it seems that for a long time there has already been opposition to the introduction of simpler and cheaper solutions to the masses using BLE technology...
And Linux has long been mired in the "copyright" introduced by its founder. As a result, there have been no innovations for many years and everything new in Linux is not supported.
BTW I recently got Motorola Edge 30 phone and can confirm it finds my thermometer advertising over Coded PHY just fine in nRF Connect. I went a bit farther away so all other BLE devices from same location could not be found and I still saw my LYWSD03-1E in the scan and could connect to it reliably and repeatedly and read various characteristics when RSSI was reported as dBm -106.
So I can confirm it works just fine when BLuez stack is not involved :-)
Has anyone ever found a single USB BT device that is capable of scanning both 1M and CODED at the same time? And connecting at minimum over LE 1M.
The closest I've got is wioth the 0bda:8771 Realtek Bluetooth Radio, I have tested quite a few other chips but few support even receiving coeded announcements (despite supposed support).
I also have one nRF52840 USB module that I flashed Zephyr too (which theoretically could support a coded PHY). Havent managed to be successful yet. However it appears the nRF52840 can only scan one PHY at a time.
The 0bda:8771 Realtek Bluetooth Radio is unable to connect to any devices as soon as CODED scanning is enabled in my testing. And is only able to scan a single PHY at a time.
Ignoring Linux support, I'm yet to find a single hardware product that can complete this element of the specification.
Realtek RTL8761 BT-USB (0bda:8771) scans all PHYs.
PHY 1M/2M/Coded:
hcitool - i hci1 cmd 08 31 03 07 07
Edit /lib/systemd/system/bluetooth.service
ExecStart=/usr/local/libexec/bluetooth/bluetoothd --experimental
USB\VID_10D7&PID_B012 scans all PHYs.
HCI 12.513 / LMP 12.513 (BTv5.3)
The 0bda:8771 Realtek Bluetooth Radio is unable to connect to any devices as soon as CODED scanning is enabled in my testing.
Linux has been dead for a long time. Bluez does not support Bluetooth 4.2 or higher. Only Bluetooth 4.0.
Bleak not support Bluetooth 4.2
Use Android. It supports BT 5.0+
@pvvx When you have time I would love a btmon dump from start to finish for comparison.
I wonder if mine has an older firmware or similar. Or perhaps its the commands I'm sending.
Whenever I try and connect over either LE_1M or LE_CODED if CODED scanning is enabled the module refuses to connect. It's fine if scanning in LE_1M mode. I just assumed the dongle had incomplete CODED support.
I'm developing a custom bt stack (using raw HCI socket) currently. Not currently open source, but hoping the stakeholder will allow release to open source in the future.
Tend to agree re; linux. HCI support is incredibly buggy.
Connection to Coded PHY in Linux does not work! BLE scanning only.
(Also, SSD “fast parking” is not called up in Linux. If there is an unexpected power outage, the SSD may become damaged! Micron also failed with kernel patches... That is, with Linux kernel everything is very bad and has been for a long time.)
The firmware in the RTL chip in Linux is loaded at startup.
The firmware in the RTL chip in Linux is loaded at startup.
Its likely loaded by the Realtek drivers, no guaruntee we are on the same versions - or even that is the cause.
Connection to Coded PHY in Linux does not work! BLE scanning only.
In raw (HCI_CHANNEL_USER) mode this might be bypassable if the cause is understood and the issue not the fault of firmware or hardware.
In raw (HCI_CHANNEL_USER) mode this might be bypassable if the cause is understood and the issue not the fault of firmware or hardware.
No support in API and Bluez.
Anyway we will never know if there could be support because I don't have my USB dongle even getting to the stage you have yours at. Nor any way to tell what the difference is.
Android connection with encoded PHY works well. Even in Explorer.
However, the Web Blutooth API cannot scan the encoded PHY. But if the device (MAC) is stored in Explorer, then Explorer is connected to Coded PHY.
The problem is the same - there is no way to provide support for Linux and Windows.
For Realtek RTL8761 BT-USB (0bda:8771) use these fw:
bt_adpter.zip
However it appears the nRF52840 can only scan one PHY at a time.
what do you mean one at a time?
The S140 stack do accept multiple PHY in a bitfield for scanning https://infocenter.nordicsemi.com/topic/com.nordic.infocenter.s140.api.v6.1.0/structble__gap__scan__params__t.html#a369859ca03e34c8220452ae95d1aa02f and it should work
As for hardware features I guess with single radio you can scan only one channel and one PHY at the same time so the solution is to alternate between them and that would be true for most/all of the BLE hardware? True that for nrf52 there is MODE register https://infocenter.nordicsemi.com/topic/ps_nrf52840/radio.html?cp=5_0_0_5_19_0#register.MODE where you select data rate but I guess most/all other hardware would use same solution?
And who will upgrade Bluez and the Linux kernel to support HCI from Nordic?
And add support for Bluetooth 5.0?
PS: At the moment, it is only possible to expect support (Linux patches) for BT5.0 for some small platforms such as RPi from the Chinese...
what do you mean one at a time?
The LE Coded PHY uses 1Mbit PHY but payload is coded at 125kbps or 500kbps. It also adds Forward Error Correction and Pattern Mapper. RF reception occurs in the same buffer and is then decoded. Purely software implementation.
And who will upgrade Bluez and the Linux kernel to support HCI from Nordic?
Actually very little is needed you get a mostly complete HCI stack from nordic, Zephyr wraps this up (and the HCI stack is "mostly to spec"). hciattach can handle the uart attachment already, no special support required.
We have our hardware wired up this way. Zephyr firmware with Nordic Radio exposing HCI over USB. udev rules to call hciattach, custom bt stack to speak hci (and gatt etc)
And add support for Bluetooth 5.0?
Our custom stack has theoretical support. However we do not have any known working bluetooth 5.0 dongles, hence the request for a btmon. Its easy enough to replay those command parameters outside the linux stack from a custom stack and verify that the hardware does indeed support BT 5.0. We would love that to be the RTL dongle, we have that in hand and got pretty close with it in the past.
With that we can begin to investigate what issues are blocking and if they are software or hardware (and share those results back to the community).
Regardless of who builds support and which stack knowledge on working hardware is incredibly useful to any further development efforts (and this issue is easily findable on Google). If theres no hardware capable of BT Coded connection for example you should not expect BlueZ or Linux BT maintainers to prioritise support.
what do you mean one at a time?
We have not (yet) been successful in getting it to receive coded announcements when both PHYs are active (only when CODED is the only one active). Or to begin join over coded (this may be our test device). This has been a side project however (which is getting consideration for some more engineer time) so it might be something simple.
Its easy for us to blame the hardware at this point, however I'm certainly willing to entertain that we might not be not exercising HCI correctly etc. Or a HCI command need hooking and handling differently within Zephyr (as we have done a couple times already).
At some point we will update our sniffer dongle to one that supports coded sniffing (any suggestions?) and it will become more obvious probably. Also planning on getting a Xiaomi Mijia or one of the other devices to act as a known working BT 5 coded device incase the issue is with our android phone we are using as the announcing & slave device.
We are using a special order nRF52840 devkit with a LNA&PA however any devkit should work. There exists plenty on marketplaces like aliexpress.
HCI from Nordic
AFAIK Nordic does not support HCI over USB or serial (or SPI) with nrf5x chips at all, typically it is standalone solution where the application runs on same CPU as the BLE stack (or in case of nrf5340 on separate application core) . Also it offers so called serialization for multi MCU solution which is similar to HCI but non standard. It is also used by their own nrfConnect software for linux/windows via their own pc-ble driver with so called connectivity firmware https://github.com/NordicSemiconductor/pc-ble-driver/ so it is completely separate from any OS BLE support (which may actually help with solving some use cases
where the OS support is poor?) I'll check whether it supports CODED PHY at all.
nrf52 chips work with linux as HCI over usb or serial only in Zephyr with the hci sample applications https://docs.zephyrproject.org/latest/samples/bluetooth/hci_usb/README.html but that is opensource solution not supported by Nordic (?) Also it can only do BLE not classic bluetooth.
nrf52 chips work with linux as HCI over usb or serial only in Zephyr with the hci sample applications https://docs.zephyrproject.org/latest/samples/bluetooth/hci_usb/README.html but that is opensource solution not supported by Nordic (?) Also it can only do BLE not classic bluetooth.
Correct, BLE only. Mostly correct on the lack of direct Nordic support (not that you should ever rely on a hardware manufacturer for software support). The Zephyr HCI implementation is very small, its mostly just framing, flow control etc of the Nordic commands for the Radio. It still requires the Nordic Radio components which provide the stack (and therefore define many of the potential limitations).
You can get it to provide a decent HCI stack. We have done so. I do not know if BlueZ, noble or similar would find it compatible, its not something I've looked into.
Its been about a year since I touched our Dongle code / hardware or looked into CODED support. I might be incorrect but I think thats accurate.
The LE Coded PHY uses 1Mbit PHY but payload is coded at 125kbps or 500kbps.
Interesting, I am not that familiar with it. If it is only about payload then it should work indeed, however the Nordic docs says even the packet preamble is different https://infocenter.nordicsemi.com/topic/ps_nrf52840/radio.html?cp=5_0_0_5_19_0#concept_hz4_cjl_4r "For MODE = Ble_LR125Kbit and MODE = Ble_LR500Kbit the PREAMBLE is 10 repetitions of 0x3C." so I am not sure the hardware could match it together with Ble_1Mbit mode which uses different preamble. All just my guesses, haven't tried it on such low level yet.
You can get it to provide a decent HCI stack. We have done so. I do not know if BlueZ, noble or similar would find it compatible, its not something I've looked into.
But mostly people use Home Assistant, which is built on Bluez...
And the situation with BLE in Linux has not changed since the release of Bluetooth v4.2. A decade has passed...
however the Nordic docs says even the packet preamble is different
The hardware features on the chips may vary, including hardware CRC packet checking...
The hardware features on the chips may vary, including hardware CRC packet checking...
yeah, but the preamble marks start of the packet, if it is not matched the packet is not received - the whole receiving flow does not start no matter how many other hardware features it has. If preamble is different for coded PHY than regular 1MBit you can't receive both types of packets at the same time. Well unless the hardware can match more of them at the same time to start receiving. Nordic hardware can match multiple addresses (which follows after preamble) but not multiple preamble patterns
https://infocenter.nordicsemi.com/topic/ps_nrf52840/radio.html?cp=5_0_0_5_19_6#concept_v4s_gkl_4r
BTW the preamble for coded phy is longer for redundancy
https://devzone.nordicsemi.com/f/nordic-q-a/70156/ble-coded-phy-preamble-and-range/287855
So do you know hardware that can match both preamble patterns (and then possibly non coded or coded addresses) at the same time?
Was checking zephyr source and for nrf there is radio_phy_set
which sets mode according to phy
https://github.com/zephyrproject-rtos/zephyr/blob/main/subsys/bluetooth/controller/ll_sw/nordic/hal/nrf5/radio/radio.c#L288
and also packet configure which sets the preamble in PCNF0
https://github.com/zephyrproject-rtos/zephyr/blob/main/subsys/bluetooth/controller/ll_sw/nordic/hal/nrf5/radio/radio.c#L435
Sadly I see no other BLE radio hardware supported in zephyr source to match similar code there. However I guess even from software point of view this would be complicated to do, you first set PHY mode and then do something with the radio - send packet receive packet, setting multiple PHYs at once would complicate it even from software POV.
In principle, periodically switching on different PHYs is quite enough.
The main current problem is that in Linux there is no connection to a device running in Coded PHY. But LE advertising packages are accepted. And the connection attempt occurs. But it ends in failure. Most likely the client does not switch the connection to Coded PHY, since there is no such thing in the API settings.
The connection is needed solely for setup and it’s easy to press a button that temporarily switches the device to legacy mode for connection.
In HA, in the BTHome integration, several thermometers are now working on LE Long Range (Coded PHY):
In the diagram - one is in the freezer of the refrigerator, the other is 500 meters away, in another house. RF TX 0 dB. Reception on BT-USB without external antenna.
There are no problems observed.
But what IoT needs most is support for “Periodic Advertising with Replies (PAwR)” and “Encrypted Advertising Data” (BT5.4).
https://www.bluetooth.com/wp-content/uploads/2023/02/2301_5.4_Tech_Overview_FINAL.pdf
And now he is busy with the Zigbee2BLE repeater (a simplified version of Zigbee-Direct).
PS: Let Linux live its own life. Many users use Android and BLE is acceptable there.
Has anyone ever found a single USB BT device that is capable of scanning both 1M and CODED at the same time? And connecting at minimum over LE 1M.
Install several USB-BT adapters. Each to their own PHY.
pc-ble driver with so called connectivity firmware https://github.com/NordicSemiconductor/pc-ble-driver/ so it is completely separate from any OS BLE support (which may actually help with solving some use cases
where the OS support is poor?) I'll check whether it supports CODED PHY at all.
While the nrfconnect gui app for windows/linux does not support coded phy for scanning/advertising the pc-ble driver with connectivity firmware does
https://devzone.nordicsemi.com/f/nordic-q-a/55337/long-range-and-coded-phy-support-for-a-pc-application-on-nrf52840-dongle/224472
I downloaded latest release from https://github.com/NordicSemiconductor/pc-ble-driver/releases and there is prebuilt connectivity_4.1.4_usb_with_s140_6.1.1.hex inside that works with my E104-BT5040U dongle from aliexpress (was US$6 thing few years ago, now it is about $10, it is dongle compatible with Nordic PCA10059 - has basically same schematics just different form factor )
So with the dongle and this pc-ble-driver one can write C code for linux/windows that can use Nordic s140 6.1.1 softdevice api which should allow using coded PHY. I'll try some basic example.
What is the problem with using Telink TLSR825x chips? They work on all PHYs, there is an SDK with BT5.2 and HCI (via USB, SPI, UART and possibly via I2C) support.
What is the problem with using Telink TLSR825x chips? They work on all PHYs, there is an SDK with BT5.2 and HCI
How that will help with long range/coded phy working in code running in linux/windows?
HCI is too low level for using in your own code, this gives bluetooth API layer (which is even compatible with NRF SDK bluetooth APIs) on top of usb serial.
Alternative can be noble on top of HCI but you'll probably still get linux kernel messing things for you if you run hciattach?
EDIT: when checking https://github.com/noble/node-bluetooth-hci-socket I see it can bypass OS HCI support and uses USB directly but it is quite old so most probably does not support newer bluetooth stuff anyway
How that will help with long range/coded phy working in code running in linux/windows?
Windows and Linux do not fully support Bluetooth version 4.2.
And doing anything there is useless, because there is no API for Bluetooth v5+ and there never will be.
In Android BLE connection and extended advertising BT5.0+ works.In addition, there is no connection delay in the form of waiting for several advertising events. They are simply not needed algorithmically, but are present in Linux and Windows...
Scans and connects to Coded PHY and other BLEs: https://apps.rustore.ru/app/ru.kompius.thermo_plung
Supports thermometers with firmware from this repository. Works also for thermometers switched to LE Long Range (Coded PHY, BT5+).
Scans and connects to Coded PHY and other BLEs: https://apps.rustore.ru/app/ru.kompius.thermo_plung Supports thermometers with firmware from this repository. Works also for thermometers switched to LE Long Range (Coded PHY, BT5+).
I tried this app but I'm having a couple of problems:
- I cannot find a way to publish to an mqtt broker.
- if I enable long range I can no longer find the thermometer with miflasher (I have to use thermo_plung to switch long range off then I can connect again with miflasher).
if I enable long range I can no longer find the thermometer with miflasher (I have to use thermo_plung to switch long range off then I can connect again with miflasher).
Web bluetooth API (Chrome) cannot scan BT5.0 devices. The reason is there is no support in Linux/Bluez. Say thank you to Linux, because... it hasn't given this to users since 2014.
https://github.com/WebBluetoothCG/web-bluetooth/blob/main/implementation-status.md#scanning-api
But if the device (running LE Long Range) is already scanned, then in Android Chrome you will be able to connect to it.
Linux does not provide the user with any long-distance connectivity capabilities with standard BLE devices and adapters released since 2014. These are decades-old problems due to incorrect implementation of functions in the kernel and reluctance to support new standards in Bluez.
Contact Linus Torvalds :P
I'm using chrome on the same android tablet where I use thermo_plung 😞
Hello,
I have bought BT dongle from Aliexpress.
I want to use it in BLE mode on my Raspberry Pi 4, where is running my HA in Supervisor mode.
Once i plugged in the dongle to USB 2.0 port on my RPi, I can see this:
#lsusb
Bus 001 Device 003: ID 0a12:0001 Cambridge Silicon Radio, Ltd Bluetooth Dongle (HCI mode)# hciconfig -a
hci0: Type: Primary Bus: USB
BD Address: 04:7F:0E:5A:C6:9E ACL MTU: 1021:9 SCO MTU: 255:4
UP RUNNING
RX bytes:12589 acl:0 sco:0 events:656 errors:0
TX bytes:7093 acl:0 sco:0 commands:462 errors:0
Features: 0xbf 0xee 0xcd 0xfe 0xdb 0xff 0x7b 0x87
Packet type: DM1 DM3 DM5 DH1 DH3 DH5 HV1 HV3
Link policy: RSWITCH SNIFF
Link mode: SLAVE ACCEPT
Name: 'raspberrypi #1'
Class: 0x4c0000
Service Classes: Rendering, Capturing, Telephony
Device Class: Miscellaneous,
HCI Version: (0xa) Revision: 0x89
LMP Version: (0xa) Subversion: 0x89
Manufacturer: not assigned (2279)I have executed also following command:
hcitool -i hci0 cmd 08 31 03 05 05
I can see the Adapter in HA:
I have already:
- flashed my LYWSD03MMC with latest ATC firmware
- allowed BT5+ PHY
- allowed LE Long Range
- setup advertising type to BTHome
- allowed AdFlags
Are these settings correct?
I'm asking because my thermometer is discovered in HA (via BTHome) only when its in the same room like my RPi with BT dongle.
When its placed in next room, behind 2 doors on the same floor of my house, there is no signal.
How can determine that its working in BLE mode? My understanding was that with BLE dongle I can cover all Xiaomi thermometers in my house. Is it correct?
My intention is to get rid off esp32_ble_trackers which I have in each floor of my house and extend thermometers battery life.
Thank you
This adapter (ID 0a12:0001) works very poorly.
After purchasing it, it should be thrown in the trash.
I would buy a new one, but i would like to have some recommendation. This one i have bough this one from the same Ali link you have shared previously. It seems to be the same like on your picture marked as V5.1 (Barrot BR865101).
I have also one more dongle, which is marked as V5.0, where is Barrot 8041A02 chip:
This one is recognized on my RPi like bellow, but its not connecting to Mi Thermometers at all:
#lsusb
Bus 001 Device 003: ID 0a12:0001 Cambridge Silicon Radio, Ltd Bluetooth Dongle (HCI mode)
#hciconfig -a
hci2: Type: Primary Bus: USB
BD Address: 00:1A:7D:DA:71:11 ACL MTU: 679:9 SCO MTU: 48:16
UP RUNNING
RX bytes:684 acl:0 sco:0 events:48 errors:0
TX bytes:3362 acl:0 sco:0 commands:48 errors:0
Features: 0xff 0xff 0x87 0xfa 0xdb 0xbf 0x7b 0x83
Packet type: DM1 DM3 DM5 DH1 DH3 DH5 HV1 HV2 HV3
Link policy: RSWITCH HOLD SNIFF PARK
Link mode: SLAVE ACCEPT
Name: 'raspberrypi #3'
Class: 0x4c0000
Service Classes: Rendering, Capturing, Telephony
Device Class: Miscellaneous,
HCI Version: 5.0 (0x9) Revision: 0x810
LMP Version: 5.0 (0x9) Subversion: 0x2512
Manufacturer: Cambridge Silicon Radio (10)
You have previously recommended to buy a Realtek RTL8761 BT adapter, but its not easy to find it on Ali according to the chipset name. Can you recommend some dongle which is tested and working well please?
Can you recommend some dongle which is tested and working well please?
For "LE Long Range" only RTL8761. Otherwise, there are either a lot of errors in the firmware, or very poor sensitivity of the receiver.
Many WiFi-BT PCIe work well...
LE Long Range -> RTL8761BU (BT5.3) The best reception with the built-in antenna: sensitivity is better than -108..110 dBm. Realtek updates firmware. The firmware is available for download. Loaded into the adapter's RAM at startup. It is possible to create an alternative firmware if you obtain information from Realtek.
USB\VID_0BDA&PID_8771&REV_0200 - https://aliexpress.com/item/4001097686145.html RTL8761B
https://www.home-assistant.io/integrations/bluetooth/#realtek-rtl8761bu-adapters
Thank you, I have ordered ASUS USB-BT500 from my local store, so I hope it will work well.
PS: your link to Ali is not valid, but its possible to search according to image (dongle with "easy idea" mark).
Hello. I have finally received my new dongle - ASUS USB-BT500, which should be based on RTL8761BU.
But it seems it doesn't work on my RPi. I can see it in hciconfig, but the HCI even LMP Version is (0xa), which I don't think its right.
# hciconfig:
hci0: Type: Primary Bus: USB
BD Address: C8:7F:54:98:B9:2B ACL MTU: 1021:6 SCO MTU: 255:12
UP RUNNING
RX bytes:16805 acl:0 sco:0 events:1389 errors:0
TX bytes:19200 acl:0 sco:0 commands:1353 errors:0
Features: 0xff 0xff 0xff 0xfe 0xdb 0xfd 0x7b 0x87
Packet type: DM1 DM3 DM5 DH1 DH3 DH5 HV1 HV2 HV3
Link policy: RSWITCH HOLD SNIFF PARK
Link mode: SLAVE ACCEPT
Name: 'raspberrypi #1'
Class: 0x4c0000
Service Classes: Rendering, Capturing, Telephony
Device Class: Miscellaneous,
HCI Version: (0xa) Revision: 0xb
LMP Version: (0xa) Subversion: 0x8761
Manufacturer: Realtek Semiconductor Corporation (93)
# lsusb:
Bus 001 Device 003: ID 0b05:190e ASUSTek Computer, Inc.
I have tested for now with one thermometer (LYWSD03MMC) and your latest firmware (v4.6) with following settings, but it's not connected via BTHome addon in HA.
Any idea what can be wrong please? Maybe a driver issue?
The only thing I have done was following settings followed by reboot of my RPi:
hcitool -i hci0 cmd 08 31 03 05 05
My RPi 4B is running on 32bit OS with Linux Kernel 5.10.103-v7l+ armv7l.
On Asus pages is a 32bit driver which support kernel 2.6.32 - 4.15 - so my Kernel seems to be not supported.
When I checked /lib/firmware/ there are already rtl8761bu_config & rtl8761bu_fw files dated from 4.4.2023
sudo apt install build-essential
says build-essential is already the newest version (12.6).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
sudo make install INTERFACE=usb
make: *** No rule to make target 'install'. Stop.
I can try to copy and overwrite the files rtl8761bu_config & rtl8761bu_fw from dwonloaded driver to /lib/firmware/, but I'm not sure, if its a good idea (because of Kernel version).
I'm sorry, I'm not a Linux guru.
Thank you @lbarjak . I have downloaded fw and config files from the link above and I have copied them to my RPi:
sudo cp -iv rtl8761bu_fw.bin /lib/firmware/rtl_bt/rtl8761bu_fw.bin
sudo cp -iv rtl8761bu_config.bin /lib/firmware/rtl_bt/rtl8761bu_config.bin
Then it was needed to switch the adapter to work with CodedPHY:
sudo hcitool -i hci0 cmd 08 31 03 04 04
and now it's finally working, even if the HCI & LMP Version is still reported as (0xa).
The main issue was probably the setting for CodedPHY. not the driver.
I have tried 3 options: for CodedPHY:
08 31 03 04 04
08 31 03 05 05
08 31 03 07 07
and seems that only 04 is working fine for me.
I do not understand, how could I determine, which settings is the best for my adapter, but I'm happy it's finally working.
One more thing I want to ensure - is the following settings the best option for BLE?
And the last question is about 'BTHome integration' vs 'Passive BLE Monitor' in HA.
I'm using BTHome, because 'Passive BLE Monitor' is not available for me - most probably because it's already somehow integrated directly in Home Assistant, but i do not understand, what it means? Where I can find it now?
PS: "Packet Id" & "Signal Strength" is not available in BTHome.
Update:
it's working only with the thermometers on the same floor of my house.
For thermometers which are downstairs, there is no signal, so I will need to keep them under ESPHome.
Thank you
Realtek has long released a firmware (rtl8761bu_fw.bin, rtl8761bu_config.bin) version with BT5.3 for RTL8761BU.
I didn't update because... It also works with the old version.
Windows already has BT5.3 firmware installed.
08 31 03 04 04 08 31 03 05 05 08 31 03 07 07
Bit0 - PHY 1Mbit (BLE Legacy)
Bit1 - PHY 2Mbit (Non-standard for BLE advertising)
Bit2 - PHY 125kbit (Coded PHY (S=8), LE Long Range)
Bit3 - PHY 500kbit (Coded PHY (S=2), Non-standard for BLE advertising)
I'm sorry Victor for a dumb question, but I really do not understand the relation between i.e. 04 04 and the Bit0 - Bit3 you have described above. Please can you share more details? I'm interesting in which case to use 04/05/07 or maybe some other? My intention is to have as long range as possible.
And if you stated that Realtek has long released a firmware with BT5.3 for RTL8761BU, do you mean that adapter based on RTL8761BU which was sold as BT5.0 will support BT5.3 with latest firmware?
Thank you
I'm sorry Victor for a dumb question, but I really do not understand the relation between i.e. 04 04 and the Bit0 - Bit3 you have described above
https://en.m.wikipedia.org/wiki/Binary_number#Binary_counting
https://en.m.wikipedia.org/wiki/Binary_number#/media/File%3ABinary_counter.gif
Thank you @fanoush
So my understanding is following:
Bit0 - PHY 1Mbit (BLE Legacy)
Bit1 - PHY 2Mbit (Non-standard for BLE advertising)
Bit2 - PHY 125kbit (Coded PHY (S=8), LE Long Range)
Bit3 - PHY 500kbit (Coded PHY (S=2), Non-standard for BLE advertising)
04 dec is 0100 bin -> PHY 2Mbit (Non-standard for BLE advertising)
05 dec is 0101 bin -> PHY 2Mbit (Non-standard for BLE advertising) & PHY 500kbit (Coded PHY (S=2), Non-standard for BLE advertising)
06 dec is 0110 bin -> PHY 2Mbit (Non-standard for BLE advertising) & PHY 125kbit (Coded PHY (S=8), LE Long Range)
07 dec is 0111 bin -> PHY 2Mbit (Non-standard for BLE advertising) & PHY 125kbit (Coded PHY (S=8), LE Long Range) & PHY 500kbit (Coded PHY (S=2), Non-standard for BLE advertising)
Then becomes a question - which PHY is supported by my adapter (RTL8761BU)?
I was trying to check it with following commands, but they are invalid.
# btmgmt phy
Invalid command in menu main: phy
# btmgmt -i 0 phy
Invalid command in menu main: phy
I can try it just by testing of each combination, so if I understand it right, the best should be 08 31 03 07 07, where all bellow options should be available:
Bit1 - PHY 2Mbit (Non-standard for BLE advertising)
Bit2 - PHY 125kbit (Coded PHY (S=8), LE Long Range)
Bit3 - PHY 500kbit (Coded PHY (S=2), Non-standard for BLE advertising)
Am I right?
PS:
# sudo apt list --installed |grep bluez
bluez-firmware/oldoldstable,now 1.2-4+rpt8 all [installed,automatic]
bluez/oldoldstable,now 5.50-1.2~deb10u4 armhf [installed,automatic]
# cat /lib/systemd/system/bluetooth.service |grep bluetoothd
ExecStart=/usr/lib/bluetooth/bluetoothd --experimental
For BLE advertising, the standard will be 0x05 to receive the advertising header on the main channels. That is, only 1M PHY and Coded S8 PHY are allowed in the Bluetooth SIG standard.
2M PHY is used only after negotiation during connection. Or with extended advertising, after receiving a header on 1M PHY or Сoded S8 PHY, which indicates that the second part will be on 2M PHY on an additional channel.
Therefore, 2 values are used - for the main channel and for the additional one. But this is not in the Linux API, which makes it impossible to support BT5.0. - Linux is dead.
Then becomes a question - which PHY is supported by my adapter (RTL8761BU)?
This chip supports all PHYs.
You can find out what your current version of Linux and the firmware loaded into the adapter supports using the command:
btmgmt phy
But that doesn't indicate what Bluez supports. Bluez is limited to pre-2014 knowledge.
@pvvx : but if bluez is frozen in 2014, and windows is in the same bad situation, and ESP32 simply don't support BLE LR.. What are you using on the remote side of your sensors? I get you have a realtek USB adapter, the one I bought too on aliexpress.. But what OS? What BLE to MQTT solution (if any)?
Thanks!
What BLE to MQTT solution (if any)?
One option could be dedicated usb dongle based on nrf52840. I do have long range/coded phy working in Espruino which is javascript interpreter acessible over usb serial (or BLE of course). There is also
https://github.com/espruino/EspruinoHub which is BLE to MQTT solution that works with Espruino based devices.
See e.g. this https://forum.espruino.com/conversations/393987/ or https://forum.espruino.com/conversations/361380/
I don't use home assistant or MQTT but I believe it should work.
One possible hardware is E104-BT5040U which is ~US$10 thing on Aliexpress https://www.aliexpress.com/item/1005004625411410.html
It is compatible with original Nordic 52840 dongle (PCA10059) so all the tools work with it just fine, you can upload new firmware over USB with nRF Connect for Desktop which is available both for Windows and Linux.
Here is quick sample of scanning for all devices advertising over coded PHY showing one of my thermometers switched to long range mode.
NRF.findDevices(function(devices) { console.log(devices);}, {timeout : 2000, active : false, phy:"coded"});
=undefined
[
BluetoothDevice: {
"id": "a4:c1:38:c9:52:1e public",
"rssi": -55,
"data": new Uint8Array([2, 1, 6, 18, 22, 26, 24, 30, 82, 201, 56, 193, 164, 122, 9, 66, 20, 217, 10, 82, 136, 4, 11, 9, 76, 89, 87, 83, 68, 48, 51, 45, 49, 69]).buffer,
"name": "LYWSD03-1E",
"serviceData": {
"181a": new Uint8Array([30, 82, 201, 56, 193, 164, 122, 9, 66, 20, 217, 10, 82, 136, 4]).buffer
}
}
]
>
EDIT: or for cheaper 52840 boards check https://github.com/joric/nrfmicro/wiki/Alternatives
there are boards like this https://www.aliexpress.com/item/1005006271779544.html
However I am not sure how good is the antenna if you really need long range. One cheap device with external antenna connector was the Particle Xenon which is now discontinued and hard to get (have Espruino working on that one too)
What BLE to MQTT solution (if any)?
I don't use MQTT. I receive BLE in Home Assistant using two simple USB-BT adapters. One works for the near zone, the second for the far zone in the “LE Long Range” mode.
Backup BLE receiving channels are organized on various small SoCs, with my programs.
NRF has not released USB-BT adapters or software for these options.
I am not talking about (normal HCI) USB-BT adapters. With espruino these dongles can run your stored javascript code, talk to anything over BLE and then write to USB serial any output you need. So the logic is in javascrit code inside the dongle, if the MQTT broker (like EspruinoHub) or the Home Assistant directly can read from serial port it should work.
No versatility. For each task on NRF you will have to create a new program.
Today there are cheaper chips with full support for BT 5.4 from WCH. Less than 1 dollar.
The CH32V208W is a little more expensive. It has BT5.4, USB, Ethernet,... The SDK contains examples of working with BLE+MQTT+Ethernet...
No versatility. For each task on NRF you will have to create a new program.
Oh! Well, I think you don't quite get it with the versatility, but never mind. if you have solution that works for you than that's great.
Here is one example of 'no versatility' https://www.espruino.com/Pixl.js+BLE+Ethernet+Bridge - it is NRF device with BLE and ethernet shield connected over SPI. In just few lines of javascript you convert any BLE advertisement seen around into MQTT message and send it to server over ethernet.
You don't need to write that stuff in C like with SDK for CH32V208W . If there would be e.g. micropython for that CH32V208W with working BLE api that can do long range then it would be similar.
Who cares - I can do it faster in C.
But no one has been able to write a normal BLE gateway on nRF. For years already. And who in the modern world needs chips with libraries in blobs? NRF - proprietary software. - Correction of errors in nRF binary libs has to wait for years.
But no one has been able to write a normal BLE gateway on nRF. For years already.
Not sure what you mean by "normal BLE gateway". If you mean HCI then we already discussed that, it is part of ZephyrOS and works. However due to poor Linux support as you said many times it is not very useful to get long range/coded phy working in Linux.
And who in the modern world needs chips with libraries in blobs? NRF - proprietary software
This was already discussed too. There are several open source production ready certified BLE stacks for nRF5x chips. Hardly any other chip has such wide open source support. There is ZephyrOS which is also current supported official SDK from Nordic (called nRF Connect SDK), there is also Apache NimBLE and there is at least one commercial one with full sources on GitHub which I think needs license for commercial use (forgot its name, maybe it was this one https://github.com/packetcraft-inc/stacks ?). SoftDevices are legacy and you are definitely not forced (or even recommended) to use them if you don't appreciate the advantages (stability, simplicity, clear separation from your code).
This was already discussed too.
Again, one nRF advertisement, but in reality there are no devices for users or DIY on nRF.
Not sure what you mean by "normal BLE gateway".
I want to enable 'PAWR' on all thermometers, but users have no adapters capable of working with PAwR.
MQTT is just suitable for working with BLE 'PAwR'.
Create a cheap NRF project for working with BLE 'PAwR', and do not write advertising posts ...
Again, one nRF advertisement, but in reality there are no devices for users or DIY on nRF.
Well I was just disproving your untruths. Would be nice if you could stop so there is no need to correct you over and over again (it is also off topic here).
Sadly you just did it again. There are of course tons of nrf5x based devices directly targeted for DYI both made by companies and also individuals and there is also a lot of hackable cheap nrf52 based stuff on aliexpress (smartwatches, fitness trackers, NFC+BLE keychains). For companies just check sparkfun or adafruit for their own nrf5x boards, adafruit even maintains Arduino and also Circuitpython for nrf52 just check list here https://github.com/adafruit/circuitpython/tree/main/ports/nrf/boards , also Microbit is NRF, the version 2 is 52833 and widely available (version 1 was bad IMO but 2 is nice). The Espruino stuff is mostly nrf52, previously I linked nrf52 boards for doing custom BLE/USB HID keyboards https://github.com/joric/nrfmicro/wiki/Alternatives For some reason they like 52840 based boards for this, not sure why, but they are probably quite popular as they got cloned and sold on aliexpress in quantities (google "ZMK firmware"). The Pinetime watch is nrf52832 - they are using Apache NimBLE there in their firmware. In fact it is one of many cheap hackable OTA updatable DaFit nrf5x based smartwatches which come in many shapes and sizes. One example is the 12 EUR "C17" nrf52840 one a.k.a. Magic 3 you can find on aliexpress now but there are many others. So you must be living in some parallel universe to say that thing I quoted. Or you just like to troll.
I want to enable 'PAWR' on all thermometers, but users have no adapters capable of working with PAwR. MQTT is just suitable for working with BLE 'PAwR'.
Oh, that is actually interesting stuff I did not know yet. Some time ago I tried to sync precise time via advertising but got bitten by that random delay and also wanted to send different scan response based on who is asking (so having bidirectional communication via advertising+scan response), it is nice someone was thinking in same direction and fixing both issues :-) GATT and connections are too heavy for a lot of stuff.
There are of course tons of nrf5x based devices directly targeted for DYI
Where is the firmware for this thermometer to work in BTHome?
You are describing some toys that are not needed in IoT.
There are many more DIY boards available with other BLE chips. And there are many more of them than with SoС from the NRF
Low cost BLE Advertisements Repeater into Ethernet TCP/IP
@fanoush - How long will you advertise nRF or will you do something? :) :)
So you must be living in some parallel universe to say that thing I quoted.
Or do you need to block access to the repository "from this universe"? :)