Does deCONZ serial protocol for ConBee/RaspBee Zigbee Coordinator adapters not support joining/pairing via install code and qr code?
Hedda opened this issue · 34 comments
@manup Reposting from zigpy/zigpy-deconz#214 and Koenkk/zigbee2mqtt#17492 as a public request to you dresden-elektronik's deCONZ (ConBee) developers because the developers of zigpy (used by Home Assistant's ZHA integration + Zigbee plugins for Domoticz and Jeedom) and zigbee-herdsman (Zigbee2MQTT + IoBroker) replied there that this function is not supported in zigpy-deconz or zigbee-herdsman since the function for install code (and qr code) is not a documented in the linked deCONZ serial protocol documentation:
https://github.com/dresden-elektronik/deconz-serial-protocol/blob/master/README.md
UPDATE: The exact same issue also applies to Zigbee2MQTT (Z2M) and IoBroker (which are both based on zigbee-herdsman):
It started with one user of Home Assistant’s ZHA integration with an ConBee Zigbee Coordinator adapter posted that they could not join/pair a Zigbee 3.0 (ZB3) device that required an "install code", and then some other user replied to that post saying "install code" (and "qr code"?) support is not implemented in the zigpy-deconz radio library for zigpy which the ZHA integration depends on. Is that true?
For reference that specific Zigbee 3.0 device that require install code to join was the "Bosch Thermostat 2":
https://community.home-assistant.io/t/bosch-thermostat-2/492845/
ZHA (and zigpy) can set an Install Code for Zigbee 3.0 devices if using Silicon Labs EmberZNet or Texas Instruments Z-Stack ZNP:
https://www.home-assistant.io/integrations/zha#services
https://wiki.st.com/stm32mcu/wiki/Connectivity:Zigbee_Install_Code
Just reposting this issue reported in the Home Assistant community forum:
https://community.home-assistant.io/t/bosch-thermostat-2/492845/10
https://community.home-assistant.io/t/bosch-thermostat-2/492845/11
The first device in question was the Bosch Thermostat II 230 V (BTH-RA / BTH-RM230Z), however, I understand that some people using deCONZ or Zigbee2MQTT have reported the same issue with a few other Bosch-branded Zigbee devices, like Bosch Smoke Alarm II (BSD-2), Bosch Door/window contact II (RBSH-SWD-ZB), and Bosch BWA-1 water sensor, all of which require adding the install code to pair/join. Another device that been mentioned to require install code is the rotary switch Aqara H1 -> Koenkk/zigbee-herdsman#420
PS: I do not have a ConBee/RaspBee Zigbee Coordinator setup myself any longer so can not replicate it to confirm or disconfirm this but think that it would be a good idea if this could be tracked as a feature request because the function is missing the deconz serial protocol documentation and for deconz firmware based Zigbee Coordinator adapters like ConBee/RaspBee but supported by other Zigbee Coordinator adapters like Silicon Labs and Texas Instruments based Zigbee Coordinator adapters via their respective radio library for zigpy.
I think that you have to set the network parameter 'Link Key' at 0x19
(page 12). However I don't have a Conbee Adapter to test it out
Unfortunately, Link Key (0x19) is the Trust Center Link Key, which is always ZigBeeAlliance09. There needs to be a specific API to add a link key for a specific IEEE address, to replace the well-known key during transport key exchange.
@ChrisHae Can you tell if deconz adapter/driver in zigbee-herdsman (used by Zigbee2MQTT and IoBroker) supports install codes?
https://github.com/Koenkk/zigbee-herdsman
https://github.com/Koenkk/zigbee-herdsman/tree/master/src/adapter/deconz
The Conbee2 fw does not support it yet. But it will come soon. And then it will also be implemented in zigbee herdsman deconz adapter.
The Conbee2 fw does not support it yet. But it will come soon. And then it will also be implemented in zigbee herdsman deconz adapter.
Great to hear! If possible please update the documentation then also ping the zigpy developers too -> zigpy/zigpy-deconz#214
Any news on this? Unfortunately the Bosch Thermostat 2 only supports pairing via install code :/
Bumping here. Still waiting for the support of pairing via install code to get my devices up and running. Any info on the current state?
@pdsccode
It seems like it's included in the latest version: https://github.com/dresden-elektronik/deconz-rest-plugin/releases/tag/v2.22.0-beta
But it's still in Beta-Status. Waiting for the version to be finally released.
Can anyone confirm or disconfirm if Zigbee install code support is enabled in the current or latest ConBee/RaspBee firmware?
dresden-elektronik/deconz-rest-plugin#6791
dresden-elektronik/deconz-rest-plugin@24c7ff9
https://github.com/dresden-elektronik/deconz-rest-plugin/releases/tag/v2.22.0-beta
If and when it has can you then please also update the deconz-serial-protocol documentation to include all the commands for it?
Developers of Home Assistant's ZHA integration and Zigbee2MQTT need it to implement it in zigpy-deconz and zigbee-herdsman:
What makes you think this has anything to do with the device firmware? You can see all the necessary changes to make the install codes work from the referenced PR. To my knowledge, last changes involving the firmware were about a year ago or so and none of the change log entries reference install codes.
Generally, I cannot comment on the serial protocol, since I have no clue about it.
So you are saying all that is missing is the documentation for the deconz serial protocol?
What makes you think this has anything to do with the device firmware?
That is what ChisHae wrote above:
Originally posted by @ChrisHae in #20 (comment)
The Conbee2 fw does not support it yet. But it will come soon. And then it will also be implemented in zigbee herdsman deconz adapter.
Again, this interface information is needed in the deconz serial protocol documentation in order zigpy and zigbee-herdsman for third parties to be able to communicate with the Zigbee devices via ConBee/RaspBee as Zigbee Coordinator adapters:
https://github.com/dresden-elektronik/deconz-serial-protocol/blob/master/README.md
zigpy is for example used by Home Assistant's ZHA integration, the Zigbee Plugin for Domoticz, and Zigbee Plugin for Jeedom.
https://github.com/zigpy/zigpy (see zigpy-deconz feature request -> zigpy/zigpy-deconz#214 )
zigbee-herdsman is for example used by Zigbee2MQTT and IoBroker:
https://github.com/Koenkk/zigbee-herdsman (see Zigbee2MQTT feature request -> Koenkk/zigbee2mqtt#17492 )
So you are saying all that is missing is the documentation for the deconz serial protocol?
I more or less said the exact opposite: I don't know anything about the deconz serial protocol. Please read my previous reply again.
Other than that, I also cannot comment if for other software any changes on the firmware might be necessary. For deconz itself, the PR documents the necessary changes to make it work. Nothing more, nothing less 🙂
We don't use the binary deCONZ library in ZHA or Z2M though and we instead directly interact with the coordinator over the documented serial protocol. The deCONZ object is a higher-level API so while this code is straightforward:
#if DECONZ_LIB_VERSION >= 0x010B00
QVariantMap m;
m["mac"] = uniqueid.toULongLong(&ok, 16);
m["key"] = mmoHash;
if (ok && mmoHash.size() == 32)
{
ok = deCONZ::ApsController::instance()->setParameter(deCONZ::ParamLinkKey, m);
}
#endifWriting the hashed link key and a joining device's IEEE address to this specific parameter and then permitting joins still results in the Conbee sending the network key as usual for me. So unless I'm making some simple mistake somewhere, there may be something else happening in the background within the library or a specific sequence in which this parameter write needs to be sent when permitting joins.
Various questions:
- Does this command need to be sent before permitting joins? After?
- How do you clear a link key? If you can't, is it cleared automatically after the join window closes? How many can you write at once?
- What firmware version was this feature introduced in?
And while I could spend the time to reverse engineer deCONZ by intercepting traffic between it and the Conbee, it's a lot nicer to not have to do this.
Hi, the MAC + derived Link Key is indeed what is used also on the serial protocol side.
The firmware doesn't know anything about Install Codes itself and the Link Key is derived as shown in above PR. Formerly this was done with a command line tool from the Zigbee Alliance but this was meh.
The Link Key parameter 0x19 is in the firmware since a long time, also for ConBee I, where we did our initial tests for a student diploma with various real devices. Here we also found that Install Codes sometimes are invalid/non standard, e.g. byte order of the checksum was wrong.
But I'd like to put a big warning on using the 0x19 parameter yet, while it does work it could also make the configuration in the firmware messy, e.g. after setting it classical joining of devices might not work any longer until the trust center MAC address + link key is set again. Fixing this is on the bucket list but doesn't have a high priority currently, since there are too many other pressing things on the todo list.
Any updates on this? ...AFAIK, install code is still not supported for ConBee in Home Assistant's ZHA integration or Zigbee2MQTT?
Support for commissioning Zigbee 3.0 devices via “Install Code” or “QR Code” via the ‘zha.permit’ service has so far only been implemented for ‘ezsp’ (Silicon Labs EmberZNet) or ‘znp’ (Texas Instruments Z-Stack) radio type adapters in the ZHA integration in Home Assistant (that depends on zigpy-deconz) and Zigbee2MQTT (that depends zigbee-herdsman) . Other radio types are missing support in their respective radio libraries for zigpy or manufacturer’s firmware commands/APIs.
So you are saying all that is missing is the documentation for the deconz serial protocol?
I more or less said the exact opposite: I don't know anything about the deconz serial protocol.
Mixed signals(?); as according to @ChrisHae support for using install code is not yet implemented in the deconz serial protocol:
Originally posted by @ChrisHae in Koenkk/zigbee2mqtt#17492 (comment)
It is not yet implemented in the serial protocol. When it is, it can be added to z2mqtt deconz adapter.
Again, deconz firmware for ConBee and RaspBee does not yet seem to implement Zigbee install code support in the deconz serial protocol and therefore ConBee/RaspBee does not currently offer feature-partity with with serial protocols for Texas Instruments Z-Stack and Silicon Labs EmberZNet for Zigbee Gateway host applications such as the ZHA integration in Home Assistant (which is turn depends on the zigpy and zigpy-deconz libraries) and Zigbee2MQTT (which in turn depends on zigbee-herdsman).
Any updates or news on this from dresden-elektronik's developers?
Can you assist with getting Zigbee install code support working in ZHA (zigpy-deconz) and Zigbee2MQTT(zigbee-herdsman)? See:
and
Can we get an update on this?
I would be also very happy this there was a solution for this. Thank you in advance!
Any update on this?
Hi, any updates on this?
Any updates of deCONZ Serial Protocol documentation to be able to support "install code" for ConBee III (ConBee 3) adapter?
As it stands, deconz "install code" and "qr code" support is not available in Home Assistant's ZHA integration of Zigbee2MQTT:
PS: Still listed as a deconz serial protocol related limitation under ZHA docs and until then any ConBee cannot be recommended:
https://www.home-assistant.io/integrations/zha#limitations
"Support for commissioning Zigbee 3.0 devices via “Install Code” or “QR Code” via the ‘zha.permit’ service has so far only been implemented for ‘ezsp’ (Silicon Labs EmberZNet) or ‘znp’ (Texas Instruments) radio type in ZHA. Other radio types are missing support in their respective radio libraries for zigpy or manufacturer’s firmware commands/APIs."
I tested yesterday with Conbee 3 and ZHA without success.
File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 1006, in _async_admin_handler await result File "/usr/src/homeassistant/homeassistant/components/zha/websocket_api.py", line 1257, in permit await application_controller.permit_with_link_key( File "/usr/local/lib/python3.12/site-packages/zigpy_deconz/zigbee/application.py", line 115, in permit_with_link_key await self._api.write_parameter( File "/usr/local/lib/python3.12/site-packages/zigpy_deconz/api.py", line 849, in write_parameter await self.send_command( File "/usr/local/lib/python3.12/site-packages/zigpy_deconz/api.py", line 508, in send_command return await self._command(cmd, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy_deconz/api.py", line 589, in _command return await fut ^^^^^^^^^ zigpy_deconz.exception.CommandError: <CommandId.write_parameter: 11>, status: <Status.UNSUPPORTED: 4>
I tested yesterday with Conbee 3 and ZHA without success.
Even if was supported in ConBee 3 firmware it would still have to be implemented in zigpy-deconz -> zigpy/zigpy-deconz#214
And that is not possible until deconz serial protocol documentation is updated to include info about "install code" + "qr code".
Again, deconz serial protocol for ConBee/RaspBee Zigbee Coordinator adapters not support joining/pairing via install code and qr code, or at least it is not yet documented. Until then you can not use ConBee/RaspBee radios with ZHA or Zigbee2MQTT if want to provision/commision via QR-code or install-code:
Any news?
@manup Any chance you or someone else from dresden-elektronik could prioritize this and updating the documentation here?
While I can only speak for myself, think I can still say that as it stands right now many of us who are a part of the overall Zigbee2MQTT and ZHA communities can no longer not with good conscience recommend ConBee II or ConBee III to new end-users as they would not be able to add any Zigbee devices that require install code / QR code for provisioning and we have not seen any (public) progress on this for over a year. 😞
This "limitation" with the ConBee/RaspBee firmware and deCONZ Serial Protocol is currently only indirectly mentioned in FAQs:
PS: Imagine though that if support for this can not be added then ConBee will be removed from lists of recommended adapters:
Sorry this took a while, I had reworked the document since the former .docx based one was messy to update. The document references are in a bit rough shape but that's for another version. The new version 1.22 has now been uploaded http://deconz.dresden-elektronik.de/raspbian/deCONZ-Serial-Protocol_en_1.22.pdf.
There is a new small section 6.2.3 Zigbee Install Codes which describes the basic usage.
As reference how it works in deCONZ the following sources might be helpful as well:
Calculating the derived Link Key from Zigbee Install Code (I think ZHA and Z2M already have similar code):
https://github.com/dresden-elektronik/deconz-rest-plugin/blob/master/crypto/mmohash.cpp
How the install code is processed by the REST-API and set to the firmware:
https://github.com/dresden-elektronik/deconz-rest-plugin/blob/0e2dc155e85893b439f354cd5d764e414ad0ed0c/rest_devices.cpp#L1090
Sorry this took a while, I had reworked the document since the former .docx based one was messy to update. The document references are in a bit rough shape but that's for another version. The new version 1.22 has now been uploaded http://deconz.dresden-elektronik.de/raspbian/deCONZ-Serial-Protocol_en_1.22.pdf
Nice but that HTTP URL does not work for me
aslo tested HTTPS without any better luck:
Are you sure? The link I've posted works here normally 🤔
Are you sure? The link I've posted works here normally 🤔
Yes I get “404 Not Found” error from your Nginx server so should not be on my (client side). See screenshot from Chrome:
The “404 Not Found” error is an HTTP status code that indicates the requested resource couldn't be found on the server. In the context of Nginx, this error typically occurs when a client requests a file or page that doesn't exist or isn't properly configured in the Nginx server block or configuration file.
Ah, can get HTTP link to work in other web browsers, but the HTTPS does not work in any webbrowsers, so still issue on your side?
http link works for me, https gives errors.
But otherwise awesome that this is now documented!
I really hope it can be implemented into zigbee2mqtt in the near future, need to get the Bosch thermostats on my network before the cold season starts ;)
Thanks for sharing the updated documentation, @manup!
I took a look at the current permit_with_link_key code used in ZHA (https://github.com/zigpy/zigpy-deconz/blob/dev/zigpy_deconz/zigbee/application.py#L115). If I am not overlooking something, then it already does the same as the code you shared above. It is just a bit more hidden. E.g. the key derivation happens in the zigpy lib (in convert_install_code https://github.com/zigpy/zigpy/blob/dev/zigpy/util.py#L242).
So I gave it a try with a Conbee III (latest firmware 0x26530900), but it seems that the stick returns 0x04 (unsupported) for the 'write parameter' 0x19. 🤔 I would expect 0x05 (Error) if the given payload would be wrong.
Is there a way to get debug logs from the Conbee III directly?