home-assistant-libs/zwave-js-server-python

Integration Roadmap

cgarwood opened this issue ยท 24 comments

Roadmap for the Z-Wave JS integration.
Note: for a list of all that is already supported see here.

Open Tasks

General

  • Complete certification requirements [Board]

Maintenance

  • #590
  • Refactor discovery so that discovery logic for each platform lives in each platform - use Matter as a reference
  • Explore using stateful metadata parameter for value notification discovery in device triggers

Platform support/features

  • set date/time entity (when support for this entity type is added)
  • add is_opening and is_closing support for covers [PR]
  • Ensure that when secure inclusion fails, the reason is passed back to the user in the UI
  • Handle inclusion initiated outside of HA
  • Add a lastSeen sensor

Service calls

None

Z-Wave control panel (UI)

WS API Ready, awaiting frontend changes

  • Refresh Node (re-interview, refresh values, refresh CC values)
  • Replace Failed Node
  • Show node metadata on the device page (exclusion, inclusion, reset, wakeup, manual, device database URL)
  • UI for managing node associations
  • Add support for changing wakeup time to config panel (separate CC from normal configuration)
  • add rfRegion to device info for controller node

Lib ready, WS API support missing

  • NVM backup and restore
  • Let users view, add, delete, and update provisioning entries
  • Node diagnostics (checking route health, testing power level, etc.)
  • Record events and commands, see options (in either case we can have a stop button to stop recording because we can see whether recording is enabled or not):
    • reload config entry after setting a parameter that tells the config entry to start recording when initializing the client during setup
    • Update the parameter live, downside is we miss past events
  • Controller hard reset

Z-Wave JS ready, lib support missing

None

zwave-js support missing

None

Unclear yet on what's needed

  • Some form of network health check, make it easier to point out dead nodes or nodes that are slow to respond or having communication failures.
  • Network visualization of the mesh (like ZHA)

Note: Full configuration of the options are already available in the zwavejs2mqtt control panel.

Other

  • Better documentation! (this will likely never be checked off unless and until the zwave-js project stops maturing)
  • Improve the Core config flow for add-on config options. Provide a list of USB devices to select device, and validate that security keys have the correct format.

Finished

Services

  • Set configuration parameter service.
  • Poll node/value service.
  • Bulk set partial configuration parameter service.
  • Multicast support
  • Reset meters

UI

  • UI for updating node configuration settings
  • Adjust logging level of zwave-js server from UI
  • Subscribe to logs in the frontend
  • Node specific page
  • Log config update handling
  • Heal network
  • Show parameter number on node config page (most device manuals refer to parameter numbers instead of descriptions)
  • Heal node
  • Remove failed node
  • metadata comments (e.g. known issues with the device)
  • QR code (SmartStart) inclusion [2021.12]
  • Controller statistics [2022.6]
  • Node tx/rx statistics [2022.7]
  • Node Firmware Updates [2022.7]
  • Controller firmware updates [2023.3]
  • DSK only QR code support [2023.3]

General

  • Add sensor for node status (awake/asleep/dead/etc)
  • SoundSwitch (siren) support [2021.8]
    • Support setting default tone and volume [2021.9]
  • Device conditions (config parameter state, any value state, node status) [2021.8]
  • Device triggers for value notifications, notifications, and node status [2021.8]
  • Protection Command Class (select entity) [2021.9]
  • Automation triggers (non-device) for any value update [2021.9]
  • Device triggers for any value update [2021.9]
  • Device actions [2021.10]
  • Migration wizard from (deprecated) zwave integration [2021.10]
  • S2 support [2021.10]
  • Automatic firmware updates [2022.9]
  • Manual idling buttons for notification CC values [2023.5]
  • perform optimistic state updates for multilevel switch entities [2023.6]
  • improve Indicator CC support [2023.6]
  • Controller and node statistics sensors [2023.6]
  • config parameter entities [2023.6]
  • Add support for window covering CC [[2023.6]]
  • add support for Energy Production CC [[2023.6]]
  • Add "identify" event handler to generate a persistent notification in HA

Obsolete

  • Some sort of UI for configuring complex device settings, like notifications for Inovelli switches: https://nathanfiscus.github.io/inovelli-notification-calc/ or aeotec energy monitor reporting configs. (solved by partial parameter service)
  • Migration wizard from ozw integration (based on current analytics and the low usage of the ozw component, we feel a migration wizard is significantly more effort than its worth)

OTA Firmware updates?
edit: nm I'm blind

Some sort of UI for configuring complex device settings, like notifications for Inovelli switches

Might not be necessary. We use partial params for this. Bulk-setting them (if necessary) is on the TODO list

Refresh Node

Should clarify what this means. Users confuse this with node.refreshInfo where zwave-js restarts the interview from zero - which might not be what users expect.

Network visualization of the mesh (like ZHA)

Might be worth to join forces: zwave-js/zwave-js-ui#384

Refresh Node

Should clarify what this means. Users confuse this with node.refreshInfo where zwave-js restarts the interview from zero - which might not be what users expect.

Yeah, we should probably add some warning there if we create that UI.

no mention of Garage Doors (which I have) that is mentioned here? https://www.home-assistant.io/integrations/zwave_js#current-limitations

Some better setup and install instructions for the zwave js server for all installation scenarios, I am on core (not supervisord) so I don't have access to addons.

Just quickly looking at it; https://zwave-js.github.io/zwavejs2mqtt/#/getting-started/quick-start does not tell you how to configure your MQTT server? Is there a file configuration appraoch?

Garage door support is atm missing in Z-Wave JS, I'm sure it will be added soon enough.
We'll do our best to provide better setup documentation of all possibilities.
You don't need mqtt, you can just only use the zwavejs2mqtt control panel, it provides the WS server too.

Wow! great guys. for me to switch to this platform this is the most important i gues to avoid to rename all entites new:

Migration wizard from zwave and ozw integrations

You don't need mqtt, you can just only use the zwavejs2mqtt control panel, it provides the WS server too.

well, I have it. ;) That said then I feel there should be more doco about what one needs in what scenarios or the pro's and con's on various approaches.

Thanks!

Edit, question: WS = Websocket?

Refresh Node

Should clarify what this means. Users confuse this with node.refreshInfo where zwave-js restarts the interview from zero - which might not be what users expect.

Yeah, we should probably add some warning there if we create that UI.

So... what is intended here by "Refresh Node" - is the the same thing as requestnodedynamic, refreshvalue, refreshnodeinfo, or other?

Can zwavejs2mqtt be used to connect/use existing another mqtt server?

@steve28 more like "re-interview from scratch as if you don't know that node"

Device triggers for Scene (activation) events.
Just to confirm, right now there are no HA events at all (and so no way to handle) central scene notifications from a device, right? I wasn't 100% clear whether this item referred to that or to some additional capability related to scene events. Thanks!

We do fire zwave_js_event for notification and value notification events.

https://zwave-js.github.io/node-zwave-js/#/api/node?id=quotnotificationquot

https://zwave-js.github.io/node-zwave-js/#/api/node?id=quotvalue-notificationquot

You should be able to see them by listening to the zwave_js_event in the dev tools in the Home Assistant GUI.

We don't yet support device automations.

@cgarwood @marcelveldt I'd add "configure polling" to the list. zwave-js doesn't do that automatically, because it should be used very sparingly.
To make legacy devices work, zwavejs2mqtt allows users to enable scheduled polling on a per-value basis.

@cgarwood @marcelveldt I'd add "configure polling" to the list. zwave-js doesn't do that automatically, because it should be used very sparingly.
To make legacy devices work, zwavejs2mqtt allows users to enable scheduled polling on a per-value basis.

Yeah, we're already looking at it. Basically two options:

1) Configure polling within HA for a node and we will poll it's primary value at every interval automatically.
Pro: Easy to configure and mostly like previous Z-Wave implementions in HA.
Con: With lot's of devices with polling enabled, a lot of polling "for nothing" will be done, causing additional traffic on the mesh.

2) Provide a poll_value service to the user, so he could poll a value within an automation. For example only poll a dimmer if there is any motion detected in that room.
Pro: Very flexible. A user decides if and how a value should be polled, and for example only poll when it's needed, the HA automation system is very flexible for this.
Con: A little more work to setup for the user and if the user abuses it it might break the network but I guess we could issue a warning in case of extensive polling...

Just to know. What are the benefits of this python plugin? Why don't simply help me improve zwavejs2mqtt ?

Just to know. What are the benefits of this python plugin? Why don't simply help me improve zwavejs2mqtt ?

This is just the library we use between Home Assistant and the Websocket server. The Home Assistant integration uses this library to talk to the mesh. I think a lot of users will use Zwave JS 2 MQTT because it includes the server (required for the native HA integration) AND provides a very powerful UI AND allows you to ALSO publish your mesh to mqtt.

For the minimal/basic implementation you can use the builtin parts, which is only the Z-Wave JS server and HA integration (with basic UI to do the standard commands).

I understand but by looking at the roadmap seems like it will be a python version of zwavejs2mqtt in the future, so IMO it's kinda like investing time in something that already exists adn works well for most users.

I don't have anything against it I just saw many (bad) tries in the past with zwave UI implementations, time wasted that if invested in one single project would have been much better ๐Ÿคทโ€โ™‚๏ธ

I understand but by looking at the roadmap seems like it will be a python version of zwavejs2mqtt in the future, so IMO it's kinda like investing time in something that already exists adn works well for most users.

I don't have anything against it I just saw many (bad) tries in the past with zwave UI implementations, time wasted that if invested in one single project would have been much better ๐Ÿคทโ€โ™‚๏ธ

I get your point. The HA configuration UI for Z-Wave will most probably never reach the state where zwavejs2mqtt is now but it also adds additional complexity. A major part of the users never touches that advanced settings and just need some basic stuff. That's so great about this direction, there's a choice. For your example: I'm running Z-Wave JS 2 MQTT myself as my daily driver for the control panel, disabled the mqtt part and enabled the WS server for the HA integration, great!
I know a few others that just want to run the bare minimum Z-Wave JS server and connect HA to it. Take your pick.
Now to only document all those choices properly ;-)

The only thing really confusing is the "2mqtt" part. Maybe we should include your control panel in the Z-Wave JS addon too (without mqtt) to make things more clear ? So have the control panel in it's own project perhaps? Just thinking out loud here btw....

The name 2mqtt is because at start it was a gateway, then it becomes also a control panel and at last I added the hass mqtt discovery.

I'm open to any kind of requests if this could save time to you in this, I can even change the name of the project if needed. I think that it's better to discuss this things now, I like the team work and open to every kind of collaboration ๐Ÿ‘

I undestand your points @marcelveldt and thanks for your explanation

I get your point. The HA configuration UI for Z-Wave will most probably never reach the state where zwavejs2mqtt is now but it also adds additional complexity. A major part of the users never touches that advanced settings and just need some basic stuff. That's so great about this direction, there's a choice. For your example: I'm running Z-Wave JS 2 MQTT myself as my daily driver for the control panel, disabled the mqtt part and enabled the WS server for the HA integration, great!
I know a few others that just want to run the bare minimum Z-Wave JS server and connect HA to it. Take your pick.
Now to only document all those choices properly ;-)

The only thing really confusing is the "2mqtt" part. Maybe we should include your control panel in the Z-Wave JS addon too (without mqtt) to make things more clear ? So have the control panel in it's own project perhaps? Just thinking out loud here btw....

Hear! Hear! I'm on zwave 1.4 and it just works, and is stable for me, but not keeping up with newer features, so I understand the push. I was waiting on OZW beta, but here we are, a year later. I understand why. I'm quite committed to zwave as one can see with my 541 entities (many are disabled). But over the past year I've renamed all of my entities for clarity, consistency, and organization, and have built solid, well-tested automations/scripts, all of which I will have to redo once I convert to this new zwave integration. I don't use MQTT (don't fully understand it), nor do I want to. That was a major sticking point with trying OZW beta and now with zwavejs2mqtt. I'm using HASSOS, so I just want an integration that I can install, like the original one, and manage everything from the UI (although I do tinker with the core repositories at times when necessary). I never really understood the benefit of MQTT sitting in between HA and Zwave, requiring a verbose translation of messages. It just seems like extra work, but I'm probably missing something. I AM looking forward to moving to v1.6, whatever form that takes. But I REALLY only want to go through this conversion ONCE.

zwave

I AM looking forward to moving to v1.6, whatever form that takes. But I REALLY only want to go through this conversion ONCE.

Well, Z-Wave JS != OpenZwave 1.6.
Z-Wave JS is a whole new Z-wave driver, written from scratch, not related to OpenZWave. The new Home Assistant integration for Z-Wave uses that new Z-Wave driver, hence the name Z-Wave JS)

So don't bet your horses on ozw 1.6 but go for Z-Wave JS instead, whenever you think it has the features you need. In the meanwhile the current zwave integration in HA will stay there (but will not be maintained anymore).

Basically two options

@marcelveldt I've just made this saved reply, which I'm probably going to post a lot under issues related to missing auto-polling:

Some legacy devices don't report all their values automatically and require polling to get updated values when controlled manually. In contrast to OZW, zwave-js does not automatically poll devices on a regular basis without user interaction. Polling can quickly lead to network congestion and should be used very sparingly and only where necessary.

zwavejs2mqtt allows you to configure scheduled polling on a per-value basis, which you can use to keep certain values updated. It also allows you to poll individual values on-demand from your automations, which should be preferred over blindly polling all the time if possible.

So in other words: I advise for both your option 2) where possible and option 1) where absolutely necessary. 1) should be configured on a per-value basis (careful with values that share a message)

@AlCalzone thanks for the additional feedback. We'll be working on the docs today to clarify things like this.

@robertsLando in the zwavejs2mqtt control panel there is an option to poll a value. Is that available for non-mqtt users too ? If that's the case, we should advice users to use this perhaps and not reinvent the wheel ?

@robertsLando in the zwavejs2mqtt control panel there is an option to poll a value. Is that available for non-mqtt users too ? If that's the case, we should advice users to use this perhaps and not reinvent the wheel ?

Yep you can select the values you want to poll and add a poll interval for them (in seconds). This can be done from settings > gateway > values table