Sensors values not updated / unknown after homeassistant restart

Question

Sensors values not updated / unknown after homeassistant restart

Closed this issue a year ago · 17 comments

Describe the bug
Shelly plus 1 devices announced and discovered via the script.
Sensors, switches, ... are created and I'm able to control the switch from home assistant.
Also the input change is detected.
However, when I restart home assistant, lot's of sensors become unknown... (screenshot below taken 1h after restart of hass)

Devices are sending the data, as I see them coming in on the status/rpc topic for every device every 30s.

Strange thing: If I reboot the shelly device, the sensors seem to be populated with a value, but I don't think it's updating any further...

Expected behavior
Sensors are available after home assistant restart

Versions:

Home Assistant: 2023.9.3
Shellies Discovery GEN2: v2.19.1
Shelly device firmware: 1.0.3

Shellies Discovery GEN2 automation:

###########################################
### Shelly gen2 announce and discovery  ###
###########################################

- id: shelly_gen2_announce
  alias: Shelly gen2 announce
  trigger:
    - platform: homeassistant
      event: start
  variables:
    device_info_payload: "{{ {'id': 1, 'src':'shellies/shellies_discovery', 'method':'Shelly.GetConfig'} | to_json }}"
    device_ids: # enter the list of device IDs (MQTT prefixes) here
      - shellies/shellyplus1/garage_door_1
      - shellies/shellyplus1/garage_door_2
      - shellies/shellyplus1/garage_door_3
  action:
    - repeat:
        for_each: "{{ device_ids }}"
        sequence:
          - service: mqtt.publish
            data:
              topic: "{{ repeat.item }}/rpc"
              payload: "{{ device_info_payload }}"
  mode: single

- id: shelly_gen2_discovery
  alias: Shelly gen2 discovery
  trigger:
    - platform: mqtt
      topic: shellies/shellies_discovery/rpc
  action:
    - service: python_script.shellies_discovery_gen2
      data:
        id: "{{ trigger.payload_json.src }}"
        device_config: "{{ trigger.payload_json.result }}"
        script_prefix: shellies
  mode: queued
  max: 999

Debug log:
https://privatebin.net/?36100cd7034a5a8e#D46adV2uW49WnFKQXqYxdvzGdBAVNw6ErSQGaHgvcfdd

Answer 1 · 2023-10-03T19:19:45.000Z

The states of these entities should be updated when the device sends a status update. It takes a few minutes.

Answer 2 · 2023-10-04T05:09:50.000Z

It doesn't happen... even an hour after the restart, the entities are still ´unknown´...
I see a lot of logs like Pending discovery for ... Could this be the reason of not being updated?

I've also configured the script with the custom mqtt prefixes, as I've changed the mqtt prefixes like this:

Because I want all shellies to be under the same root topic:

Same for the discovery topic, it's also prefixed with 'shellies' so it's get published inside this root topic.

Answer 3 · 2023-10-04T06:50:40.000Z

The binary sensor input state will update when use a physical input connected to the device.
The temperature sensor state will update when you turn on/off the relay.

The device doesn't send those statuses periodically:

shellies/shellyplus1/garage_door_1/status/input:0
shellies/shellyplus1/garage_door_1/status/switch:0

Answer 4 · 2023-10-04T07:05:26.000Z

I'm a bit confused now... What's the purpose then of the installed script on the shelly that pubilishes the device status every 30s on the status/rpc topic?
´json { "ble": {}, "cloud": { "connected": false }, "input:0": { "id": 0, "state": true }, "mqtt": { "connected": true }, "script:1": { "id": 1, "running": true, "mem_used": 1498, "mem_peak": 4648, "mem_free": 17206 }, "script:2": { "id": 2, "running": true, "mem_used": 6496, "mem_peak": 8610, "mem_free": 17206 }, "switch:0": { "id": 0, "source": "init", "output": false, "temperature": { "tC": 45.6, "tF": 114 } }, "sys": { "mac": "441793AB5FFC", "restart_required": false, "time": "09:02", "unixtime": 1696402939, "uptime": 3360, "ram_size": 244656, "ram_free": 142564, "fs_size": 458752, "fs_free": 143360, "cfg_rev": 52, "kvs_rev": 4, "schedule_rev": 0, "webhook_rev": 0, "available_updates": {}, "installed_version": "1.0.3" }, "wifi": { "sta_ip": "192.168.10.51", "status": "got ip", "ssid": "MyWiFi", "rssi": -80 }, "ws": { "connected": false } } ´

It contains all the info about the switch, input, temp, ... so I would expect that every 30s this data is updated in the sensor?

Answer 5 · 2023-10-04T07:23:12.000Z

We cannot use the status/rpc topic for the input/switch state because the device doesn't send this topic as a response when the input/relay state changes. Using this topic would result in a state flip-flop in HA.
status/rpc topic is used for other entities, cloud, update etc.
I need to check if I can use the status/rpc topic for the relay temperature sensor. I'm not sure if this won't cause problems for devices with multiple relays.

Answer 6 · 2023-10-04T07:32:20.000Z

Ok, starting to undestand...
So for switch and input, you depend on it's own topic where data is published when switch and input changes, right?
-> switch:0 and input:0 topics below?

Currently the temp sensor is also only updated when something is published on the switch:0 topic?

Another thing I saw is that the discovery automation is not executed after a hass restart.
I've restarted 39 minutes ago, the announce has run, but the discovery is still 13 hours ago.

Answer 7 · 2023-10-04T08:44:14.000Z

So for switch and input, you depend on it's own topic where data is published when switch and input changes, right?
-> switch:0 and input:0 topics below?

Exactly

Currently the temp sensor is also only updated when something is published on the switch:0 topic?

Yes but I will try to change this.

but the discovery is still 13 hours ago

Discovery automation will start when the device will send Shelly.GetConfig payload to the topic shellies/shellies_discovery/rpc.

Answer 8 · 2023-10-04T11:06:41.000Z

Discovery automation will start when the device will send Shelly.GetConfig payload to the topic shellies/shellies_discovery/rpc.

Any idea why the discovery part is not executed on restart of home assistant? (See one of the screenshots above)
Discovery part should occur right after the annouce, right?
Only when I manually run the announce, also the discovery seems to be executed, not after a restart of home assistant...

And is there any way to get the current value of the relay and the input upon restart?

Answer 9 · 2023-10-04T13:52:27.000Z

Any idea why the discovery part is not executed on restart of home assistant?

The device doesn't send Shelly.GetConfig service or send it to the wrong topic.

And is there any way to get the current value of the relay and the input upon restart?

Out of the box, no. You can try to send Switch.GetStatus and Input.GetStatus request via MQTT or write a Shelly script to send input/switch information periodically.

Answer 10 · 2023-10-04T14:07:45.000Z

The device doesn't send Shelly.GetConfig service or send it to the wrong topic.

How can I check this?

Isn't sending of the Shelly.GetConfig the purpose of the announce automation?
This automations sends a message to the <device_topic_prefix>/rpc to call the Shelly.GetConfig method?
To which topic is this Shelly.GetConfig sent?

- id: shelly_gen2_announce
  alias: Shelly gen2 announce
  trigger:
    - platform: homeassistant
      event: start
  variables:
    device_info_payload: "{{ {'id': 1, 'src':'shellies/shellies_discovery', 'method':'Shelly.GetConfig'} | to_json }}"
    device_ids: # enter the list of device IDs (MQTT prefixes) here
      - shellies/shellyplus1/garage_door_1
      - shellies/shellyplus1/garage_door_2
      - shellies/shellyplus1/garage_door_3
  action:
    - repeat:
        for_each: "{{ device_ids }}"
        sequence:
          - service: mqtt.publish
            data:
              topic: "{{ repeat.item }}/rpc"
              payload: "{{ device_info_payload }}"
  mode: single

Answer 11 · 2023-10-04T14:13:56.000Z

How can I check this?

On MQTT broker (You can use MQTT-Explorer).

Isn't sending of the Shelly.GetConfig the purpose of the announce automation?

Announce automation sends request for Shelly.GetConfig to <device_topic_prefix>/rpc topic and the device should response to the topic shellies/shellies_discovery/rpc.

Answer 12 · 2023-10-04T14:40:33.000Z

I only see the message on shellies/shellies_discovery/rpc when I manually trigger the automation... not after restart...

Answer 13 · 2023-10-04T14:45:48.000Z

Automation trace is your friend :)

Answer 14 · 2023-10-04T16:36:15.000Z

Hmm, I think I'm going crazy...
Just checked it with independant mqtt explorer (not the HA addon) and now it seems to work upon restart:

GetConfig is sent to the devices and they respond on the shellies/shellies_discovery/rpc topic.

Thanks for your great and quick support to get this cleared out. 👍

Answer 15 · 2023-10-04T16:43:53.000Z

Maybe the MQTT integration is not ready when the announce automation is triggered? You can add to this automation wait 30 seconds as the first action.

Answer 16 · 2023-10-04T16:50:32.000Z

Could be something like that...
I have restarted now about 3 times and seems to have executed the discovery every time...
I'll keep an eye on it, if it doesn't work again, I'll add the delay.

So just some final conclusion to be sure I understood it all correctly:

Some sensors are updated every 30s with the script that is put on the shelly
But switch (and temp) and input are only sent/updated when there is a change in their state,

So I have to find a solution to get the current state upon reboot of home assistant...
Otherwise the values for the switch and input remain unknown until they are used.

EDIT: I managed to get it working by publishing a status_update command during discovery.

- id: shelly_gen2_discovery
  alias: Shelly gen2 discovery
  trigger:
    - platform: mqtt
      topic: shellies/shellies_discovery/rpc
  action:
    - service: python_script.shellies_discovery_gen2
      data:
        id: "{{ trigger.payload_json.src }}"
        device_config: "{{ trigger.payload_json.result }}"
        script_prefix: shellies
    # Publish status update upon discovery
    # This requires 'MQTT Control' to be enabled in the MQTT settings on the shelly device
    # See https://shelly-api-docs.shelly.cloud/gen2/ComponentsAndServices/Mqtt#mqtt-control
    - service: mqtt.publish
      data:
        topic: "{{ trigger.payload_json.result.mqtt.topic_prefix }}/command"
        payload: "status_update"
  mode: queued
  max: 999

Thanks for your support!

Answer 17 · 2023-10-06T04:46:23.000Z

Some sensors are updated every 30s with the script that is put on the shelly

But switch (and temp) and input are only sent/updated when there is a change in their state,

Exactly