Devices discovered, but not showing up in MQTT
Closed this issue · 12 comments
Describe the bug
Upon running the Shellies Announce Gen2 automation, we can trace the output of the subsequent Shellies Discovery Gen2 that is being called.
params:
domain: python_script
service: shellies_discovery_gen2
service_data:
id: shelly1pmminig3-5432046c9db0
device_config:
ble:
enable: false
rpc:
enable: true
observer:
enable: false
cloud:
enable: true
server: shelly-100-eu.shelly.cloud:6022/jrpc
input:0:
id: 0
name: null
type: switch
enable: true
invert: false
factory_reset: true
mqtt:
enable: true
server: xx.xx.xx.xx:1883
client_id: shelly1pmminig3-5432046c9db0
user: mqtt
topic_prefix: shelly
rpc_ntf: true
status_ntf: true
use_client_cert: false
enable_rpc: true
enable_control: true
script:1:
id: 1
name: shellies_discovery_gen2_script_20240216
enable: true
...We can see that the shellies are being found.
The log file from MQTT itself, also suggest so:
2024-07-11 00:21:03.748 DEBUG (MainThread) [homeassistant.components.mqtt.client] Mosquitto broker: register write 84
2024-07-11 00:21:03.748 DEBUG (MainThread) [homeassistant.components.mqtt.client] Transmitting message on shelly/rpc: '{"id":1,"src":"shellies_discovery","method":"Shelly.GetConfig"}', mid: 431, qos: 0
2024-07-11 00:21:03.763 DEBUG (MainThread) [homeassistant.components.mqtt.client] Mosquitto broker: unregister write 84
2024-07-11 00:21:03.814 DEBUG (MainThread) [homeassistant.components.mqtt.client] Received message on shellies_discovery/rpc (qos=0): b'{"id":1,"src":"shellyplus1-e465b8f30000","dst":"shellies_discovery","result":{"ble":{"enable":false,"rpc":{"enable":true},"observer":{"enable":false}},"cloud":{"enable":true,"server":"shelly-100-eu.shelly.cloud:6022/jrpc"},"input:0":{"id":0,"name":null,"type":"switch","enable":true,"invert":false,"factory_reset":true},"mqtt":{"enable":true,"server":"xx.xx.xx.xx:1883","client_id":"shellyplus1-e465b8f30000","user":"mqtt","topic_prefix":"shelly","rpc_ntf":true,"status_ntf":true,"use_client_cert":false,"enable_rpc":true,"enable_control":true},"script:1":{"id":1, "name":"shellies_discovery_gen2_script_20240216", "enable":true},"switch:0":{"id":0, "name":null,"in_mode":"follow","initial_state":"match_input", "auto_on":false, "auto_on_delay":60.00, "auto_off":false, "auto_off_delay":60.00},"sys":{"device":{"name":null,"mac":"E465B8F30000","fw_id":"20231031-152227/1.0.7-g5db02bd","discoverable":true,"eco_mode":false,"addon_type":"sensor"},"location":{"tz":null,"lat":null,"lon":null},"debug":{"level":2,"file_level":null,"mqtt":{"enable":false},"websocket":{"enable":false},"udp":{"addr":null}},"ui_data":{},"rpc_udp":{"dst_addr":null,"listen_port":null},"sntp":{"server":"time.google.com"},"cfg_rev":13},"temperature:100":{"id":100, "name":null, "report_thr_C": 0.1, "offset_C": 0.0},"wifi":{"ap":{"ssid":"ShellyPlus1-E465B8F30000","is_open":true, "enable":false, "range_extender": {"enable":false}},"sta":{"ssid":"xxxxxx","is_open":false, "enable":true, "ipv4mode":"dhcp","ip":null,"netmask":null,"gw":null,"nameserver":null},"sta1":{"ssid":null,"is_open":true, "enable":false, "ipv4mode":"dhcp","ip":null,"netmask":null,"gw":null,"nameserver":null},"roam":{"rssi_thr":-80,"interval":60}},"ws":{"enable":false,"server":null,"ssl_ca":"ca.pem"}}}'
2024-07-11 00:21:03.961 DEBUG (MainThread) [homeassistant.components.mqtt.client] Received message on shellies_discovery/rpc (qos=0): b'{"id":1,"src":"shelly1pmminig3-5432046c9db0","dst":"shellies_discovery","result":{"ble":{"enable":false,"rpc":{"enable":true},"observer":{"enable":false}},"cloud":{"enable":true,"server":"shelly-100-eu.shelly.cloud:6022/jrpc"},"input:0":{"id":0,"name":null,"type":"switch","enable":true,"invert":false,"factory_reset":true},"mqtt":{"enable":true,"server":"xx.xx.xx.xx:1883","client_id":"shelly1pmminig3-5432046c9db0","user":"mqtt","topic_prefix":"shelly","rpc_ntf":true,"status_ntf":true,"use_client_cert":false,"enable_rpc":true,"enable_control":true},"script:1":{"id":1, "name":"shellies_discovery_gen2_script_20240216", "enable":true},"switch:0":{"id":0, "name":null,"in_mode":"follow","initial_state":"match_input", "auto_on":false, "auto_on_delay":60.00, "auto_off":false, "auto_off_delay":60.00,"power_limit":2240,"voltage_limit":280,"autorecover_voltage_errors":false,"current_limit":8.000},"sys":{"device":{"name":null,"mac":"5432046C9DB0","fw_id":"20231121-110955/1.1.99-minig3prod1-ga898543","discoverable":true,"eco_mode":false},"location":{"tz":null,"lat":null,"lon":null},"debug":{"level":2,"file_level":null,"mqtt":{"enable":false},"websocket":{"enable":false},"udp":{"addr":null}},"ui_data":{},"rpc_udp":{"dst_addr":null,"listen_port":null},"sntp":{"server":"time.google.com"},"cfg_rev":10},"wifi":{"ap":{"ssid":"Shelly1PMMiniG3-5432046C9DB0","is_open":true, "enable":false, "range_extender": {"enable":false}},"sta":{"ssid":"xxxxxx","is_open":false, "enable":true, "ipv4mode":"dhcp","ip":null,"netmask":null,"gw":null,"nameserver":null},"sta1":{"ssid":null,"is_open":true, "enable":false, "ipv4mode":"dhcp","ip":null,"netmask":null,"gw":null,"nameserver":null},"roam":{"rssi_thr":-80,"interval":60}},"ws":{"enable":false,"server":null,"ssl_ca":"ca.pem"}}}'
2024-07-11 00:21:04.034 DEBUG (MainThread) [homeassistant.components.mqtt.client] Received message on shellies_discovery/rpc (qos=0): b'{"id":1,"src":"shellypro3em-34987a45b1dc","dst":"shellies_discovery","result":{"ble":{"enable":false,"rpc":{"enable":true},"observer":{"enable":false}},"cloud":{"enable":true,"server":"shelly-100-eu.shelly.cloud:6022/jrpc"},"em:0":{"id":0, "name":null,"blink_mode_selector":"active_energy" ,"phase_selector":"all", "monitor_phase_sequence":false,"reverse":{"a":true,"b":true,"c":true}},"emdata:0":{},"eth":{"enable":true,"ipv4mode":"dhcp","ip":null,"netmask":null,"gw":null,"nameserver":null},"modbus":{"enable":true},"mqtt":{"enable":true,"server":"xx.xx.xx.xx:1883","client_id":"shellypro3em-34987a45b1dc","user":"mqtt","ssl_ca":null,"topic_prefix":"shelly","rpc_ntf":true,"status_ntf":true,"use_client_cert":false,"enable_rpc":true,"enable_control":true},"script:1":{"id":1, "name":"shellies_discovery_gen2_script_20240216", "enable":true},"sys":{"device":{"name":null,"mac":"34987A45B1DC","fw_id":"20240625-123006/1.3.3-gbdfd9b3","discoverable":true,"eco_mode":false,"profile":"triphase","addon_type":null},"location":{"tz":"Europe/Copenhagen","lat":55.446700,"lon":11.807600},"debug":{"level":2,"file_level":null,"mqtt":{"enable":false},"websocket":{"enable":false},"udp":{"addr":null}},"ui_data":{},"rpc_udp":{"dst_addr":null,"listen_port":null},"sntp":{"server":"time.google.com"},"cfg_rev":18},"temperature:0":{"id":0, "name":null, "report_thr_C": 5.0, "offset_C": 0.0},"wifi":{"ap":{"ssid":"ShellyPro3EM-34987A45B1DC","is_open":true, "enable":false, "range_extender": {"enable":false}},"sta":{"ssid":"xxxxxx","is_open":false, "enable":true, "ipv4mode":"dhcp","ip":null,"netmask":null,"gw":null,"nameserver":null},"sta1":{"ssid":null,"is_open":true, "enable":false, "ipv4mode":"dhcp","ip":null,"netmask":null,"gw":null,"nameserver":null},"roam":{"rssi_thr":-80,"interval":60}},"ws":{"enable":false,"server":null,"ssl_ca":"ca.pem"}}}'
2024-07-11 00:21:04.175 DEBUG (MainThread) [homeassistant.components.mqtt.client] Mosquitto broker: register write 84
2024-07-11 00:21:04.175 DEBUG (MainThread) [homeassistant.components.mqtt.client] Transmitting retained message on shelly/switch/shellyplus1-e465b8f30000-0/config: '{"name": "Relay 0", "cmd_t": "~rpc", "pl_off": "{\"id\":1,\"src\":\"home-assistant\",\"method\":\"Switch.Set\",\"params\":{\"id\":0,\"on\":false}}", "pl_on": "{\"id\":1,\"src\":\"home-assistant\",\"method\":\"Switch.Set\",\"params\":{\"id\":0,\"on\":true}}", "stat_t": "~status/switch:0", "val_tpl": "{%if value_json.output%}on{%else%}off{%endif%}", "stat_off": "off", "stat_on": "on", "avty": [{"t": "~online", "pl_avail": "true", "pl_not_avail": "false"}, {"t": "~status/rpc", "val_tpl": "{%if value_json.mqtt.connected%}online{%else%}offline{%endif%}"}], "uniq_id": "shellyplus1-e465b8f30000-0", "qos": 0, "dev": {"cns": [["mac", "E4:65:B8:F3:00:00"]], "name": "Shelly Plus 1", "mdl": "Shelly Plus 1", "sw": "20231031-152227/1.0.7-g5db02bd", "hw": "gen2 (SNSW-001X16EU)", "mf": "Allterco Robotics", "cu": "http://shellyplus1-e465b8f30000.local/"}, "o": {"name": "Shellies Discovery Gen2", "sw": "2.30.0", "url": "https://github.com/bieniu/ha-shellies-discovery-gen2"}, "~": "shelly/"}', mid: 432, qos: 0
2024-07-11 00:21:04.179 DEBUG (MainThread) [homeassistant.components.mqtt.client] Mosquitto broker: unregister write 84
2024-07-11 00:21:04.185 DEBUG (MainThread) [homeassistant.components.mqtt.client] Mosquitto broker: register write 84
2024-07-11 00:21:04.187 DEBUG (MainThread) [homeassistant.components.mqtt.client] Transmitting retained message on shelly/light/shellyplus1-e465b8f30000-0/config: '', mid: 433, qos: 0
2024-07-11 00:21:04.192 DEBUG (MainThread) [homeassistant.components.mqtt.client] Mosquitto broker: unregister write 84
2024-07-11 00:21:04.195 DEBUG (MainThread) [homeassistant.components.mqtt.client] Mosquitto broker: register write 84
2024-07-11 00:21:04.196 DEBUG (MainThread) [homeassistant.components.mqtt.client] Transmitting retained message on shelly/fan/shellyplus1-e465b8f30000-0/config: '', mid: 434, qos: 0
2024-07-11 00:21:04.198 DEBUG (MainThread) [homeassistant.components.mqtt.client] Mosquitto broker: unregister write 84
2024-07-11 00:21:04.206 DEBUG (MainThread) [homeassistant.components.mqtt.client] Mosquitto broker: register write 84
2024-07-11 00:21:04.208 DEBUG (MainThread) [homeassistant.components.mqtt.client] Transmitting retained message on shelly/sensor/shellyplus1-e465b8f30000-0-temperature/config: '{"name": "Relay 0 Temperature", "en": "false", "uniq_id": "shellyplus1-e465b8f30000-0-temperature", "qos": 0, "dev": {"cns": [["mac", "E4:65:B8:F3:00:00"]], "name": "Shelly Plus 1", "mdl": "Shelly Plus 1", "sw": "20231031-152227/1.0.7-g5db02bd", "hw": "gen2 (SNSW-001X16EU)", "mf": "Allterco Robotics", "cu": "http://shellyplus1-e465b8f30000.local/"}, "o": {"name": "Shellies Discovery Gen2", "sw": "2.30.0", "url": "https://github.com/bieniu/ha-shellies-discovery-gen2"}, "~": "shelly/", "avty": [{"t": "~online", "pl_avail": "true", "pl_not_avail": "false"}, {"t": "~status/rpc", "val_tpl": "{%if value_json.mqtt.connected%}online{%else%}offline{%endif%}"}], "val_tpl": "{{value_json[\"switch:0\"].temperature.tC}}", "stat_t": "~status/rpc", "unit_of_meas": "°C", "dev_cla": "temperature", "ent_cat": "diagnostic", "stat_cla": "measurement", "sug_dsp_prc": 1}', mid: 435, qos: 0
2024-07-11 00:21:04.210 DEBUG (MainThread) [homeassistant.components.mqtt.client] Mosquitto broker: unregister write 84
2024-07-11 00:21:04.215 DEBUG (MainThread) [homeassistant.components.mqtt.client] Mosquitto broker: register write 84
However, no devices show up under MQTT.
Are we doing something wrong? as far as I can tell, we have followed the instructions in the readme, and have spent a few hours trying to debug it ourselves.
However, we still see no devices within MQTT.
Expected behavior
The 3 shellies to show up in MQTT
Versions:
- Home Assistant: 2024.5.5
- Shellies Discovery GEN2: newest script, fetched today
- Shelly device firmware: 20240625-123006/1.3.3-gbdfd9b3
Shellies Discovery GEN2 automation:
- id: shellies_announce_gen2
alias: "Shellies Announce Gen2"
trigger:
- platform: homeassistant
event: start
variables:
device_info_payload: "{{ {'id': 1, 'src':'shellies_discovery', 'method':'Shelly.GetConfig'} | to_json }}"
device_ids: # enter the list of device IDs (MQTT prefixes) here
- shelly
action:
- repeat:
for_each: "{{ device_ids }}"
sequence:
- service: mqtt.publish
data:
topic: "{{ repeat.item }}/rpc"
payload: "{{ device_info_payload }}"
- id: shellies_discovery_gen2
alias: "Shellies Discovery Gen2"
mode: queued
max: 999
trigger:
- platform: mqtt
topic: shellies_discovery/rpc
action:
- service: python_script.shellies_discovery_gen2
data:
id: "{{ trigger.payload_json.src }}"
device_config: "{{ trigger.payload_json.result }}"
discovery_prefix: "shelly"
- service: mqtt.publish
data:
topic: "{{ trigger.payload_json.result.mqtt.topic_prefix }}/command"
payload: "status_update"Shelly device MQTT settings:
Debug log:
discovery_prefix: "shelly"
I bet this is wrong.
Using shelly as a device ID is also a very bad idea. Device ID should be unique.
Hey,
Alright.
Both of them are listed as prefixes, (device_ids: # enter the list of device IDs (MQTT prefixes) here), which made us hope that we didn't have to list the exact device ids manually.
I will try to remove the discovery prefix, and use unique names, and see if that helps.
That worked. Brilliant. Thanks!
Do you know if we can change the name of the client Id to include spaces and match the name of the device in Shelly?
Currently it gets added with the device type (ie Shelly Pro 3EM) instead of "Solar Panels", which it is called within Shelly.
Do you know if we can change the name of the client Id to include spaces and match the name of the device in Shelly?
Client ID also must be unique, probably it cannot contain spaces, I'm not sure. The best solution is not to change the client ID and leave it as default.
Client ID also must be unique, probably it cannot contain spaces, I'm not sure. The best solution is not to change the client ID and leave it as default.
Perfect, we've done that. We're still not getting the original name from Shelly, but that may not be possible?
It is me being daft, because I am trying to help a friend with HA/Shelly without really having worked with it before.
"Device configuration", does that mean the configuration within shelly, or the configuration of device-ids in the automations.yaml?
We did try changing the second name here "shellypro3em..." to solar panels, but still without luck unfortunately.
"Device configuration", does that mean the configuration within shelly, or the configuration of device-ids in the automations.yaml?
"Device configuration" means the device configuration in the device web panel. I think you configure the device via the Shelly cloud and the configuration looks different there, I don't know, I don't use the cloud. Go to http://<DEVICE_IP>/ and then Settings -> Device name
Awesome!
There is no way to automatically detect every device that connects to MQTT. You have to input the device-ids into the list, manually?
Currently, there is such an option (it was not there when the script was created) but implementing it would be a breaking change for all users, so I don't want to go in this direction.
I see. Thanks for the swift replies. If you end up making a guide or change to allow automatic discovery without whitelisting,can you ping me, if you remember?
Sure