LYWSD03MMC configuration Error in Z2M
Opened this issue · 73 comments
Since the Z2M update to 1.35.3-1 I get the following configuration-error for all my LYWSD03MMC:
"Failed to configure""TempHumLCD_Schlafzimmer",
"attempt 2 (Error":Read 0xa4c138a130e2c2f3/1 hvacUserInterfaceCfg([
16
],
{
"timeout":10000,
"disableResponse":false,
"disableRecovery":false,
"disableDefaultResponse":true,
"direction":0,
"srcEndpoint":null,
"reservedBits":0,
"manufacturerCode":null,
"transactionSequenceNumber":null,
"writeUndiv":false
}")
failed (Status""UNSUPPORTED_ATTRIBUTE"")
at Endpoint.checkStatus (/app/node_modules/zigbee-herdsman/src/controller/model/endpoint.ts":"319":"28)
at Endpoint.read (/app/node_modules/zigbee-herdsman/src/controller/model/endpoint.ts":"469":"22)
at setupAttributes (/app/node_modules/zigbee-herdsman-converters/src/lib/modernExtend.ts":"63":"13)
at configure (/app/node_modules/zigbee-herdsman-converters/src/lib/modernExtend.ts":"84":"17)
at Object.configure (/app/node_modules/zigbee-herdsman-converters/src/index.ts":"141":"25)
at Configure.configure (/app/lib/extension/configure.ts":"120":"13)
at EventEmitter.wrappedCallback (/app/lib/eventBus.ts":"174":17))
I have already tested this external converter - same problem
There are no problems with this external converter, but all settings are missing (smiley off etc.):
const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const ota = require('zigbee-herdsman-converters/lib/ota');
const e = exposes.presets;
const definition = {
zigbeeModel: ['LYWSD03MMC'],
model: 'LYWSD03MMC',
vendor: 'Xiaomi',
description: 'Temperature & humidity sensor',
fromZigbee: [fz.temperature, fz.humidity, fz.battery],
toZigbee: [],
configure: async (device, coordinatorEndpoint, logger) => {
const endpoint = device.getEndpoint(1);
const bindClusters = ['msTemperatureMeasurement', 'msRelativeHumidity', 'genPowerCfg'];
await reporting.bind(endpoint, coordinatorEndpoint, bindClusters);
await reporting.temperature(endpoint, {min: 10, max: 300, change: 10});
await reporting.humidity(endpoint, {min: 10, max: 300, change: 50});
await reporting.batteryVoltage(endpoint);
await reporting.batteryPercentageRemaining(endpoint);
},
exposes: [e.temperature(), e.humidity(), e.battery()],
ota: ota.zigbeeOTA,
};
module.exports = definition;
There is only this warning:
OTA: Images currently unavailable for device 'LYWSD03MMC', hardwareVersion '16', manufacturerName Xiaomi, {"fieldControl":0,"manufacturerCode":4417,"imageType":522,"fileVersion":18427905}'
I'm sure that all the settings worked until recently even without an external converter.
However, I can't find any issue or report on this problem.
Am I doing something wrong or is it a known problem?
Is there a way to solve this?
Installed Version on devices:
1141-020a-01193001-Z03MMC.zigbee
(until yesterday all devices were on v0.1.1.8 - as the problem was recognized)
Had the same thing after upgrade. Make sure binds look like this:
For me the hvacuserinterfacecfg wasnt bound anymore.
nevermind...
Failed to configure 'test_pvvx', attempt 2 (Error: Read 0xa4c138222d75d9cf/1 hvacUserInterfaceCfg([16],
{"timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":true,"direction":0,"srcEndpoint":null,"
eservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (Status
UNSUPPORTED_ATTRIBUTE') at Endpoint.checkStatus (/app/node_modules/zigbee-
herdsman/src/controller/model/endpoint.ts:319:28) at Endpoint.read (/app/node_modules/zigbee-
herdsman/src/controller/model/endpoint.ts:469:22) at setupAttributes (/app/node_modules/zigbee-herdsman-
converters/src/lib/modernExtend.ts:63:13) at configure (/app/node_modules/zigbee-herdsman-
converters/src/lib/modernExtend.ts:84:17) at Object.configure (/app/node_modules/zigbee-herdsman-
converters/src/index.ts:141:25) at Configure.configure (/app/lib/extension/configure.ts:120:13) at EventEmitter.wrappedCallback
(/app/lib/eventBus.ts:174:17))
For me the hvacuserinterfacecfg wasnt bound anymore.
you are right - hvacuserinterfacecfg wasnt bound at all my LYWSD03MMC-Sensors, too
But that didn't solve the problem with failed config
New ver 0.1.2.0 - zigbeeModel: ['LYWSD03MMC-z']
#50 (comment)
New ver 0.1.2.0 - zigbeeModel: ['LYWSD03MMC-z'] #50 (comment)
Do you want to say there will be a new Version soon, that will fix this Problem?
The devices will have different names.
"-z" will be added to the name Zigbee devices, "-bz" will be added to the BLE & Zigbee devices, "-zb" to the Zigbee2BLE devices.
Z2M identifies a device by name, since it does not know how to poll clusters and identifiers to accurately determine which version it is connected to.
This causes many errors in Z2M, especially with Tuya devices. With the same names, Tuya has different versions of firmware and functions.
And LYWSD03MMC in Z2M gets confused with the devbis version, also due to the fact that it cannot poll the version. Devbis cannot change the device name. :)
Z2M users must write their own scripts. This is how Z2M is designed. Z2M also requires too many resources to operate. That's why I don't support or use Z2M.
thank you for clarification.
just for my understanding:
And LYWSD03MMC in Z2M gets confused with the devbis version, also due to the fact that it cannot poll the version. Devbis cannot change the device name. :)
I realise you're not a friend of Z2M - that's OK.
I don't want to start a discussion about why you prefer ZHA :)
So if I understand you correctly, the external converter from devbis simply doesn't work because we Z2M users have to build our own external converter for your PVVE version?
And the official integration for the LYWSD03MMC from Z2M probably refers to the devbis-FW, which is probably why this is the same as the external converter from it.
Knowing that would be a good approach, then I could deal with it and see if I can get it built such an converter somehow, if I want to stay on your pvvx-FW with Z2M.
I assume you don't have anything ready in the drawer that I just couldn't find?
And the official integration for the LYWSD03MMC from Z2M probably refers to the devbis-FW
Yes.
I'm on ZHA, devbis on Z2M.
In ZHA the name is indifferent.
But I alone cannot support all types of Zigbee applications and write scripts for them... At the moment, there are already 20 devices with different BLE/Zigbee functions - different supported firmware...
I realise you're not a friend of Z2M - that's OK.
I don't want to start a discussion about why you prefer ZHA :)
Preference has nothing to do with it.
I have the same problem. A few days ago I did a fw update to the zigbee version and everything was OK. I updated another same device today but it shows as unsupported with "-z". And I see that what works has a slightly older fw, see picture. Does anyone have any ideas how to fix this when the bluetooth is not working? I don't have the option to flash it somehow via cables. And I have 3 more thermometers that I wanted to convert to zigbee as well, but I don't have the courage now and I don't want to turn them into another useless device.
I uploaded the fw via this page https://pvvx.github.io/atc_mithermometer/telinkmiflasher.html
Sorry, I'm a beginner. Will it be fixed somehow in cooperation with zigbee2mqtt and it will start working again on its own, or is it necessary to reset something somewhere? If so, what exactly? Thank you.
thank you for clarification. just for my understanding:
And LYWSD03MMC in Z2M gets confused with the devbis version, also due to the fact that it cannot poll the version. Devbis cannot change the device name. :)
I realise you're not a friend of Z2M - that's OK. I don't want to start a discussion about why you prefer ZHA :) So if I understand you correctly, the external converter from devbis simply doesn't work because we Z2M users have to build our own external converter for your PVVE version? And the official integration for the LYWSD03MMC from Z2M probably refers to the devbis-FW, which is probably why this is the same as the external converter from it.
Knowing that would be a good approach, then I could deal with it and see if I can get it built such an converter somehow, if I want to stay on your pvvx-FW with Z2M. I assume you don't have anything ready in the drawer that I just couldn't find?
Did you manage to get this to work? Is there any guide to adding a new custom converter to Z2M for us newbies? I have half of all my thermomethers with fw 119 working fine and the other hald with fw 120 not working at all... :(
So if I understand you correctly, the external converter from devbis simply doesn't work because we Z2M users have to build our own external converter for your PVVE version?
And the official integration for the LYWSD03MMC from Z2M probably refers to the devbis-FW, which is probably why this is the same as the external converter from it.
I just took devbis converter and changed the name to -z in it.
Seems to not work with special features like turnoff smiley or display, but otherwise fine.
Might look in to how to flash devbis firmware using OTA, but for now this is good enough.
I don't want to start religious discussion either, but I don't understand the augment about Z2M using too many resources.
On my system it is at 0.0% cpu and 0.2% memory (of my 2GB), for the whole container...
Might look in to how to flash devbis firmware using OTA, but for now this is good enough.
python3 zb_bin_ota.py z03mmc.bin z03mmc -m0x1141 -i0x020a -v0x09993001 -s"Zigbee ver: ZigbeeTLc to devis"
https://github.com/pvvx/ZigbeeTLc/tree/master/bin/devbis
Added to TelinkMiFlasher.html :
Warning: The devbis version consumes twice as much as ZigbeeTlc!
I don't want to start religious discussion either, but I don't understand the augment about Z2M using too many resources.
On my system it is at 0.0% cpu and 0.2% memory (of my 2GB), for the whole container...
You forgot MQTT and stuff. 200 MB RAM minimum
With 2 GB of RAM, Home Assistant does not work well. The database does not fit into the cache.
For comfortable operation, the media speed (PCIe NVME) must be at least 1 GB/s.
So far I have solved it like this https://www.reddit.com/r/homeassistant/comments/1aok4l4/help_im_stuck_with_an_unknown_zigbee_device/ . Apparently my posts were published "Hairy-Plum-3118", so you need to be logged in.
@TreborR2 - The device names have been renamed because the devbis converter (at https://github.com/devbis/z03mmc/blob/master/converters/lywsd03mmc.js) is not compatible with the firmware from the ZigbeeTlc repository.
I don't understand your message. Does that mean you don't recommend this solution for some reason? Is not working? Or why? For now, it's blaming me for the data, see picture.
Might look in to how to flash devbis firmware using OTA, but for now this is good enough.
python3 zb_bin_ota.py z03mmc.bin z03mmc -m0x1141 -i0x020a -v0x09993001 -s"Zigbee ver: ZigbeeTLc to devis"https://github.com/pvvx/ZigbeeTLc/tree/master/bin/devbis
Added to TelinkMiFlasher.html :
Sorry I'm going to need my hand held here. Is the python script above something I run from the command line on the box with my zigbee coordinator in it and if so how does it know which device to actually update as I have a few and only one is not working or is it something incorporated into the TelinkMiFlasher which I understands needs Bluetooth to connect and the device I want to flash no longer has Bluetooth available.
What I want to achieve is to get one device I flashed OTA that is now zigbee only and now appearing with a -z suffix back to the state that Z2M recognises it. If that's using the devis firmware that I'm all for it.
Disable other thermometers during Zigbee OTA. Remove the battery. Then delete the Zigbee OTA file and reboot the system.
If that's using the devis firmware that I'm all for it.
Update all thermometers to devbis version. The batteries in Devbis will last up to 6 months.
Disable other thermometers during Zigbee OTA. Remove the battery. Then delete the Zigbee OTA file and reboot the system.
If that's using the devis firmware that I'm all for it.
Update all thermometers to devbis version. The batteries in Devbis will last up to 6 months.
So
1 copy python script to folder on linux box hosting z2m which has sonos stick plugged into it
2 copy z03mmc.bin to same folder
3 execute python3 zb_bin_ota.py z03mmc.bin z03mmc -m0x1141 -i0x020a -v0x09993001 -s"Zigbee ver: ZigbeeTLc to devis"
4 reboot my linux box
Sorry I'm really lost here Is that correct ?
Is that correct ?
No. In Zigbee2MQTT OTA is more difficult to do.
Like this:
#43 (comment)
but you need to edit the links and numbers in accordance with the new OTA file.
In ZHA you need to copy the *.zigbee file to the directory for OTA and enable OTA.
I don't understand your message. Does that mean you don't recommend this solution for some reason? Is not working? Or why?
The "Calibration" and "comfort" settings do not match the ZigbeeTlc version.
I'm still puzzled by the discussion about the name - I don't really understand something here.
I have updated one of my 3 thermometers to 0.1.2.0.
All 3 still show up as LYWSD03MMC, not as LYWSD03MMC-z,
If I connect it via the following external converter, everything works without errors and can be configured.
So this part of the script seems to be correct.
Of course, the settings for temp calibration, smiley and so on are missing here
const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const ota = require('zigbee-herdsman-converters/lib/ota');
const e = exposes.presets;
const definition = {
zigbeeModel: ['LYWSD03MMC'],
model: 'LYWSD03MMC',
vendor: 'Xiaomi',
description: 'Temperature & humidity sensor',
fromZigbee: [fz.temperature, fz.humidity, fz.battery],
toZigbee: [],
configure: async (device, coordinatorEndpoint, logger) => {
const endpoint = device.getEndpoint(1);
const bindClusters = ['msTemperatureMeasurement', 'msRelativeHumidity', 'genPowerCfg'];
await reporting.bind(endpoint, coordinatorEndpoint, bindClusters);
await reporting.temperature(endpoint, {min: 10, max: 300, change: 10});
await reporting.humidity(endpoint, {min: 10, max: 300, change: 50});
await reporting.batteryVoltage(endpoint);
await reporting.batteryPercentageRemaining(endpoint);
},
exposes: [e.temperature(), e.humidity(), e.battery()],
ota: ota.zigbeeOTA,
};
The current external converter vor devbis-FW runs into our known problems - that is clear
Well, that's 147 very manageable lines of code, the last part of which is no problem.
If I understand everything correctly, we would only have to use this converter as a template and adapt it to the corresponding variables/parameters/attributes of the pvvx-FW.
Then all Z2M users would be served by simply using this converter.
The advantage for all of us would be that we would benefit from the more energy-efficient pvvx-FW.
Do I understand that correctly?
Let's make an example:
@pvvx
how or where can we find the correct attribut for this part in your sourcecode?
binary({
name: 'show_smiley',
valueOn: ['SHOW', 1],
valueOff: ['HIDE', 0],
cluster: 'hvacUserInterfaceCfg',
attribute: {ID: 0x0010, type: dataType.boolean},
description: 'Whether to show a smiley on the device screen.',
})
Maybe the ID is the only thing to change?!
I think if we tackle this all together, everyone will benefit from making this firmware even more popular :)
Maybe the ID is the only thing to change?!
ZigBee Cluster Library Specification -> 2.6.2 Data Types -> Table 2-10. Data Types
Setting offsets for temperature and humidity
Switch temperature between Celsius and Fahrenheit, Show smiley, Comfort parameters
Finally got some time to look in to this.
Might look in to how to flash devbis firmware using OTA, but for now this is good enough.
python3 zb_bin_ota.py z03mmc.bin z03mmc -m0x1141 -i0x020a -v0x09993001 -s"Zigbee ver: ZigbeeTLc to devis"https://github.com/pvvx/ZigbeeTLc/tree/master/bin/devbis
Added to TelinkMiFlasher.html :
Warning: The devbis version consumes twice as much as ZigbeeTlc!
Thank you so much for your help 👍
But I have run in to a couple of problems:
- Your python script seems to add some gibberish at the begging of the firmware file (just removed in editor)
- Zigbee2MQTT complain about some memory mapping:
"Update of '0xa4a1281ad787cb93' failed (Attempt to access memory outside buffer bounds)"
I don't want to start religious discussion either, but I don't understand the augment about Z2M using too many resources.
On my system it is at 0.0% cpu and 0.2% memory (of my 2GB), for the whole container...You forgot MQTT and stuff. 200 MB RAM minimum With 2 GB of RAM, Home Assistant does not work well. The database does not fit into the cache. For comfortable operation, the media speed (PCIe NVME) must be at least 1 GB/s.
Ahh sorry I didn't look the at the right place.
My Zigbee2MQTT node.js process is consuming 6.5% of the RAM, but HA is using 25%. So still rather insignificant all in all.
I'm using MQTT for many other things besides Z2M, but it is only using 0.6% of the RAM.
My HA is rock stable and super fast running on the 2GB Proxmox is allocating to it. It is not actively using any swap, and it only has 650MB swap anyway:
# free -h
total used free shared buff/cache available
Mem: 1.9Gi 1.0Gi 79Mi 3.0Mi 837Mi 883Mi
Swap: 650Mi 331Mi 319Mi
zwave is using 3.1% RAM, and I only have 2 zwave devices (as compared with 41 zigbee.
Don't you think ZHA is about just as demanding, only not as visible because it buried in the rest of HA python processes?
Maybe the ID is the only thing to change?!
ZigBee Cluster Library Specification -> 2.6.2 Data Types -> Table 2-10. Data Types
Setting offsets for temperature and humidity Switch temperature between Celsius and Fahrenheit, Show smiley, Comfort parameters
That is awesome.
I have changed this to get the smiley working again:
binary({
name: 'show_smiley',
valueOn: ['SHOW', 0],
valueOff: ['HIDE', 1],
cluster: 'hvacUserInterfaceCfg',
attribute: 'programmingVisibility',
description: 'Whether to show a smiley on the device screen.',
}),
havn't looked at calibration yet, but it seems to pretty straight forward.
available 883Mi
Consequently, assistant.db no longer fits into the RAM buffers. When you set to display the graph for a year, it will slow down mercilessly...
free -h
total used free shared buff/cache available
Mem: 3.6Gi 590Mi 2.7Gi 5.6Mi 441Mi 3.0Gi
Swap: 1.8Gi 0B 1.8Gi
hdparm -tT /dev/nvme0n1
Timing cached reads: 2062 MB in 2.00 seconds = 1031.08 MB/sec
Timing buffered disk reads: 1198 MB in 3.00 seconds = 399.29 MB/sec
NanoPi_R5S
home-assistant_v2.db 3.5G
@pvvx :
thank you very much for this info!
I have changed this to get the smiley working again:
binary({ name: 'show_smiley', valueOn: ['SHOW', 0], valueOff: ['HIDE', 1], cluster: 'hvacUserInterfaceCfg', attribute: 'programmingVisibility', description: 'Whether to show a smiley on the device screen.', }),
hmm...
why:
attribute: 'programmingVisibility'
?
devbis:
attribute: {ID: 0x0010, type: dataType.boolean},
ZHA-Settings from pvvx say ENUM8 (id:0x30)
so why not:
attribute: {ID: 0x30, type: dataType.boolean}
or at least:
attribute: 'ScheduleProgrammingVisibility'
?
How do you come up with 'programmingVisibility'?
I'm still puzzled by the discussion about the name - I don't really understand something here.
I have updated one of my 3 thermometers to 0.1.2.0. All 3 still show up as LYWSD03MMC, not as LYWSD03MMC-z,
If I connect it via the following external converter, everything works without errors and can be configured. So this part of the script seems to be correct. Of course, the settings for temp calibration, smiley and so on are missing here
const fz = require('zigbee-herdsman-converters/converters/fromZigbee'); const exposes = require('zigbee-herdsman-converters/lib/exposes'); const reporting = require('zigbee-herdsman-converters/lib/reporting'); const ota = require('zigbee-herdsman-converters/lib/ota'); const e = exposes.presets; const definition = { zigbeeModel: ['LYWSD03MMC'], model: 'LYWSD03MMC', vendor: 'Xiaomi', description: 'Temperature & humidity sensor', fromZigbee: [fz.temperature, fz.humidity, fz.battery], toZigbee: [], configure: async (device, coordinatorEndpoint, logger) => { const endpoint = device.getEndpoint(1); const bindClusters = ['msTemperatureMeasurement', 'msRelativeHumidity', 'genPowerCfg']; await reporting.bind(endpoint, coordinatorEndpoint, bindClusters); await reporting.temperature(endpoint, {min: 10, max: 300, change: 10}); await reporting.humidity(endpoint, {min: 10, max: 300, change: 50}); await reporting.batteryVoltage(endpoint); await reporting.batteryPercentageRemaining(endpoint); }, exposes: [e.temperature(), e.humidity(), e.battery()], ota: ota.zigbeeOTA, };The current external converter vor devbis-FW runs into our known problems - that is clear
Well, that's 147 very manageable lines of code, the last part of which is no problem.
If I understand everything correctly, we would only have to use this converter as a template and adapt it to the corresponding variables/parameters/attributes of the pvvx-FW. Then all Z2M users would be served by simply using this converter. The advantage for all of us would be that we would benefit from the more energy-efficient pvvx-FW.
Do I understand that correctly?
Let's make an example: @pvvx how or where can we find the correct attribut for this part in your sourcecode?
binary({ name: 'show_smiley', valueOn: ['SHOW', 1], valueOff: ['HIDE', 0], cluster: 'hvacUserInterfaceCfg', attribute: {ID: 0x0010, type: dataType.boolean}, description: 'Whether to show a smiley on the device screen.', })Maybe the ID is the only thing to change?! I think if we tackle this all together, everyone will benefit from making this firmware even more popular :)
How is this possible? And are you really using zigbee2mqtt or ZHA
How is this possible? And are you really using zigbee2mqtt or ZHA
That is what I don't understand
Maybe because my devices were already paired?
definetly Z2M:
And are you using the conversion file? If Z2M fix it??? And I still ask, perhaps stupidly, what good is the conversion file if the thermometer has the right name and the Z2M recognizes it? The picture shows the thermometer with the functional FW, where I don't use the conversion file, and I have all the setting options there. On the other hand, I don't see some of these options with the new FW and converter???
Otherwise, I tried putting 3 thermometers in one place (1. with original BT FW, 2. with older FW without conversion file and 3. with new FW and preview file) and the temperature and humidity data were almost identical. Just for fun.
I think I got it.
Here I have modified the devbis version of the ext. converter so that it works with Z2M for the pvvx-FW.
I have tested everything so far and could not find any errors.
Open questions:
- Why is the -z still missing on my devices?
I can't explain that.
@pvvx do you have any ideas?
Perhaps lines 29 and 30 still need to be adjusted for other users who want to use this converter
I left a lot of comments in to have a comparison to the devbis version - these can all still be removed
- line 51 VS line 55:
attribute: {ID: 0x0000, type: dataType.enum8},
attribute: 'tempDisplayMode',
I don't understand where to get the attributes like 'tempDisplayMode'
both works fine, the first one with ID fits better to the publications of pvvx, could therefore be adapted more easily
- Line 77 ff. enable_display:
couldn't find the right attribute for pvvx-FW
@pvvx is "display off" implemented in your FW? (do you know the attribute?)
"display off" implemented in your FW? (do you know the attribute?)
There is no display shutdown.
The devbis implementation of "Display off" does not turn off the LCD, but rather displays a blank screen. This does not reduce battery consumption.
No documentation found for all display chip variants...
Thanks for the clarification.
Then this option really makes no sense and can be omitted.
(I would actually have thought that an LCD display that doesn't show anything would also require less energy...but anyway, if you want sensors without a display, the Aquara is a better option anyway).
so here is "my" converter with everything cleaned up so far.
only open questions are line 29/30
zigbeeModel: ['LYWSD03MMC'],
model: 'LYWSD03MMC',
[Edit] updated version below
(I would actually have thought that an LCD display that doesn't show anything would also require less energy...but anyway, if you want sensors without a display, the Aquara is a better option anyway).
TS0201_TZ3000 + soon TH03Z, ...
finally got it.
I had to remove all of my three Devices and then I started a fresh Interview.
Now all devices have the -z extension.
It seems that Z2M somehow identify the devices, even when they get a new zigbee-name with an OTA-update. ... I'm not sure what to make of this
So this is the final converter, that should work well with v.0.1.2.0 for every new paired devices.
Thanks a lot for your support @pvvx !
Is that suitable?
https://github.com/pvvx/ZigbeeTLc/tree/master/z2m
Consequently, assistant.db no longer fits into the RAM buffers. When you set to display the graph for a year, it will slow down mercilessly...
free -h total used free shared buff/cache available Mem: 3.6Gi 590Mi 2.7Gi 5.6Mi 441Mi 3.0Gi Swap: 1.8Gi 0B 1.8Gi
That's a good point!
My db is currently only 420MB. I only have about half a year of data, can't remember why, but at some point I had to wipe it.
But reading in that half year is snappy and dosen't seem to consume much memory.
But you got me thinking. I will keep an eye on it when my db grows.
hdparm -tT /dev/nvme0n1 Timing cached reads: 2062 MB in 2.00 seconds = 1031.08 MB/sec Timing buffered disk reads: 1198 MB in 3.00 seconds = 399.29 MB/secNanoPi_R5S
home-assistant_v2.db 3.5G
hdparm does not work in the HA VM for some reason, but directly on the Proxmox OS it is:
root@pve:~# hdparm -tT /dev/sda
/dev/sda:
Timing cached reads: 26538 MB in 1.98 seconds = 13389.94 MB/sec
Timing buffered disk reads: 1576 MB in 3.00 seconds = 524.85 MB/sec
hmm... why:
attribute: 'programmingVisibility'
?
devbis:
attribute: {ID: 0x0010, type: dataType.boolean},
ZHA-Settings from pvvx say
ENUM8 (id:0x30)so why not:
attribute: {ID: 0x30, type: dataType.boolean}
or at least:
attribute: 'ScheduleProgrammingVisibility'
? How do you come up with 'programmingVisibility'?
I didn't actually notice the difference.
programmingVisibility I got from the Development console in Z2M:
Don't why it's called like that, but it works...
Zigbee Alliance Cluster Library Specification, Revision 6..8
https://github.com/Koenkk/zigbee-herdsman/blob/master/docs/07-5123-08-Zigbee-Cluster-Library.pdf
6.6.2.1 Attributes
The attributes of this cluster are summarized in Table 6-49.
and Table 2-11. Data Types
My db is currently only 420MB. I only have about half a year of data, can't remember why, but at some point I had to wipe it.
I have a base depth limit set to 1 month.
Next, HA performs data averaging.
Rewriting (trimming) of the database in HA occurs every day. This causes all 3.5Gb to be overwritten.
Zigbee Alliance Cluster Library Specification, Revision 6..8 https://github.com/Koenkk/zigbee-herdsman/blob/master/docs/07-5123-08-Zigbee-Cluster-Library.pdf
6.6.2.1 Attributes The attributes of this cluster are summarized in Table 6-49.
and Table 2-11. Data Types
Yes I see. Funny why Koenkk has decided not to use the full name...
My db is currently only 420MB. I only have about half a year of data, can't remember why, but at some point I had to wipe it.
I have a base depth limit set to 1 month. Next, HA performs data averaging. Rewriting (trimming) of the database in HA occurs every day. This causes all 3.5Gb to be overwritten.
Okay it sounds like you have spend whole lot more time tuning your recorder. Honestly I haven't touched a thing.
But I do sync some of the data to InfluxDB though.
finally got it. I had to remove all of my three Devices and then I started a fresh Interview. Now all devices have the -z extension. It seems that Z2M somehow identify the devices, even when they get a new zigbee-name with an OTA-update. > ... I'm not sure what to make of this
Could it be related to this in the 1.35 release:
https://github.com/Koenkk/zigbee2mqtt/releases/tag/1.35.0
This release contains a quite fundamental change on how Zigbee2MQTT handles unsupported devices.
Zigbee2MQTT will now attempt to automatically detect features supported by the device and discover those to e.g. the frontend and Home Assistant.
This means that if you, for example pair an unsupported light, there is no need to wait for official support. Basic features will work out-of-the-box.
If you read the modelID from the dev console after update to 0.1.2.0 and restart with the new converter devices will be updated and don't have to be readded.
If the functionality has changed after the OTA, the Zigbee coordinator does not know this. In the Zigbee coordinator, the device is registered by MAC address.
I didn't actually notice the difference.
programmingVisibility I got from the Development console in Z2M
And I didn't know the dev-console yet, thank you :D
Zigbee Alliance Cluster Library Specification, Revision 6..8
That's exactly what I meant.
In the docu it's called 'ScheduleProgrammingVisibility'
But in the converter you should use only 'ProgrammingVisibility'
So I did not find a docu where the correct atrribute-names for the converters are listed.
That is why I just used the hex-IDs as in the official docu (or in your examples for ZHA :) )
If you read the modelID from the dev console after update to 0.1.2.0 and restart with the new converter devices will be updated and don't have to be readded.
Didn't Work for me, nor restart of Z2M than the complete host
In the Zigbee coordinator, the device is registered by MAC address.
That would explain everything, I almost thought so.
Aktualizoval jsem jeden ze svých 3 teploměrů na 0.1.2.0. Všechny 3 se stále zobrazují jako LYWSD03MMC, ne jako LYWSD03MMC-z
- Thank you for the conversion file! Works great with the new FW!
- I have a question about the update to the latest FW. If you did the update from 0119 to 0120 and via OTA and via Z2M? Could you write the exact procedure how to do this? I'm not entirely sure how to do this and would hate to brick the device.
When I try to update via Z2M now, it fails and reports an error, see image, but if I understand correctly, this is "normal" behavior?
Can anyone please tell me or make a short summary what to do if I have a Z2M and thermometer with error "Exception converter: LYWSD03MMC-z has no endpoint"? OTA isn't possible in Z2M now.
that's my OTA-config:
make sure that you store the files in the "zigbee2mqtt" subfolder (not directly next to the HA-config-File)
What a nightmare in Z2M...
Well, it was easier to write everything from scratch in C... :)
OK guys, but I don't have an "update" button on this thermometer. In the OTA panel there's no this thermometer.
@pvvx maybe it's a nightmare, but in ZHA half of my devices aren't working ;)
So I set it up according to your instructions (I also tried force false) and it gives me this error, see picture. I tried searching on Google and couldn't find anything interesting. Anyone have any ideas or advice?
@pvvx maybe it's a nightmare, but in ZHA half of my devices aren't working ;)
With Zigbee this is true everywhere. Therefore, for a long time I did not want to make firmware for thermometers on Zigbee.
@TreborR2
I notice that the device with V0.1.1.9 is recognised as a "Custom Device (DIY)".
This indicates that the z2m-devbis-converter has been loaded here.
Try the following:
Additionally save this external converter in your z2m folder (name it "lywsd03mmc.js")
Then add this converter to your Z2M config
After that reboot your Z2M.
Your "Teplomer-Koupelna" should now show "Xiaomi_TEST" as vendor. if not so -> reconfigure this device (yellow button)
If everything up to this point is correct, Z2M is already using a converter that is not completely wrong for the pvvx-FW
(but has only minimal functionality)
Than try to do a OTA-Update
When everything is fine, you can delete this converter and remove it from Z2M config
@pvvx maybe it's a nightmare, but in ZHA half of my devices aren't working ;)
With Zigbee this is true everywhere. Therefore, for a long time I did not want to make firmware for thermometers on Zigbee.
So put the firmware file in the dir, point at it in a json file the same place, and add the json file in Z2M GUI…?
Just read up on ZHA, looks not much different 🤷♂️
So I set it up according to your instructions (I also tried force false) and it gives me this error, see picture. I tried searching on Google and couldn't find anything interesting. Anyone have any ideas or advice?
I got the same when I tried to go the other way: ie. from ZigbeeTLC/pvvx firmware to devbis.
I guess the to firmware images are not compatible with each other?
Guess that's why pvvx explains how to create a conversion file here: #58 (comment)
Based on the LYWSD03MMC external converter for Zigbee2MQTT posted above, here is one for the CGDK2 (Qingping Temp & RH Monitor Lite)
Since the CGDK2 does not have Smiley or Comfort settings, these are removed from the convertor.
ZCGDK2 external convertor:
ZCGDK2.js.txt
@TreborR2 I notice that the device with V0.1.1.9 is recognised as a "Custom Device (DIY)". This indicates that the z2m-devbis-converter has been loaded here.
Try the following: Additionally save this external converter in your z2m folder (name it "lywsd03mmc.js") Then add this converter to your Z2M config
After that reboot your Z2M. Your "Teplomer-Koupelna" should now show "Xiaomi_TEST" as vendor. if not so -> reconfigure this device (yellow button) If everything up to this point is correct, Z2M is already using a converter that is not completely wrong for the pvvx-FW (but has only minimal functionality)
Than try to do a OTA-Update
When everything is fine, you can delete this converter and remove it from Z2M config
I'm only responding now because I was away. Thank you for answer. I did what you wrote and I think it shows what you think it should show?
But to be honest, I don't quite understand now, what did we verify / find out with it? Even with this conversion file it gives the same error "Failed to check if update available for 'Teploměr-Koupelna' (Attempt to access memory outside buffer bounds)" when I try to OTA update the FW? Any advice? I admit that I am not completely knowledgeable about this. Thanks in advance for any further help.
@Bodengriller
What exactly was the conversion file supposed to test? Because I deleted it and removed it from the Z2M settings as well and that device is still showing me as "Xiaomi_TEST". I tried resetting the button several times and restarted the whole system.
I have one more question. Do you also find that the newer FW 3001-0120 drains the battery faster than the older 3001-0119? I started using the one with the newer FW later than the older FW and it shows that the one with the newer one already has 81.5% and the other one still has 100%.
And the strength in the zigbee signal is not that different to be the reason???
I'm just asking? Maybe there was a weak battery or another problem?
What exactly was the conversion file supposed to test? Because I deleted it and removed it from the Z2M settings as well and that device is still showing me as "Xiaomi_TEST". I tried resetting the button several times and restarted the whole system.
My thought was that your OTA update might fail because the device is configured with the wrong converter (Custom Device DIY is definitely the Z2M converter for the devbis firmware, which is stored at Z2M, but does not work correctly for the pvvx firmware)
So that you can distinguish it better, I have changed the name in the converter to "xiaomi_test".
However, since the OTA update does not work for you even with the stripped-down, functioning converter, it was probably not due to this.
After you have deleted the converter, you must reconfigure the device (yellow button) - then the completely wrong converter from Z2M should be loaded again
I've no clue why you cannot update OTA, sorry
I have one more question. Do you also find that the newer FW 3001-0120 drains the battery faster than the older 3001-0119? I started using the one with the newer FW later than the older FW and it shows that the one with the newer one already has 81.5% and the other one still has 100%.
I can't make a judgement on this, all the updates and gimmicks have almost drained my batteries :D
In any case, an update consumes significantly more power than normal operation
Do you also find that the newer FW 3001-0120 drains the battery faster than the older 3001-0119? I started using the one with the newer FW later than the older FW and it shows that the one with the newer one already has 81.5% and the other one still has 100%.
Keep in mind that the OTA can drain your battery a lot already.
Also I noticed that some of my devices used wrong report intervals, so I'd advice to occassionally reconfigure the thermometer through zigbee2mqtt (yellow reconfigure / refresh button at bottom in "About" tab of device).
I'd definitely do this after pairing / reflashing.
Therefore I'm still not 100% confident that this firmware stores the configuration (such as report intervals!) correctly in flash because I also noticed different battery drain and report intervals on some of my devices - usually resolved by manually reconfiguring them.
There was a similar issue with the smiley in #47. For the other settings we appear to trust the telink SDK to store/load the intervals etc.
I too was using LYWSD03MMC with ZigbeeTLc <= 0.1.1.9, updated to 0.1.2.0 using the OTA suggested by z2m, and ran into the UNSUPPORTED_ATTRIBUTE error.
I wanted to stay on ZigbeeTLc rather than devbis (mostly for battery life) but was unsure whether the z2m OTA had updated my devices to the latter. A major source of confusion for me was that z2m kept reporting the model as LYWSD03MMC and not as LYWSD03MMC-z, even after deleting all my affected devices from z2m and attempting to re-pair them.
FWIW, what finally worked for me was adding the custom converter from here, but twice (one as-is, and a second copy edited without the -z suffix). After that, z2m finally reported model as LYWSD03MMC-z and all affected devices re-paired successfully. Once that was done, I removed the second copy of the converter leaving only the one with the -z and restarted z2m one last time.
@Nephiel I have the same issue, that the device with the new firmware is still recognized as the one without -z. I tried to use the external converters twice, but unfortunately even the converter without -z is used.
I have done the following:
- added both external converters
- restarted z2m
- removed the current device
- activated pairing mode by removing the battery
- re-paired in z2m
- device is still listed without -z and the external converter file without -z isn't used either
Do you have any further hints? Thanks!
A similar problem exists in ZHA. After updating, the device name is not updated. Removing and re-pairing does not help.
Although the device name changes.
Something is not completed in the Home Assistant software with Z2M and ZHA.
But in ZHA, the device name has no effect. The device operates according to the Zigbee 3.0 standard and does not require converters or “quirks” for basic operation.
Converters or "quirks" are required solely to handle non-Zigbee 3.0 device functions.
In this case, non-standard functions are setting up a smiley face and shifting the measurements of the temperature and humidity sensor. But they are available in the “Services” menu or configured in the “yaml” of Home Assistant.
I'm using FHEM, not Home Assistant, but the issue is within zigbee2mqtt I think. The model is not updated. I checked the data directory where z2m saves all data, but even there the device was removed completely before re-pairing. I don't know where z2m gets the wrong model when pairing a new device. Another z2m installation, which hasn't been paired with any LYWSD03MMC devices before, works.
This means that the command to remove the device from the list has not been sent to the Zigbee coordinator. And old data is used.
After updating the firmware, the Zigbee 3.0 device may change functionality. For example, expand. But this is not tracked in z2m. This greatly limits the expansion of the device's functionality.
In z2m, users are tied to a list of supported devices approved by z2m programmers. Everything depends only on the wishes of the z2m developers.
It was recently announced that support for the Zigbee 3.0 standard will be implemented. But I don’t know how things are in z2m.
PS: Similar to BLE, it typically takes open source writers 10 years to implement a new standard. Linux still does not support Bluetooth 5.0, which was released in 2016.
The coordinator information is also stored in the data directory of z2m. I don't know of any other place.
Just one thing to mention regarding the external converter. Since 1.36.1 (released a few hours ago) the batteryPercentage function isn't implemented any more https://github.com/pvvx/ZigbeeTLc/blob/master/z2m/lywsd03mmc-z.js#L8
battery({percentage: true}) needs to be called instead
I can't help you with anything other than corrections or additions to the thermometer firmware. There is no such opportunity to monitor and participate in writing the code for each user application.
And additions to old products usually take a long time due to previously incorrectly chosen algorithms and other conditions. This time is significantly reduced if someone creates a new product. Often, an old product simply cannot be adapted to new formats...
There is always a workaround - you can introduce a function to change the MAC device. But this will be a “plug” for the sake of compatibility with outdated software and will not add functionality to the device, and may create new problems.
You can also introduce the ability for the user to change the name and manufacturer of the device. Then z2m will be a mess and fixes will be needed faster :) But this is a more popular feature among users, as it will allow you to tag devices with a unique name for any application.
In general, the software for Zigbee is very crude and unfinished.
Example - a device for relaying BLE thermometers in Zigbee has 3 temperatures, 3 humidity, 3 batteries. According to Zigbee 3.0 this is described as 3 devices on different endpoints.
ZHA thinks and creates 3 temperatures, 3 humidity, but 1 battery. But there is no way to distinguish which endpoint number this temperature is from. The assignment of the sensor name in HA occurs according to unclear conditions - random. :)
And there is still a lot of such chaos in the existing software for users in Zigbee...
Do you have any further hints? Thanks!
I was just trying to recall any other steps I took to get it to work for me. I do remember not enabling the "force remove" and "prevent from joining again" options in z2m web UI when removing the old devices. Other than that, I'm not sure, but I'd probably try restarting z2m right after removing all the non -z LYWSD03 devices, before pairing them again. Hopefully that ensures they're fully removed and z2m picks them up as the correct model.
@fhempy
Removing the device, rebooting Home Assistant, “resetting” to GND in Xiaomi LYWSD03MMC for a few seconds, new pairing with ZHA. This led to the name being updated.
All "Object IDs" in Home Assistant for the LYWSD03MMC-z device have been updated. All panels and connections had to be rewritten.
In the ZHA, these actions are not required, since there is no connection to the name of the device and crutches are not required to work with the device. But after Zigbee OTA update nothing is updated.
I removed the device in z2m and did the "resetting" with the GND-RST pin connection. After that I had to remove the battery again to pair it again in z2m. Unfortunately it still remains without -z.
I never disconnected the batteries. A temporary GND-reset contact or pressing a button on thermometers with a button is always sufficient.
To save battery, polling of all Zigbee channels for binding occurs with a random period of several minutes. This process requires a lot of energy and, taking into account pauses of several minutes, the average consumption still increases to 0.1 mA, which is a lot for the CR2032. If there is no pairing, the battery will drain quickly...