Solar production and power distribution showing values in hex
lordCONAN opened this issue · 63 comments
I've managed to get my Solar System added, which include two entities - Solar power generation and power distribution, but all values are showing in hex.
I was wondering if this could be fixed to display the correct units.
I also wanted to say thank you so much for your continuing work on this project, I posted on the home assistant board asking you to continue the work, and I'm so thankful that you have developed this to encompass more than just air conditioners.
It’s an absolute pleasure to have helped so many people. I am blown away with what people are using this for, well outside my own simple needs.
I can certainly fix this issue. It’s missing functionality in the ‘pychonet’ to be able to parse this data in a human readable format for these two classes - your the first who need this so I’ll add the missing classes. Not a problem. Give me a day or two and I will try to fix the back end and then we can look at configuring the sensors in the front end to belong to the energy classes.
Looking at the distribution panel class, what I would be proposing is that we ignore any sensors which are returning culmative data, as you will be collecting culmative data using Home Assistant itself. For now I will focus on instantaneous measurements only. I’m going to have to come up with sensor logic specifically for the distribution panel as reading the specs it combines individual values together to come up with more accurate readings. Give me a few days
No worries, take as much time as you need. It'd be fantastic to be able to add it to energy class, that was my ultimate plan. I also don't see what use I would have for cumulative, so ignore away!
Can you pull 3.4.2 and let me know how your home solar generation is looking. Distribution meter I will get to after.
Just pulled 3.4.2 and it looks like maybe you've done the distribution meter (breaker box) and not the solar panels. I removed the previous solar system entry and on re-adding only the distribution meter is showing up (where previously both the solar panel generation and distribution meter showed up from the one ip).
Are there any error messages in the logs?
Also, can you enable debug logging in configuration.yaml?
logger:
default: warning
logs:
custom_components.echonetlite: debug
I've pushed 3.4.3 as I realised there was a minor issue in the sensor configuration. Can you please pull, turn on debug logging as per above, remove and re-add your system and let me know of any errors in your logs and log dump.
It's working much better now. After the update I re-added the system, this time only the solar generation appeared (I assume you've disabled the power distribution for now) and has good looking numbers. This is my first time working with the home assistant energy, but only cumulative is available to be chosen ... maybe that's how it works? I honestly don't know.
As for the logs, I'm not seeing any errors from echonet, there is a warning, but that is to do with the aircon, but I'll attach the logs for you to check if you want.
echonet debug logs.txt
OK very interesting! strange that the power distribution panel has dropped off.. Let me have another look and get back to you.
Ok, having looked at the code, I think I worked out why 1. your solar panel sensors disappeared in the first place, and 2. why the distribution panel disappeared after I fixed the solar panel sensors. Can you update to 3.4.4 and let me know if both sets of sensors are back?
EDIT: that cumulative power value is off by about a factor of 1000 lol - which I'll fix shortly.
Distribution panel is back now, with the same data as last time.
Edit: And still no errors in the logs.
no worries, you can turn off debug logging now. Can you please update to 3.4.5 - I have fixed up your solar panel cumulative sensors to show the correct value in Wh.
Once that is done, can you add your cumulative sensor to your energy panel as the solar panel input sensor?
Grid import and Grid export we should hopefully be able to resolve with potentially a pair of template sensors combining two of your distribution panels sensor together. That is actually going to be a LOT easier than me messing about with custom code.
Looks like that did the trick, the energy dashboard is looking much more reasonable now!
Okay, awesome, well I've just made some updates to support your distribution panel. Can you download 3.4.6 latest and give me a peek at how your distribution panel sensors are looking? I've had to be a bit tricky with the logic and I'm sort of working a little blind at the moment as you can appreciate. Im hoping it works first go.
Done and done.
Looks pretty good to me ... although I honestly have no idea.
Okay, i've spotted a minor bug which I have fixed in 3.4.7 (measured instant amount of energy was not displayed correctly).
Now, what you should be able to do in the energy dashboard is configure grid consumption to be your "normal flow" cumulative reading, and return to grid should be your "reverse flow" cumulative reading. You may need to trial this as this is just guesswork on my part. Let me know how you go and make sure you show some dashboard pictures!
Also, if you don't mind can you please let me know what the make and model of your solar system and distribution panel is?
Instantaneous is now showing correctly. I'm currently trying to work out the energy dashboard, I think it only updates on the hour, so currently waiting for the next hour to roll round to see if it's working properly.
Edit: I'm actually at work right now, so I can't get you the details of my system, but I'll post them when I get home!
It updated on the hour and is working like a charm! ... At least I think it is, I'm not at home to check the values with the actual monitor screen, but it looks like it'd be correct.
If you ever add a "buy me a coffee" button, I'll definitely through a few yen your way!
No problem, glad it’s working let’s keep an eye on it.
Also, this is my buy me a coffee. 😊
https://www.buymeacoffee.com/RgKWqyt
How's she looking almighty lord CONAN? 😃
Also, for your non-fossil fuel value, go to https://co2signal.com and sign up your email for an API key and then add the co2signal integration into your home assistant.
It's looking great! I also made some template sensors and did a little maths so that I could get instantaneous readings use the tesla-style-solar-power-card custom card.
Also for future reference the solar panel system I'm using is a Lixil RLE-MU1P-SET R-Type, which is just a rebranded Omron product, sorry I can't give you any better info than that right now. I'll try googling for a proper product name later.
Edit: I question the non-fossil fuel part cause it seems my part of Japan isn't scrapped by energy map ... but some how it's giving a value.
Yes, I think it’s a little inaccurate too, but I find it interesting nonetheless.
I’m glad it’s working well! I have a similar setup using Enphase inverters and its better then the vendor dashboard.
I’ll leave the ticket open for another couple of days but otherwise if you are happy I’ll close it later this week.
Yeah, I'm happy with this. Now I guess I have to think of the next echonet lite device I want integrated :P (currently my only other echonet device is my hot water heater ... but I don't really need to use home assistant to start running a bath)
Have you tried to add your hot water heater to home assistant?
When I try to add the hot water heater, this is the messages in the home assistant log
2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IP address is IPREDACTED 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] API listener has already been setup previously.. 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] Beginning ECHONET node discovery 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] ECHONET Node Discovery Successful! 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] instance is 1 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-114-1 map attributes discovered! 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-114-1 Identification number discovered! 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] instance is 1 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-128-1 map attributes discovered! 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-128-1 Identification number discovered! 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] instance is 1 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-129-1 map attributes discovered! 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-129-1 Identification number discovered! 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] instance is 1 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-130-1 map attributes discovered! 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-130-1 Identification number discovered! 2022-03-20 15:18:36 DEBUG (MainThread) [custom_components.echonetlite.config_flow] instance is 1 2022-03-20 15:18:36 ERROR (MainThread) [aiohttp.server] Error handling request Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request resp = await request_handler(request) File "/usr/local/lib/python3.9/site-packages/aiohttp/web_app.py", line 504, in _handle resp = await handler(request) File "/usr/local/lib/python3.9/site-packages/aiohttp/web_middlewares.py", line 117, in impl return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 100, in forwarded_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 79, in ban_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 219, in auth_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 137, in handle result = await result File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 166, in post return await super().post(request, flow_id) File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 62, in wrapper result = await method(view, request, *args, **kwargs) File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 111, in post result = await self._flow_mgr.async_configure(flow_id, data) File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 252, in async_configure result = await self._async_handle_step(flow, cur_step["step_id"], user_input) File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 325, in _async_handle_step result: FlowResult = await getattr(flow, method)(user_input) File "/config/custom_components/echonetlite/config_flow.py", line 116, in async_step_user self.instance_list = await validate_input(self.hass, user_input) File "/config/custom_components/echonetlite/config_flow.py", line 67, in validate_input await server.getAllPropertyMaps(host, eojgc, eojcc, instance) File "/usr/local/lib/python3.9/site-packages/pychonet/echonetapiclient.py", line 90, in getAllPropertyMaps return await self.echonetMessage( File "/usr/local/lib/python3.9/site-packages/pychonet/echonetapiclient.py", line 72, in echonetMessage payload = buildEchonetMsg(message_array) File "/usr/local/lib/python3.9/site-packages/pychonet/lib/functions.py", line 84, in buildEchonetMsg raise ValueError( ValueError: Value 0xa6 not a valid SEO class code
Challenge accepted
Awesome! In case it matters, checking the newest echonet lite appendix, ours is a hybrid water heater, but ours is only used for heating water, not for underfloor heating and such (the picture at the bottom of page 510 in the pdf, not the one at the top of 511).
ive made an update to add support for 0xA6, do you want to pull latest and see if your hot water service can be added?
Just updated, still gives an error when trying to add, but it is a new error.
2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IP address is IPREDACTED 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] API listener has already been setup previously.. 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] Beginning ECHONET node discovery 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] ECHONET Node Discovery Successful! 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] instance is 1 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-114-1 map attributes discovered! 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-114-1 Identification number discovered! 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] instance is 1 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-128-1 map attributes discovered! 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-128-1 Identification number discovered! 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] instance is 1 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-129-1 map attributes discovered! 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-129-1 Identification number discovered! 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] instance is 1 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-130-1 map attributes discovered! 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-130-1 Identification number discovered! 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] instance is 1 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-166-1 map attributes discovered! 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] IPREDACTED - ECHONET Instance 2-166-1 Identification number discovered! 2022-03-22 18:30:20 DEBUG (MainThread) [custom_components.echonetlite.config_flow] instance is 1 2022-03-22 18:30:20 ERROR (MainThread) [aiohttp.server] Error handling request Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request resp = await request_handler(request) File "/usr/local/lib/python3.9/site-packages/aiohttp/web_app.py", line 504, in _handle resp = await handler(request) File "/usr/local/lib/python3.9/site-packages/aiohttp/web_middlewares.py", line 117, in impl return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 100, in forwarded_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 79, in ban_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 219, in auth_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 137, in handle result = await result File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 166, in post return await super().post(request, flow_id) File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 62, in wrapper result = await method(view, request, *args, **kwargs) File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 111, in post result = await self._flow_mgr.async_configure(flow_id, data) File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 252, in async_configure result = await self._async_handle_step(flow, cur_step["step_id"], user_input) File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 325, in _async_handle_step result: FlowResult = await getattr(flow, method)(user_input) File "/config/custom_components/echonetlite/config_flow.py", line 116, in async_step_user self.instance_list = await validate_input(self.hass, user_input) File "/config/custom_components/echonetlite/config_flow.py", line 67, in validate_input await server.getAllPropertyMaps(host, eojgc, eojcc, instance) File "/usr/local/lib/python3.9/site-packages/pychonet/echonetapiclient.py", line 90, in getAllPropertyMaps return await self.echonetMessage( File "/usr/local/lib/python3.9/site-packages/pychonet/echonetapiclient.py", line 72, in echonetMessage payload = buildEchonetMsg(message_array) File "/usr/local/lib/python3.9/site-packages/pychonet/lib/functions.py", line 81, in buildEchonetMsg if data["DEOJCC"] in EOJX_CLASS[data["DEOJGC"]]: KeyError: 15
Hooboy... you have:
2-114-1 - "Hot water generator",
2-128-1 - "Electric energy meter",
2-129-1 - "Water flow meter",
2-130-1 - ""Gas meter"
and 2-166-1 "Hybrid water heater" (yay)
Now for reasons unknown, it has also returned some unknown entity with group code "0xF" - which is throwing the error. Let me do some reading
0x0F in the group codes is "user defined" class group (aka some non-standard thing). Ill have to do the following:
- fix up support for the other classes.
- work out how to ignore 0x0f.
I'm surprised that it found so many instances, as the terrible online only HEMS controller that we were given only lists 2 instances. The hybrid heater and the instantaneous gas heater.
Edit: Take as long as you need with this!
Hey, you want to try again and update to 3.4.9. I think I should have hopefully suppressed the 0x0F group during discovery so more of your gadgets should work
Alright, it now pulls in all 5 (!) of the devices and shows their values. Obviously there'll need to be some tweaking to set the sensor types and show values, but it's looking good!
Yeah that's good. Ill have to populate the other classes but that should be straightforward.
Hey mate can you update to 3.5.0 and let me know if water flow meter is working properly now and not showing hex?
Once we crack that then we can clean up how the sensors look.
Just updated it, and the water heater system is throwing up an error now. This is from the logs.
2022-05-14 11:25:19 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry Eco One for echonetlite Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 335, in async_setup result = await component.async_setup_entry(hass, self) File "/config/custom_components/echonetlite/__init__.py", line 148, in async_setup_entry await echonetlite.async_update() File "/config/custom_components/echonetlite/__init__.py", line 274, in async_update _LOGGER.debug(polling_update_debug_log(update_data, self._eojgc, self._eojcc)) File "/config/custom_components/echonetlite/__init__.py", line 64, in polling_update_debug_log debug_log = debug_log + f' - {EPC_CODE[eojgc][eojcc][value]} ({value:#x}): {values[value]}\n' KeyError: 128
hey mate, yeah another issue uncovered that gremlin, can you update to 3.5.1 please
Sorry, was away from home the whole day, update no longer produces an error, and there don't seem to be any values in hex anymore.
Not going to lie that is so cool.
I might try to see if I can include unit specific details for some of those sensors, your hot water electric energy sensor we should be able to include that in the energy panel.
Noice! I added an energy monitor plug to my aircon to measure it's usage too. It's all coming together!
Hi mate, can you please download the 3.5.2-beta. Ive update a few of those HWS sensors of yours, but you have to be the guinea pig for testing. Stuff like the power meter, gas meter and water meter should now show the calculated correct values.
You may need to go to hacs -> click three dots on echonetlite -> select 'redownload' -> select 'allow beta versions' -> select 3.5.2-beta.
https://hacs.xyz/docs/faq/select_version
On it. Looking good so far.
Looks like I overcooked the water calculations by 1000 and need to tweak the gas calculation too.
Can you directly update the 'sensor.py' to the latest version from git?
Done and done.
Those ‘unavailable’ sensors - they were not there previously where they? When did they start appearing?
Yeah, I don't think they were there previously ... the security data information one popped out to me last night, but I couldn't tell you when they started appearing, sorry, although my guess would be with the beta version. Do you want me to downgrade to check?
Just try to remove and re-add it if you can first. The devices I mean, not the custom component
Can you also enable debug logging and send through the log
I tried removing and readding the device and the sensors still showed up, then I downgraded and the sensors were still there. Then I removed and readded the device again and the sensors were gone on the latest non-beta release, so they definitely came with the beta.
I'm just about to head out for the day, I'll get the log to you when I get back.
no worries, when you get back just pull the latest from Git I think I found the bug.
Pulled the latest straight from git and things are looking good. The extra sensors are gone, and the usage looks better (I really have know idea how much water or gas we've been using cause I haven't really been checking anything on the bills except for the price!)
Back to the solar production part of the component, I seem to have come across a bug where my cumulative production drops to 0 then goes back to where it was, cause a huge spike in registered production in HA. This is the first time this has happened since you got the values showing properly, so I'm wondering if some failsafe isn't necessary to avoid spikes like that.
i think you need to sort that one out yourself with a template sensor, or put in a PR
Cool Cool, I'll do that then.
Edit: In case this happens to anyone else there's a quick and easy fix using the developer>statistics page. https://github.com/macxq/foxess-ha/wiki/How-to-fix-Energy-Dashboard-data-(statistic-data)
Latest update is looking good.
The sensors won’t exist any more they have been replaced with switches so you might want to delete them..
You can thank @nao-pon for all his hard work!