Load Energy Report
Closed this issue · 13 comments
Hi,
Is there a reason for not fetching the report variable 'loads'?
I think this should provide the house load in addition to the feedin, generation, gridConsumption, chargeEnergyToTal and dischargeEnergyToTal that you are displaying in the Parameters tab.
Looked at this a bit further. I think its just that Load Power is not included in the default selection, presumably as there isn't a total to show?
Hi, that's right. It's not in the defaults - I chose them prety arbitrarily to match the values in the report tab. I think it would be useful to store the most recent selection (or let users define and save selections).
Also, I've been adding descriptions to the parameters taken from your very useful GitHub page to reduce the Facebook questions (primarily the "what is 'ambient temperature'?"). That should be in the next release in a few days.
The app is really good, thanks for your work. Happy for you to lift any useful stuff.
I've been playing with the cloud in Python / Jupyter Lab to see what it returns. The code is under the FoxESS-Cloud repo under my github.
Looking at the report interface, I'm getting values using other variables, like PVPower that might give total solar production before system losses. Still looking at this to see if a day sums to a sensible value.
Will let you know if anything useful turns up...
Thanks, that'll be interesting to see what you dig up.
It's a shame FoxESS aggregate so few variables so we're unable to show stats for PV-only generation. I had wondered whether you could take output - discharge + charge to synthesis the value but haven't got around to doing the maths to check.
I've spotted FoxESS have a websocket too which is used by the website for getting more current data. If you haven't already you can sniff the traffic with Chrome tools. Unfortunately the values in there aren't accessible via their REST api though.
The main difference between pv power and output power, ignoring battery for a moment, is system losses, so you would under estimate pv generation. They have a pv energy counter on the inverter but they hide it. Can only think this is because they don't want you to be able to work out system losses. My daily efficiency calculation shows about 85% overnight when load is small, rising to 92% by the end of the day.
Not looked at web sockets, will aggregate the report data next and see how it compares to known data.
Running some queries against the cloud data. Reporting on variables like pvPower doesn't seem to produce sensible results. It seems like it might be sampling points in the raw power data but not doing any integration.
Doing some checks, reporting on 'generation' also seems to show some inconsistencies that I didn't expect: if I report by 'day' for yesterday and sum the hourly results, I get 22kwh. If I report by 'month', the result for yesterday is 25kwh. My 'Output Energy Today' value in HA is 25kWh. Trying to make sense of this, it would be useful to understand what queries you use to get the energy totals displayed in Parameters and if you validated the results?
Instead of using reporting, I've done some queryies on the raw 5 minute data and run an integration on this to convert kW to kWh to get PV Energy Today, which seemed to produce some promising results... I'm assuming you get the raw parameter data for a day in any case to draw the graphs?
BTW, I also tried to query raw data using all the variables for a day (1 value for each variable, every 5 minutes) and it took about 30 seconds to return a sizeable block of data. Not sure if this will be too slow if you query everything and cache into memory and select the data, especially if Energy Stats is running on a more limited phone platform?
Tided up and checked my results. Here's an example of what I see when querying the fox cloud data using Python / Jupyter Lab (hope this makes sense, f is my library of python fox ess cloud functions):
Cell [2] is the result of getting the raw kW data and integrating (sum/12) to convert to kWh, based on 5 minute samples. In addition to the standard results, this also allows me to estimate the PV energy generated using pvPower.
Cell [3] is the result of reporting the energy production for the same day and summing the hourly results. Cell [4] is the result of reporting the energy production for the month and extracting the day. There are some decimal place differences between the [2] (integration) and [4] (day/month report) but these disappear if the data is rounded to 0 places.
Conclusion is that Fox cloud report by hour / day produces the wrong result for the day?
Looking at the same day in Energy Stats, generation reports the day as 22kWh but HA reports 25kWh.
Yep this tallies with the data you see on the Fox cloud website too, except they don't make it so easy to see that it's wrong.
So if you select a day and manually sum the values it's less than the amount which is returned from their API when you view the monthly view. Additionally as you say their daily total is around ~2kWh low versus Home Assistant on a few days I've sampled.
The stats tab is graphing the reports API providing either day/month/year as the reporting interval.
The parameters tab is graphing the raw 5 minute data and then sum's it which is unfair unless the entire 5 minutes was at that rate which seems unlikely. It only sums those 5 values I might remove them to avoid confusion. On the iOS version you can tap the graph at gives you the output at the given time. I'm hoping to add this to Android too which makes more sense than totals - and also makes it clear the totals issue lies with FoxESS.
Makes sense - the Stats and Parameters tab are duplicating the energy totals but both are currently giving inaccurate results.
I think for many people, the totals are what they are really interested in and having these not be accurate could be quite a big thing. Personally, I don't find the default stats graphs very useful as you can't really see what the hourly number are and the daily totals are wrong. If you flip to months, you get the monthly total but the daily totals are then lost in the graph.
The Parameters graphs are useful as you can see changes in generation / load through the day but the daily totals need to be correct as well.
I've done some checking and it's just the fox cloud report by day/hour that has the problem. If you report month/day, the daily values are correct and add up to the month and are consistent with the year/month report.
For me, the ideal stats would give the ability to see accurate daily totals (using month/day report) and monthly totals (using year/month report).
This is the iOS app when you tap the stats graph, so you can see the totals for each hour. I never got around to adding this on Android (iOS is my day job and more expertise) but i'll add it as a priority as you say without it, the stats are much less useful.
Interesting that only the day/hour has the issue. I can't understand what the problem is though from a technical perspective, e.g. I initially wondered if they were just counting the first 23 hours but that doesn't make sense either. Just plain wrong on FoxESS with no route to raise an issue.
Thats nice. I'd even go as far as to suggest remembering the settings, so you can set the default view to month/day with yesterday or today selected, so you then get the daily totals in a list (and that are correct values). You can select a different day by clicking on the chart or with forward/ backward buttons to move through the days. You could obviously still select day/hour if you wanted and do the same thing.
Another option, if screen space allows would be to display day / month values side by side, or toggle between them
The start of a year or month does not show much data, but a tick box to show a rolling view so you get at least 12 months / 30 days displayed, where data is available, would be cool?
Parameters can then just be 24 hour raw data graphs of power, volts, amp, C or %, giving more screen space for the charts. You could look at highlighting max / min parameter values, like foxesscloud.com does? Very useful, for example, when looking for RVolt max.
Interesting that only the day/hour has the issue. I can't understand what the problem is though from a technical perspective, e.g. I initially wondered if they were just counting the first 23 hours but that doesn't make sense either. Just plain wrong on FoxESS with no route to raise an issue
If I had to guess... it may help to understand what the inverter is doing. The energy counters are meters that increment for every 0.1kwh from the day the inverter is installed. Fox samples this data every 5 minutes. To work out the energy in a given interval, you have to get the difference between a start and end value.
Roll-over of hours at the start and end of the day is tricky, so my guess is that they take the first and last sample for each hour and subtract them... but this leaves a 5 minute gap between hours. The error is 5 minutes per hour so, over a 24 hour period, they loose 2 hours worth of energy data, spread throughout the day when the inverter is most active.
When they do the same for a day or month, the error is 5 minutes per day or month and this is usually around 00:00:00 GMT when inverter activity is low. They also only seem to show the day, month, year data in the app so it's possible they didn't even notice the error in the hourly data. When they show daily info, they tend to use the raw data .
That's an interesting insight and would certainly make sense. I've just pushed an update which pulls the stats totals from the monthly feed so now the Total Yield in app matches the value in the stats graph. This update also removes the totals from the parameters graph to avoid confusion.
I've got separate notes to make things sticky - remembering toggles etc. so i'm going to close this issue now.