Prometheus Support
adamcstephens opened this issue · 31 comments
Would you consider adding an endpoint to expose the data in Prometheus format?
Thanks!
It's possible. I was thinking of adding something to the Actuator endpoints. Prometheus format would just be an added jar.
Some questions first.
- How much data do you want (just basic consumption & production or full panel by panel)
- How do you use the app (docker or jar)?
- Willing to help test?
Hello
- I'd prefer full panel by panel
- Docker
- Yes I'd definitely be willing to help test
There is a test image dlmcpaul/enphasecollector:experimental that exposes a Prometheus endpoint at /solar/actuator/prometheus
You should see
solar.collection.time
solar.meter.production
solar.meter.consumption
solar.meter.voltage
solar.panel.production (for each panel tagged with the panel id)
Something to be aware of. the sum of the panels will not necessarily equal the production value. The panel reporting does not seem to be in sync with the phase monitoring data.
Let me know your thoughts.
I got this scraping in prometheus and it looks great so far! It's night time so I'll follow up after a couple days.
My only feedback so far would be to lowercase the unit in the name, if possible. Currently the units start with a capital letter, e.g. Watts and Volts. This seems more convention than recommendation, but prometheus metrics are generally all lowercase. Right now this outputs metrics such as solar_panel_production_Watts
Feedback on the naming is good. Once chosen it is hard to change.
I have made the units lowercase. I have stuck with watts though as joules would need calculation and I think solar generally uses watts.
I am also adding import (from grid) and export (to grid) metrics. Although they can be calculated I found when I used grafana it was easier if they were pre-calculated.
Any chance you can push another docker image? I went to build it, but there are some pre-docker steps that I'm clearly missing.
I did a new build and pushed to docker hub. I am traveling at the moment so did it all manually instead of the build server so not much testing.
Try it out, I am back Saturday so can do a real build then and maybe add some build instructions to the documentation.
Sorry, had some computer issues over the weekend that delayed me getting a good set of data. Here's today's daylight hours. The import/export calculation looks off if I'm reading it right.
Yes the grid import is definitely wrong. It should go to zero when not importing. I will also make it a positive value.
I think grid export is ok, at least it is zero at the right times.
Will let you know when I push a new image.
pushed another image dlmcpaul/enphasecollector:experimental
Let me know your thoughts.
All looks good to me including the import. Let me know if you have anything else you'd like me to test.
Good to see.
My only questions/thoughts are:
- Any additional stats you think could be added?
- Naming of the stats are ok?
Otherwise I will close this issue off in a few days and potentially push a new release.
The only other metric I'd need would be kWh. Would it make sense to do this calculation and expose it? If not I can calculate it in Grafana.
The naming all works for me.
Thanks again for adding this support and being so responsive!
I should at least expose the sample rate as a metric so that it can be calculated.
I assume for the kWh values you mean daily total production & consumption
I've gotten pretty close to what Enphase is displaying in their interface using promql to do the kWh calculation. This works for me, and I don't feel the need to expose daily summaries. For example any of the metrics can be converted from watts to kWh using this formula in Grafana:
(sum_over_time(solar_meter_export_watts[$__range]) / (count_over_time(scrape_samples_scraped{job="metrics-solar"}[1m]) * 60)) / 1000
Note that $__range is a Grafana variable and if using pure promql would be replaced with something like 24h. Using this variable allows for the calculation to adapt to the time range. Once summed, then it's divided by the number of scrapes and finally converts to kWh.
Here's yesterday using the prometheus data and the corresponding Enphase view:
Is it possible there's an error in calculating the imported and consumption watts?
Can you attach a screenshot of the Week tab in the internal web view and hover over the same day as above.
The internal web view matches Enphase's data. I dug into this some last night but all of those numbers in grafana screenshot are using the same formula. I'll continue to look, but wanted to share this in case you had any ideas.
Hmm, well I generate the graphs from the internal database primarily based on consumption and productions values using these formula (I have highlighted what I think is a key part of the calc)
Grid Import = select cast(time as date) as date, nvl(sum(consumption - production),0) as value from Event where time < ?1 and (consumption - production) >= 0 group by date order by date
Grid Export = select cast(time as date) as date, nvl(sum(production - consumption),0) as value from Event where time < ?1 and (production - consumption) >= 0 group by date order by date
Maybe see what you get using similar queries?
My only other thought is rounding issues in the calculations I do when exporting to prometheus.
Heya, did you manage to sort this out?
Hi. Sorry I haven't gotten back to this yet. I'll look at the data again this week and see if I can figure out where the disparity is.
I did make some changes after we last spoke. Are you using the experimental image or building your own as I have not updated the image yet.
Last I tried I wasn't able to build the image, so I've been relying on your experimental image. If you wouldn't mind pushing an update I can pull and check it out.
Ok I will push a new image.
Did you try using this file to build - https://github.com/dlmcpaul/EnphaseCollector/blob/master/Dockerfile-build
something like
docker build -f Dockerfile-build -t name:tag
should work
New image should be there now.
I verified I can build this now, so you won't have to push them again in the future.
Upgrading and will gather some data for a couple days.
Thanks!
My kWh calculations are much closer to what Enlighten displays, though not exactly the same. For my purposes this is good enough, but I'm happy to give any data you're interested in.
As far as I'm concerned, this is good enough to release and close this issue. Let me know if there's anything else you'd like to see.
Ok. Will try and cut a release soon.
You can also try changing the refresh setting from 60000 (60s) to say 15000 (15s) (and change your formulae as well) which should improve accuracy but it does make the device work harder.
Support is now available in release 0.25
Thank you! I'd be more than happy to buy you a coffee, beer, meal, etc, if you have a way to send you money. Otherwise, please accept my gratitude.
@adamcstephens can you share your Grafana dashboards?
@adamcstephens can you share your Grafana dashboards?
Yep, here you go: https://gist.github.com/adamcstephens/6433028aa4fd8f1d992f25c77dceb618