Missing stats after restart on RPi
solrak44 opened this issue · 12 comments
Kia ora,
I had to restart my Raspberry Pi after a power cut, and despite using an InfluxDB the stats are missing. So are some of the stats on the right-hand side.
This is the command I am using to kick it off:
docker run -it
-e TZ=Pacific/Auckland
-e ENVOY_CONTROLLER_PASSWORD=passwordishere
-e ENVOY_CONTROLLER_HOST=envoy.ip.is.here
-e ENVOY_INFLUXDBRESOURCE_HOST=host.ip.is.here
-e ENVOY_INFLUXDBRESOURCE_PORT=8086
-e SPRING_PROFILES_ACTIVE=influxdb
-e ENVOY_REFRESHSECONDS=5000
-e ENVOY_PAYMENTPERKILOWATT=0.05
-e ENVOY_CHARGEPERKILOWATT=0.16
-e ENVOY_DAILYSUPPLYCHARGE=2.30
dlmcpaul/enphasecollector:arm
Is there something I am missing? I can see the InfluxDBs have some data in them, but it looks like it didn't read it back in.
Up until now it's been running sweet as a nut!
Thanks heaps
Sorry, the influxdb profile sends data to an influx database so you can make your own display (generally with grafana or similar). It was the original mechanism before I did the UI and does not hold the data for the UI.
The UI shown was designed for a standalone mode and so uses an local database inside the image. If an image is reset then it starts clean unless you link the database to an external file. From the read me
You can also link the internal database to an external file system, so the database kept on upgrade of the image using the mount point /internal_db
Try adding
--mount source=/internal_db,target=host_path
replacing host_path with a location on your pi drive.
The internal db is available at /solar/h2 so you might be able to repopulate it from the influx database afterwards.
Thanks for the super-fast reply!!
Gotcha - apologies for the misunderstanding. I have made sure the db writes out to the local drive now.
Why have some of the stats (like daily import from the grid, savings of not importing etc...) disappeared? How do I get them back?
I need to work on the documentation some more. It is a little confusing with all the options.
docker images are designed to reset to clean although you can usually stop/start an image without issue but if the old image is replaced then all the internal data is reset.
If you really need to get the data back then maybe you can export it from the influxdb and import it into the internal db.
If you use the /solar/h2 link you can login into the internal db by replacing the jdbc url with jdbc:h2:/internal_db/solar_stats_db
Historical data is written to a table called "SUMMARY"
The values are the totals over the day for each column.
Ahh, so they'll appear again when there's a days worth?
I'm not stressed about the missing data. If I have to start again then that's OK. I appreciate your help – I know enough to be dangerous, but not a lot more! This has been an enjoyable learning experience for me.
Yes, at about 5mins past midnight the daily values are summarised into SUMMARY table and that is what the other tabs query to produce the graphs
What about the circled stats? There are fewer of them than before. Are they subject to the same 5mins past midnight summary?
No that I find curious. There are 13 stats that are populated and shuffle about on that display. They are based on current data being collected.
I think there is an error occurring on when trying to add the next stat in the list.
Is there a log output you have for your docker instance. I expect there is a message like "populateStatusList Exception:"
2020-10-15 11:24:53.025 ERROR 1 --- [nio-8080-exec-3] com.hz.controllers.EnphaseController : populateStatusList Exception: Non-terminating decimal expansion; no exact representable decimal result. java.lang.ArithmeticException: Non-terminating decimal expansion; no exact representable decimal result.
Looks like something weird with getting data from the enphase maybe?
java.lang.ArithmeticException: Non-terminating decimal expansion; no exact representable decimal result. at java.base/java.math.BigDecimal.divide(BigDecimal.java:1723) ~[na:na] at com.hz.configuration.EnphaseCollectorProperties.getRefreshAsMinutes(EnphaseCollectorProperties.java:33) ~[classes!/:development-SNAPSHOT] at com.hz.services.LocalDBService.calculateTodaysPayment(LocalDBService.java:199) ~[classes!/:development-SNAPSHOT] at com.hz.services.LocalDBService$$FastClassBySpringCGLIB$$6d4cbcad.invoke(<generated>) ~[classes!/:development-SNAPSHOT] at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.8.RELEASE.jar!/:5.2.8.RELEASE] at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:687) ~[spring-aop-5.2.8.RELEASE.jar!/:5.2.8.RELEASE] at com.hz.services.LocalDBService$$EnhancerBySpringCGLIB$$103a4d2f.calculateTodaysPayment(<generated>) ~[classes!/:development-SNAPSHOT] at com.hz.controllers.EnphaseController.populateStatusList(EnphaseController.java:58) ~[classes!/:development-SNAPSHOT] at com.hz.controllers.EnphaseController.status(EnphaseController.java:145) ~[classes!/:development-SNAPSHOT] at jdk.internal.reflect.GeneratedMethodAccessor110.invoke(Unknown Source) ~[na:na] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na] at java.base/java.lang.reflect.Method.invoke(Method.java:564) ~[na:na] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) ~[spring-web-5.2.8.RELEASE.jar!/:5.2.8.RELEASE] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) ~[spring-web-5.2.8.RELEASE.jar!/:5.2.8.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(Se
Just a bit more (noticed it was talking about the same issue)
Actually - I noticed that there was a typo in my run script. I had changed it to poll every 5 secs (my silly wee keyboard hadn't typed the '1' in '15000'). Fixing it to poll every 15 secs made it work perfectly.
I knew that something must have changed because it was sweet as before. I assume it didn't have time to bring the data through before it was polling again. Sorry to bother you!!
Oh, and saving the data to the local drive meant that it stayed when I restarted the image. Yay
Yes, there is a bug in the way I convert the refresh rate to minutes. If the result has a repeating value (like 5000/60000 = 0.083333) then that error is thrown.
I have a fix I will test out but 15000 is probably a better value.
I will warn you that today's summaries are probably going to be wrong as you collected some data at 5s intervals and others at 15s. The summary will be calculated only at 15s. Probably not an issue for you now but the frequency is currently assumed to stay the same forever.
Thank you for helping me find and fix a bug. It is always nice to hear from people using my application.