kerberos-io/agent

Kerberos Agent not retaining setting across reboots.

kprose3154 opened this issue · 11 comments

Hi everyone,

So I just started working on getting Kerberos working at home and I've ran into a big issue. It's possible I misconfigured something but from how to install guide goes, I did not miss anything. When I deploy a camera agent using Factory it works, until I reboot the system. After everything starts back up none of the settings for the agent are there anymore. I'm still able to see it in Factory just all of the camera settings are missing. Can anyone help with this?

Screenshot 2023-07-12 at 1 39 33 AM Screenshot 2023-07-12 at 1 39 42 AM

EDIT: Added Screenshots

@kprose3154 can you confirm the mongodb pod is still working?

@cedricve I can confirm it is still working. I'm able to port-forward the pod and talk to it using compass. The only collection in the KeberosFactory schema is configuration. What I'm noticing though is the information gets pushed like normal and it shows up in the DB (First Screenshot) but as soon as I reboot all that data goes away. The document is still present but all the variables are blank. (Second Screenshot, this one was taken earlier) Another thing is when I delete an agent out of factory, it does not delete the document in the DB.
Screenshot 2023-07-12 at 10 48 09 PM
Screenshot 2023-07-12 at 10 43 23 PM

This looks like a MongoDB/persistence issue , how are you persisting your data, are you using OpenEBS as well?

I am using OpenEBS at the default path /var/openebs/local/. It's only the cameras getting lost, the global config stays around after reboots.
Screenshot 2023-07-12 at 10 31 53 PM

Thank you, the thing is that the configuration collection is auto created on start when the collection is not available.

Can you create a collection in your mongodb manually and insert some data into it? the restart the machine and check if it still persist. I strongly believe the underlaying PV is not persisting properly.

I created both a temp DB (Screenshot 1) and a temp collection in the KerberosFactory DB (Screenshot 2) and all the data persisted. Maybe I have something misconfigured but I followed the guide. The only thing I changed was the namespace things were put under so instead of KerberosFactory I put them under Kerberos. MongoDB is still in its own namespace as helm puts it there. For the data I just grabbed the camera data before a reboot so it would be as Factory put it.

Screenshot 2023-07-16 at 7 49 20 PM Screenshot 2023-07-16 at 7 49 08 PM

Thank you for validating, this is interesting. So once you add a new Kerberos Agent through the UI it will boot up, but it will not persist it configuration in the mongodb.

Are you able to modify the "global" settings page of the Kerberos Factory and reflect your changes in the configuration setting?

Yes, the agent pod will always boot up. The only difference is the configuration for it no longer exists in the DB after reboots, the config gets stored like it is supposed to before reboots like shown 5-6 days ago. I am able to modify "global" settings and it does reflect in the DB (Screenshot 1). I bought a temp hub subscription to get keys to put in, and they persisted. So it only looks like the configuration for cameras gets erased, the global configuration persists.

Screenshot 2023-07-17 at 7 42 04 PM

@cedricve Just checking to see if there is any update on this?

Will have a look @kprose3154, just came back from a long holiday! ;)

@kprose3154 is your system/behaviour still the same?