truenas/charts

Stuck on Job Waiting for pods to be scaled to 0 replica(s) in TrueNAS

Closed this issue · 24 comments

After updating my nextcloud to newest version (App Version: 29.0.4 / Chart Version 2.0.11] it's stuck on job "Waiting for pods to be scaled to 0 replica(s)". I tried reverting the update to the version I had before but it did not work. II have no idea what to change at this point. It won't deploy and I don't have access to my files anymore. Is there a way to bring it all back? Thanks!

image
image

Or if possible I would like to know if it's possible to retrieve all my data back to my PC

Hello,
The data are in the host path you have defined, or if you used ix volumes, its in your /mnt/<apps-pool>/ix-applications/releases/nextcloud/...

What does these command return?

k3s kubectl get pods -n ix-nextcloud
k3s kubectl get jobs -n ix-nextcloud

Thanks

Hey thanks for the reply. I'm sorry to be a burden but I have no idea where to put this command. Do you mean that I connect with ssh for example through linux to the Nextcloud's IP address? If so - connection is refused. I am not able to connect through ssh. Unless there is an option to do it from the TrueNAS itself where I can do that

Hello, you can run those on the TrueNAS itself!
Either using ssh or the web shell on the menu.

Ok I managed and this is what I got:

k3s kubectl get pods -n ix-nextcloud

image

k3s kubectl get jobs -n ix-nextcloud

image

Can you please share output of this

k3s kubectl logs -n ix-nextcloud --selector pod.name=postgres

Thanks

Looks like the permissions on the postgres data directory are not right.

Do you use host path for postgres data?
If yes, you have to make sure the permissions for that directory are 999:999

I switched postgres backup and data from host path to ixvolume (since I have no idea to which user/group I should give permissions to) and it still doesnt work

IMG_20240731_145047

IMG_20240731_145317

... you can't just switch postgres data to another type.
the ixvolume is empty, without postgres data.

Changing type wont move data.

Please switch back to host paths and update permissions
eg on the host you can run something like this. (update with the correct paths)

# chown 999:999 /path/to/postgres/data

Ok now it says "running" instead of "deploying". Ill let this run for few hours and let you know since i have a lot of data. For now when I go to the NextCloud website it says the site cant be reached

You can check logs of nextcloud, maybe there are hints on whats wrong.

Either from the UI click the "logs" icon
or

k3s kubectl logs -n ix-nextcloud --selector pod.name=nextcloud

2024-07-31 15:22:25
Readiness probe failed: HTTP probe failed with statuscode: 502
2024-07-31 15:22:25
Liveness probe failed: HTTP probe failed with statuscode: 502

Ok so i can access the website but I get the same thing like I did last time. Non stop maintenance mode

IMG_20240731_154409

Shell in the nextcloud pod/container (there is a button on the UI once you select nextcloud app)

And run

occ maintenance:mode --off

I think I'm blind because i cant see that

Screenshot_20240731-161915

image

The icon with the >_ symbol

Make sure on the popup the nextcloud pod and nextcloud container are selected

This is what I selected

IMG_20240731_162557

And this is what i get

IMG_20240731_162627

These are my pod options before selecting:

IMG_20240731_162455

When selecting this

IMG_20240731_163055

I get

IMG_20240731_163117

Select this one
image

When selecting this

IMG_20240731_163055

I get

IMG_20240731_163117

Yeah I did here

Can you try please

su -p "www-data" -s /bin/bash -c 'php /var/www/html/occ maintenance:mode --off'

This seems to have worked. I can access my files again. Thank you so so much!