Mounted PVCs are sometimes not deleted.

[rbollar]

I've managed to create a situation where I have mounted directories that I can't delete. The directories have been unmounted using heavyscript, but this empty directory remains. Mounting / unmounting the app again doesn't help, nor does rebooting.

I have switched to the args branch as recommended.

root@scale[/mnt/ixapps/mounted_pvc]# bash -x heavyscript pvc --unmount 
+ set -Eeuo pipefail
+ script_dir=/root/heavy_script
+ [[ ! -d /root/heavy_script ]]
+ cd /root/heavy_script
+ bash ./heavy_script.sh pvc --unmount
find: ‘/mnt/ixapps/ix-applications/releases//volumes/’: No such file or directory
cannot open '': empty component or misplaced '@' or '#' delimiter in name
Failed to unmount tautulli-config.

[Heavybullets8]

Sorry for the confusion. But I will need the output for bash -x /path/to/heavyscript/heavy_script.sh pvc --unmount

If you used the one liner script it will be:

bash -x /root/heavy_script/heavy_script.sh pvc --unmount

[rbollar]

No problem, but...

root@scale[/mnt/ixapps/mounted_pvc]# bash /root/heavy_script/heavy_script.sh pvc --unmount
find: ‘/mnt/ixapps/ix-applications/releases//volumes/’: No such file or directory
cannot open '': empty component or misplaced '@' or '#' delimiter in name
Failed to unmount tautulli-config.

[Heavybullets8]

I just woke up, i forgot the -x flag in my comment. It will actually be:

bash -x /root/heavy_script/heavy_script.sh pvc --unmount

the -x turns on debugging for bash. Which allows me to bugtrace.

[rbollar]

That works better! 🤣

Attached.

heavyscript.txt

[Heavybullets8]

@rbollar

I think I may have fixed your issue, preform a self update on the args branch

heavyscript self-update

Then run the unmount command again and let me know if that worked or not.

Edit:
If that does not work I will need the output of:

k3s kubectl get pvc --all-namespaces | grep -E "tautulli-config|mylar-config"

[rbollar]

The output was the same for me. I've attached an updated dump. Here's the other result (there is no output):

root@scale[~]# k3s kubectl get pvc --all-namespaces | grep -E "tautulli-config|mylar-config"
root@scale[~]# 

heavyscript.txt

[Heavybullets8]

Were tautulli and mylar deleted? Name changed?
Was anything done other than mounting and un-mounting the applications with HeavyScript

[rbollar]

Yes, at this point, neither is on this machine. However, I do have an example on another machine that is similar and plex is still installed. It says the selection is successful, but the directory remains. Plex is running currently, so I don't want to replace the current pvc with the blank directory... 🤣 :

ix-plex                  plex-config                           Bound    pvc-8fd58a5a-8c5d-477b-8bf1-0a9fe5f598db   750Gi      RWO            ix-storage-class-plex                  37h
root@ozzy[/mnt/ixapps/mounted_pvc]# 

root@ozzy[/mnt/ixapps/mounted_pvc]# rm -rf plex-config
rm: cannot remove 'plex-config': Device or resource busy
root@ozzy[/mnt/ixapps/mounted_pvc]# 

heavyscript.txt

[Heavybullets8]

So when an application is removed from the system, it no longer has anywhere to mount to. So unmounting from your temporary mountpoint will never work.
You may just have to delete the PVC if that is what you were wanting.

Plex however, I am not entirely sure. I don't know exactly what you did.. The application should not start at all if the PVC is mounted to your host path.

Heavyscript is meant to just mount a pvc from an application
The user will make their edits, or backups or whatever
Then unmount the PVC so the PVC is allocated to the pod again

It sounds like you have done something outside of that scope, and therefore I cannot support it.

However, I am willing (and i am sure others are too) to help you out if you want to open a thread on the truenas-scale channel on discord. Maybe tag me and we can see if we can solve this for you.

I will close this though as I am sure this is the results of preforming a task this script is not meant to handle. If you believe this to be confusion on my part, please re-open this. Good luck!