netgroup-polito/CrownLabs

Upload to NextCloud no longer works

giorio94 opened this issue · 1 comments

giorio94 commented

Describe the bug
It is impossible to upload new files to NextCloud (both via the web interface and through webdav). Indeed, the file appears to be uploaded but then NextCloud displays a popup reading "An unknown error has occurred" and the file is deleted. Instead, empty folders and files can be created from the web interface correctly.

To Reproduce
Steps to reproduce the behavior:

  1. Go to https://crownlabs.polito.it/cloud
  2. Log-in
  3. Try to upload a new file
  4. NextCloud will show a popup reading "An unknown error has occurred"

Expected behavior
The file should be uploaded correctly

Desktop (please complete the following information):

  • OS: Ubuntu/Debian
  • Browser Firefox/Chrome
  • Version Up-to-date

Additional context
One of the database pods is returning strange logs (may or may not be related to this issue):

$ kubectl logs -n nextcloud nextcloud-db-cluster-1

/var/run/postgresql:5432 - rejecting connections
2020-05-15 09:07:29,766 INFO: Lock owner: nextcloud-db-cluster-2; I am nextcloud-db-cluster-1
2020-05-15 09:07:29,767 INFO: Still starting up as a standby.
2020-05-15 09:07:29,768 INFO: Lock owner: nextcloud-db-cluster-2; I am nextcloud-db-cluster-1
2020-05-15 09:07:29,768 INFO: does not have lock
2020-05-15 09:07:29,769 INFO: establishing a new patroni connection to the postgres cluster
2020-05-15 09:07:30,492 INFO: establishing a new patroni connection to the postgres cluster
2020-05-15 09:07:30,518 WARNING: Retry got exception: 'connection problems'
2020-05-15 09:07:30,519 INFO: Error communicating with PostgreSQL. Will try again later

Additionally, if I remember correctly, the nextcloud-db-cluster-1 pod has been killed forcefully while draining the node to upgrade the system O.S.

giorio94 commented

The problem related to file uploads in NextCoud should have been fixed. Hence, I am closing this issue. Please, feel free to reopen if it occurs again.

Apparently, there where two main problems (still unclear the root cause):

  1. The database cluster was in a corrupted state and one of the members (nextcloud-db-cluster-1) remained in starting state (as confirmed by the output of patronictl list). This issue has been fixed executing patronictl -c postgres.yml reinit nextcloud-db-cluster nextcloud-db-cluster-1 from inside the pod itself to reinitialize the member.
  2. The folders used by NextCloud to temporarily save the files during the upload where no longer present. These folders are configured in the php configuration here:

    CrownLabs/infrastructure/file-sharing/manifests/nextcloud-php-configmap.yaml

    Lines 6 to 7 in 2763f20

    sys_temp_dir=/var/www/html/tmp
    upload_tmp_dir=/var/www/html/upload_temp_dir

    After having created the two folders in the mounted PVC, the file upload started working again normally.
  • 🚀1