Backups (via task-runner/backup-utility) error on invalid .s3cfg
adambar opened this issue · 1 comments
adambar commented
Hello,
The configuration for backups via the kubernetes secret gitlab_gcs_credentials is not in the required format.
Documentation of the correct format is available at:
https://gitlab.com/charts/gitlab/blob/master/doc/advanced/external-object-storage/index.md#backups-storage-example
This presents in the following way:
kubectl exec -it gitlab-task-runner-65c756f75b-bwtx7 backup-utility
WARNING: This version of GitLab depends on gitlab-shell 9.0.0, but you're running Unknown. Please update gitlab-shell.
2019-04-24 18:47:28 +0000 -- Dumping database ...
Dumping PostgreSQL database gitlabhq_production ... [DONE]
2019-04-24 18:47:31 +0000 -- done
WARNING: This version of GitLab depends on gitlab-shell 9.0.0, but you're running Unknown. Please update gitlab-shell.
2019-04-24 18:47:46 +0000 -- Dumping repositories ...
* root/test1 ... [SKIPPED]
[SKIPPED] Wiki
2019-04-24 18:47:47 +0000 -- done
Bucket not found: xxx-gitlab-registry. Skipping backup of registry ...
Bucket not found: xxx-gitlab-uploads. Skipping backup of uploads ...
Bucket not found: xxx-gitlab-artifacts. Skipping backup of artifacts ...
Bucket not found: xxx-git-lfs. Skipping backup of lfs ...
Bucket not found: xxx-gitlab-packages. Skipping backup of packages ...
WARNING: This version of GitLab depends on gitlab-shell 9.0.0, but you're running Unknown. Please update gitlab-shell.
Packing up backup tar
WARNING: Ignoring invalid line in '/home/git/.s3cfg': {
WARNING: Ignoring invalid line in '/home/git/.s3cfg': "type": "service_account",
WARNING: Ignoring invalid line in '/home/git/.s3cfg': "project_id": "xxx",
WARNING: Ignoring invalid line in '/home/git/.s3cfg': "private_key_id": "xxx",
WARNING: Ignoring invalid line in '/home/git/.s3cfg': "private_key": "-----BEGIN PRIVATE KEY-----\nhola\n-----END PRIVATE KEY-----\n",
WARNING: Ignoring invalid line in '/home/git/.s3cfg': "client_email": "hi",
WARNING: Ignoring invalid line in '/home/git/.s3cfg': "client_id": "szia",
WARNING: Ignoring invalid line in '/home/git/.s3cfg': "auth_uri": "https://accounts.google.com/o/oauth2/auth",
WARNING: Ignoring invalid line in '/home/git/.s3cfg': "token_uri": "https://oauth2.googleapis.com/token",
WARNING: Ignoring invalid line in '/home/git/.s3cfg': "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
WARNING: Ignoring invalid line in '/home/git/.s3cfg': "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/gitlab-gcs%40xxx.iam.gserviceaccount.com"
WARNING: Ignoring invalid line in '/home/git/.s3cfg': }
ERROR: /home/git/.s3cfg: None
ERROR: Configuration file not available.
ERROR: Consider using --configure parameter to create one.
command terminated with exit code 78
This can be fixed by enabling interoperability (https://console.cloud.google.com/storage/settings) and updating the terraform code.
variable "gitlab_bucket_interoperable_access_key" {}
variable "gitlab_bucket_interoperable_secret_key" {}
resource "kubernetes_secret" "gitlab_gcs_credentials" {
metadata {
name = "google-application-credentials"
}
data {
gcs-application-credentials-file = <<EOT
[default]
host_base = storage.googleapis.com
host_bucket = storage.googleapis.com
use_https = True
signature_v2 = True
access_key = ${var.gitlab_bucket_interoperable_access_key}
secret_key = ${var.gitlab_bucket_interoperable_secret_key}
# Multipart needs to be disabled for GCS !
enable_multipart = No
EOT
}
}
github-actions commented
This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days