Passwords not set up correctly
chrislovecnm opened this issue · 11 comments
Summary
When using vault the passwords where not set correctly and the git operator is not starting.
Steps to reproduce the behavior
Install via terraform and use vault.
My main.tf
module "eks-jx" {
source = "jenkins-x/eks-jx/aws"
region = var.region
use_vault = var.use_vault
use_asm = var.use_asm
cluster_name = var.cluster_name
is_jx2 = var.is_jx2
create_eks = var.create_eks
create_vpc = var.create_vpc
create_nginx = var.create_nginx
jx_git_url = var.jx_git_url
apex_domain = var.apex_domain
subdomain = var.subdomain
tls_email = var.tls_email
use_kms_s3 = var.use_kms_s3
registry = var.registry
nginx_chart_version = var.nginx_chart_version
cluster_version = var.cluster_version
enable_backup = var.enable_backup
jx_bot_username = var.jx_bot_username
jx_bot_token = var.jx_bot_token
enable_external_dns = var.enable_external_dns
jx_git_operator_values = var.jx_git_operator_values
production_letsencrypt = var.production_letsencrypt
create_and_configure_subdomain = var.create_and_configure_subdomain
}
Expected behavior
The system is set up to use vault
Actual behavior
jx secret verify returns this
jx-production/tekton-container-registry-auth key secret/data/tekton/container/registry/auth missing properties: .dockerconfigjson
jx-staging/tekton-container-registry-auth key secret/data/tekton/container/registry/auth missing properties: .dockerconfigjson
jx/jenkins-maven-settings key secret/data/jx/mavenSettings missing properties: settingsXml, securityXml
jx/jenkins-x-chartmuseum key secret/data/jx/adminUser missing properties: password, username
jx/jx-basic-auth-htpasswd key secret/data/jx/basic/auth/htpasswd missing properties: auth
jx/jx-basic-auth-user-password key secret/data/jx/basic/auth/user missing properties: password
jx/jx-basic-auth-user-password key secret/data/jx/basic/auth/user/password missing properties: username
jx/lighthouse-hmac-token key secret/data/lighthouse/hmac missing properties: token
jx/lighthouse-oauth-token key secret/data/lighthouse/oauth missing properties: token
jx/nexus key secret/data/nexus missing properties: password
jx/tekton-container-registry-auth key secret/data/tekton/container/registry/auth missing properties: .dockerconfigjson
jx/tekton-git key secret/data/jx/pipelineUser missing properties: token, username
The majority of the secrets are in vault. I checked.
$ k -n jx-git-operator get po
NAME READY STATUS RESTARTS AGE
jx-boot-db5ff8d2-8bff-4771-9734-daa9b248c475-7fs7z 0/1 Error 0 20h
jx-boot-db5ff8d2-8bff-4771-9734-daa9b248c475-995lv 0/1 Error 0 20h
jx-boot-db5ff8d2-8bff-4771-9734-daa9b248c475-c7vgn 0/1 Error 0 20h
jx-boot-db5ff8d2-8bff-4771-9734-daa9b248c475-hfmzr 0/1 Error 0 20h
jx-boot-db5ff8d2-8bff-4771-9734-daa9b248c475-t982z 0/1 Error 0 20h
jx-git-operator-7bc44fc4c-26bm7 1/1 Running 1 20h
I found logs somewhere to run jx devops update. Which I am getting working on the m1.
Terraform version
The output of terraform version is:
See other bugs I have filed.
Module version
Current release
Operating system
Linux in container
Since the boot job failed, the secret generation never worked I think. Once the boot jobs works, try this guide: https://jenkins-x.io/v3/admin/troubleshooting/install/#issues-with-secret-generation
jx secret populate
Did not work previously. But I will try it again. Just got jx gitops update to work on the m1. I am getting:
jenkins-x-chartmuseum: key secret/data/jx/adminUser missing properties: password, username
jx-basic-auth-user-password: key secret/data/jx/basic/auth/user missing properties: password, key secret/data/jx/basic/auth/user/password missing properties: username
lighthouse-hmac-token: key secret/data/lighthouse/hmac missing properties: token
lighthouse-oauth-token: key secret/data/lighthouse/oauth missing properties: token
nexus: key secret/data/nexus missing properties: password
tekton-container-registry-auth: key secret/data/tekton/container/registry/auth missing properties: .dockerconfigjson
tekton-git: key secret/data/jx/pipelineUser missing properties: token, username
In the operator logs.
Hopefully the secrets command will work.
You dont need to run jx secret populate manually, the boot job runs it for you. If you do a direct push to master, it will run the step where it re-generates the secrets. But first you need to fix the boot job (unless that was fixed) 🤔
VAULT_ADDR=https://vault.jx-vault:8200 VAULT_NAMESPACE=jx-vault EXTERNAL_VAULT=false jx secret populate --secret-namespace jx-vault
Ran so let me read the docs you gave me and see if they help.
Yah the docs did not help. How are the secrets set in vault? Because that evidently failed during the terraform setup.
Did a direct push to master fail? Can you post the logs from the bootjob?
Can we chat on slack?
@ankitm123 I am @chrislovecnm on k8s slack. I have meetings. Ping me and I will respond when I can
Join the #jenkins-x-dev channel in the k8s slack, that's where most of the devs are. My slack handle is @ankit
Closing this. This was an issue with vault, and most likely an older version of k8s.
@chrislovecnm and @ankitm123 posting here since we're running into a similar issue. I am new to Jenkins-x and having trouble with the installation on EKS using Secrets Manager with Terraform installation approach. Seeing these errors. I don't see any references in the document where and how these passwords need to be set up. Any help is really appreciated.
I did make a dummy commit to our internal bitbucket cluster repo(jx3-eks-asm) and still seeing same errors.
jx secret verify
SECRET STATUS
jx-production/tekton-container-registry-auth key tekton-container-registry-auth missing properties: .dockerconfigjson
jx-staging/tekton-container-registry-auth key tekton-container-registry-auth missing properties: .dockerconfigjson
jx/jenkins-maven-settings key jx-maven-settings missing properties: settingsXml, securityXml
jx/jenkins-x-chartmuseum valid: jx-admin-user/BASIC_AUTH_PASS, jx-admin-user/BASIC_AUTH_USER
jx/jx-basic-auth-htpasswd key jx-basic-auth-htpasswd missing properties: token
jx/jx-basic-auth-user-password valid: jx-basic-auth-user/password, jx-basic-auth-user/username
jx/lighthouse-oauth-token key lighthouse-oauth missing properties: token
jx/nexus valid: jx-admin-user/password
jx/tekton-container-registry-auth key tekton-container-registry-auth missing properties: .dockerconfigjson
jx/tekton-git key jx-pipeline-user missing properties: token, username