deis/workflow

Chart provenance verification is broken

mboersma opened this issue · 10 comments

Seems that when we migrated charts.deis.com, somehow it broke the gpg verification of our signed charts, even older ones:

helm fetch --verify workflow/workflow --version v2.14.0
Error: openpgp: signature made by unknown entity

For now, I'll comment out the documentation in that encourages helm --verify so users aren't led astray. When this is fixed, revert the last commit made in #821.

vdice commented

I'm not currently able to reproduce, nor is CI which downloads the signing key from scratch and verifies, etc. (https://ci.deis.io/job/helm-chart-verify/53/console). Will try to see what the issue is...

Seems to boil down to a local misconfiguration with the latest gpg2? At least this suggestion from @fibonacci1729 seems to have worked around it here:

$ cd ~/.gnupg
$ gpg --export-ownertrust >otrust.lst
$ mv pubring.gpg publickeys
$ gpg --import-options import-local-sigs --import publickeys
$ gpg --import-ownertrust otrust.lst
$ mv pubring.kbx pubring.kbx~
$ gpg --export >pubring.gpg

Hi,

I'm still getting the error: Error: openpgp: signature made by unknown entity
when trying to run helm fetch --verify deis/workflow

I had a previous releases and after I deleted it, now when I try to install workflow again I encounter this issue.

vdice commented

@IulianParaian just to confirm, the error still occurs after attempting the gpg2 fix mentioned in #822 (comment)?

I tried to run those steps for gpg fix but no luck.
I also tried to remove all the deis repositories with helm repo remove deis and get it again helm repo add deis https://charts.deis.com/workflow
When I was checking the existing keys with gpg --list-keys I am not getting anything. So probably the keys are not downloaded?
Just to mention my installed version of gpg is 2.1.15

vdice commented

@IulianParaian I see -- it looks like you hadn't downloaded the public signing key? Try one of the methods from this doc and then the gpg --list-keys should show the key and helm fetch --verify deis/workflow should work.

@vdice Thanks, I managed to make it work. This is what I used:
gpg1 --keyserver pgp.mit.edu --recv-keys 1D6A97D0
I needed to use an older version of gpg, v 1.4.21. With v2.1.15 it's not getting the pubring.gpg and I couldn't make it work using the gpg2 fix from in #822 (comment).

PS: In the official Deis documentation I didn't find an explicit way of how to download the public signing key. It might be useful to be added there as well.

I don't think it is related to this issue, but now after the installation passes the verification step I am getting a new error:
Error: secrets "builder-key-auth" already exists
I deleted everything related to deis even the namespace but nothing seems to be working.
If I'm checking with kops the secrets list I am getting this:

Keypair         ca                              
Keypair         kops                            
Keypair         kube-controller-manager         
Keypair         kube-proxy                      
Keypair         kube-scheduler                  
Keypair         kubecfg                         
Keypair         kubelet                         
Keypair         master                          
SSHPublicKey    admin                           
Secret          admin
Secret          kube
Secret          kube-proxy
Secret          kubelet
Secret          system:controller_manager
Secret          system:dns
Secret          system:logging
Secret          system:monitoring
Secret          system:scheduler

Is there something I can delete to make it work?

@IulianParaian if you did helm delete of a previous Workflow install, it may leave the secrets around. One way to work around this is to delete the namespace before reinstalling:

$ kubectl delete ns deis

@mboersma, I already deleted the deis namespace. The only namespaces I still have are
kube-public and kube-system.