gitpod-io/gitpod

Have Gitpod-based commits GPG-signed

sr229 opened this issue · 66 comments

sr229 commented

There are some projects that requires commits to be GPG-signed to verify the authenticity of a contributor. Gitpod should be able make commits GPG-signed as a preference or automatically like browser-editing.

Hi,

If home directories were persistent (see #340) then gpg keys may be imported into a persistent home area./home/gitpod/.gnupg/ or /home/<github username>/.gnupg/

If home directories were persistent (see #340) then gpg keys may be imported into a persistent home area./home/gitpod/.gnupg/ or /home/<github username>/.gnupg/

Good suggestion, thanks!

But since persisting the home directory seems a bit complicated, I have found a potential work-around (that I'd like to use for SSH keys):

  1. Set SSH_PUBLIC_KEY and SSH_PRIVATE_KEY environment variables in https://gitpod.io/environment-variables/ (you may restrict this to repos you trust, e.g. yourname/*)
  2. Add something like this to your projects' .gitpod.yml:
tasks:
  - before: >
      mkdir -p ~/.ssh &&
      echo $SSH_PUBLIC_KEY > ~/.ssh/id_rsa.pub &&
      chmod 644 ~/.ssh/id_rsa.pub &&
      echo $SSH_PRIVATE_KEY > ~/.ssh/id_rsa &&
      chmod 600 ~/.ssh/id_rsa

This will re-setup your SSH keys automatically in every workspace for that project, and may be adapted for GPG keys too.

It's a bit cumbersome, because it requires a project configuration change, but may work. I wonder if SSH and GPG could also accept keys as environment variables, removing the need for creating special files in the home directory.

Hi,

The following would work for GPG keys so something similar will also work for .ssh.

  1. Convert your source .gnugpg directory contents to base64 data:
    tar -czvf - ./.gnupg | base64 -w 0

  2. Place this data into a gitpod environment varaiable called GNUGPG

  3. Add following to project's .gitpod.yml:

tasks:
  - before: >
      [[ ! -z $GNUGPG  ]] &&
      cd ~ &&
      rm -rf .gnupg &&
      echo $GNUGPG | base64 -d | tar --no-same-owner -xzvf -

Thanks for the tip!

  1. Place this data into a gitpod environment varaiable called GNUGPG

You can even do this directly from a Gitpod Terminal, like so:

gp env GNUGPG=$(tar -czvf - ~/.gnupg | base64 -w 0)

I don't like to store my private GPG (or SSH) keys in a (unencrypted) third-party database (aka “not my computer”), but I really would like to sign my commits. I think one should rather use a secure solution like Krypton for that (see also #782, thx @Vlaaaaaaad for the hint). With Krypton, the private key never leaves the security chip on your phone.

We could add Krypton to the Gitpod base image gitpod/workspace-full. I was curious how well Krypton works in Gitpod. Thus, I created a simple sample repository (with some notes in the README.md): https://github.com/corneliusludmann/gitpod-krypton-example You still need to pair (kr pair + scan QR code with your phone) and enable (kr codesign) Krypton in every workspace via the terminal. (It's totally fine for me, however, it shouldn't be to hard to build a plugin/widget for GUI support for this.)

Pros of adding Krypton to the Gitpod base image:

  • Having a secure (!) way to sign your commits.
  • Little effort to support signed commits in Gitpod.
  • Could encourage users to sign their commits.

Cons of adding Krypton to the Gitpod base image:

  • Additional dependency in the base image for a feature that may not be used by many people.
  • Feature is “hidden” behind the terminal. A proper documentation on the Gitpod website could help users to discover this useful feature. Or even a widget.

How do you feel about it? Any opinions?

(Probably there are even alternatives to Krypton?!?)

I very much like the idea( it's what I use now due to me switching tons of devices for work) and it also adds secure SSH for people who need that buuuuut krypt.co has been acquired and is now a part of Akamai. No comment has been made regarding the future of krypt.co so I would be against adding it in the base image until the future of the project is known.

CC @kcking and @agrinman for a comment on this.

I don't like to store my private GPG (or SSH) keys in a (unencrypted) third-party database (aka “not my computer”),

@corneliusludmann If the keys are password protected then they are encrypted .

If the keys are password protected then they are encrypted .

You're right. That makes it much better. The only thing left is that you have to decrypt the keys in the (probably “untrusted”) workspace to sign your commits, don't you? It's still a (small) attack vector that services like Krypton solve.

sr229 commented

I would prefer much to have an option to have Gitpod generate a one-time GPG key for me just like GitHub were they provide a GPG key for each account when you perform a browser-based edit as well.

There's reasons why this would be feasible since not everyone likes to provide their own keys and would probably prefer to have a service generate it for them.

sr229 commented

while the solutions you have might be good, again, in my opinion, this increases complexity if we wanted Gitpod to be just a out of the box experience, meaning user should just code and test, and that's it. I think this should be a optional integration for services like Krypton since I don't feel like its worth the configuration for many just to have their commits authenticated. Having options is a nice little thing.

I agree with @sr229 the idea of the service is "frictionless coding" I feel it would compromise the goal of the project to force this it should be recommended but not forced upon users.

Any info on this? Gitpod being unable to sign the commits is quite annoying when i am trying to contribute to things that require CLA, etc.. alike tldr-pages/tldr#4136

So for me i have to clone local and remake the commit..

Note that github has access token for this

image

So i would think that just adapting gitpod's logic to use this would be enough?

sr229 commented

Note that github has access token for this

image

So i would think that just adapting gitpod's logic to use this would be enough?

Yep! That's what I was going for, We already have the API for it.

Any updates?

Any updates?

Hi @wusatosi and @crazyuploader!

While I agree this would definitely be more convenient with a built-in feature in Gitpod, you can already set up GPG-signed commits in Gitpod yourself as explained in #666 (comment)

Please let me know if this doesn't work or if you're getting any errors.

sr229 commented

Just wanna let you know @jankeromnes GitHub Codespaces now sign commits so you guys might want to catch up now.

image

Hi @wusatosi and @crazyuploader!

While I agree this would definitely be more convenient with a built-in feature in Gitpod, you can already set up GPG-signed commits in Gitpod yourself as explained in #666 (comment)

Please let me know if this doesn't work or if you're getting any errors.

Oh it worked, thanks!

stale commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

FWIW, I just tried the instructions on #666 (comment) and they don't work in the latest Gitpod builds (with rebranding and VS Code) via the gitpod user. It requires sudo to sign anything. Does anyone know how to fix this?

FWIW, I just tried the instructions on #666 (comment) and they don't work in the latest Gitpod builds (with rebranding and VS Code) via the gitpod user. It requires sudo to sign anything. Does anyone know how to fix this?

I'm having difficulty with this as well. Initially I had difficulty generating keys in the container without sudo. I learned that the gpg needed write access to the terminal (owned by root:nogroup with file mode 640) for the passphrase prompt. I can change the file mode with sudo chmod 666 $(tty) which allowed key generation to work. I've also seen errors when the terminal window was too small for the passphrase prompt. Even with gpg reporting access to the secret key, the terminal file mode changed, and the terminal window maximized I still dont get code signing to work.

Another solution here that I think is much closer to being a reality is to use Remote GPG Agent Forwarding (https://wiki.gnupg.org/AgentForwarding).

This would be the ideal solution for those already using the local VS Code feature.

We can piggy back on the SSH connection to forward the local gpg-agent to the remote gitpod.

woss commented

I would also like to mention that I checked today GitLab access token and I couldn't find the GPG/PGP permission scope ( relates to #666 (comment))

That sounds like a viable solution, but it seems that would only work for the desktop Vscode and not for the browser ( no local system to connect to)

Another suggestion would be https://www.agwa.name/projects/git-crypt/ , where users can create repository-specific GPG keys ( for example I have a GPG key just for signing commits ) then share it with the repo. i know, it's cumbersome and prob not good, but it's an example.

Another example would be the Browser extension ( there is none ) that would hold or even connect to a local GPG agent and then sign then somehow let the VsCode ( browser ) know the result ...

I'm just throwing the ideas :)

woss commented

maybe look into this https://keys.pub/ and their https://keys.pub/docs/specs/wormhole.html approach

Those also look like other possible solutions too.

However I’d like to point out that for those orgs which use PGP smartcards, remote forwarding the GPG agent will be required, none of the other solutions (gpg-agent in the browser, extra private keys, etc) will address that use case.

The use case for PGP in gitpod is not just for signing commits. It’s also for secrets management (e.g., with mozilla/sops and signing packages/releases.

Ok, good news :) I have managed to get gpg-agent forwarding working from my local workstation to the remote gitpod. I can successfully perform pgp operations (decrypting, signing) on my USB smartcard (plugged into to my local workstation) inside the gitpod.

The procedure is a bit of a manual one, and would greatly benefit with a little polish from the local companion I suspect.

  1. Copy your ~/.gnupg locally to the remote gitpod, or export/import a subset. In my case I needed to have my personal key to the gitpod. In my case I was not transferring any secret material, because that’s all on the smartcard. The key file I import on gitpod only has the secret “stubs”
  2. In the gitpod workspace, kill any existing gpg-agents with gpgconf --kill gpg-agent
  3. Open the gitpod workspace in a local VSCode
  4. In local VSCode, use the command palette (default: Ctrl+Shift+P) to run “Remote-SSH: Open SSH Configuration File”
  5. On your local workstation, copy the contents into another file, say gitpod.ssh
  6. On your local workstation, add the line RemoteForward /home/gitpod/.gnupg/S.gpg-agent /path/to/your/local/gpg-agent/extra-socket (I assume you already are running gpg-agent locally)
    • To get the path to your local gpg-agent extra socket run this locally: gpgconf --list-dir agent-extra-socket
  7. On your local workstation, SSH into the gitpod with ssh -F gitpod.ssh <gitpod hostname>
  8. Inside the gitpod you can then gpg --list-secret-keys and try out decryption/signing operations

🎉

Security warning: this exposes your entire gnupg keyring and smartcard to the gitpod. So you probably don’t want to leave this open more than necessary, or configure a very short caching time for your smartcard pin.

You’ll need to perform this procedure every time you connect, because the ~/.gnupg folder is not persisted (you could put it in the workspace and use the GNUPG_HOME env var to workaround that). Also the connection details inside the ssh config file change every time you open the local vscode.

I implemented it as follows based on this blog post from @adangel.

But I made some changes:

tasks:
  - before: >
      [[ ! -z $GNUPG_KEY  ]] &&
      gpg --verbose --batch --import <(echo $GNUPG_KEY|base64 -d) &&
      echo 'pinentry-mode loopback' >> ~/.gnupg/gpg.conf &&
      git config --global user.signingkey $GNUPG_SIGNING_KEY &&
      git config commit.gpgsign true   

I want to make a signed commit by default when there is a GPG key. For my GPG signed commits I use a subkey, so I need to include that.

image

The error message I get has been the following.

gpg failed to sign the data

Note that github has access token for this
image
So i would think that just adapting gitpod's logic to use this would be enough?

Yep! That's what I was going for, We already have the API for it.

This only gives you access to your public gpg key.
Even GitHub doesn't have your private key, and to sign your commit you need your private key.

I don't think Gitpod would want to store your private key as well, due to security and compliance complications.

One possible solution would be for gitpod to generate gpg private key for each user, and you would add public key to your github account. Then gitpod would use that private key to sign your commits, with the idea that even user wouldn't be able to see that private key. I think github is probably doing something similar when you use web interface.

I personally do want to be able to sign my commits in gitpod as well, but would want this to be done securely.

@sagor999 thank you for the feedback and clear explanation 🙏 😍 !

@gitpod-io/engineering-meta , I've just added this item to your inbox for triage. In talking with @csweichel , this would need to be done in the web app space, so I've added to your inbox accordingly.

I support the idea of gitpod creating its own GPG keys! Sounds like a good solution.

In the interest of userbility would it be possible for gitpod to pop up a dialogue when a user has a key generated, that asks them if they would like the key added to their app of choice? I can already imagine users getting frustrated when they go to commit their work and git throws an error about GPG signing being required, and they'd have to break their workflow to go and resolve it and add the key manually to GitLab etc.

Through the OAuth mechanisms I think this should be possible. You should already have access to the gitlab api for example.

It might be feature creep though, but it would be a much better user experience than just silently supporting it.

We currently don't really have the time to look into this. Secret management will come up on our roadmap sooner or later.

Until then I would
0) Generate keys yourself

  1. Store the keys in a user environment variable in Gitpod
  2. Use a personal dotfile repo to read that variable and configure git (as in the .gitpod.yml above). This way you can make them appear in all your Gitpod workspaces

I implemented it as follows based on this blog post from @adangel.

But I made some changes:

tasks:
  - before: >
      [[ ! -z $GNUPG_KEY  ]] &&
      gpg --verbose --batch --import <(echo $GNUPG_KEY|base64 -d) &&
      echo 'pinentry-mode loopback' >> ~/.gnupg/gpg.conf &&
      git config --global user.signingkey $GNUPG_SIGNING_KEY &&
      git config commit.gpgsign true   

I want to make a signed commit by default when there is a GPG key.

The error message I get has been the following.

gpg failed to sign the data

This is the same place I get stuck trying to set up code signing in the browser.

woss commented

Hi guys, I was playing around with the idea to host my own little server which would have the keys and do the git commit signing with GPG. This way gitpod wouldn't have to host users GPG keys. This works from the system and there is no need to implement anything in the Vscode for now since all the passphrase is on the server.

I know that's not best practice in the security sense but I have no ide how to make the OS app talk to vscode :) maybe someone knows.

Here is the repo, let me know if you have any questions :)

https://github.com/woss/git-pgp-remote-sign

Hi guys, I was playing around with the idea to host my own little server which would have the keys and do the git commit signing with GPG. This way gitpod wouldn't have to host users GPG keys. This works from the system and there is no need to implement anything in the Vscode for now since all the passphrase is on the server.

I know that's not best practice in the security sense but I have no ide how to make the OS app talk to vscode :) maybe someone knows.

Here is the repo, let me know if you have any questions :)

https://github.com/woss/git-pgp-remote-sign

Looks really interesting, @woss ! Great work, I would love to test it out sometime.

After using my previous instructions for some months now, I've finally wrapped it into a simple script:

forward-gpg-agent-to-gitpod.sh

Example:

# open local vscode connected to remote gitpod
# then..
./forward-gpg-agent-to-gitpod.sh --ssh

And you'll have a shell into your gitpod with your local gpg-agent forwarded. Works great for smartcards.

It's unfortunate that this appears to not have any intention of being supported in an official capacity. Signing commits is not an uncommon practice and is used across many organizations. The lack of this feature is prohibiting me from adopting Gitpod and I suspect has been a gating factor for others as well.

woss commented

As for the native support, i totally get the gitpod tech team being reluctant to do this. First, if they would keep your private key they are responsible for its security, storage, retrieval etc ... This would also make them a highly centralized solution and very likely a target of the baddies.

@jmgilman them holding your key, or generating one keypair for you would go to the same bucket as i me asking you to send me your main gpg key and i will set it up on my server with my remote solution.

You don't trust me more than i trust you :)

If you are eager to have the signed commits, go here https://github.com/woss/git-pgp-remote-sign and set it up. If you need help you can reach me on matrix @woss:matrix.org

🙌🏽

First, if they would keep your private key they are responsible for its security, storage, retrieval etc

Understood, but I wouldn't expect them to tackle it this way. As others have mentioned, if you're working via the browser, tying into the Github API would be a nice alternative. However, like others here, I use a smart card to sign my commits so that solution wouldn't work for my setup. In that case, support for forwarding the GPG agent to the container would work. If I recall, the VSCode Remote Container plugin has this implemented already.

sr229 commented

In my personal case, I use Codespaces which uses GitHub-generated GPG key, while yes, most people do not trust that system, this is considered enough for most people - so much so even GitLab has considered this too

In my personal case, I use Codespaces which uses GitHub-generated GPG key, while yes, most people do not trust that system, this is considered enough for most people - so much so even GitLab has considered this too

Sounds like a good idea. Does that mean we will have the GitHub GPG key reused by Gitpod (which I doubt, because GitHub won't provide its private key), or we will have a custom GPG key for our particular Gitpod environments?

The latest discussion on the GitLab thread pointed to a tool called GitSign which might or might not be applicable to this exact situation. Looks like it could be uniquely suited to Gitpods environment.

sr229 commented

The latest discussion on the GitLab thread pointed to a tool called GitSign which might or might not be applicable to this exact situation. Looks like it could be uniquely suited to Gitpods environment.

If GitHub Apps are allowed to generate per-account keys or a register a service-level key as a verified key for supported VCS providers it may just work.

With the latest updates making gitpod remote IDE connections use SSH, and the just added "bring your own ssh publickey" to gitpod.io, combined that means to enable gpg signing via agent forwarding you can use these steps:

  • add publickey to your profile
  • update your .ssh/config to forward gpg keys to *.ssh.ws.gitpod.io servers, this is my example:
Host *.ssh.ws*.gitpod.io
    ServerAliveInterval 15
    IdentityFile /Users/cbarker/.ssh/gitpod
    RemoteForward /home/gitpod/.gnupg/S.gpg-agent /Users/cbarker/.gnupg/S.gpg-agent.extra
    StreamLocalBindUnlink yes
  • Have a build task in your gitpod.yml to inject your public key, stop the gpg-agent (started because we initialized a public key), and configure git for your key:
tasks:
  - before: |
      gpg --keyserver keys.openpgp.org --recv-keys 5993621FD9D214E212E421C3AC3A6CB9E464DBDA
      gpgconf --kill gpg-agent
      git config --global user.signingkey 5993621FD9D214E212E421C3AC3A6CB9E464DBDA

Now when you launch your workspace via VS Code IDE or Jetbrains, it should automatically forward a socket for GPG over SSH to your workspace. If you're in the web IDE, starting an ssh session from your workstation would also suffice.

In my configuration, my yubikey (setup from drduh's guide) hosts my private key. A side effect of this is that any commit signing or gpg action requires either pin + physical touch, or physical touch, before it will proceed. That makes me feel relatively secure knowing that while the socket is on a remote machine, the yubikey means I have to approve any use / access to the socket (I always require touch to sign and you can set a timeout for the pin approval).

If VS Code Web Supports signing, why not GitPod? GitPod is also a wrapper of VS Code... Please search VS Code Source to get how they sign commits.

woss commented

@RudraSen2

If VS Code Web Supports signing, why not GitPod? GitPod is also a wrapper of VS Code... Please search VS Code Source to get how they sign commits.

The VScode relies on the git + gpg program that YOU install on YOUR computer. you have the private key and public key; VSCode is , only using it. Also, the way HOW it is using it is via the git itself, especially via the gpg.program key in the .gitconfig file. Which you can set to any gpg compliant program. What you think is simple is far from it.

You have to understand that Gitpod is not just a wrapper for VScode, it is also the infrastructure that runs in the cloud, and the ONLY state that is persistent is the /workspace directory ( unless specified diff in the config ), which also means that you don't have many options if you are going to use it purely in the browser as i do. many people use it via the ssh on the desktop, which i personally think it's pointless since you can use the vscode directly ( of course, minus all the deps setup, etc ... then again, .devcontainer is good enough)

Having said that, i am using the remote signer for 6 months already, which signs my commits, and I am using ONLY the web version. maybe you can look into that before you start telling them what to do ;)

@mrzarquon's approach +1

1 issue though... StreamLocalBindUnlink should in theory force the remote socket to get created? For some reason it didn't seem to work unless I went into the box to unlink /home/gitpod/.gnupg/S.gpg-agent first... anyone else find this?

With the latest updates making gitpod remote IDE connections use SSH, and the just added "bring your own ssh publickey" to gitpod.io, combined that means to enable gpg signing via agent forwarding you can use these steps:

  • add publickey to your profile
  • update your .ssh/config to forward gpg keys to *.ssh.ws.gitpod.io servers, this is my example:
Host *.ssh.ws*.gitpod.io
    ServerAliveInterval 15
    IdentityFile /Users/cbarker/.ssh/gitpod
    RemoteForward /home/gitpod/.gnupg/S.gpg-agent /Users/cbarker/.gnupg/S.gpg-agent.extra
    StreamLocalBindUnlink yes
  • Have a build task in your gitpod.yml to inject your public key, stop the gpg-agent (started because we initialized a public key), and configure git for your key:
tasks:
  - before: |
      gpg --keyserver keys.openpgp.org --recv-keys 5993621FD9D214E212E421C3AC3A6CB9E464DBDA
      gpgconf --kill gpg-agent
      git config --global user.signingkey 5993621FD9D214E212E421C3AC3A6CB9E464DBDA

Now when you launch your workspace via VS Code IDE or Jetbrains, it should automatically forward a socket for GPG over SSH to your workspace. If you're in the web IDE, starting an ssh session from your workstation would also suffice.

In my configuration, my yubikey (setup from drduh's guide) hosts my private key. A side effect of this is that any commit signing or gpg action requires either pin + physical touch, or physical touch, before it will proceed. That makes me feel relatively secure knowing that while the socket is on a remote machine, the yubikey means I have to approve any use / access to the socket (I always require touch to sign and you can set a timeout for the pin approval).

I'm planning on having a look at GitSign in my own time to see if there's anything smoother that can be done. In particular I'm keen to see how a Self hosted Fulcio and GitLab instances can be used with Gitpod.

The real unknown is whether I can bypass the browser auth of GitSign by using the token that Gitpod uses. I know I can get an access_token from http://localhost:22999/_supervisor/v1/token/git/my.host.name/ but I don't know if that's enough or if I need a full jwt or id_token, or even if Fulcio will accept one from Gitpod rather than from itself. I'll see what I can figure out.

One question I had for the dev's whilst I figure this out is: is there a way to get the full oidc id_token including the aud field?

One question I had for the dev's whilst I figure this out is: is there a way to get the full oidc id_token including the aud field?

I made some progress on this front. Turns out you can do the following:

curl --user oauth2:$(curl 'http://127.0.0.1:22999/_supervisor/v1/token/git/gitlab.com/' -s | jq -r '.token') "https://gitlab.com/jwt/auth?service=container_registry"

This retrieves a valid jwt (at least for gitlab) however it has the wrong audience to be useful for sigstore or fulcio. I've raised https://gitlab.com/gitlab-org/gitlab/-/issues/393134 to ask for this feature.

Not sure if the same is possible for github however

woss commented

i've been following this discussion for a while, and i don't what's the big deal, i have been using signed commits since last year may, and where gitpod does not host my keys.

@ChevronTango as @woss says, this is already possible (but undocumented). If you take a look at my Dotfiles, you'll see how you can do it - you need to configure the envvars GPG_PRIVATE_KEY_BASE64 and GPG_SIGNING_KEY. The only problem then is that, if you have a passphrase, you have to enter it each time (I could probably get around that, but my passphrase is easy to type so the effort ain't worth going to)

woss commented

expanding on the @mrsimonemms comment...
or you can use a remote-signer, which i built for the purpose of having the signed commits.

It consists of 2 components; a CLI and the server. All you need to do is download the CLI in the init script, set a few ENV vars that point to your server, run the express app with your keys on the server, and that's it.

the server you can run on the cheapest and slowest VM you can find :)

Also with this setup you can have repos from Github and Gitlab using the same key

@woss nice work. Have you got any source code you can share?

And could you run the server in the background of a Gitpod workspace to avoid having the expense ($5 is one whole beer after all)?

woss commented

@mrsimonemms

And could you run the server in the background of a Gitpod workspace to avoid having the expense ($5 is one whole beer after all)?

sure, but what's the point? in this case, you have your PVT key in the gitpod env, which is the thing i wanted to avoid 🙂

Have you got any source code you can share?

Unfortunately, i closed the source the repo because i accidentally pushed the sensitive info ☹️

but i can send you the CLI and the docker server image so you can play around

$5 is one whole beer after all

dude, where do you live???? where i live you get TWO beers for that 😊

sure, but what's the point? in this case, you have your PVT key in the gitpod env, which is the thing i wanted to avoid :)

Yeah, but where's the fun in that?

Unfortunately, i closed the source the repo because i accidentally pushed the sensitive info ☹️

Oooops, but we've all been there. If you can share something, I'd be very interested in looking at how you solved it.

dude, where do you live???? where i live you get TWO beers for that 😊

UK. A pint for me is usually about £4. In that there London, it can be double that.

woss commented

@mrsimonemms i composed the gist with hopefully clear guidelines so you can test it :)
https://gist.github.com/woss/8a2e9921729280ce0e07bed459ecd7ef

All feedback is welcomed ( beers too ) 😊

@mrsimonemms So my key point here is not that I want Gitpod to sign my commits, as you are quite correct that that can be done right now, but instead I want all commits from Gitpod signed for all of my teams, and most of all, I don't want Gitpod to store those credentials. Any time you have a service storing credentials that's just asking for a breach to occur, and whilst I trust the Gitpod devs to some extent, I'd rather not have to. I am not an individual, but a part of a larger userbase and I want to solve the problem for all of them.

The approach I'm prusing is aimed at providing what you and @woss were talking about, a remote service that gitpod can call out to and use to sign the commits, but without the user having to store anything sensitive in either of those systems. I don't want each user to have one service each, I want it centralised and managed. Fulcio and Gitpod both use OIDC and if they could be made to work in tandem then that means I can have every single user in my org signing all of their commits, without any of them having to do anything special, nor us having to do extra risk mitigation for potential loss of GPG keys stored in one or other of those systems.

To go a step further, if this were to work, it could be used for all users in Gitpod, not just my teams, which would be a big win for security around Gitpod.

woss commented

@ChevronTango
let me see if i got you correctly. Do you want a centralized 3rd party server to store your pvt keys and you don't trust a centralized 3rd party system ( gitpod ) to store your keys in the workspace that is literally alive for the duration of the task implementation? This doesn't make much sense to me.

Also, owning the PGP key is a personal responsibility, not companies'.

@woss That's not what Fulcio does. It creates ephemeral keys based on your OIDC token. In the same way that you don't use your password for everything, you use a cookie or ephemeral token, Fulcio provides the same ephemeral signing key approach. All you do is provide your OAuth JWT token and it returns to you a temp key that you can sign your commits with. That's what I want. It's not that I don't trust 3rd party services, but more that I don't trust storing things in 3rd party services. There shouldn't ever exist a database of everyone's GPG keys just waiting to be attacked, and that includes Gitpod itself.

Agreed on the simplicity of the approach @ChevronTango has brought up. An OpenID Connect token with Fulcio signing makes a ton of sense, and is something I'm advocating for internally.

In the meantime, we've released a blog post detailing how to use 1Password for biometric confirmation of signing with SSH keys: https://www.gitpod.io/blog/signing-git-commits-on-gitpod-with-1-password

The main caveat here is that it only works on desktop clients, and doesn't work in the browser, like an OpenID Connect based approach would.

When you use GitLab Repositories, you can now use SSH Code Signing.
https://docs.gitlab.com/ee/user/project/repository/ssh_signed_commits/

Agreed on the simplicity of the approach @ChevronTango has brought up. An OpenID Connect token with Fulcio signing makes a ton of sense, and is something I'm advocating for internally.

In the meantime, we've released a blog post detailing how to use 1Password for biometric confirmation of signing with SSH keys: https://www.gitpod.io/blog/signing-git-commits-on-gitpod-with-1-password

The main caveat here is that it only works on desktop clients, and doesn't work in the browser, like an OpenID Connect based approach would.

Just noticed #16482 which makes Gitpod itself an OIDC provider. If it can support a sigstore JWT token, and the user can retrieve that from the workspace supervisor, then we may be in with a chance of getting fulcio working and signing our commits. That's a much better approach than getting a different token from GitLab and Github etc.

@mrzarquon's approach +1

1 issue though... StreamLocalBindUnlink should in theory force the remote socket to get created? For some reason it didn't seem to work unless I went into the box to unlink /home/gitpod/.gnupg/S.gpg-agent first... anyone else find this?

@agibralter StreamLocalBindUnlink depends on both client and server honoring it, and it looks like our latest revisions might not at the moment. I updated my example here to use a little more robust script that I'm actually using to consistently get my yubikey based gpg key forwarded to my workspaces in VSCode, without having to start a second ssh session.

@mrsimonemms

And could you run the server in the background of a Gitpod workspace to avoid having the expense ($5 is one whole beer after all)?

sure, but what's the point? in this case, you have your PVT key in the gitpod env, which is the thing i wanted to avoid 🙂

Have you got any source code you can share?

Unfortunately, i closed the source the repo because i accidentally pushed the sensitive info ☹️

but i can send you the CLI and the docker server image so you can play around

$5 is one whole beer after all

dude, where do you live???? where i live you get TWO beers for that 😊

@woss It is probably irrelevant but where I live, you can survive for 3-4 days. :')

So I got very close to making this work:

I set up Fulcio with the following Config

{
  "OIDCIssuers": {
    "https://api.gitpod.io/idp": {
      "IssuerURL": "https://api.gitpod.io/idp",
      "ClientID": "sigstore",
      "Type": "email"
    }
  }
}

and in my gitpod workspace I setup Gitsign with the following instructions.

brew tap sigstore/tap
brew install gitsign

git config --global commit.gpgsign true  # Sign all commits
git config --global tag.gpgsign true  # Sign all tags
git config --global gpg.x509.program gitsign  # Use Gitsign for signing
git config --global gpg.format x509  # Gitsign expects x509 args
git config --global gitsign.fulcio https://5555-my-workspace.ws-eu96b.gitpod.io/

export SIGSTORE_ID_TOKEN=$(gp idp token --audience sigstore)

unfortunately it looks like the JWT token doesn't include the right fields for fulcio to approve the certificate for signing:

ERROR server/error.go:45 returning with error {"requestID": "O7hSTLqo", "code": "InvalidArgument", "clientMessage": "There was an error processing the identity token", "error": "token missing email claim"}

looks like fulcio has a number of different OIDC types and I chose email but it looks like none of them are compatible with the existing JWT claims:

{
  "iss": "https://api.gitpod.io/idp",
  "aud": [
    "sigstore"
  ],
  "azp": "sigstore",
  "c_hash": "",
  "exp": 1683410309,
  "iat": 1683406709,
  "auth_time": 1683406709,
  "sub": "https://gitlab.com/ChevronTango/gpg-test/-/tree/main/",
  "name": "ChevronTango"
}

so for the Gitpod dev's, if those extra "email" and "email_verified" claims could be added in, I can use a private sigstore instance to sign all of my commits. If you wanted to go one step further you could ask fulcio to add Gitpod to their config for SaaS fulcio, and you could then add the above git config to your static layers. Then every commit in all gitpod workspaces would all be signed. We are very close to making this a very powerful feature and it's not much more work to achieve it.

+1
It would make GitHub integration even better.