KMS Setup

[chapati23]

We need to allow users to pass in configuration for the KMS system they'd like to use.
We want to support Google Cloud, AWS, and Azure.

This could be handled as a config param upon initialisation:

new StrongConfig({ kmsProvider: 'aws', otherKmsParam: 'blub' }).load()

Ultimately, the provider-specific parameters need to be made available to the sops binary

[mohoff]

Having researched and thought about this more, I think this is the wrong approach.

There's no primary use-case in doing

const s = new StrongConfig({ kmsProvider: 'aws', otherKmsParam: 'blub' })
s.load()

This is because at the time of .load() (=decryption) , we have all the kms/key setup already contained in the sops metadata field. Thus, passing these options to the class constructor doesn't bring anything.

What we actually want:

• Encrypt configs with a CLI command. Here we pass the key/kms details as params.
• Decrypt configs automatically on load() (and with a CLI command in the future but this is out of scope of this issue.)

Now the line between @strong-config/node and @strong-config/cli gets very narrow. To make use of strong-config, which includes an encryption and a decryption, we would need both of thes packages. Moving forward, I think we have two options:

1. Keep two distinct packages. This means that /node is pretty much ready for a first releae at its current state. But it also means we require /cli for the release. This brings users the possibility to put /node as dependency, while /cli is in devDependenices.
2. Merge /cli into /node. We would make /node both an in-code imported module as well as a CLI tool. The main reason to do this is to have one package to make actual use of strong-config without needing a second package (which adds confusion too).

[barathvk]

Unless I'm missing something, the KMS provider is part of the sops encrypted file. It is already inferred by sops when it tries to decrypt the file

myapp1: ENC[AES256_GCM,data:Tr7o=,iv:1=,aad:No=,tag:k=]
app2:
    db:
        user: ENC[AES256_GCM,data:CwE4O1s=,iv:2k=,aad:o=,tag:w==]
        password: ENC[AES256_GCM,data:p673w==,iv:YY=,aad:UQ=,tag:A=]
    # private key for secret operations in app2
    key: |-
        ENC[AES256_GCM,data:Ea3kL5O5U8=,iv:DM=,aad:FKA=,tag:EA==]
an_array:
- ENC[AES256_GCM,data:v8jQ=,iv:HBE=,aad:21c=,tag:gA==]
- ENC[AES256_GCM,data:X10=,iv:o8=,aad:CQ=,tag:Hw==]
- ENC[AES256_GCM,data:KN=,iv:160=,aad:fI4=,tag:tNw==]
sops:
    kms:
    -   created_at: 1441570389.775376
        enc: CiC....Pm1Hm
        arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e
    -   created_at: 1441570391.925734
        enc: Ci...awNx
        arn: arn:aws:kms:ap-southeast-1:656532927350:key/9006a8aa-0fa6-4c14-930e-a2dfb916de1d
    pgp:
    -   fp: 85D77543B3D624B63CEA9E6DBC17301B491B3F21
        created_at: 1441570391.930042
        enc: |
            -----BEGIN PGP MESSAGE-----
            hQIMA0t4uZHfl9qgAQ//UvGAwGePyHuf2/zayWcloGaDs0MzI+zw6CmXvMRNPUsA
                            ...=oJgS
            -----END PGP MESSAGE-----

see the kms key in the above example.

[mohoff]

Yeah exactly! Sorry, probably I didn't make my point clear enough.

This means that we don't need to pass any kms setup to StrongConfig, as it's contained in the sops field. We only need the kms setup for encrypting configs, which is done from CLI, not inside the StrongConfig class.

[chapati23]

Hmm 🤔, agree with your reasoning so far @mohoff.

I'm always a fan of clear separation of concerns (= 2 repos), but I do agree that it's not that clear in this case what the best tradeoff is.

What about a compromise solution:

• Keep @strong-config/node and @strong-config/cli separate repos
• Add the CLI as a peerDependency to @strong-config/node to make the link between the two a bit clearer, without forcing developers to install it even if they don't want any encryption

[mohoff]

peerDependency seems to express tight coupling, where the package needs/uses a peer dependency to function [1,2]. We don't have that case. IMO, we have a workflow that requires 1-2 packages that are used at different places. --> Users always need /node, and optionally /cli if they want encrypted secrets in their configs. In other words, /node as dependency and /cli as devDependency in the users' projects.

So far, I agree that keeping two repos is clean and separates concerns. /cli could live as optionalDependency in /node's package.json. The biggest downside for me is the impact on the README and telling a story of 2 packages, while we want strong-config to be super easy to get started with.

[1] https://yarnpkg.com/lang/en/docs/dependency-types/
[2] https://stackoverflow.com/a/34645112

[chapati23]

Hmm, and what if we just made @strong-config/cli a normal dependency of @strong-config/node`?

Pros:

• Getting started with strong-config remains easy, just one package install
• We could still separate our concerns into two repos

Cons:

• @strong-config/node package install would get slightly bigger
  • Depending on HOW much bigger it gets, this could be an ok-compromise, because this is not a client-side package where every last KB of bundle-size matters

[mohoff]

Yeah I'm also not worried about package size.

But why separate into two repos in the first place then? It looks bad to advertise two packages, 'clean separation of concerns', and 'use at different places in the workflow' but at the same time define the first as a dependency of the second. That's a good intention killed by the execution, just to make the How-to simpler.

I like a folder mapping more and more:

• Have one strong-config repo, say sc, with subfolders /node and /cli.
• Install sc if you want both, or install sc/node if you want only node functionality in dependencies.
• Import sc/cli globally or install it as a local devDependency.

This would require a repo restructure.

[crosby]

As inspiration here are how a few other popular packages are tackling the CLI tool:

• Jest has a single repo but appears to have a separate jest-cli package within it.
• ESLint just has a JS binary. So yarn global add eslint would allow for eslint to be run as a command.
• Webpack has a separate Webpack CLI. They don't depend on each other from what I can tell.
• Babel takes a similar approach to Jest here by using a monorepo and publishing multiple packages to NPM.
• Typescript does the same as ESLint and adds the executable JS binaries to the repo under /bin.

Seems there's no fixed standard here. My vote would be for a simple /bin folder with the binary because:

1. It's 1 less dep for the user to add i.e. I don't have to do yarn add @strong-config/node @strong-config/cli to get both deps
2. It allows for the binary and library to use the same codebase e.g. both can use the same sops wrapper functionality
3. Having both the binary and library version up independently is probably not required as both would be very tightly coupled anyway.
4. Users can just do yarn global add @strong-config/node and then have a binary strong-config that they use.

[mohoff]

Thanks for the research! Haven't thought of the bin approach but agree that it's a sweet option. It requires the repo merge as well though.

We also need to be aware that the example projects you listed do pretty much the same in their CLI tool as they do in their node module. In our case, both have complementary use.

[chapati23]

ok, let's just start with having the CLI live in this repo. if it later becomes clear that there's natural boundaries between /node and /cli they'll emerge over time and then we can always extract /cli into its own project

[mohoff]

Sounds good to me. I'm closing this issue as it's not relevant in it's current form anymore.