Encrypt dotenv file inplace more than once should be failsafe.
r10r opened this issue · 11 comments
Oh no It hit me again. After changing the configuration in .sops.yaml I usually run sops -i -d; sops -i -e on the encrypted files to apply the changes. (Maybe there is a better way to do this ?)
But if I run sops -i -e twice on a file, then the file can not be encrypted afterwards and sops returns the error MAC mismatch. File has .... Any succeeding sops -i -e call on a file should abort if the file is already encrypted.
I have tested @r10r issue and confirm with SOPS v3.8.1
Error Response
$ sops -i -e testing.enc.sops.yaml
The file you have provided contains a top-level entry called 'sops'. This
is generally due to the file already being encrypted. SOPS uses a top-level
entry called 'sops' to store the metadata required to decrypt the file. For
this reason, SOPS can not encrypt files that already contain such an entry.
If this is an unencrypted file, rename the 'sops' entry.
If this is an encrypted file and you want to edit it, use the editor mode,
for example: `sops my_file.yaml`
I would suspect this is a current expected output.
You are interacting with the file humanistically so you're getting a humanistic message back. If you just are editing a SOPS encrypted file, just use sops file, it will open in an editor and when you exit the editor, it'll save it.
You could also update your command: sops -i -d testing.enc.sops.yaml; sops -i -e testing.enc.sops.yaml
To something more safe: sops -i -d testing.enc.sops.yaml; if [[ $(cat testing.enc.sops.yaml | grep -c 'sops:') -gt 0 ]]; then echo 'file already encrypted'; else sops -i -e testing.enc.sops.yaml;fi
When interacting with a SOPS encrypted/decrypted file programmatically, you can build this kind of logic into the program, and it is a noop at that point.
Hope this helps ever so slightly while you keep working while waiting on the maintainers to decide how they want to proceed and get back to you.
TTYL,
Peter
@r10r which version of sops are you using? Sops 3.8.1 does not seem to show the behavior you claim.
@silentpete Thanks for testing this. I'll consider to create a script with the proposed update command.
@r10r which version of sops are you using? Sops 3.8.1 does not seem to show the behavior you claim.
sops v3.8.1-49-g418ea23e3
I use a development version to be able set yaml indent to 2 to avoid re-indentation for our yaml files.
You should be able to reproduce the issue with the following command sequence:
echo HELLO=world > foo.env
echo "stores:" > .sops.yaml
sops -i -e foo.env
sops -i -e foo.env
sops -i -d foo.env error message:
MAC mismatch. File has 712A6E486181E64F8949DAF476EAD3FF0924F26D69A28DADCA03B52BA9DF573C0D2E704FC17985EF0763DBF98C0FA8C3C81A20CF322FBB8AD57E2E600070E3C6, computed 1ADDF28E3BCC682983D0FD44D7C1DD81C15D24B22904B9858E6E7CE8D3C1F30C5FDE2F06A37041C0102E40F201B61EB4264C941F427F2A75C051A710727BB853
NOTE echo stores: > .sops.yaml is required or I get the following exception
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xdfa520]
goroutine 1 [running]:
github.com/getsops/sops/v3/cmd/sops/common.newDotenvStore(0xf145e0?)
/workdir/cmd/sops/common/common.go:46
github.com/getsops/sops/v3/cmd/sops/common.StoreForFormat(...)
/workdir/cmd/sops/common/common.go:162
github.com/getsops/sops/v3/cmd/sops/common.DefaultStoreForPathOrFormat(0x10bb462?, {0xc000451520, 0xd}, {0x0?, 0x10c35db?})
/workdir/cmd/sops/common/common.go:177 +0x9d
main.inputStore(0xc000187a20, {0xc000451520, 0xd})
/workdir/cmd/sops/main.go:1085 +0x65
main.main.func8(0xc000187a20)
/workdir/cmd/sops/main.go:796 +0x770
github.com/urfave/cli.HandleAction({0xec9b80?, 0x1125d88?}, 0xc00046d6c0?)
/go/pkg/mod/github.com/urfave/cli@v1.22.14/app.go:524 +0x50
github.com/urfave/cli.(*App).Run(0xc00046d6c0, {0xc0000400c0, 0x4, 0x4})
/go/pkg/mod/github.com/urfave/cli@v1.22.14/app.go:286 +0x766
main.main()
/workdir/cmd/sops/main.go:1018 +0x35be
I can try to reproduce the issue with the current HEAD if required.
Ok, I've tested it with a yaml file and then I'll get the expected error response. So it seems to be limited to dotenv files.
My guess is that it happens both with INI and dotenv files, since both of them employ flattening of metadata.
(wrong tab)
NOTE
echo stores: > .sops.yamlis required or I get the following exceptionpanic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xdfa520] goroutine 1 [running]: github.com/getsops/sops/v3/cmd/sops/common.newDotenvStore(0xf145e0?) /workdir/cmd/sops/common/common.go:46 github.com/getsops/sops/v3/cmd/sops/common.StoreForFormat(...) /workdir/cmd/sops/common/common.go:162 github.com/getsops/sops/v3/cmd/sops/common.DefaultStoreForPathOrFormat(0x10bb462?, {0xc000451520, 0xd}, {0x0?, 0x10c35db?}) /workdir/cmd/sops/common/common.go:177 +0x9d main.inputStore(0xc000187a20, {0xc000451520, 0xd}) /workdir/cmd/sops/main.go:1085 +0x65 main.main.func8(0xc000187a20) /workdir/cmd/sops/main.go:796 +0x770 github.com/urfave/cli.HandleAction({0xec9b80?, 0x1125d88?}, 0xc00046d6c0?) /go/pkg/mod/github.com/urfave/cli@v1.22.14/app.go:524 +0x50 github.com/urfave/cli.(*App).Run(0xc00046d6c0, {0xc0000400c0, 0x4, 0x4}) /go/pkg/mod/github.com/urfave/cli@v1.22.14/app.go:286 +0x766 main.main() /workdir/cmd/sops/main.go:1018 +0x35beI can try to reproduce the issue with the current HEAD if required.
I cannot reproduce this part. Can you provide instructions on how to reproduce it, including the exact git commit you have been using?
#1393 should fix the main issue from this thread (not detecting that DotEnv files are already encrypted).
Once I can reproduce the stores config problem I can also fix that one, but right now I cannot reproduce it...
Thanks @felixfontein for fixing this so fast. I'll double check for the nil pointer issue and open a separate issue if required. Happy new Year!
@r10r thanks for checking it! And a Happy New Year to you as well :)
I've upgraded sops to sops v3.8.1-109-g28ecd70c7 on my development machine and the dotenv file encoding now works as expected.
The runtime error does not happen anymore. Thanks again for your work 👍