Secret Values special characters are interpreted

[nichcuta]

Hi,

Running v0.7.0, and noticed that secret values are getting interpreted rather than passed as is.
Example:

"value": "password<"

is JSON rendered as:
"value": "password\u003c"

When using base as format, if newline (\n) is in value, this is also interpreted resulting in the below incorrect syntax

          │   ├── example-keystore-jks_base64=/u3+7QQAAAYtIxCqRAAAFATCCBP0wDgYKKwAAAAIAAAABAAAAAQABMYBBAEqAhEBAQUABIIE6Z9q
            │   │   w2lvG+76o2JhexbzP2qA8EnUwzRorbDkbgRvTrsp3dMFBEnYqFurMXJMVnY8qffoD4OC19OvMkFg
            │   │   /am4W0GdZV05AQh+OTmEgDPfyVNKZCosmGl5Zr6zYuxSR1a9jVOjIPh+KIgDFy/Q021S7R+1fHrR
            │   │   6eKX5JDtl3TWDSphgInCsNN+h8LVk+nC6DtJoAQSqC6+t+b1SRzvVOWk+EmnRHTLULmjG7zTCNx0

[FalcoSuessgott]

Oh interesting! Will have a look, thanks for reporting.

[FalcoSuessgott]

Special characters like < and > are now preserved.

I cant reproduce the \n breaking the base format output. Can you provide me an example secret value that leads to the output you have provided?

[nichcuta]

I'm out right now so can't provide the actual value but it was something on the lines of: "value": "dhaisjckakqhdjckwk2940wjz\nkskfifoak48skfkalqorjc\nsksiwi3ifockqk1ofocowksfi\nfjskwi3if92oskvkzlap1p5010" The secret itself is stored in a nested sub directory structure. I expect that the value is rendered as a single line. I will provide the actual secret later. Thanks, Nic

…

On Sat, 11 May 2024, 12:28 FalcoSuessgott, ***@***.***> wrote: Special characters like < and > are now preserved. I cant reproduce the \n breaking the base format output. Can you provide me an example secret value that leads to the output you have provided? — Reply to this email directly, view it on GitHub <#255 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGCKD56DF6J2AFC5DAACEXLZBXXEXAVCNFSM6AAAAABHRUG5USVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBVGY3DOOBUGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[nichcuta]

So value is:

"example-keystore-jks_base64": "/u3+EqAhEBAQUABII7QAAAAIAAAABAAAAAQABMQAAAYtIxCqRAAAFATCCBP0wDgYKKwYBBAE6Z9q\nw2lvbDkbgRvTrsp3dMFBEnYqFurMXJMVnG+76o2JhexbzP2qA8EnUwzRorY8qffoD4OC19OvMkFg\n/am4W0GdZVPfyVNKZCosmGl5Zr6zYuxSR1a9j1fHrR05AQh+OTVOjIPh+KIgDFy/Q021S7R+mEgD\n6eKX5JDtl3TWDSphgInCsN1SRzvVOWk+EmnRHTLULmjG7zTCN+h8LVk+nC6DtJoAQSqC6+t+bNx0\nukVSh0CPv7tjX5z6XeMJW+PNQUR3pe/w4QbH3MAzEQhkCg1l93CWxoooRrgXmIdSf0qq4joA2kt/\nzn65p359dOEx+owXCMBh/JRZ3Gf253m+jyLTYp0kPHzJGwXwD+6GVs6dum3B+lZuc8v45VvXTE4B"

This is rendered like so:

            │   ├── example-keystore-jks_base64=/u3+EqAhEBAQUABII7QAAAAIAAAABAAAAAQABMQAAAYtIxCqRAAAFATCCBP0wDgYKKwYBBAE6Z9q
            │   │   w2lvbDkbgRvTrsp3dMFBEnYqFurMXJMVnG+76o2JhexbzP2qA8EnUwzRorY8qffoD4OC19OvMkFg
            │   │   /am4W0GdZVPfyVNKZCosmGl5Zr6zYuxSR1a9j1fHrR05AQh+OTVOjIPh+KIgDFy/Q021S7R+mEgD
            │   │   6eKX5JDtl3TWDSphgInCsN1SRzvVOWk+EmnRHTLULmjG7zTCN+h8LVk+nC6DtJoAQSqC6+t+bNx0
            │   │   ukVSh0CPv7tjX5z6XeMJW+PNQUR3pe/w4QbH3MAzEQhkCg1l93CWxoooRrgXmIdSf0qq4joA2kt/
            │   │   zn65p359dOEx+owXCMBh/JRZ3Gf253m+jyLTYp0kPHzJGwXwD+6GVs6dum3B+lZuc8v45VvXTE4B

Expected to be rendered as:

            │   ├── example-keystore-jks_base64=/u3+EqAhEBAQUABII7QAAAAIAAAABAAAAAQABMQAAAYtIxCqRAAAFATCCBP0wDgYKKwYBBAE6Z9q\nw2lvbDkbgRvTrsp3dMFBEnYqFurMXJMVnG+76o2JhexbzP2qA8EnUwzRorY8qffoD4OC19OvMkFg\n/am4W0GdZVPfyVNKZCosmGl5Zr6zYuxSR1a9j1fHrR05AQh+OTVOjIPh+KIgDFy/Q021S7R+mEgD\n6eKX5JDtl3TWDSphgInCsN1SRzvVOWk+EmnRHTLULmjG7zTCN+h8LVk+nC6DtJoAQSqC6+t+bNx0\nukVSh0CPv7tjX5z6XeMJW+PNQUR3pe/w4QbH3MAzEQhkCg1l93CWxoooRrgXmIdSf0qq4joA2kt/\nzn65p359dOEx+owXCMBh/JRZ3Gf253m+jyLTYp0kPHzJGwXwD+6GVs6dum3B+lZuc8v45VvXTE4B

Note: the above example is not a valid base64. Cant provide the actual base64 value for security reasons.

[FalcoSuessgott]

Thanks for the example. I dont really think I can avoid that .. I think the terminal will simply break the line once the line is longer than the terminal width which will lead to the underlaying library (gotree) to add another | in the front. But I will play around and see what I can do and get back to you

[FalcoSuessgott]

Bonjour @nichcuta!

I think I found a nice solution:

with vkv v0.7.0 secrets across multiple lines breakes the base output (Githubs Markdown Codeblocks do actually not break the lines like a terminal does ..)

> vkv export -p secret --show-values
secret/ [desc=key/value secret storage] [type=kv2]
├── admin [v=1] [key=value]
│   └── sub=password
├── demo [v=1]
│   └── foo=bar
└── sub
    ├── demo [v=1]
    │   ├── demo=hello world
    │   ├── password=s3cre5
    │   └── user=admin
    └── sub2
        └── demo [v=3] [admin=false key=value]
            ├── admin=key
            ├── foo=bar
            ├── key=/u3+EqAhEBAQUABII7QAAAAIAAAABAAAAAQABMQAAAYtIxCqRAAAFATCCBP0wDgYKKwYBBAE6Z9q\nw2lvbDkbgRvTrsp3dMFBEnYqFurMXJMVnG+76o2JhexbzP2qA8EnUwzRorY8qffoD4OC19OvMkFg\n/am4W0GdZVPfyVNKZCosmGl5Zr6zYuxSR1a9j1fHrR05AQh+OTVOjIPh+KIgDFy/Q021S7R+mEgD\n6eKX5JDtl3TWDSphgInCsN1SRzvVOWk+EmnRHTLULmjG7zTCN+h8LVk+nC6DtJoAQSqC6+t+bNx0\nukVSh0CPv7tjX5z6XeMJW+PNQUR3pe/w4QbH3MAzEQhkCg1l93CWxoooRrgXmIdSf0qq4joA2kt/\nzn65p359dOEx+owXCMBh/JRZ3Gf253m+jyLTYp0kPHzJGwXwD+6GVs6dum3B+lZuc8v45VvXTE4B
            ├── password=password
            └── user=user

with the changes from #257 secret values are now correctly indented by interpreting any \n:

> go run main.go export -p secret --show-values
secret/ [desc=key/value secret storage] [type=kv2]
├── admin [v=1] [key=value]
│   └── sub=password
│   
├── demo [v=1]
│   └── foo=bar
│   
└── sub
    ├── demo [v=1]
    │   ├── demo=hello world
    │   ├── password=s3cre5
    │   └── user=admin
    │   
    └── sub2
        └── demo [v=4] [admin=false key=value]
            ├── admin=key
            ├── foo=bar
            ├── key=/u3+EqAhEBAQUABII7QAAAAIAAAABAAAAAQABMQAAAYtIxCqRAAAFATCCBP0wDgYKKwYBBAE6Z9q
            │   w2lvbDkbgRvTrsp3dMFBEnYqFurMXJMVnG+76o2JhexbzP2qA8EnUwzRorY8qffoD4OC19OvMkFg
            │   /am4W0GdZVPfyVNKZCosmGl5Zr6zYuxSR1a9j1fHrR05AQh+OTVOjIPh+KIgDFy/Q021S7R+mEgD
            │   6eKX5JDtl3TWDSphgInCsN1SRzvVOWk+EmnRHTLULmjG7zTCN+h8LVk+nC6DtJoAQSqC6+t+bNx0
            │   ukVSh0CPv7tjX5z6XeMJW+PNQUR3pe/w4QbH3MAzEQhkCg1l93CWxoooRrgXmIdSf0qq4joA2kt/
            │   zn65p359dOEx+owXCMBh/JRZ3Gf253m+jyLTYp0kPHzJGwXwD+6GVs6dum3B+lZuc8v45VvXTE4B
            ├── password=password
            └── user=user

would that work for you?

[nichcuta]

Good morning @FalcoSuessgott ,

> vkv export -p secret --show-values
secret/ [desc=key/value secret storage] [type=kv2]
├── admin [v=1] [key=value]
│   └── sub=password
├── demo [v=1]
│   └── foo=bar
└── sub
    ├── demo [v=1]
    │   ├── demo=hello world
    │   ├── password=s3cre5
    │   └── user=admin
    └── sub2
        └── demo [v=3] [admin=false key=value]
            ├── admin=key
            ├── foo=bar
            ├── key=/u3+EqAhEBAQUABII7QAAAAIAAAABAAAAAQABMQAAAYtIxCqRAAAFATCCBP0wDgYKKwYBBAE6Z9q\nw2lvbDkbgRvTrsp3dMFBEnYqFurMXJMVnG+76o2JhexbzP2qA8EnUwzRorY8qffoD4OC19OvMkFg\n/am4W0GdZVPfyVNKZCosmGl5Zr6zYuxSR1a9j1fHrR05AQh+OTVOjIPh+KIgDFy/Q021S7R+mEgD\n6eKX5JDtl3TWDSphgInCsN1SRzvVOWk+EmnRHTLULmjG7zTCN+h8LVk+nC6DtJoAQSqC6+t+bNx0\nukVSh0CPv7tjX5z6XeMJW+PNQUR3pe/w4QbH3MAzEQhkCg1l93CWxoooRrgXmIdSf0qq4joA2kt/\nzn65p359dOEx+owXCMBh/JRZ3Gf253m+jyLTYp0kPHzJGwXwD+6GVs6dum3B+lZuc8v45VvXTE4B
            ├── password=password
            └── user=user

The key above is the current output. Perhaps zsh (the terminal i use) is causing this issue for me?
I think vkv should never interpret the '\n' so the secret value is preserved

Thanks,
Nic

[FalcoSuessgott]

Hi,

you're right, interpreting \n is probably not a good idea. I decided to stick to the behavior of the official vault CLI:

Which does not interpret special character. This equals now with vkv export:

As you can see in vault and vkv output, the terminal & shell simply break long lines at whatever width the current terminal session has ( I use zsh + alacritty). I think this is fine and actually inevitable.

[nichcuta]

Thats the behaviour i expected. Looks good to me :D

Thanks for fixing