Multiline secrets are not correctly parsed
djuarezg opened this issue · 11 comments
Bug:
What happened:
If you follow https://github.com/openshift/ansible-service-broker/blob/master/docs/secrets.md and try to add a multiline secret as in:
---
apiVersion: v1
kind: Secret
metadata:
name: test
namespace: openshift-automation-service-broker
stringData:
"test1": "test1"
"test2": "test2"
"test_multiline": |-
-----BEGIN RSA PRIVATE KEY-----
<FIRST LINE OF THE SSH KEY>
<SECOND LINE OF THE SSH KEY>
the Ansible Playbook Bundle will see an error while loading the secrets YAML file, as if it was using newlines to separate secrets:
ERROR! Syntax Error while loading YAML.
could not find expected ':'
The error appears to have been in '/tmp/secrets': line 6, column 1, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
<FIRST LINE OF THE SSH KEY>
<SECOND LINE OF THE SSH KEY>
^ here
This happens as well if you use the base64 data secret.
What you expected to happen:
The secret should keep the newlines and be used as a parameter on the APB.
Mounted secrets are copied to /tmp/secrets so they can be passed as parameters to the playbook, but instead of producing this expected secrets file:
---
ACCESS_KEY: blah
SECRET_KEY: blah
SWARM_CLUSTER_KEYPAIR: |-
-----BEGIN RSA PRIVATE KEY-----
blah
blah
blah
-----END RSA PRIVATE KEY-----
openstack_admin__user: blah
openstack_admin_password: blah
They produce something like this, which will fail during parsing:
---
ACCESS_KEY: blah
SECRET_KEY: blah
SWARM_CLUSTER_KEYPAIR: -----BEGIN RSA PRIVATE KEY-----
blah1 blah2
blah3 ...
-----END RSA PRIVATE KEY-----
openstack_admin__user: blah
openstack_admin_password: blah
The python tool used to create the secret does indeed use a newline for the data. I will fix this.
@djuarezg a pre-release of 1.4 is out. There is still quite a bit of time to get 1.4 bugs fixed. We don't have a good version mechanism for Release Candidates or betas.
Created bugzilla to track this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1649075
@djuarezg FINALLY looking into this. I can't seem to recreate the scenario you've mentioned above. I took the snippet of the tool that creates the secret to try to debug it. My secret ends up being base64 encoded for the values.
additional_keys.yml
---
ACCESS_KEY: blah
SECRET_KEY: blah
SWARM_CLUSTER_KEYPAIR: |-
-----BEGIN RSA PRIVATE KEY-----
blah
blah
blah
-----END RSA PRIVATE KEY-----
openstack_admin__user: blah
openstack_admin_password: blahThe secret in openshift looks like this:
apiVersion: v1
data:
ACCESS_KEY: ImJsYWgi
SECRET_KEY: ImJsYWgi
SWARM_CLUSTER_KEYPAIR: Ii0tLS0tQkVHSU4gUlNBIFBSSVZBVEUgS0VZLS0tLS0KYmxhaApibGFoCmJsYWgKLS0tLS1FTkQgUlNBIFBSSVZBVEUgS0VZLS0tLS0i
key1: ImhlbGxvIg==
key2: IndvcmxkIg==
openstack_admin__user: ImJsYWgi
openstack_admin_password: ImJsYWgi
kind: Secret
metadata:
creationTimestamp: 2019-02-11T21:23:54Z
name: testname
namespace: broker-ns
resourceVersion: "3238"
selfLink: /api/v1/namespaces/broker-ns/secrets/testname
uid: 55c378a5-2e43-11e9-a6ca-64006a559cc9
type: OpaqueIf I decode the SWARM_CLUSTER_KEYPAIR with base64 I see the following:
"-----BEGIN RSA PRIVATE KEY-----
blah
blah
blah
-----END RSA PRIVATE KEY-----"
This is the script I was using to test it is a subset of the create_broker_secret.py script.
https://gist.github.com/jmrodri/169be3a1fc2cb232df81604481f9a6b6
#! /usr/bin/env python
import sys
import base64
import subprocess
# Output some nicer errors if a user doesn't have the required packages
try:
import yaml
except Exception:
print("No yaml parsing modules installed, try: pip install pyyaml")
sys.exit(1)
# Work around python2/3 input differences
try:
input = raw_input
except NameError:
pass
USAGE = """USAGE:
{command} NAME NAMESPACE IMAGE [BROKER_NAME] [KEY=VALUE]* [@FILE]*
NAME: the name of the secret to create/replace
NAMESPACE: the target namespace of the secret. It should be the namespace of the broker for most usecases
IMAGE: the docker image you would like to associate with the secret
BROKER_NAME: the name of the k8s ServiceBroker resource. Defaults to ansible-service-broker
KEY: a key to create inside the secret. This cannot contain an "=" sign
VALUE: the value for the KEY in the secret
FILE: a yaml loadable file containing key: value pairs. A file must begin with an "@" symbol to be loaded
EXAMPLE:
{command} mysecret ansible-service-broker docker.io/ansibleplaybookbundle/hello-world-apb key1=hello key2=world @additional_keys.yml
"""
DATA_SEPARATOR = "\n "
SECRET_TEMPLATE = """---
apiVersion: v1
kind: Secret
metadata:
name: {name}
namespace: {namespace}
data:
{data}
"""
def main():
name = sys.argv[1]
namespace = sys.argv[2]
apb = sys.argv[3]
if '=' not in sys.argv[4] and '@' not in sys.argv[4]:
broker_name = sys.argv[4]
idx = 4
else:
broker_name = None
idx = 3
keyvalues = list(map(
lambda x: x.split("=", 1),
filter(lambda x: "=" in x, sys.argv[idx:])
))
files = list(filter(lambda x: x.startswith("@"), sys.argv[idx:]))
data = keyvalues + parse_files(files)
create_secret(name, namespace, data)
def parse_files(files):
params = []
for file in files:
file_name = file[1:]
with open(file_name, 'r') as f:
params.extend(yaml.load(f.read()).items())
return params
def create_secret(name, namespace, data):
encoded = [(quote(k), base64.b64encode(quote(v))) for (k, v) in data]
secret = SECRET_TEMPLATE.format(
name=name,
namespace=namespace,
data=DATA_SEPARATOR.join(map(": ".join, encoded))
)
with open('/tmp/{name}-secret'.format(name=name), 'w') as f:
f.write(secret)
print('oc create -f /tmp/{name}-secret'.format(name=name))
print('Created secret: \n\n{}'.format(secret))
def quote(string):
return '"{}"'.format(string)
if __name__ == '__main__':
if len(sys.argv) < 5 or sys.argv[1] in ("-h", "--help"):
print(USAGE.format(command=sys.argv[0]))
sys.exit()
try:
main()
except Exception:
print("Invalid invocation")
print(USAGE.format(command=sys.argv[0]))
raiseCan you provide a the sample input file and exactly how you were invoking the create_broker_secret.py script? I might be missing something.
No feedback in 22 days and I was not able to recreate the problem.
Sorry I could not come back to this issue before. Thank you for your script for local secret generation, it is really useful.
@jmrodri Just one question, secrets are eventually stored in /opt/apb/env/passwords. How do I access these variables from my Ansible roles?
Extravars are accessible through env vars, but passwords are not. If I want to access them I have to manually add a task including this file as vars.