libs get wrapped
Closed this issue · 9 comments
Hi, some of our templates get libs that are minified, resulting in long strings that we can't post-process.
My assumption is that konstraint shrinks output based on some limit.
Example:
- libs:
- "package lib.core\n\ndefault is_gatekeeper = false\n\nis_gatekeeper {\n\thas_field(input,
\"review\")\n\thas_field(input.review, \"object\")\n}\n\nresource = input.review.object
{\n\tis_gatekeeper\n}\n\nresource = input {\n\tnot is_gatekeeper\n}\n\nreview
= input.review {\n\tis_gatekeeper\n}\n\nreview = {\"object\": resource, \"kind\":
{\"group\": group, \"kind\": kind, \"version\": version}} {\n\tnot is_gatekeeper\n}\n\nparameters
= input.parameters {\n\ttrace(sprintf(\"has input.parameters: %v\", [input.parameters]))\n\tis_gatekeeper\n}\n\nopa_upstream_bug_1046
:= true\n\nparameters = data.parameters {\n\ttrace(sprintf(\"has data.parameters:
%v\", [data.parameters]))\n\tnot is_gatekeeper\n}\n\nformat(msg) = {\"msg\":
msg}\n\nformat_with_id(msg, id) = msg_fmt {\n\tmsg_fmt := {\n\t\t\"msg\": sprintf(\"%s:
%s\", [id, msg]),\n\t\t\"details\": {\"policyID\": id},\n\t}\n}\n\napiVersion
:= resource.apiVersion\n\nname := resource.metadata.name\n\ngv := split(apiVersion,
\"/\")\n\ngroup = gv[0] {\n\tcontains(apiVersion, \"/\")\n}\n\ngroup = \"core\"
{\n\tnot contains(apiVersion, \"/\")\n}\n\nversion := gv[minus(count(gv), 1)]\n\nkind
:= resource.kind\n\nlabels := resource.metadata.labels\n\nannotations := resource.metadata.annotations\n\nhas_field(obj,
field) {\n\tnot object.get(obj, field, \"N_DEFINED\") == \"N_DEFINED\"\n}\n\nmissing_field(obj,
field) {\n\tobj[field] == \"\"\n}\n\nmissing_field(obj, field) {\n\tnot has_field(obj,
field)\n}"If that is the case we would like to turn off this behaviour as we have the need to strip out certain lines because those interfere with gatekeeper parser which does not allow data.parameters refs to pass. (See issue open-policy-agent/gatekeeper#1046). If you need this shrinking behaviour, then please allow us to run it a as a post-step.
@Morriz are you running the latest version of Konstraint (v0.12.1)? We've seen this behavior in the past, but they should be resolved now. It's more to do with how some Go libraries parse YAML than the size of the file.
If you are and still seeing this behavior, which OS are you running and do you have an example to replicate this?
I am not sure it happened in your lib now. Also managed to work around it. Closing for now. Tnx
@Morriz Hey there. How did you managed to work around the issue? I got similar thing right now.
@usualstuff can you provide more information about your scenario? The version of Konstraint that you're running and your Operating System would be really helpful!
@Morriz Hey there. How did you managed to work around the issue? I got similar thing right now.
After inspecting the konstraint output files I noticed it wasn't minifying the output at all. It was a subsequent step translating it. (kubectl?) Anyways, we have control over the output so I closed this.
@jpreese I'm using v0.13.0 installed with command from the readme. OS is Ubuntu 18.04, Go version 1.16.
I have a simple policy which imports a couple of libraries which themselves have some imports. Overall libs size is around 8k characters.
@Morriz thanks. Looks like I stumbled upon something different.
@jpreese I think I found the culprit: it's tab characters :) Once I removed them from an every involved file the generated code became readable again.
@jpreese I think I found the culprit: it's tab characters :) Once I removed them from an every involved file the generated code became readable again.
@usualstuff Brilliant! We should be able to fix that. See here (https://github.com/plexsystems/konstraint/blob/main/internal/rego/rego.go#L238) how we remove newlines. I'll add tabs to the list and see if that fixes it for you.