hairyhenderson/gomplate

plugin function is executed within an if block that evaluates to false

Opened this issue · 5 comments

cofiem commented 
$ gomplate --version
gomplate version 4.1.0

This may be the intended behaviour, but it was surprising to me.

I expected that anything within an if block would not be executed when the pipeline evaluated to empty.

The plugin function pluginThatIsNotDefined does not exist, and I expected that it would not be executed (see below).

if this is intended, please document it somewhere.

From the go template doc:

{{if pipeline}} T1 {{end}}
	If the value of the pipeline is empty, no output is generated;
	otherwise, T1 is executed. The empty values are false, 0, any
	nil pointer or interface value, and any array, slice, map, or
	string of length zero.
	Dot is unaffected.

Command:

gomplate --verbose -i '{{ if false }}{{ pluginThatIsNotDefined "arg1" }}{{end}}'

Output:

13:47:29 DBG starting gomplate
13:47:29 DBG config is:
---
datasources:
  testDs:
    header: {}
    url: https://example.com
in: '{{ if fa...'
 version=4.1.0 build=cc2584028866967a39b096265d5b9af4516c734f
13:47:29 DBG completed rendering templatesRendered=0 errors=0 duration=231.987µs
13:47:29 ERR  err="renderTemplate: parse template <arg>: template: <arg>:1: function \"pluginThatIsNotDefined\" not defined"
hairyhenderson commented

The issue here is not that the plugin is executed... this error is happening while the template is parsed, before any of the logic is evaluated.

  • 👍1
cofiem commented

Ok. I didn't understand the difference between "parsing the template" and "executing the T1 in the conditional block".

Am I correct that this means there is no way to use some external way to create a gomplate config file, which may or may not define plugins, and then only execute a plugin if it exists, similar to datasources?

hairyhenderson commented

The reason datasourceExists works is because datasources are referenced by name (as strings). Plugins are different, in that they're referenced as keywords, and so must be parseable.

I suppose it would be possible to create a new way of executing plugins, where plugins were referenced by name rather than as a keyword. But that would be contrary to the main purpose of plugins, which is to provide a very simple way to add functionality to gomplate.

Can you give me some more concrete reasoning on why you feel this is necessary? What's the exact use-case?

cofiem commented

First, thanks very much for your time and for considering this.

My motivation is to create a config file that can access values the same way regardless of which cloud provider the file is used with. Currently AWS and OpenStack, soon also GCP.

I've tried a few approaches to this.
If you have suggestions for other ways, I'd be interested.

For now, I have solved this problem by templating both the gomplate config file and the templated config file using ansible first, so that only the cloud provider-specific plugin functions are present.
If you think this approach - templating the files first using some other process - is the way to go, then I'm happy with that, and this issue can be closed.

The approach that raised this issue was like this:

  • create a gomplate config file specific to each cloud provider
  • use go templating conditionals to decide which datasources / plugins to use
  • build the values that can be used the same way in the rest of the config file (e.g. simple variables, a map variable that has items added to it)
# gomplate config file for AWS
datasources:
  awsSecret:
    url: 'aws+sm:'
  awsAppConfig:
    url: 'aws+smp:'
  awsVmMetadata:
    url: 'aws+imds:'

# gomplate config file for OpenStack
datasources:
  openstackVmMetadata:
    url: 'http://169.254.169.254/openstack/2018-08-27/meta_data.json'

plugins:
  openstackSecret:
    cmd: '/opt/gomplate/barbican.sh'  # e.g. bash script like 'openstack secret get "$1" -p -f value'
    pipe: false
    timeout: '5s'
  openstackAppConfig:
    cmd: '/opt/gomplate/swift.sh'
    pipe: false
    timeout: '5s'

Then in a config file, for example:

{{- $common_value := "" -}}
{{- $secret_value := "" -}}

{{- if (datasourceExists awsVmMetadata) -}}
  {{- $common_value = aws.EC2Tag "SomeTag" -}}
  {{- $secret_value = (ds "awsSecret" "SomeKey") -}}
{{- else if (datasourceExists openstackVmMetadata) -}}
  {{- $common_value = (ds "openstackVmMetadata").meta.SomeTag -}}
  {{- $secret_value = (openstackSecret "SomeKey") -}}
{{- end -}}

On an AWS EC2 instance, with the AWS gomplate config file, the templated config file gave the 'openstackSecret not defined' error.

hairyhenderson commented

Hrm interesting... it seems to me the right way to approach this would be to have a datasource for Barbican. That being said, it's a fairly big lift and not something I have the time to work on any time soon.

From the openstackAppConfig plugin though - from the command it looks like it's pulling data from Swift? There's an S3-compatible API for that, maybe you could use an S3 datasource for that (with a custom endpoint)?

Also an alternative to using aws.EC2Tag could be to use the AWS IMDS datasource - I don't think I ever got around to documenting it, but it is available (in gomplate 4.1) with the aws+imds scheme - see https://pkg.go.dev/github.com/hairyhenderson/go-fsimpl/awsimdsfs for the docs for the go-fsimpl filesystem that backs it.

In general though, I see the dilemma.

One alternative you might consider, since these really are datasource-like things, is to pull the data you need from OpenStack out-of-band and store in a local json file. It should be possible then to arrange the template such that you don't need to call datsourceExists or have much logic around deciding which provider you're rendering for.

  • 👍1