Incorporate HIL into Gru

[dnaeon]

https://github.com/hashicorp/hil

[dnaeon]

Thinking about this one a bit more, it may turn out that hil is probably not the most appropriate interpolation package we might want to use in Gru.

Consider the following example where we declare the variables to use in a separate file, where each variable is expressed in HCL.

var "bar" {
  value = "foo"
}

var "baz" {
  value = "qux"
}

And a simple Gru module which uses the above variables:

package "${var.bar}" {
  state = "present"
}

In order to get the ${var.bar} variable interpolated we must do a 3-step parsing.

1. First parse the variable declarations from HCL and deserialize into proper HIL AST tree.
2. Use the result variables from step one and parse the Gru module using HIL in order to interpolate any variable references
3. Parse the result HIL program using HCL in order to deserialize into the proper Go types

One problem with this approach is that we will not be able to use intermediate variable interpolation, e.g. a variable which is constructed from another variable. This makes things such as this one not possible, unless doing some kind of recursive HCL-to-HIL interpolation, which is something that we should never consider doing.

var "foo" {
  value = "bar"
}

var "baz" {
  value = "/path/to/${var.foo}"
}

Another shortcoming of HIL is the fact that it is not a full-featured interpolation language, and certain things such iteration, conditionals, etc. are missing in it, which does not allow for doing complex login within the Gru modules.

All and all, HIL is not suitable for the purposes we want to accomplish with Gru, so closing this issue. It seems that Terraform is also suffering from these issues as seen in hashicorp/terraform#4084, hashicorp/terraform#444, hashicorp/terraform#4397 and hashicorp/terraform#2772

A better approach might be to use lua as the configuration and data language for Gru modules.

[rdeusser]

Hey @dnaeon! I'm loving Gru right now. I know this is mostly irrelevant since you switched Gru over to use Lua as the configuration language, but instead of HCL and HIL did you consider using HCL with text/template? At a glance, it seems that would have suited your needs.

[dnaeon]

Hey,

I thought about text/html as well, but at the same time I wanted to avoid the strange mix of languages used as the DSL. Take Ansible for example - mixes yaml and jinja2, which I personally don't like at all.

Having Lua onboard would also allow for writing custom modules and plugins, maybe even have resources completely written in Lua. In general I think that having Lua would allow the DSL to evolve, which is always a good thing :)

[kaey]

Having working with ansible and salt, I too would prefer to avoid that yaml+jinja horror.

While I would prefer javascript, just because much more people know it and learn it, lua is a great choice as well.

[rdeusser]

While I agree yaml+jinja is a nightmare, I think HCL+text/template would be quite beautiful together.

lua is a great choice though.

[dnaeon]

Hey folks,

Thanks for your input on this one.

One of the nice things about the current implementation of resources is that they are written in pure Go and are not bound to any specific config language. We simply register new functionality into the Lua VM.

That means we should be able to easily switch to something else, if there's a need for it - be that JS, HCL+text/template, etc.

If anyone wants to experiment with a different language for the DSL, please feel free to submit a separate issue where we can further discuss it.

Thanks,
Marin