Feat: add variables, possibly using default values for substitutions

[mikemccracken]

Is your feature request related to a problem? Please describe.

a common pattern is a build container that pulls some tag of something, builds it and then munges the result so that a final container can be built by importing a directory from the build container and copying out of it.

often the tag is repeated lots of places, like this

imagename:
  import:
    - https://github.com/org/repo/archive/TAG.tar.gz
  run:
    tar zxvf /stacker/TAG.tar.gz
    cd TAG
    make

it's annoying and error prone to have to update TAG in multiple places.

if you have TAG set up as a substitution then this is solved, but that means that the stacker file is not really self contained, and in order to repeat a build you need the stacker file and whatever stacker command line you used to build it.

Ideally there would be a way to define a substitution and value explicitly in the stackerfile itself

Describe the solution you'd like

One possibility would be to add a top level substitutions map

substitutions:
  - { key: "VERSION", default: "v1.2.2"}

then ${{VERSION}} could be used anywhere in the stackerfile just as if it was a regular substitution. If we provide a --substitution VERSION=whatever on the command line, it would override the file's default (and stacker would log that)

Describe alternatives you've considered

we have projects that have variations on the theme of a separate subs file that declares things that then get added to the final stacker command line, but as mentioned above this makes the stackerfile not self contained, and makes it harder to track down where values come from

Additional context

No response

[rchincha]

Wouldn't --substitute-file not work for this?

[mikemccracken]

Not really, that's still not a self contained stackerfile, and I will probably need to dig around in some build system to understand how the substitute file gets generated.

I basically just want a global variable for the stackerfile. It is actually less important to me that it is a substitution - if it was just a #define that couldn't be overridden, that'd also be OK.

[mikemccracken]

I will probably need to dig around in some build system to understand how the substitute file gets generated.

What if instead of substitute-file, we had this substitutions dictionary in each stacker file and we could import them from other stacker files using the prerequisite name. then we could be able to tell where substitutions were coming from without having to trace through some build pipeline to figure out how the command line was being built up and what files it was generating

[smoser]

I will probably need to dig around in some build system to understand how the substitute file gets generated.

What if instead of substitute-file, we had this substitutions dictionary in each stacker file and we could import them from other stacker files using the prerequisite name. then we could be able to tell where substitutions were coming from without having to trace through some build pipeline to figure out how the command line was being built up and what files it was generating

Maybe even just replace the "substitute file" with your suggestion entirely.

• All stacker files can have a substitutions dictionary
• substitutions from stacker file A that imports stacker file B will override the substitutions in stacker file B.

[mikemccracken]

yeah I think having a substitutions dict in stacker files (and allowing a stackerfile that is only a substitutions dict to be imported into others) would remove the need for substitute-file and probably remove the need for passing 32 --substitute args to stacker like we do in some internal builds.

IIUC the only reason that internal build has a ton of substitute args like that is that the cmdline is the only place to put substitutions, and the choice was to either have a different stacker cmdline for each layer (madness) or do what it does there, which is to have one cmdline that is a superset of all required substitutions. if there were ever conflicts between subs needed for layers this would get worse...