virtuald/pyhcl

Multiple keys with the same name

Closed this issue · 11 comments

According to: https://github.com/hashicorp/consul-template, HCL supports constructs like the following:

template {
  source = "/path/on/disk/to/template"
  destination = "/path/on/disk/where/template/will/render"
  command = "optional command to run when the template is updated"
}

template {
  // Multiple definitions are supported
}

When I parse such a config file using pyhcl (0.18), only the first template key is available:

{u'consul': u'someurl:8500', u'template': {u'source': u'/tmp/first.conf', u'destination': u'/tmp/config/first.conf', u'command': u'curl localhost || true'}}

The right way would probably to make the key template a list within the python dict:

{ ..., 'template': [{ _my_first_template_}, {_my_second_template_}], .... }

Thanks for the bug report, I'll look into it tomorrow.

Any progress on this issue?

Sorry about that, it totally slipped my mind. I'll look at it now.

Confirmed bug, used hcl2json to generate expected json, will add a test/fix for this.

Ok, I'm going to step back from this until hashicorp/terraform#807 and hashicorp/hcl#24 are resolved. HCL appears to be conflicted internally about what the output should actually be.

Just stumbled across this when discussing pyhcl with @phrawzty and we realized that https://github.com/mozilla/socorro-infra/blob/master/terraform/main.tf can't be easily converted to yaml because it's got a large number of resource definitions for the same kinds of resources, and they end up collapsed.

provider:
  aws:
    access_key: ${var.access_key}
    region: ${var.region}
    secret_key: ${var.secret_key}
resource:
  aws_elb:
    elb_for_webheads:
      availability_zones:
      - ${aws_instance.webheads.*.availability_zone}
      instances:
      - ${aws_instance.webheads.*.id}
      listener:
        instance_port: 80
        instance_protocol: http
        lb_port: 80
        lb_protocol: http
      name: elb-for-webheads
      security_groups:
      - ${aws_security_group.internet_to_elb__http.id}
  aws_instance:
    admin_host:
      ami: ${lookup(var.base_ami, var.region)}
      count: 1
      instance_type: t2.micro
      key_name: ${lookup(var.ssh_key_name, var.region)}
      security_groups:
      - ${aws_security_group.internet_to_any__ssh.name}
      - ${aws_security_group.private_to_private__any.name}
  aws_security_group:
    internet_to_snowflakes__http:
      description: Allow HTTP access to some oddball nodes.
      ingress:
        cidr_blocks:
        - 0.0.0.0/0
        from_port: 80
        protocol: tcp
        to_port: 80
      name: internet_to_snowflakes__http

hashicorp/terraform#807 was closed with a fix in terraform and hashicorp/hcl#24 seems to be awaiting code review, but it seems pretty clear that this is going to be a common way that HCL files are written for Terraform.

Is there something that can be done to handle this in the mean time? Knowing that pyhcl can handle Terraform configuration files would greatly help me as I could really use it in some Ansible modules I'm working on to help drive Terraform based deploys.

I have zero bandwidth to work on this until after February 18.

Feel free to submit a PR with additional fixtures -- I think more examples need to be created that show this issue and their equivalent in json, so that we can be sure that both hcl and pyhcl are on the same page.

Here's another example of this issue.

There are 2 resources in this template: https://gist.github.com/gposton/27d5f63d2317eb5f75cb

But when parsed with pyhcl, the json only includes 1 resource.

Here's the output: https://gist.github.com/gposton/3fb0055348b4d8b79147

I'm not motivated to fix this at the moment, but I'm happy to accept any PRs that will resolve this issue.

PR for this here: #6

Excited to see this get in!

Fixed in #6, pushed to pypi.