local exec fails to parse terraform.py

[dnitsch]

There seems to be a problem parsing the terraform.py inside a local-exec provisioner in Terraform

Taking this as an example trigger in terraform to run ansible

resource "null_resource" "jenkins_ansible" {
  // count = 2 ### "${length(list(var.jenkins_private_ip))}"
  triggers {
    // jenkins_ips = "${var.jenkins_private_ip[count.index]}"
    jenkins_ips = "${var.jenkins_private_ip}"
  }

  provisioner "local-exec" {
    interpreter = ["/bin/bash", "-c"]
    command = "cd ${var.ansible_dir}; ANSIBLE_TF_DIR=${var.ANSIBLE_TF_DIR} ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook --inventory-file=${var.ansible_dir}/terraform.py jenkins/master.yml"
  }
}

gives me this output

[WARNING]:  * Failed to parse /Users/moi/ansible/sandbox-
test/terraform.py with script plugin: Inventory script
(/Users/moi/ansible/sandbox-test/terraform.py) had an execution
error: 2018/03/14 13:44:52 [INFO] Terraform version: 0.11.3
3802b14260603f90c7a1faf55994dcc8933e2069 2018/03/14 13:44:52 [INFO] Go runtime
version: go1.9.1 2018/03/14 13:44:52 [INFO] CLI args: []string{"terraform",
"state", "pull", "-input=false"} 2018/03/14 13:44:52 [DEBUG] Attempting to open
CLI config file: /Users/moi/.terraformrc 2018/03/14 13:44:52
[DEBUG] File doesn't exist, but doesn't need to. Ignoring. 2018/03/14 13:44:52
[INFO] CLI command args: []string{"state", "pull", "-input=false"} 2018/03/14
13:44:52 [DEBUG] command: loading backend config file:
/Users/moi/foo/git-dev/idam/idam-iac/devops-
idam-static-dev 2018/03/14 13:44:52 [WARN] BackendOpts.Config not set, but
config found 2018/03/14 13:44:52 [TRACE] Preserving existing state lineage
"a34ef702-b45c-49ee-ab06-985baa37f7eb" 2018/03/14 13:44:52 [TRACE] Preserving
existing state lineage "a34ef702-b45c-49ee-ab06-985baa37f7eb" 2018/03/14
13:44:52 [INFO] Building AWS region structure 2018/03/14 13:44:52 [INFO]
Building AWS auth structure 2018/03/14 13:44:52 [INFO] Setting AWS metadata API
timeout to 100ms 2018/03/14 13:44:53 [INFO] Ignoring AWS metadata API endpoint
at default location as it doesn't return any instance-id 2018/03/14 13:44:53
[INFO] AWS Auth provider used: "SharedCredentialsProvider" 2018/03/14 13:44:53
[INFO] Initializing DeviceFarm SDK connection 2018/03/14 13:44:53 [DEBUG]
Trying to get account ID via iam:GetUser 2018/03/14 13:44:54 [INFO] command:
backend initialized: *s3.Backend 2018/03/14 13:44:54 [DEBUG] checking for
provider in "." 2018/03/14 13:44:54 [DEBUG] checking for provider in
"/terraform" 2018/03/14 13:44:54 [DEBUG] checking for provider in
".terraform/plugins/darwin_amd64" 2018/03/14 13:44:54 [DEBUG] found provider
"terraform-provider-aws_v1.11.0_x4" 2018/03/14 13:44:54 [DEBUG] found provider
"terraform-provider-null_v1.0.0_x4" 2018/03/14 13:44:54 [DEBUG] checking for
provider in "/Users/moi/.terraform.d/plugins" 2018/03/14 13:44:54
[DEBUG] checking for provider in
"/Users/moi/.terraform.d/plugins/darwin_amd64" 2018/03/14
13:44:54 [DEBUG] found provider "terraform-provider-ansible_v0.0.2" 2018/03/14
13:44:54 [DEBUG] found valid plugin: "aws", "1.11.0",
"/Users/moi/foo/git-dev/idam/idam-iac
/devops-idam-static-dev/.terraform/plugins/darwin_amd64/terraform-provider-
aws_v1.11.0_x4" 2018/03/14 13:44:54 [DEBUG] found valid plugin: "null",
"1.0.0", "/Users/moi/foo/git-dev/idam/idam-
iac/devops-idam-static-dev/.terraform/plugins/darwin_amd64/terraform-provider-
null_v1.0.0_x4" 2018/03/14 13:44:54 [DEBUG] found valid plugin: "ansible",
"0.0.2", "/Users/moi/.terraform.d/plugins/darwin_amd64/terraform-
provider-ansible_v0.0.2" 2018/03/14 13:44:54 [DEBUG] checking for provisioner
in "." 2018/03/14 13:44:54 [DEBUG] checking for provisioner in "/terraform"
2018/03/14 13:44:54 [DEBUG] checking for provisioner in
".terraform/plugins/darwin_amd64" 2018/03/14 13:44:54 [DEBUG] checking for
provisioner in "/Users/moi/.terraform.d/plugins" 2018/03/14
13:44:54 [DEBUG] checking for provisioner in
"/Users/moi/.terraform.d/plugins/darwin_amd64" 2018/03/14
13:44:54 [INFO] command: backend *s3.Backend is not enhanced, wrapping in local
2018/03/14 13:44:54 [TRACE] Preserving existing state lineage "294a1e0c-
cb5d-4365-90e6-0f6a9cecb4fc" 2018/03/14 13:44:54 [TRACE] Preserving existing
state lineage "294a1e0c-cb5d-4365-90e6-0f6a9cecb4fc" 2018/03/14 13:44:54
[TRACE] Preserving existing state lineage "294a1e0c-
cb5d-4365-90e6-0f6a9cecb4fc" 2018/03/14 13:44:54 [DEBUG] plugin: waiting for
all plugin processes to complete...
 [WARNING]:  * Failed to parse /Users/moi/ansible/sandbox-
test/terraform.py with ini plugin: /Users/moi/ansible/sandbox-
test/terraform.py:3: Expected key=value host variable assignment, got: json
 [WARNING]: Unable to parse /Users/moi/ansible/sandbox-
test/terraform.py as an inventory source
 [WARNING]: No inventory was parsed, only implicit localhost is available
 [WARNING]: provided hosts list is empty, only localhost is available. Note
that the implicit localhost does not match 'all'
 [WARNING]: Could not match supplied host pattern, ignoring: jenkins_hosts

when the same command is run in the terminal it works just fine

$ cd /Users/moi/ansible/sandbox-test; ANSIBLE_TF_DIR=/Users/moi/foo/git-dev/idam/idam-iac/devops-idam-static-dev ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook --inventory-file=/Users/moi/ansible/sandbox-test/terraform.py jenkins/master.yml

Should it be used like that within Terraform?

As I am very close to creating a separate application to that is triggered on remote state callback that will then trigger the ansible scripts, and have a feeling it could be a more logical way to separate the concern and domains.

Wondering if anyone else came across the same issue.

[nbering]

This inventory script is meant to be run after Terraform is done. It uses terraform state pull to get your Terraform State, so if you run while changes are being applied, the state will be locked.

That said, it looks like you don't have executable permissions on terraform.py.

Try chmod +x terraform.py.

[nbering]

I could be wrong above. Can you share your ansible version and ansible.cfg?

[dnitsch]

Ok, that could have been part of the problem (the locking, d'oh, I mean, not the perms that was fine) - in fact it was most likely that when ran as part of the null_resource on trigger. I have moved it out to separate callback application so that it's guaranteed to run after the state has been released.

[nbering]

If that isn't the problem, I'd next look at whether you've somehow configured Ansible 2.4+ to restrict the Inventory parsing mechanism. There's a bunch of things to look at there if this still doesn't work after you run outside the Terraform session.

[nbering]

Sounds like no changes were required to fix this issue. Closing... will reopen if there's still a problem.