Action not working with Terraform Cloud

Question

Action not working with Terraform Cloud

alex-feel opened this issue 3 years ago · 12 comments

First I used this guide. After I got the error:

Error: terraform plan failed: exit status 1

Try setting the --path to a Terraform plan JSON file. See https://infracost.io/troubleshoot for how to generate this.
Error: Process completed with exit code 1.

I have tried several other options, all without success.

My last GitHub Actions configuration looks like this:

name: Terraform

on:
  push:
    branches:
      - main
    paths:
      - '**.tf'
  pull_request:
    paths:
      - '**.tf'

jobs:
  terraform:
    name: Terraform
    runs-on: ubuntu-latest
    env:
      working-directory: .
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1
        with:
          cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}
          terraform_version: 1.1.5

      - name: Terraform Format
        id: fmt
        run: terraform fmt -check -recursive
        working-directory: ${{ env.working-directory }}

      - name: Terraform Init
        id: init
        run: terraform init
        working-directory: ${{ env.working-directory }}

      - name: Terraform Validate
        id: validate
        run: terraform validate -no-color
        working-directory: ${{ env.working-directory }}

      - name: Setup TFLint
        uses: terraform-linters/setup-tflint@v1
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}

      - name: Run TFLint
        run: tflint --format compact
        working-directory: ${{ env.working-directory }}

      - name: Terraform Plan
        id: plan
        if: github.event_name == 'pull_request'
        run: terraform plan -no-color -compact-warnings
        working-directory: ${{ env.working-directory }}
        continue-on-error: true

      - name: Truncate Terraform Plan
        if: github.event_name == 'pull_request'
        run: |
          plan=$(cat <<'EOF'
          ${{ format('{0}{1}', steps.plan.outputs.stdout, steps.plan.outputs.stderr) }}
          EOF
          )
          echo "PLAN<<EOF" >> $GITHUB_ENV
          echo "${plan:0:65536}" >> $GITHUB_ENV
          echo "EOF" >> $GITHUB_ENV

      - name: Update Pull Request
        uses: actions/github-script@v5
        if: github.event_name == 'pull_request'
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          script: |
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: `
            **Terraform Format and Style**: \`${{ steps.fmt.outcome }}\`
            **Terraform Initialization**: \`${{ steps.init.outcome }}\`
            **Terraform Plan**: \`${{ steps.plan.outcome }}\`
            **Terraform Validation**: \`${{ steps.validate.outcome }}\`

            <details><summary>Plan Details</summary>

            \`\`\`terraform
            ${process.env.PLAN}
            \`\`\`\

            </details>

            **Pusher**: @${{ github.actor }}
            **Action**: \`${{ github.event_name }}\`
            `
            })

      - name: Terraform Plan Status
        if: steps.plan.outcome == 'failure'
        run: exit 1

      - name: Setup Infracost
        if: github.event_name == 'pull_request'
        uses: infracost/actions/setup@v1
        with:
          api-key: ${{ secrets.INFRACOST_API_KEY }}

      - name: Generate Infracost JSON
        if: github.event_name == 'pull_request'
        run: infracost breakdown --path ${{ env.working-directory }} --format json --out-file /tmp/infracost.json
        working-directory: ${{ env.working-directory }}

      - name: Post Infracost Comment
        if: github.event_name == 'pull_request'
        uses: infracost/actions/comment@v1
        with:
          path: /tmp/infracost.json
          behavior: new

      - name: Terraform Apply
        if: github.ref == 'refs/heads/main' && github.event_name == 'push'
        run: terraform apply -auto-approve
        working-directory: ${{ env.working-directory }}

All *.tf files are located in the root of the repository.

Each run of Generate Infracost JSON step looks like this:

Run infracost breakdown --path . --format json --out-file /tmp/infracost.json
  infracost breakdown --path . --format json --out-file /tmp/infracost.json
  shell: /usr/bin/bash -e {0}
  env:
    working-directory: .
    TERRAFORM_CLI_PATH: /home/runner/work/_temp/f790a945-b2d2-4cfb-b698-7f567a359822
    PLAN: Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
  will stop streaming the logs, but will not stop the plan running remotely.
  
  Preparing the remote plan...
  
  To view this run in a browser, visit:
  https://app.terraform.io/app/website/test/runs/run-9d2nNaVjbf4sPWG5
  
  Waiting for the plan to start...
  
  Terraform v1.1.5
  on linux_amd64
  Configuring remote state backend...
  Initializing Terraform configuration...


(removed for brevity)

  
  Plan: 4 to add, 0 to change, 0 to destroy.
  ╷
  │ Warning: Experimental feature "module_variable_optional_attrs" is active
  │ 
  │   on .terraform/modules/acme_com/main.tf line 2, in terraform:
  │    2:   experiments = [module_variable_optional_attrs]
  │ 
  │ Experimental features are subject to breaking changes in future minor or
  │ patch releases, based on feedback.
  │ 
  │ If you have feedback on the design of this feature, please open a GitHub
  │ issue to discuss it.
  ╵
    INFRACOST_GITHUB_ACTION: true
    INFRACOST_SKIP_UPDATE_CHECK: true
    INFRACOST_VCS_REPOSITORY_URL: https://github.com/website/terraform-test
    INFRACOST_VCS_PULL_REQUEST_URL: https://github.com/website/terraform-test/pull/13
    INFRACOST_LOG_LEVEL: info
time="2022-02-22T18:00:31Z" level=info msg="Detected Terraform directory at ."
time="2022-02-22T18:00:31Z" level=info msg="Starting: Running terraform plan"

Error: terraform plan failed: exit status 1

Try setting the --path to a Terraform plan JSON file. See https://infracost.io/troubleshoot for how to generate this.
Error: Process completed with exit code 1.

I have several questions:

It says that you can point to the Terraform configuration directory, how do you specify that the directory is the top level directory in the repository? I've tried different options - none work.
How exactly to specify parameters when using Action? Here = sign is used between the parameter and its value like this: infracost breakdown --path=examples/terraform-cloud-enterprise/code --format=json --out-file=/tmp/infracost.json, and on the main page the parameter and its value are separated by a space like this: infracost breakdown --path plan.json --format json --out-file /tmp/infracost.json. Are both options acceptable or is the documentation out of date?
How to make it work with Terraform Cloud?

Answer 1 · 2022-02-22T20:45:50.000Z

hi @alex-feel thanks for reporting this. Something looks awry here. To answer your questions:

Setting . should work fine. You can see from the log line: time="2022-02-22T18:00:31Z" level=info msg="Detected Terraform directory at ." that infracost has detected this. So this isn't the problem.
Both = and spaces work to set flags
I need some more info to figure out whats going on here. Could you set a repository secret ACTIONS_RUNNER_DEBUG and set it to true (as outlined here: https://docs.github.com/en/actions/monitoring-and-troubleshooting-workflows/enabling-debug-logging) and then run the workflow again. This should give you some further debug logs from infracost which can hopefully find the issue.

Alternatively you can just try and generate a plan out file and pass this to infracost. This is what the Try setting the --path to a Terraform plan JSON file is referring to. This can be done by running:

terraform plan -out tfplan.binary 
terraform show -json tfplan.binary > plan.json

and passing the plan.json to infracost:

infracost breakdown --path plan.json --format json --out-file /tmp/infracost.json

So in your instance you could modify the terraform plan step:

     - name: Terraform Plan
        id: plan
        if: github.event_name == 'pull_request'
        run: |
            terraform plan -no-color -compact-warnings -out /tmp/tfplan.binary
            terraform show -json /tmp/tfplan.binary > /tmp/plan.json
        working-directory: ${{ env.working-directory }}
        continue-on-error: true

then infracost step:

    - name: Generate Infracost JSON
       if: github.event_name == 'pull_request'
       run: infracost breakdown --path /tmp/plan.json --format json --out-file /tmp/infracost.json
       working-directory: ${{ env.working-directory }}

Answer 2 · 2022-02-22T20:55:36.000Z

Hi @hugorut, thanks for all the info! I'll be able to get extra logs tomorrow.

As for the file generation option, nothing will work here - Terraform cannot save files locally if it runs remotely, I already tried this option.

Answer 3 · 2022-02-22T20:59:54.000Z

@alex-feel ah yes you are correct. Ping me with the debug output tomorrow and we can see if this gives us any more info.

Answer 4 · 2022-02-22T22:23:36.000Z

As for the file generation option, nothing will work here - Terraform cannot save files locally if it runs remotely, I already tried this option.

Yeah Terraform Cloud doesn't work with -out but if it helps to debug, a plan JSON be generated using some bash like this: https://www.infracost.io/docs/troubleshooting/#terraform-cloud

Answer 5 · 2022-02-23T07:53:00.000Z

Hi guys, I found the reason and it is unexpected - Infracost is trying to run Terraform with the -out key, but we all know that this will not work 🤷‍♂️

Here is the detailed log of this step:

##[debug]Evaluating condition for step: 'Generate Infracost JSON'
##[debug]Evaluating: (success() && (github.event_name == 'pull_request'))
##[debug]Evaluating And:
##[debug]..Evaluating success:
##[debug]..=> true
##[debug]..Evaluating Equal:
##[debug]....Evaluating Index:
##[debug]......Evaluating github:
##[debug]......=> Object
##[debug]......Evaluating String:
##[debug]......=> 'event_name'
##[debug]....=> 'pull_request'
##[debug]....Evaluating String:
##[debug]....=> 'pull_request'
##[debug]..=> true
##[debug]=> true
##[debug]Expanded: (true && ('pull_request' == 'pull_request'))
##[debug]Result: true
##[debug]Starting: Generate Infracost JSON
##[debug]Loading inputs
##[debug]Evaluating: format('infracost breakdown --path {0} --format json --out-file /tmp/infracost.json', env.working-directory)
##[debug]Evaluating format:
##[debug]..Evaluating String:
##[debug]..=> 'infracost breakdown --path {0} --format json --out-file /tmp/infracost.json'
##[debug]..Evaluating Index:
##[debug]....Evaluating env:
##[debug]....=> Object
##[debug]....Evaluating String:
##[debug]....=> 'working-directory'
##[debug]..=> '.'
##[debug]=> 'infracost breakdown --path . --format json --out-file /tmp/infracost.json'
##[debug]Result: 'infracost breakdown --path . --format json --out-file /tmp/infracost.json'
##[debug]Evaluating: env.working-directory
##[debug]Evaluating Index:
##[debug]..Evaluating env:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'working-directory'
##[debug]=> '.'
##[debug]Result: '.'
##[debug]Loading env
Run infracost breakdown --path . --format json --out-file /tmp/infracost.json
  infracost breakdown --path . --format json --out-file /tmp/infracost.json
  shell: /usr/bin/bash -e {0}
  env:
    working-directory: .
    TERRAFORM_CLI_PATH: /home/runner/work/_temp/e1d41faf-1d69-4808-95e5-b77f3c9617ba
    PLAN: Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
  will stop streaming the logs, but will not stop the plan running remotely.
  
  Preparing the remote plan...
  
  To view this run in a browser, visit:
  https://app.terraform.io/app/website/test/runs/run-dfMrExgB5MEox71a
  
  Waiting for the plan to start...
  
  Terraform v1.1.5
  on linux_amd64
  Configuring remote state backend...
  Initializing Terraform configuration...


(removed for brevity)

    
  Plan: 4 to add, 0 to change, 0 to destroy.
  ╷
  │ Warning: Experimental feature "module_variable_optional_attrs" is active
  │ 
  │   on .terraform/modules/acme_com/main.tf line 2, in terraform:
  │    2:   experiments = [module_variable_optional_attrs]
  │ 
  │ Experimental features are subject to breaking changes in future minor or
  │ patch releases, based on feedback.
  │ 
  │ If you have feedback on the design of this feature, please open a GitHub
  │ issue to discuss it.
  ╵
    INFRACOST_GITHUB_ACTION: true
    INFRACOST_SKIP_UPDATE_CHECK: true
    INFRACOST_VCS_REPOSITORY_URL: https://github.com/website/terraform-test
    INFRACOST_VCS_PULL_REQUEST_URL: https://github.com/website/terraform-test/pull/13
    INFRACOST_LOG_LEVEL: debug
##[debug]/usr/bin/bash -e /home/runner/work/_temp/0e9d2bc2-f9d2-41a1-b3a2-76f1a12f6a5b.sh
time="2022-02-23T07:34:20Z" level=info msg="Detected Terraform directory at ."
time="2022-02-23T07:34:20Z" level=info msg="Starting: Running terraform plan"
time="2022-02-23T07:34:20Z" level=debug msg="Running command: /home/runner/work/_temp/e1d41faf-1d69-4808-95e5-b77f3c9617ba/terraform plan -input=false -lock=false -no-color -out=/tmp/.tfplan-580801c7-4d46-413b-90c8-4706fc445ea3"
2-02-23T07:34:20Z" level=debug binary=terraform msg="[command]/home/runner/work/_temp/e1d41faf-1d69-4808-95e5-b77f3c9617ba/terraform-bin plan -input=false -lock=false -no-color -out=/tmp/.tfplan-580801c7-4d46-413b-90c8-4706fc445ea3"
time="2022-02-23T07:34:21Z" level=debug binary=terraform
time="2022-02-23T07:34:21Z" level=debug binary=terraform msg="Error: Saving a generated plan is currently not supported"
time="2022-02-23T07:34:21Z" level=debug binary=terraform
time="2022-02-23T07:34:21Z" level=debug binary=terraform msg="The \"remote\" backend does not support saving the generated execution plan"
time="2022-02-23T07:34:21Z" level=debug binary=terraform msg="locally at this time."
time="2022-02-23T07:34:21Z" level=debug binary=terraform msg="::debug::Terraform exited with code 1."
time="2022-02-23T07:34:21Z" level=debug binary=terraform msg="::debug::stdout: "
time="2022-02-23T07:34:21Z" level=debug binary=terraform msg="::debug::stderr: %0AError: Saving a generated plan is currently not supported%0A%0AThe \"remote\" backend does not support saving the generated execution plan%0Alocally at this time.%0A"
time="2022-02-23T07:34:21Z" level=debug binary=terraform msg="::debug::exitcode: 1"
time="2022-02-23T07:34:21Z" level=debug binary=terraform msg="::set-output name=stdout::"
time="2022-02-23T07:34:21Z" level=debug binary=terraform msg="::set-output name=stderr::%0AError: Saving a generated plan is currently not supported%0A%0AThe \"remote\" backend does not support saving the generated execution plan%0Alocally at this time.%0A"
time="2022-02-23T07:34:21Z" level=debug binary=terraform msg="::set-output name=exitcode::1"
time="2022-02-23T07:34:21Z" level=debug binary=terraform msg="::error::Terraform exited with code 1."

Error: terraform plan failed: exit status 1

Try setting the --path to a Terraform plan JSON file. See https://infracost.io/troubleshoot for how to generate this.
Error: Process completed with exit code 1.
##[debug]Finishing: Generate Infracost JSON

Answer 6 · 2022-02-23T09:28:40.000Z

morning @alex-feel 👋 . Thanks for the additional debug information. Yes this behaviour is expected. We try and run a plan with an out file but infracost should catch the stderr message and init a remote run instead. This doesn't appear to be happening, which is strange. I'm wondering if the terraform wrapper in the setup terraform step is messing with the output. Could you try adding the terraform_wrapper: false to your Setup Terraform step and run it again. Let's see if that fixes anything.

Answer 7 · 2022-02-23T09:49:22.000Z

@hugorut, It worked 👍

Meanwhile, I already managed to try "Local Execution Mode" in Terraform Cloud - this option, of course, allows you to save and pass a plan file to Infracost. Now I don't even know which is better.

Thanks for the help!

Answer 8 · 2022-02-23T09:56:55.000Z

Amazing, glad we helped. Thanks for bringing the terraform wrapper inconsistency to our attention! Have a great day.

Answer 9 · 2022-02-23T10:40:09.000Z

I missed one point. Unfortunately, when using terraform_wrapper: false, I can't access the output of terraform plan, so the Update Pull Request step, which creates a comment with a plan in a pull request, does not work in this scenario. Need to think about what to do with it 🤔

Answer 10 · 2022-02-23T10:55:47.000Z

@alex-feel if you want to run with terraform wrapper until we fix why the output isn't being properly caught by infracost. I'd recommend downloading the plan.json from terraform cloud using the script that @aliscott highlighted before. This can be found here: https://www.infracost.io/docs/troubleshooting/#terraform-cloud. This is probably better in the longer run anyway as it avoids you running plan twice.

Answer 11 · 2022-02-23T10:57:47.000Z

@hugorut thanks again! That's probably what I'll do.

Answer 12 · 2022-02-23T10:59:36.000Z

I'm going to close this for now - feel free to reopen or add another issue if you run into any further problems. Alternatively, you can ping me & the team directly on our slack