Support for `infracost --sync-usage-file` and notification / warning if usage is not set for a given resource

[brettcurtis]

What problem are you facing?

It takes a bit of work to set up usage files, and it's often forgotten when adding new resources to a repo. As a developer, I'd like the usage file to sync and notify when values are not set/commented out.

For example, if you look at this repo: https://github.com/osinfra-io/google-cloud-networking, we have to sync up usage files across multiple environments as well as regions with the understanding usage will vary across them. This isn't the greatest example, but imagine something similar like a Kubernetes repo, one might imagine the usage across environments and regions would be very different depending on workloads.

Maybe this isn't the place to solve this problem and @alikhajeh1 has some better insight.

How could pre-commit-terraform help solve your problem?

Pre-commit could potentially be a good place to catch these problems early. 🦃

[MaxymVlasov]

I see that this flag is part of https://fig.io/manual/infracost/breakdown, which is a good sign.

Did you try to provide --sync-usage-file as an argument?
https://github.com/antonbabenko/pre-commit-terraform#infracost_breakdown

- id: infracost_breakdown
  args:
    - --args=--sync-usage-file

[brettcurtis]

Oh yeah, that gets me what I want regarding developers not running sync.

Is there a better way to do this vs. the cp/paste a for_each, for example?

osinfra-io/google-cloud-networking@9977485#diff-63a9c44a44acf85fea213a857769990937107cf072831e1a26808cfde9d096b9R17

Maybe something like this is better run as a custom script from pre-commit?
osinfra-io/google-cloud-networking@9977485#diff-f192491c98fa414fcbd235f133b909016eaea6b45e9f6e9c93da11e0ffe9d69f

I'm just messing around with a few ideas here...

Regardless, this solves the problem of developers needing to run sync. Unfortunately, that's only part of the problem; the usage files need to have some values set to reflect actual costs. So, if everything is commented out a developer will just see this:

Infracost breakdown......................................................Passed
Infracost breakdown......................................................Passed
Infracost breakdown......................................................Passed

Thanks for the help!

[alikhajeh1]

Would this help?

1. Move all of the project args into 1 infracost.yml config file in the repo root:

version: 0.1
projects:
  - path: global/infra
    name: global-infra-sandbox
    usage_file: global/infra/infracost/sandbox.yml
    terraform_var_files:
      - tfvars/sandbox.tfvars

  - path: global/infra
    name: global-infra-non-production
    usage_file: global/infra/infracost/non-production.yml
    terraform_var_files:
      - tfvars/non-production.tfvars
 
...

2. Then in the pre-hook, just have this:

    hooks:
      - id: infracost_breakdown
        args:
          - --args=--config-file=infracost.yml
          - --args=--sync-usage-file

You can then get a bit fancier with the infracost.yml and use the config file template syntax to generate the project list from the repo structure. The pre-commit script would then run:

infracost generate config --repo-path=. \
    --template-path=infracost.yml.tmpl \
    --out-file=infracost.yml

infracost breakdown --config-file infracost.yml --sync-usage-file

That can do advanced stuff like only add a usage-file attribute if it exists, and otherwise use a top-level one with just the resource_type_default_usage:

{{- if pathExists $project._dir "infracost-usage.yml" }}
  usage_file: {{ $project._dir }}/infracost-usage.yml
{{- else }}
  usage_file: infracost-usage.yml
{{- end }}

[brettcurtis]

Thanks @alikhajeh1 - Yes, that's a much cleaner way to do it thanks! I'll mess around with this in a PR later today. @MaxymVlasov thanks for poking at this with me. I'll close this issue out.

If folks think being notified via pre-commit for default usage files is useful and pre-commit is the place to solve it I can open up a new issue on that.