antonbabenko/pre-commit-terraform

[fedora, grep 3.8] grep: warning: stray \ before /

dmitry-mightydevops opened this issue · 4 comments

dmitry-mightydevops commented

Describe the bug

Terragrunt validate......................................................Failed
- hook id: terragrunt_validate
- exit code: 2

grep: warning: stray \ before /
Success! The configuration is valid.

How can we reproduce it?

Environment information

  • OS:
Linux desktop 6.6.8-100.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Dec 21 04:01:45 UTC 2023 x86_64 GNU/Linux
  • Tools availability and versions:
pre-commit 3.5.0
Terraform v1.7.2
Python 3.11.6
Python 3.11.6
checkov checkov SKIPPED
terraform-docs version v0.16.0 linux/amd64
terragrunt version v0.54.22
terrascan terrascan SKIPPED
TFLint version 0.47.0
+ ruleset.aws (0.27.0)
+ ruleset.terraform (0.4.0-bundled)
tfsec You are running a locally built version of tfsec.
tfupdate 0.8.0
hcledit hcledit SKIPPED
  • .pre-commit-config.yaml:
file content 
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.86.0
    hooks:
      - id: terragrunt_fmt
      - id: terragrunt_validate
        stages: [commit]
        args:
          - --args=--terragrunt-log-level=info
MaxymVlasov commented

Hi

  1. Which command did you use to get that error?
  2. Please provide the output of 
    grep --version
cat /etc/os-release
  3. Provide any additional info about the environment where it's run or any unique system configurations, which you can mind as helpful

We have grep call which could cause that issue in the function that runs when you check the whole repo

pre-commit-terraform/hooks/_common.sh

Lines 160 to 164 in 995a59d

if [ -z "$excluded_files" ]; then
all_files_that_can_be_checked=$(git ls-files | sort | grep -e "$included_files" | tr '\n' ' ')
else
all_files_that_can_be_checked=$(git ls-files | sort | grep -e "$included_files" | grep -v -e "$excluded_files" | tr '\n' ' ')
fi

The excluded pattern is the same for 14 of 17 hooks, so it's weird that you are the first who faces this problem from addition of that functional 24 months ago

pre-commit-terraform/hooks/_common.sh

Lines 151 to 154 in 995a59d

# get included and excluded files from .pre-commit-hooks.yaml file
local -r hook_config_block=$(sed -n "/^- id: $hook_id$/,/^$/p" "$root_config_dir/.pre-commit-hooks.yaml")
local -r included_files=$(awk '$1 == "files:" {print $2; exit}' <<< "$hook_config_block")
local -r excluded_files=$(awk '$1 == "exclude:" {print $2; exit}' <<< "$hook_config_block")

pre-commit-terraform/.pre-commit-hooks.yaml

Line 86 in 995a59d

exclude: \.terraform\/.*$

Also, I can't reproduce it in Ubuntu 22.04 
16:03 Oslo git:(test $✘!+?) 
➜ terragrunt --version
terragrunt version v0.54.22

16:03 Oslo git:(test $✘!+?) 
➜ terraform --version
Terraform v1.7.2
on linux_amd64

Your version of Terraform is out of date! The latest version
is 1.7.3. You can update by downloading from https://www.terraform.io/downloads.html


16:04 Oslo git:(test $✘!+?) 
➜ pre-commit run -a 
Terragrunt fmt...........................................................Passed
Terragrunt validate......................................................Failed
- hook id: terragrunt_validate
- exit code: 1

Success! The configuration is valid.

ERRO[0000] Did not find any Terraform files (*.tf) in /home/vm/code/Oslo/.config 
ERRO[0000] Unable to determine underlying exit code, so Terragrunt will exit with error code 1 

Warning: Argument is deprecated

  on ../../modules/aws-environment/instances.tf line 3, in resource "aws_eip" "bastion":
   3:   vpc      = true

use domain attribute instead

(and one more similar warning elsewhere)

Success! The configuration is valid, but there were some validation warnings as shown above.


Warning: Argument is deprecated

  on instances.tf line 3, in resource "aws_eip" "bastion":
   3:   vpc      = true

use domain attribute instead

(and one more similar warning elsewhere)

Success! The configuration is valid, but there were some validation warnings as shown above.
dmitry-mightydevops commented

@MaxymVlasov

➜ grep --version

grep (GNU grep) 3.8
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and others; see
<https://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.

➜ cat /etc/os-release
NAME="Fedora Linux"
VERSION="38 (Workstation Edition)"
ID=fedora
VERSION_ID=38
VERSION_CODENAME=""
PLATFORM_ID="platform:f38"
PRETTY_NAME="Fedora Linux 38 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:38"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f38/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=38
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=38
SUPPORT_END=2024-05-14
VARIANT="Workstation Edition"
VARIANT_ID=workstation

.pre-commit-config.yaml

default_install_hook_types: ["pre-commit", "pre-push"]

repos:
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.86.0
    hooks:
      - id: terragrunt_fmt
        stages: [commit,push]
      - id: terragrunt_validate
        stages: [push]
        args:
          - --args=--terragrunt-log-level=info
MaxymVlasov commented

Reporoduced

Reproduction instructions

Run from pre-commit repo root

mkdir -p tests/fedora-38/hooks
cd tests/fedora-38


ln ../../hooks/_common.sh hooks/_common.sh
ln ../../hooks/terragrunt_fmt.sh  hooks/terragrunt_fmt.sh
ln ../../lib_getopt lib_getopt
ln ../../.pre-commit-hooks.yaml .pre-commit-hooks.yaml


cat <<EOF > .pre-commit-config.yaml
default_install_hook_types: ["pre-commit", "pre-push"]

repos:
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.86.0
    hooks:
      - id: terragrunt_fmt
        stages: [commit,push]
        verbose: true
      - id: terragrunt_validate
        stages: [push]
        args:
          - --args=--terragrunt-log-level=info
        verbose: true
EOF

cat <<EOF > Dockerfile
FROM fedora:38


RUN dnf update -y \
    && dnf install -y \
        git \
        python3 \
        python3-pip \
        python-is-python3 \
    && python3 -m pip install --upgrade pip \
    && pip3 install --no-cache-dir pre-commit \
    && dnf clean all

RUN DIR=~/.git-template \
    && git config --global init.templateDir ${DIR} \
    && pre-commit init-templatedir -t pre-commit ${DIR}

ARG TARGETOS
ARG TARGETARCH


RUN curl -L "$(curl -s https://api.github.com/repos/gruntwork-io/terragrunt/releases/latest | grep -o -E -m 1 "https://.+?/terragrunt_${TARGETOS}_${TARGETARCH}")" > terragrunt \
    && chmod +x terragrunt \
    && sudo mv terragrunt /usr/bin/

WORKDIR /app

COPY . ./

RUN git init \
    && git add -A \
    && pre-commit install-hooks


# CMD pre-commit run --all-files

CMD bash /hooks/terragrunt_fmt.sh /app/.
EOF

cat <<EOF > .dockerignore
hooks/
lib_getopt
.pre-commit-hooks.yaml
EOF
cat <<EOF > .terragrunt.hcl
inputs = {}
EOF

docker build -t fedora38 .

docker run -ti   -v "$(pwd)/hooks/:/hooks/:ro" -v "$(pwd)/lib_getopt:/lib_getopt:ro" -v "$(pwd)/.pre-commit-hooks.yaml:/.pre-commit-hooks.yaml:ro" fedora38

Warning comes from

pre-commit-terraform/hooks/_common.sh

Line 163 in 8bcd633

all_files_that_can_be_checked=$(git ls-files | sort | grep -e "$included_files" | grep -v -e "$excluded_files" | tr '\n' ' ')

pre-commit-terraform/.pre-commit-hooks.yaml

Line 86 in 8bcd633

exclude: \.terraform\/.*$

I suppose that's a grep enhancement introduced in grep 3.8, because grep 3.7 does not show grep: warning: stray \ before /

antonbabenko commented

This issue has been resolved in version 1.86.1 🎉

  • 👍1