chainguard-dev/melange

git-checkout pipeline fails with some tags [fatal: Needed a single revision]

smoser opened this issue · 3 comments

smoser commented

Here is a reproducing melange yaml (yajl.yaml)

package:
  name: yajl
  version: 2.1.0
  epoch: 3
  description: Yet Another JSON Library (YAJL)
  copyright:
    - license: MIT

environment:
  contents:
    packages:
      - busybox

pipeline:
  - uses: git-checkout
    with:
      expected-commit: a0ecdde0c042b9256170f2f8890dd9451a4240aa
      repository: https://github.com/lloyd/yajl
      tag: ${{package.version}}

  - runs: echo hi world

here is a script that does the same thing as melange git-checkout pipeline. Try it with arguments 'good' or 'bad'.

#!/bin/sh
if [ $# -eq 1 ]; then
    case "$1" in
        # fails 2.1.0=a0ecdde0c042b9256170f2f8890dd9451a4240aa
        bad) set -- https://github.com/lloyd/yajl 2.1.0 yajl-bad;;
        # passes 1.0.2=4a5a7c889cf2ec5daf0f29c31c0401ca77e596ff
        good) set -- https://github.com/lloyd/yajl 1.0.2 yajl-good;;
        *) echo "unknown input '$1'";;
    esac
fi
repo="$1"; tag="$2"; dest="$3"

if [ -z "$dest" ]; then
    tmpd=$(mktemp -d "$PWD/gitcheckout.XXXXXX")
    trap "rm -Rf '$tmpd'" EXIT
    dest="$tmpd"
fi
echo "cloning $tag from $repo to $dest"

# so user's git config doens't polute anything.
export HOME="$PWD"

set -ex
git clone --branch "$tag" --depth 1 "$repo" "$dest"
cd "$dest"
git tag -l
git rev-parse --verify --end-of-options "refs/tags/$tag"
smoser commented

Here is output of a wolfi-dev/os 'make package/yajl' with the above yajl.yaml.

+ make package/yajl
getting package dir for ./yajl.yaml
For package pkgdir found dir: .
found package dir as .
getting source dir for package yajl with dir .
found source dir as --source-dir ./yajl
yamlfile is ./yajl.yaml
Building package yajl with version yajl-2.1.0-r3 from file ./yajl.yaml
make yamlfile=./yajl.yaml srcdirflag="--source-dir ./yajl" pkgname=yajl packages/x86_64/yajl-2.1.0-r3.apk
make[1]: Entering directory '/home/smoser/src/wolfi/os'
@SOURCE_DATE_EPOCH=1718226947 /home/smoser/go/bin/melange build ./yajl.yaml --repository-append /home/smoser/src/wolfi/os/packages --keyring-append local-melange.rsa.pub --signing-key local-melange.rsa --arch x86_64 --env-file build-x86_64.env --namespace wolfi --generate-index false  --pipeline-dir ./pipelines/  -k https://packages.wolfi.dev/os/wolfi-signing.rsa.pub -r https://packages.wolfi.dev/os --source-dir ./yajl --log-policy builtin:stderr,packages/x86_64/buildlogs/yajl-2.1.0-r3.log
2024/06/12 22:15:57 INFO melange is building:
2024/06/12 22:15:57 INFO   configuration file: ./yajl.yaml
2024/06/12 22:15:57 INFO   workspace dir: /tmp/melange-workspace-1736824283
2024/06/12 22:15:57 INFO evaluating pipelines for package requirements
2024/06/12 22:15:57 INFO   adding packages [git] for pipeline "Check out sources from git"
2024/06/12 22:15:57 INFO populating workspace /tmp/melange-workspace-1736824283 from ./yajl
2024/06/12 22:15:57 INFO --cache-dir ./melange-cache/ not a dir; skipping
2024/06/12 22:15:57 INFO building workspace in '/tmp/melange-guest-1036530814' with apko
2024/06/12 22:15:57 INFO image configuration:
2024/06/12 22:15:57 INFO   contents:
2024/06/12 22:15:57 INFO     repositories: []
2024/06/12 22:15:57 INFO     keyring:      []
2024/06/12 22:15:57 INFO     packages:     [busybox git]
2024/06/12 22:15:57 INFO   accounts:
2024/06/12 22:15:57 INFO     runas:  
2024/06/12 22:15:57 INFO     users:
2024/06/12 22:15:57 INFO       - uid=1000(build) gid=1000
2024/06/12 22:15:57 INFO     groups:
2024/06/12 22:15:57 INFO       - gid=1000(build) members=[build]
2024/06/12 22:15:58 INFO installing ca-certificates-bundle (20240315-r3)
2024/06/12 22:15:58 INFO installing wolfi-baselayout (20230201-r11)
2024/06/12 22:15:58 INFO installing ld-linux (2.39-r6)
2024/06/12 22:15:58 INFO installing glibc-locale-posix (2.39-r6)
2024/06/12 22:15:58 INFO installing glibc (2.39-r6)
2024/06/12 22:15:58 INFO installing libxcrypt (4.4.36-r7)
2024/06/12 22:15:58 INFO installing libcrypt1 (2.39-r6)
2024/06/12 22:15:58 INFO installing busybox (1.36.1-r10)
2024/06/12 22:15:58 INFO installing libexpat1 (2.6.2-r1)
2024/06/12 22:15:58 INFO installing libpcre2-8-0 (10.44-r0)
2024/06/12 22:15:58 INFO installing zlib (1.3.1-r2)
2024/06/12 22:15:58 INFO installing libunistring (1.2-r2)
2024/06/12 22:15:58 INFO installing libidn2 (2.3.7-r2)
2024/06/12 22:15:58 INFO installing libpsl (0.21.5-r2)
2024/06/12 22:15:58 INFO installing libbrotlicommon1 (1.1.0-r2)
2024/06/12 22:15:58 INFO installing libbrotlidec1 (1.1.0-r2)
2024/06/12 22:15:58 INFO installing libnghttp2-14 (1.62.1-r0)
2024/06/12 22:15:58 INFO installing libcrypto3 (3.3.0-r9)
2024/06/12 22:15:58 INFO installing libssl3 (3.3.0-r9)
2024/06/12 22:15:58 INFO installing libcurl-openssl4 (8.8.0-r0)
2024/06/12 22:15:58 INFO installing git (2.45.2-r0)
2024/06/12 22:15:58 INFO built image layer tarball as /tmp/apko-temp-2993088703/apko-x86_64.tar.gz
2024/06/12 22:15:58 INFO using /tmp/apko-temp-2993088703/apko-x86_64.tar.gz for image layer
2024/06/12 22:15:59 INFO ImgRef = /tmp/melange-guest-2285279181
2024/06/12 22:15:59 INFO running step "git-checkout"
2024/06/12 22:15:59 INFO running step "Check out sources from git"
2024/06/12 22:15:59 WARN Cloning into '/tmp/tmp.SkyGAz'...
2024/06/12 22:15:59 WARN fatal: Needed a single revision
2024/06/12 22:15:59 ERRO ERROR: failed to build package. the build environment has been preserved:
2024/06/12 22:15:59 INFO   workspace dir: /tmp/melange-workspace-1736824283
2024/06/12 22:15:59 INFO   guest dir: /tmp/melange-guest-1036530814
2024/06/12 22:15:59 INFO error during command execution: failed to build package: unable to run pipeline: exit status 128
make[1]: *** [Makefile:150: packages/x86_64/yajl-2.1.0-r3.apk] Error 1
make[1]: Leaving directory '/home/smoser/src/wolfi/os'
make: *** [Makefile:140: package/yajl] Error 2
jonjohnsonjr commented

Looking at https://git.dag.dev/?url=https%3A%2F%2Fgithub.com%2Flloyd%2Fyajl

I see:

66cb08ca2ad8581080b626a75dfca266a890afb2	refs/heads/2.1.0
...
a0ecdde0c042b9256170f2f8890dd9451a4240aa	refs/tags/2.1.0

So there is a branch and a tag, both named 2.1.0, so my guess is that this ref is ambiguous to git and we need to disambiguate somehow.

smoser commented

Adding this line would "fix" the problem:

git fetch origin "+refs/tags/$tag:refs/tags/$tag"

but also in this case (yajl) increases the size of .git from 236k to 820k , defeating much of the value of --depth 1