actions/runner-images

fatal: detected dubious ownership in repository at

Closed this issue ยท 60 comments

Description

I started seeing this error yesterday

image

It does not make sense, because the checkout action is already running the suggested git command to set the repo as safe

image

What could the issue be in this case?

Platforms affected

  • Azure DevOps
  • GitHub Actions - Standard Runners
  • GitHub Actions - Larger Runners

Runner images affected

  • Ubuntu 18.04
  • Ubuntu 20.04
  • Ubuntu 22.04
  • macOS 11
  • macOS 12
  • Windows Server 2019
  • Windows Server 2022

Image version and build link

image

Is it regression?

yes. Version: 20221125.1 https://github.com/VodafoneIS/sf-metadata/actions/runs/3601296318/jobs/6066928420

Expected behavior

Since actions/checkout@v3 already run /usr/bin/git config --global --add safe.directory /__w/sf-metadata/sf-metadata I was not expecting to have issues with safe directories.

Actual behavior

image

image

Repro steps

  1. Create a docker container with git installed
  2. Add this image to a job
  3. Run a git command
...

jobs:

  upload-lwc-converage:
    name: Test
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/vodafoneis/salesforce-build-image:main
      credentials:
        username: ${{ github.actor }}
        password: ${{ secrets.GITHUB_TOKEN }}

  steps:
      - name: Checkout
        uses: actions/checkout@v3
        
       - name: run my sh script located in the docker container
         run:  /scripts/do_something_with_git.sh

Hi @AllanOricil we will take a look, thanks.

@igorboskovic3 I've set all my workflows to run on ubuntu-20.04 but the same issue happened, maybe because it is using Version: 20221212.2. The last job that ran without a problem was using ubuntu-20.04 20221125.1. So, how can I also set the build version?

Hi @AllanOricil, unfortunately you cannot set an old build version, just current one.

@igorboskovic3
I dont understand why we started seeing this issue. This safe dir feature was introduced to git in v2.35.2 https://confluence.atlassian.com/bbkb/git-command-returns-fatal-error-about-the-repository-being-owned-by-someone-else-1167744132.html
but the checkout action shows that we are using git v2.20.1
image

Hello @AllanOricil, we have noticed that the "checkoutv3" action uses a git version that is already installed no matter in the container or in the runner. Below you can find our simple workflow, which went successfully.

jobs:
  run_from_container:
    name: test1
    runs-on: ubuntu-latest
    container:
      image: alpine/git

    steps:
      - name: Checkout
        uses: actions/checkout@v3
        
      - name: git clone
        run: | 
          git clone --depth=1 https://github.com/actions/runner-images
          ls -la runner-images
          
  run_from_runner:
    name: test2
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v3
        
      - name: Check the data
        run: ls $GITHUB_WORKSPACE

From this small experiment, you can see that container and runner have different versions. of git. Please pay close attention to your rights in the container and the git version installed in your container as well.

It seems like the issue does not directly relate to a runner, so we will close the issue for now. Feel free to reach us if you have any questions.

@Alexey-Ayupov @igorboskovic3 I did not change anything in any of my workflows since yesterday and this issue started. There is something wrong somewhere and I cant figure out what is it...

@Alexey-Ayupov @igorboskovic3 could this action be the reason?
https://github.com/fregante/setup-git-user/blob/master/index.js

I run it after running git checkout. I need it, because inside my workflow I create commits and tags.
This action was in my workflow already, and It should not be causing the issue.

I ran this workflow, and again I had the same problem

image

...
jobs:
  validate-changes:
    name: Validate Metadata Changes
    environment: dev
    runs-on: ubuntu-20.04
    container:
      image: ghcr.io/vodafoneis/salesforce-build-image:main
      credentials:
        username: ${{ github.actor }}
        password: ${{ secrets.GITHUB_TOKEN }}

    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          fetch-depth: 2

      - name: Print Git Version
        run: |
          git --version
          touch a.txt
          git add .
          git commit -m "test"

@igorboskovic3 @Alexey-Ayupov tell me it is not a bug somewhere...
Look, I added a step Add safe.directory /__w/sf-metadata/sf-metadata and later on the my other git commands fail? Something weird is happening. I insist you reopen this ticket and investigate more. This was not happening, my workflows were working without a problem few days ago without this extra step to set the safe.directory.

image

It is something on my image. I dont know what is it yet, but I just used an image from 7 days ago, and it is working again. will check all my dependencies and then I come back here to tell you what I found.

Hi @AllanOricil . Thank you for not giving up on searching for root cause. It would be great if you provide your findings.

These are the only diffs between my images.

IMAGE 1 => is the one which I have no issues
IMAGE 2 => image where the problem started

image

So, since there aren't any other changes, I have no clue what is causing the problem. Would you like to have access to both images? That may help you to find the real issue.

@AllanOricil
I don't think this is the issue of checkout v3.
I started getting the issue today without changing any of my workflows as well.
The git version is exactly the same for both the previous successful runner and today's fail runner.
No container image build within my runner. Only trying to push a new tag.
Did you get a way around it?

@benzsuankularb @Alexey-Ayupov @igorboskovic3
No idea. I just used an older docker image and my problem was gone. Im not sure exactly what is wrong here because there are people reporting the same error and they are not even using docker images. Im just luck, I guess?

In the image below, every commit represents a new container image build. I used the image from b937316 and my issue was gone. Previously, when the problem started, my workflows were using @main.

image

Why did I choose that one? I just picked up the image from the week where all my workflows were not facing that issue.

I just don't know why changing the image fixed my problem, as the only changes between both images are these

image

To prove I did not change anything else on my workflows, but the container images only, here is are some images which you can see the only changes I did on the workflow were the container refs

image

image

This is my docker file:

FROM node:14

RUN apt-get update -y \
 && apt-get upgrade -y \
 && apt-get install -y --no-install-recommends gnupg apt-utils apt-transport-https curl git openssh-client ca-certificates jq bc gettext xmlstarlet bash python3 libxml2 libxml2-utils \
 && rm -rf /var/lib/apt/lists/*

# add an apt repository of chrome
# google-chrome.list will be overwritten by installing google-chrome-stabe
RUN curl -fsSL https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN echo "deb http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list

RUN curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg
RUN echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | tee /etc/apt/sources.list.d/github-cli.list > /dev/null


# install several packages for CI
RUN apt-get update -y \
  && apt-get install -y --no-install-recommends wget gnupg google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 gh \
  && rm -rf /var/lib/apt/lists/*

ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/google-chrome

COPY setup_npm_global_config.sh /
COPY global_node_dependencies_installer.sh /
COPY package.json /

RUN chmod +x /setup_npm_global_config.sh
RUN chmod +x /global_node_dependencies_installer.sh
RUN /setup_npm_global_config.sh
RUN /global_node_dependencies_installer.sh
RUN sfdx plugins:link /usr/local/lib/node_modules/sfdx-git-delta


ENV SFDX_AUTOUPDATE_DISABLE=false
ENV SFDX_JSON_TO_STDOUT=true
ENV SFDX_USE_GENERIC_UNIX_KEYCHAIN=true
ENV SFDX_DOMAIN_RETRY=true
ENV SFDX_PROJECT_AUTOUPDATE_DISABLE_FOR_PACKAGE_CREATE=true
ENV SFDX_PROJECT_AUTOUPDATE_DISABLE_FOR_PACKAGE_VERSION_CREATE=true
ENV SFDX_DISABLE_DNS_CHECK=true
ENV SHELL /bin/bash

COPY scripts/ scripts/

RUN echo 'y'
RUN sfdx --version
RUN sf --version
RUN sfdx plugins --core
RUN vlocity -v
RUN sfdx sgd:source:delta -h
RUN gh --version

I am having the same issue and haven't done any changes. I can see that this issue has been closed with no solution

this is fixed by using
sudo git config --global --add safe.directory /is/htdocs/wp1090589_IC9AIO06VX/your_project_name
it will ask you for the sudo password and there you go :)

this is fixed by using sudo git config --global --add safe.directory /is/htdocs/wp1090589_IC9AIO06VX/your_project_name it will ask you for the sudo password and there you go :)

this isn't a fix for all scenarios

@igorboskovic3 @Alexey-Ayupov could you please reopen this and investigate more. Another random guy reported this in Reddit.
https://www.reddit.com/r/github/comments/znfag2/new_dubious_ownership_error_in_github_workflow/

@igorboskovic3 @Alexey-Ayupov I believe the issue comes from this debian updgrade

image

image

Do you have any ideas how I can set this in the dockerfile?

This post also thinks this debian upgrade is the cause of the problem
https://phabricator.wikimedia.org/T325128

image

@Alexey-Ayupov @igorboskovic3 I HAVE FINALLY SORTED THIS ISSUE. I built an image which does not upgrade git packages to deb10u6

image

For those that are not using docker images in your workflows, you are probably facing the same issue because the official runners might have upgraded their git package with this security patch. And because you can't change the official runners, your only option is to create a docker image or your own runner, and then ensure the installed git version does not include this security patch mentioned here https://phabricator.wikimedia.org/T325128. In my case, I just had to do the following:

FROM node:14

RUN apt-mark hold git
RUN apt-mark hold git-man

Thanks @AllanOricil !
Another temporary fix is to add the suggested command in the CI/CD pipeline:
git config --global --add safe.directory /__w/{repo_name}/{repo_name}

Thanks @AllanOricil !
Another temporary fix is to add the suggested command in the CI/CD pipeline:
git config --global --add safe.directory /__w/{repo_name}/{repo_name}

@Nicoowr this does not work when using containers. I tried it already

Got it. It worked in our case though, we're using self-hosted runners with the version 2.299 of Github Actions Runner.

Exito peluche!

As with @Nicoowr, we encountered this for the first time today on the self-hosted 2.299.1 version of the GitHub Actions Runner. In our case, we were using a python:3.8-buster Docker Container inside the action. We observed that the issue disappeared after switching to a python:3.8-bullseye container which presumably has an updated version of Git. As suggested by @AllanOricil above, try to find a newer container image, currently that seems to be resolving the issue!

More info about the security update here: https://www.debian.org/lts/security/2022/dla-3239

From the above link:

Note: Due to new security checks, access to repositories owned and accessed by different local users may now be rejected by Git; in case changing ownership is not practical, git displays a way to bypass these checks using the new safe.directory configuration entry.

To be honest this should not have being closed because it is 100% related to runners. I see no reason why the permissions should be messed on a runner, it is just bad runner configuration or bad volument mounting for action images.

Yep, it is a side effect of getting a newer git, one that is more sensitive to bad permissions, one that uncovers an issue that was ignored for a very long time.

Related:

@ssbarnea setting the permissions correctly should be a better solution. A workaround is to do so manually in the CI when running as root:

     steps:
      - name: Change Owner of Container Working Directory
        run: chown root:root .

@ssbarnea setting the permissions correctly should be a better solution. A workaround is to do so manually in the CI when running as root:

     steps:
      - name: Change Owner of Container Working Directory
        run: chown root:root .

Thanks. I did not thought about it. Does it really work? The downside of this approach is that we must add this step on every workflow in multiple repositories.

yaqwsx commented

I also think this issue shouldn't be closed as, clearly, the permissions are set wrongly. The failing git is only a symptom.

i have similar error to yours , but i can't solve? may you helpp me.
Screenshot (23)

i have similar error to yours , but i can't solve? may you helpp me. Screenshot (23)

share your workflow here

Problem started again, even holding git packages upgrades.

  • name: Change Owner of Container Working Directory
    run: chown root:root .

This works if you are running jobs using containers.

Previously I did not have to add these lines. I simply did not upgrade git packages in my images. But now I need this :/ It kind sucks because I have to add it to multiple files.

Started having this issue as well.. out-of-the-blue..

๐Ÿ‘ Running into this issue now as well

@joelnotified if you are installing additional packages in your build one of them might be causing this. What I noticed is that previous "succesful" builds no longer work anymore as well.

Same error for us

@Alexey-Ayupov could you reopen this issue please?

Same issue today after I force pushed my repository, the github pages build failed giving out this error.

added this line after my checkout step, which resolved it...but i too started getting this weird error

    steps:
      - name: Checkout Code
        uses: actions/checkout@v3
      - run: git config --global --add safe.directory "$GITHUB_WORKSPACE"

@jmmclean sorry, did that fix the issue or did you get the errors despite that? ๐Ÿค”

@joelnotified @jmmclean, it doesn't fix the issue for us (I'm part of the @filamentphp team with @danharrin).

We just started seeing this problem today for the first time. For our Docker-based actions the git config needs to run inside the container.

Yup, this is likely to crash all 3rd party container-based actions that do any git work, and are based on an image that has a recent enough version of git. This is what happened to us at py-cov-action/python-coverage-comment-action#127 : suddenly, users all started to report this error.

Yeah, I've realized that is the issue as well.

In our case we have a Dockerfile for a custom GitHub Action that uses Git, which I guess has received the latest security updates, causing this.

For anyone else with similar issues. I couldn't figure out how to properly set the config from GitPython (which we use within the container), so the easiest solution for us was to put this in the Dockerfile for the custom GitHub Action:

RUN git config --system --add safe.directory *

I realize it's not the best solution, but it works and I don't think there's any risk involved in our case.

For anyone else with similar issues. I couldn't figure out how to property set the config from GitPython (which we use within the container), so the easiest solution for us was to put this in the Dockerfile for the custom GitHub Action:

RUN git config --system --add safe.directory *

I realize it's not the best solution, but it works and I don't think there's any risk involved in our case.

Thanks ๐Ÿ˜Š

I realize it's not the best solution, but it works and I don't think there's any risk involved in our case.

Honnestly, I think it's much much cleaner to do it in the Dockerfile like this than inside the action application code. Great idea :)

fishy commented

Our project just randomly started to see this error today (we use actions/checkout@v2 with ubuntu-latest and it never had this issue before).

I'm not sure why this issue is closed. Yes there're workarounds available, but those are not actual fixes. This is still a bug that should be fixed?

metjm commented

This also started happening to us yesterday.
We're using self-hosted runners, and nothing had been merged since last week, and no update was made to the self-hosted runners.

We also started seeing this error in our CIs.

We are using the base image mambaorg/micromamba:bullseye-slim.

Adding git config --global --add safe.directory <your directory> temporarily fixes the issue (but should we consider this directory safe?).

I suspect the problem is that the WORKDIR in the base image is created by a different user than the one executing the problematic command. My 2 cents.

(but should we consider this directory safe?)

As far as I can tell, the class of problems that this check protects you from is not something that may happen in the context of an action (though I can be mistaken).

The idea is if you're evil user A and want to execute code as B on the same machine, you can write that evil code as a git hook, and try and convince user B to do git something on the repo. B would think they're only doing safe git stuff, but in reality, they'll be executing hooks that you wrote. So the idea is that git won't let you execute git on a repo whose .git doesn't belong to you.

I don't see how this would be an issue in GHA so from my point of view, it's ok to bypass this check (unless you have a scenario where this check would prevent people from getting attacked)

I don't think this is a runner-images issue, but I certainly think that this is a checkout issue:
actions/checkout#1048

checkout@v3 actually is attempting to fix this with

/usr/bin/git config --global --add safe.directory /__w/.../...

But it doesn't necessarily work sometimes due to permissions (this would make sense) or in any container based build.

If you are experiencing this failure in container-based GitHub-hosted Actions runs, I believe this is the cause
actions/checkout#1169

I had the same fail, but my solution was different. I use cmder as terminal shell and when I started my git I started terminal as Administrator; when I get the fail message I had started as normal user. Just restart terminal as Administrator solved in my case.

Reopen this, lol, @Alexey-Ayupov.

FFR: actions/runner#2033 (comment)

Just use the users option to solve the issue.