bachp/git-mirror

Cannot skip a repository with a dot

Closed this issue · 7 comments

NoxInmortus commented

Hello,

I'm trying to sync an organization repositories from GitHub to GitLab. One of the repository is named ".github". When running git-mirror, the following error occurs:

🔴 2022-07-01 08:19:37,253 - gitlamirator.GitHubLab - ERROR - Error from GitLab (repo "wantedorganization/.github"): {'name': ["can contain only letters, digits, emojis, '_', '.', '+', dashes, or spaces. It must start with a letter, digit, emoji, or '_'."], 'path': ['must not start or end with a special character and must not contain consecutive special characters.']} (GitHubLab.py:215 add_project_to_gitlab)
🔴 2022-07-01 08:19:37,253 - gitlamirator.GitHubLab - ERROR - in case of error "namespace is not valid" check if the gitlab user is a member of the target group (GitHubLab.py:217 add_project_to_gitlab)
❌ 2022-07-01 08:21:03,102 - gitlamirator.mirrorIt - CRITICAL - ** an error was reported with sync'ing group 'wantedorganization'' :  (mirrorIt.py:203 call_git_mirror_tool)
❌ 2022-07-01 08:21:03,103 - gitlamirator.mirrorIt - CRITICAL - [2022-07-01T06:19:44Z ERROR git_mirror] Unable to sync repo https://github.com/wantedorganization/jobs.git -> git@gitlab.mydom.org:wantedorganization/jobs.git (Command "git" "remote" "update" "--prune" failed with exit code: 1, Stderr: fatal: could not read Username for 'https://github.com': terminal prompts disabled
    error: Could not fetch origin
    )
[2022-07-01T06:19:44Z ERROR git_mirror] Unable to sync repo https://github.com/wantedorganization/frontend-core.git -> git@gitlab.mydom.org:wantedorganization/frontend-core.git (Command "git" "remote" "update" "--prune" failed with exit code: 1, Stderr: fatal: could not read Username for 'https://github.com': terminal prompts disabled
    error: Could not fetch origin
    )
[2022-07-01T06:20:29Z ERROR git_mirror] Unable to sync repo https://github.com/wantedorganization/wantedorganization-frontend-workshop-2.git -> git@gitlab.mydom.org:wantedorganization/wantedorganization-frontend-workshop-2.git (Command "git" "remote" "update" "--prune" failed with exit code: 1, Stderr: fatal: could not read Username for 'https://github.com': terminal prompts disabled
    error: Could not fetch origin
    )
[2022-07-01T06:21:02Z ERROR git_mirror] Unable to sync repo https://github.com/wantedorganization/wantedorganization.git -> git@gitlab.mydom.org:wantedorganization/wantedorganization.git (Command "git" "push" "git@gitlab.mydom.org:wantedorganization/wantedorganization.git" "+refs/heads/*:refs/heads/*" "+refs/tags/*:refs/tags/*" failed with exit code: 1, Stderr: remote: GitLab: You are not allowed to force push code to a protected branch on this project.
    To gitlab.mydom.org:wantedorganization/wantedorganization.git
     ! [remote rejected]       21.04.x -> 21.04.x (pre-receive hook declined)
     ! [remote rejected]       21.10.x -> 21.10.x (pre-receive hook declined)
     ! [remote rejected]       22.04.x -> 22.04.x (pre-receive hook declined)
     ! [remote rejected]       MON-12409 -> MON-12409 (pre-receive hook declined)
     ! [remote rejected]       MON-12837 -> MON-12837 (pre-receive hook declined)
     ! [remote rejected]       MON-12873 -> MON-12873 (pre-receive hook declined)
     ! [remote rejected]       MON-13409-severities-feature -> MON-13409-severities-feature (pre-receive hook declined)
     ! [remote rejected]       MON-13484-Reduce-spacing-and-align-user-menu-access-buttons -> MON-13484-Reduce-spacing-and-align-user-menu-access-buttons (pre-receive hook declined)
     ! [remote rejected]       dev-21.04.x -> dev-21.04.x (pre-receive hook declined)
     ! [remote rejected]       dev-21.10.x -> dev-21.10.x (pre-receive hook declined)
     ! [remote rejected]       dev-22.04.x -> dev-22.04.x (pre-receive hook declined)
     ! [remote rejected]       develop -> develop (pre-receive hook declined)
     ! [remote rejected]       master -> master (pre-receive hook declined)
     ! [remote rejected]       update-readme -> update-readme (pre-receive hook declined)
     ! [remote rejected]       MON-10790 -> MON-10790 (pre-receive hook declined)
     ! [remote rejected]       MON-11874 -> MON-11874 (pre-receive hook declined)
     ! [remote rejected]       MON-12296-updates -> MON-12296-updates (pre-receive hook declined)
     ! [remote rejected]       MON-13409-Manage-properly-severities-on-Resource-Status -> MON-13409-Manage-properly-severities-on-Resource-Status (pre-receive hook declined)
     ! [remote rejected]       MON-13452-and-MON-13449 -> MON-13452-and-MON-13449 (pre-receive hook declined)
     ! [remote rejected]       MON-13452-and-MON-13449-for-2104 -> MON-13452-and-MON-13449-for-2104 (pre-receive hook declined)
     ! [remote rejected]       MON-13452-and-MON-13449-for-2110 -> MON-13452-and-MON-13449-for-2110 (pre-receive hook declined)
     ! [remote rejected]       MON-13829 -> MON-13829 (pre-receive hook declined)
     ! [remote rejected]       MON-13954-severities-feature -> MON-13954-severities-feature (pre-receive hook declined)
     ! [remote rejected]       MON-14121-Add-button-to-export-timeline-to-csv -> MON-14121-Add-button-to-export-timeline-to-csv (pre-receive hook declined)
     ! [remote rejected]       MON-14128 -> MON-14128 (pre-receive hook declined)
     ! [remote rejected]       MON-14145-integrate-form-builder-component -> MON-14145-integrate-form-builder-component (pre-receive hook declined)
     ! [remote rejected]       MON-14160-add-missing-wantedorganization-database-plugin-require -> MON-14160-add-missing-wantedorganization-database-plugin-require (pre-receive hook declined)
     ! [remote rejected]       MON-14160-add-missing-wantedorganization-database-plugin-require-dev-22.04.x -> MON-14160-add-missing-wantedorganization-database-plugin-require-dev-22.04.x (pre-receive hook declined)
     ! [remote rejected]       MON-14174 -> MON-14174 (pre-receive hook declined)
     ! [remote rejected]       add-useCopyToClipboard-as-utils -> add-useCopyToClipboard-as-utils (pre-receive hook declined)
     ! [remote rejected]       add_debian_packs -> add_debian_packs (pre-receive hook declined)
     ! [remote rejected]       dependabot/composer/guzzlehttp/guzzle-7.4.5 -> dependabot/composer/guzzlehttp/guzzle-7.4.5 (pre-receive hook declined)
     ! [remote rejected]       develop2master -> develop2master (pre-receive hook declined)
     ! [remote rejected]       master-2022-06-21 -> master-2022-06-21 (pre-receive hook declined)
     ! [remote rejected]       release-debian-22.04 -> release-debian-22.04 (pre-receive hook declined)
     ! [remote rejected]       reorginize_open_api_documentation -> reorginize_open_api_documentation (pre-receive hook declined)
     ! [remote rejected]       select2-hg-display -> select2-hg-display (pre-receive hook declined)
     ! [remote rejected]       update-unattended-to-2210 -> update-unattended-to-2210 (pre-receive hook declined)
    error: failed to push some refs to 'gitlab.mydom.org:wantedorganization/wantedorganization.git'
    )
 (mirrorIt.py:206 call_git_mirror_tool)

This is related to the name of one of the GitHub repository: .github: GitLab requires repository name to start with a letter or a number (no other characters).

I tried creating a github repository and added

origin: https://github.com/wantedorganization/.github.git
skip: true

but nop

bachp commented

If I understand correctly you are using a separate tool to mirror a whole organization and this organization contains a repo (.github) that is not valid on gitlab side.

Also the skip: true needs to be set on the pull side. It's not clear from your post if this is the case.

NoxInmortus commented

If I understand correctly you are using a separate tool to mirror a whole organization and this organization contains a repo (.github) that is not valid on gitlab side.

Yes, we are using a tool which takes a list of remote organizations or repositories endpoints defined within a YAML file and sync them to a GitLab instance with git-mirror. Nothing too fancy.

Also the skip: true needs to be set on the pull side. It's not clear from your post if this is the case.

On our gitlab instance we created an empty github repository and added the skip: true. As the original repository is on GitHub, I guess that our Gitlab is our "pull side"

NoxInmortus commented

@bachp is my issue understandable ?

bachp commented

@NoxInmortus What do you suggest the program should do in the situation where the target repo is not valid?

NoxInmortus commented

@bachp I guess, we can either:

  • Ignore the repository as it cannot be created in this state (add some log saying it was ignored with INFO level)
  • Make some mecanism able to recognize that the .github repository from Github, has his equivalent on gitlab as just github
    • which would make the skip: true work btw
NoxInmortus commented

Hello, i'm closing this issue as we are able to resolve it on our side with our separate tool to mirror.