Support mirroring of LFS files
Closed this issue · 2 comments
It would be very nice if files managed with LFS were mirrored. Mirroring repositories that have LFS pointers results in the following error:
[2020-09-17T15:51:41Z ERROR git_mirror] Unable to sync repo git@gitlab.com:some/repo.git -> git@example.com:some/repo-mirror.git (Command "git" "push" "git@example.com:some/repo-mirror.git" "+refs/tags/*:refs/tags/*" "+refs/heads/*:refs/heads/*" failed with exit code: 1, Stderr: remote: GitLab: LFS objects are missing. Ensure LFS is properly set up or try a manual "git lfs push --all".
We've worked around this with
# Repositories with LFS managed files need special attention in order to mirror the file contents
- |
LFS_REPOS="repo"
for r in $LFS_REPOS; do
origin_dir="$MIRROR_DIR/git-gitlab-com-some-$r-git"
mkdir -p "$origin_dir"
git clone --mirror git@gitlab.com:some/$r.git "$origin_dir"
pushd "$origin_dir"
git lfs fetch --all
git lfs push git@example.com:some/$r-mirror.git --all
popd
done
# Perform a mirror, reusing any existing cloned repos in $MIRROR_DIR
- ./git-mirror -m "$MIRROR_DIR" -g some%2Fgroup -u ssh://git@example.com --refspec "+refs/tags/*:refs/tags/*" "+refs/heads/*:refs/heads/*"
But a more elegant approach would be welcomed!
@hobti01 Thank you for the suggestin. I think supporting Git LFS would be nice.
Maybe it would be sufficient to make sure git lfs is properly setup during the cloning.
Suggestions are welcome.
Thanks for the feedback @bachp. In our case even with lfs setup large files are not pulled due to an .lfsconfig that ignores all by default. This is why git lfs fetch --all
is needed to override the lfsconfig. I don't see an alternative to "get inside" the git-mirror copy loop other than coding it in, or coding in a "pre copy" option. I think the pre copy is more extensible, but also opens the door to possible security issues.
However, pre/post call outs might solve other open issues with a general solution. I look forward to your opinion.