Add support for treeless and/or blobless clones
jemerald opened this issue · 2 comments
What feature do you want to see added?
In multi-branch pipeline setup, add options to support treeless and/or blobless clones
See what they are: https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/
Why?
We are a relatively large repo where a full clone takes several minutes and transfers more than 2GB of data.
And to avoid invisible merge issue, we have opt for building pull requests with the strategy of Merging the pull request with the current target branch revision.
Unfortunately this means that the shallow clone option is not available to us as merging is not possible (and the plugin defaults back to doing a full clone).
The (relatively) new treeless/blobless git feature should greatly improve clone performance for large repos while still cater for merge strategy. I'd even recommend to use the blobless clone as the default option as it does not compromise on feature (unlike shallow clone).
Here's a comparison of transfer size and resulting folder size for our repo:
| Clone method | Network transfer size | Directory size |
|---|---|---|
| full | 2.57 GiB | 2.9G |
| shallow | 88.82 MiB | 380M |
| blobless | 91.21 MiB + 88.24 MiB | 505M |
| treeless | 20.88 MiB + 768.26 KiB + 88.24 MiB | 405M |
Upstream changes
No response
Have you tried a reference repository as described in Jenkins hints for large git repos?
yes, reference repo does speed up the pipeline checkout on Jenkins master (down to ~1m for the first build of each new branch), but does not help with checkout scm in the pipeline as it runs on ephemeral k8s slaves