Process repositories locally by cloning
damevski opened this issue · 7 comments
OAuth token rate limits are set in stone and it's probably not a good idea to request each commit individually, so we should look at this if we care about large or private repositories.
Cloning is not available in Octokit. libgit2sharp looks like a good alternative. We've incurred some debt on this, so at least for the time being, it may be wise to cut-off at a threshold number of commits or throw an error when the user reaches their limit and ask them to try later. Thoughts? @damevski
Let's go with a threshold for now, and save the issue so we can take care of this down the road sometime.
instead of ripping out octokit we could just get the clone URL from each repo and process using some other process in the background
We definitely can do that, but we would have to figure out some way to parse it ourselves. libgit2sharp can do that fo us. We should pass the clone URL into that while still using Octokit to communicate with the API.
As of f08b76b, local parsing is fully working. I'll merge after I do some more thorough testing.
Libgit2Sharp.NativeBinaries does not currently support distros that are not Ubuntu-derived, so follow these instructions to build the needed library for CentOS:
git clone --recursive https://github.com/libgit2/libgit2sharp.nativebinariescd libgit2sharp.nativebinaries- Make sure you have the necessary OpenSSL version with
yum install openssl-devel ./build.libgit2.sh- Copy
libgit2-2d2a602.soor similar to the package install usingcp libgit2-2d2a602.so ~/.nuget/packages/libgit2sharp.nativebinaries/1.0.2xx/runtimes/linux-x64/native
I will script this out when I have the time. It looks like they should be merging in support for multiple distros soon.