Incrementally update dependencies
ellenhp opened this issue · 3 comments
Is your feature request related to a problem? Please describe.
I've been trying to get bazel_maven_repository working in my project for probably close to 12 hours now, and it's frustrating that whenever I change my dependencies at all, it has to regenerate the entire list which takes nearly 10 minutes. I'm considering moving back to gradle because I know once I get this working I'm never going to touch my dependency list again.
Describe the solution you'd like
I'd like to see an update to the version of a single dependency regenerate only the bzl files generated for that specific dependency. I don't see the reason to go back out to the internet and re-fetch all 100+ dependencies.
Describe alternatives you've considered
It's possible it could also be parallelized? At the end of the day I just don't want to be waiting 10 minutes to see if I did successfully fix all the typos, rinse and repeat for hours on end.
Additional context
I'm also curious how updates are handled internally at Square. This seems like a pretty big oversight for a tool like this. Do you have a local cache of all the artifacts you use or something like that to speed things along?
So - a few things here. First, there's a feature request which I have pending to cache pom.xml files on first read, which should make a lot of that annoyance easier. The pom stuff can't be parallelized, as pom analysis happens during analysis phase, since it builds out the set of worker actions which needs to be present during hte build - that's all inherent in the pom metadata. Regenerating only parts of the @maven// workspace is impossible as far as I know. But the pom caching will do most of what you want. I'm planning on this within the next couple of weeks.
I'm not sure why regenerating the project itself takes 10 minutes. We have a heck of a lot of artifacts and we end up with about 30 seconds of reconfiguration (which itself is too long, but again: caching - it's coming). From home, either over a VPN to work, or pulling from maven central, it still takes less than a minute. Can you do some latency testing to the maven repository servers you're using, and see if you have slow links there - maybe try faster mirrors, in the mean-time?
Also, for the .jar files, if you use the sha256 hashes, you'll never download them again (unless you blow away the content addressable cache). But the jar files don't get downloaded until you're actually building.