Support S3 Mirroring
Closed this issue · 6 comments
The NuGetMirror executable could upload directly to Amazon S3.
Would you tell me a bit more about what you are expecting for this?
Would it upload the nupkgs only, or would it clone all json files, rewrite the urls to match the s3 location and then upload them there?
Upload the nupkgs would be a nice start, but cloning json files and rewrite URLs would be awesome.
If I get spare time I can work towards this.
I like the idea. CatalogReader library I think a way to download/clone the json files themselves is needed. Right now they go into a temporary cache, but they could be mirrored also to speed up incremental reads.
There are some helpers for rewriting URLs either in this project or in Sleet.
Uploading to s3 is something I have been planning to add to sleet also. Getting the credentials in a config file and everything it needs to upload files was the biggest piece last time I looked at it.
I'm not sure where all of this should go, I think some of it could be in CatalogReader and then potentially another tool could exist to do the actual syncing to S3.
@emgarten you could use https://github.com/aloneguid/storage to support more destinations.
@aloneguid that looks like a great project! @skarllot added support for S3 in Sleet, otherwise I'd consider using this.
This issue has been open a while and there isn't a clear path forward for this. I think a 3rd tool or script is needed to run NuGetMirror and push to S3 using Sleet since this would be a very specific scenario and it would likely need a whole other set of configuration options to customize things.
I'm happy to help anyone interested in creating this, but currently I don't think it would be a good fit for NuGetMirror to build this functionality into the current tool.