external data package
kostrzewa opened this issue · 6 comments
In order to include some more example data in hadron, especially loops and/or gradient flow files, it will be necessary to externalise this due to the CRAN size limitations (https://thecoatlessprofessor.com/programming/r/size-and-limitations-of-packages-on-cran/)
We've already started something along these lines in https://github.com/HISKP-LQCD/hadron_example_data, but we clearly need a mechanism in hadron to load this data from github (or elsewhere).
My example data set for the gradient flow observables, for example, is 3.2 MB by itself...
anyone feeling inspired to do this?
I'll take a look into it.
As it is set up now, it is not much work to incrementally add data.
How would we deal with lots of small files in the webserver approach? Would be package them as ZIP files beforehand? The advantage would be that the users could just download the data they want, not having to install all sample data.
In this case one could just make a GitHub repository and directly publish the master git GitHub Pages. I am not sure about automatic directory listings, perhaps we would need to create a table of contents ourselves, which is easy to automate.