An on-boarding process for 'research compendia'?
Opened this issue · 3 comments
A late submission here, but the notion of a research compendium has been a frequent theme at previous unconfs, including the reproducibility guide (@iamciera et al), rrrpkg (@jennybc et al), checkers (@noamross et al), also @benmarwick's rrtools etc. Despite this, I think there are still quite a lot of open questions as to what exactly a compendium is, what tooling we need to support it, etc.
Given the success of the ropensci onboarding process in fostering discussion, promoting norms and boosting the visibility of packages, I wonder if a similar approach would be viable for reviewing/collating/promoting research compendia? Just as a compendium is generally viewed as something less than/simpler than a package, I imagine the review process would be somewhat of a lower bar; primarily a way to verify: "can I reproduce the outputs presented here"? and "can I understand what's going on here and how it's organized?" At the same time, it could be a good venue for learning about ways to improve (i.e. "this looks computationally intensive, you might want to talk to @wlandau about using drake here, or "you might want to put the associated data on Zenodo", etc.
So I have this vague notion that a similar on-boarding process could help build momentum/community/examples of research compendia, but there are still plenty of open questions as to how to pull this off. Should this be done under the auspices of rOpenSci or only synergistically? Should there be a more explicit journal connection, or a JOSS-like journal/index of compendia? How well would this work across domains? And most importantly, is there sufficient interest (editors/reviewers) to pull this off at all?
(idea originally based on discussion with @benmarwick at a DataONE meeting last fall, summarized here: https://github.com/benmarwick/onboarding-reproducible-compendia, but could go in different directions).
tagging @annakrystalli
Much of the work of the checkers project was actually in starting a research compendium review guide which might be of use.
I'm also interested in this, and would be willing to work on this from the humanities/social sciences end.