Back in 2016, there was discussion and excitement for data containers.
Two recent developments have told me that now is the time to address this once more:
- An article that details this idea, that containers don't necessary need an operating system.
- The ability to create a container from scratch supported by Singularity (pull request here)
Before we can build a data container, we need to decide what is important for it to have, or generally be. If we think of a "normal" container as providing a base operating system to support libraries, small data files, and ultimately running scientific software, then we might define a data container as:
- a container to support the provenance, management, and query of data.
- container should work bound as a volume or on it's own
- it should be possible to customize how metadata is extracted from the dataset
See the devel folder for early development work.
This repository will soon be populated with specific docker and singularity examples, generated by way of vsoch/cdb, the container database metadata generator.