bdbag
BDDS Big Data Bag Utilities
The bdbag utilities are a collection of software programs for working with BagIt packages that conform the BDDS Bagit and BDDS Bagit/RO profiles.
These utilities combine various other components such as the BDDS forks of the Bagit-Python bag creation utility and the BDDS Bagit-Profiles-Validator utility into a single, easy to use software package.
Enhanced bag support includes:
- Update-in-place functionality for existing bags.
- Automatic archiving and extraction of bags using ZIP, TAR, and TGZ formats.
- Automatic generation of remote file manifest entries and fetch.txt via configuration file.
- Automatic file retrieval based on the contents of a bag's fetch.txt file with multiple protocol support.
- Built-in profile validation.
- Built-in support for creation of bags with Bagit/RO profile compatibility.
Dependencies
- Python 2.7 is the minimum Python version required.
- The code and dependencies are currently compatible with Python 3. Globus Transfer is not currently supported using Python 3.
Installation
Download the current bdbag source code from GitHub or alternatively clone the source from GitHub if you have git installed:
git clone https://github.com/ini-bdds/bdbag
From the root of the bdbag source code directory execute the following command:
python setup.py install --user
Note that if you want to make bdbag available to all users on the system, you should run the following command:
python setup.py install
If you are on a Unix-based system (including MacOS) you should execute the above command as root or use sudo.
Testing
The unit tests can be run by invoking the following command from the root of the bdbag source code directory:
python setup.py test
Configuration
See the Configuration guide.
Usage
This software can be used from the command-line environment by running the bdbag script. For detailed usage instructions, see the CLI guide.
Application Programming Interface
It is also possible to use bdbag from within other Python programs via an API. See the API guide for further details.