Glenn Stampalia, CC BY 3.0 https://creativecommons.org/licenses/by/3.0, via Wikimedia Commons
Few tools can elegantly manage the following packages:
- tensorflow - is bloated and pins specific versions of many of its dependencies
- cartopy - has many system dependencies
<random pypi package>
. This can be easily installed bypip install
, but isn't in most distributions. The tool xgcm is a good example.
This situation usually requires using multiple package managers like this:
apt-get install python3-cartopy python3-pip
pip install tensorflow <random pypi package>
However, using multiple package managers leads to ugly/non-portable installation scripts that are hard to maintain and not very composable. Moreover, it is hard to lock dependencies, manage license, etc, across multiple tools.
This Rodeo is designed to find ONE tool that can manage these three dependencies in an elegant way. To manage a dependency a tool must
- be able to install a working environment
- in a reasonable amount of time
- be able to lock this installed environment so that it can reproduced at a future date.
This project tests point (1) using continuous integration (CI).
- anaconda
- pip
- apt-get (on an ubuntu system)
- Nix
- nixpkgs
- poetry2nix
- mach-nix
The goal is to compare these on Mac and Linux, but only Linux is currently implemented.
The more packages a tool can install the better. Passing CI tests is good.
Package Manager | CI Status | Packages Attempted to install 1 |
---|---|---|
pip | tensorflow, xgcm | |
conda | cartopy, tensorflow | |
nixpkgs | cartopy, tensorflow, xgcm | |
nixpkgs (machnix) | cartopy, tensorflow, xgcm | |
nixpkgs (poetry2nix) | cartopy, tensorflow, xgcm | |
apt-get | cartopy |
Please contribute a new packaging tool or modify an existing tool's configuration to improve its results. That said, we will need to discuss what a "tools configuration" consists of. Some rough thoughts are:
- conda can install pip dependencies, but to my knowledge it does not ensure that pip versions installed are consistent with the resolved conda versions, and cannot be frozen very easily
- Nix-based tools have more leeway in this regard since its "configuration" is the nix programming language
- Manually packaging the
<random pip package>
(e.g. writing an explicit nix derivation, conda recipe, etc) is not allowed.
Footnotes
-
Readers should assume that the tool CANNOT install more than this ↩