Facilitating Python Usage: Proposal for s5cmd Wheels
Opened this issue · 2 comments
Firstly, huge thanks for your work on the s5cmd tool and for contributing to the community with such a valuable tool. 🙏
Goal
We're keen on enhancing the accessibility of s5cmd within Python projects. Our aim is to streamline its integration by enabling installation via a simple pip command:
pip install s5cmd
Upon installing the wheel package, we envision an automatic creation of a convenient executable script1, allowing seamless invocation of the underlying s5cmd
executable.
Proposed Next Steps
To foster cohesion and prevent fragmentation, we propose the creation of a dedicated project repository, tentatively named peak/s5cmd-python-distributions
. This draws inspiration from analogous projects like scikit-build/cmake-python-distributions
2 and scikit-build/ninja-python-distributions
3.
We're committed to aiding in the setup and maintenance of this infrastructure. With this in place, we can align our project, ImagingDataCommons/idc-index
, to become a pure Python wheel, contingent upon the s5cmd
project.
The implementation approach mirrors our current practices within our package:
- idc_index/s5cmd/init.py: A Python module with a function used as entry point for the launcher script generation.
- CMakeLists.txt: CMake configuration for downloading the go executable using URL and SHA specified in s5cmdUrls.cmake.
Furthermore, we propose the implementation of a GitHub workflow, automatically triggered upon each new release of the corresponding go
executable. We plan to adapt the bump
function from cmake-python-distributions
4 for this purpose.
Alternative Project Repository
We are also considering ImagingDataCommons/s5cmd-python-distributions
as an alternative home for this project.
Background Details
-
@fedorov is leading the https://github.com/ImagingDataCommons initiative
-
I (@jcfr) along with @henryiii, @thewtex and many others are developing and maintaining the
cmake
2,ninja
3 andscikit-build-core
5 Python projects.
Footnotes
After running pip install s5cmd
, the following will be possible from Python without having to manually download/extract/install:
import shutil
import subprocess
s5cmdPath = shutil.which("s5cmd")
if s5cmdPath is None:
raise FileNotFoundError(
"s5cmd executable not found. Please install s5cmd from https://github.com/peak/s5cmd#installation"
)
print(f"Found s5cmd executable: {s5cmdPath}")
subprocess.check_call([s5cmdPath, "--help"])
To follow-up on this, a dedicated project has been created at https://github.com/jcfr/s5cmd-python-distributions
Wheels have been published at https://pypi.org/project/s5cmd/
The following platforms are supported by the binary wheels:
OS | Arch |
---|---|
Windows | 64-bit 32-bit ARM64 |
Linux Intel | manylinux 64-bit musllinux 64-bit manylinux 32-bit musllinux 32-bit |
Linux ARM | manylinux AArch64 musllinux AArch64 |
Linux PowerPC | manylinux ppc64le musllinux ppc64le |
macOS 10.10+ | Intel |
macOS 11+ | Apple Silicon |