Fedora packgaing challenges
Opened this issue · 4 comments
I've tried to package this as an RPM to replace python-certfi.
I know it is a hack and I have no intentions to put it in Fedora proper, this is merely an experiment now.
In this issue, I'll post the challenges I've encountered.
This is the first spec file:
Name: python-certifi-system-store
Version: 3021.3.13
Release: 1%{?dist}
Summary: A certifi hack to use system trust store on Linux/FreeBSD
License: MPLv2.0
URL: https://github.com/tiran/certifi-system-store
Source0: %{url}/archive/v%{version}/certifi-system-store-%{version}.tar.gz
BuildArch: noarch
BuildRequires: python3-devel
BuildRequires: pyproject-rpm-macros
%global _description %{expand:
...}
%description %_description
%package -n python3-certifi-system-store
Summary: %{summary}
%description -n python3-certifi-system-store %_description
%prep
%autosetup -p1 -n certifi-system-store-%{version}
%generate_buildrequires
%pyproject_buildrequires -e %{toxenv}-test
%build
%pyproject_wheel
%install
%pyproject_install
%pyproject_save_files certifi
%check
%tox
%files -n python3-certifi-system-store -f %{pyproject_files}
%doc README.md
%license LICENSE
%changelog
* Tue Mar 16 2021 Miro Hrončok <mhroncok@redhat.com> - 3021.3.13-1
- Initial packaging attempt
First challenge: The autopatching happens in wrong time/place:
I see form the output of tox:
+ /usr/bin/python3 -m tox --current-env -q --recreate -e py39-test
certifi-system store 3021.03.13
Patched certifi.dist-info -> certifi_system_store.dist-info
/etc/ssl/cert.pem
... pytest output here ...
___________________________________ summary ____________________________________
py39-test: commands succeeded
congratulations :)
It would really be convenient to see where is the dist info "patched", I needed to track that down myself.
Apparently, it is in $PWD:
$ env LANG=en_US.utf-8 \ls -ld *.dist-info
lrwxrwxrwx. 1 churchyard mock 30 Mar 16 10:34 certifi.dist-info -> certifi_system_store.dist-info
drwxr-xr-x. 2 churchyard mock 4096 Mar 16 10:34 certifi_system_store.dist-info
That's... not good.
I can run the patching myself and set some envvars to fool it to do it in what I otherwise pass as --root e.g. to pip install in %pyproject_install but I don't think the patching command supports that.
Now let's pretendt I haven't noticed this and I install the RPM package that only has the normal dist-info, not the symbolic link:
$ rpm -ql python3-certifi-system-store-3021.3.13-1.fc35.noarch.rpm | grep dist-info
/usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info
/usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info/INSTALLER
/usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info/LICENSE
/usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info/METADATA
/usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info/WHEEL
/usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info/top_level.txt
/usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info/zip-safe
When invoked as regular user:
$ python3.9 -m certifi -v
Traceback (most recent call last):
File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/lib/python3.9/site-packages/certifi/__main__.py", line 14, in <module>
_patched = _patch_dist_info()
File "/usr/lib/python3.9/site-packages/certifi/_patch.py", line 45, in _patch_dist_info
_relsymlink(target=abs_css_distinfodir, linkname=abs_certifi_distinfodir)
File "/usr/lib/python3.9/site-packages/certifi/_patch.py", line 14, in _relsymlink
os.symlink(rel_target, linkname_file, dir_fd=dir_fd)
PermissionError: [Errno 13] Permission denied: 'certifi_system_store-3021.3.13.dist-info' -> 'certifi-3021.3.13.dist-info'
As root:
# python3.9 -m certifi -v
certifi-system store 3021.03.13
Patched certifi.dist-info -> certifi_system_store.dist-info
/etc/ssl/cert.pem
# ls -l /usr/lib/python3.9/site-packages/certifi-3021.3.13.dist-info
lrwxrwxrwx. 1 root root 40 Mar 16 10:46 /usr/lib/python3.9/site-packages/certifi-3021.3.13.dist-info -> certifi_system_store-3021.3.13.dist-info
# rpm -qf /usr/lib/python3.9/site-packages/certifi-3021.3.13.dist-info
file /usr/lib/python3.9/site-packages/certifi-3021.3.13.dist-info is not owned by any package
This is bad from packaging perspective.
In the meantime, let me try to create the symbolic link manually from the spec instead:
%install
%pyproject_install
%pyproject_save_files certifi
ln -sr %{buildroot}%{python3_sitelib}/certifi_system_store-%{version}.dist-info %{buildroot}%{python3_sitelib}/certifi-%{version}.dist-info
%check
%tox
%files -n python3-certifi-system-store -f %{pyproject_files}
%doc README.md
%license LICENSE
%{python3_sitelib}/certifi-%{version}.dist-info
Except now tox fails:
Traceback (most recent call last):
File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/builddir/build/BUILDROOT/python-certifi-system-store-3021.3.13-1.fc35.x86_64/usr/lib/python3.9/site-packages/certifi/__main__.py", line 14, in <module>
_patched = _patch_dist_info()
File "/builddir/build/BUILDROOT/python-certifi-system-store-3021.3.13-1.fc35.x86_64/usr/lib/python3.9/site-packages/certifi/_patch.py", line 34, in _patch_dist_info
shutil.rmtree(certifi_dist.egg_info)
File "/usr/lib64/python3.9/shutil.py", line 728, in rmtree
onerror(os.path.islink, path, sys.exc_info())
File "/usr/lib64/python3.9/shutil.py", line 726, in rmtree
raise OSError("Cannot call rmtree on a symbolic link")
OSError: Cannot call rmtree on a symbolic link
ERROR: InvocationError for command /builddir/build/BUILD/certifi-system-store-3021.3.13/.tox/py39-test/bin/python -m certifi -v (exited with code 1)
So I need to create the link after running the tests, which feels dirty:
%install
%pyproject_install
%pyproject_save_files certifi
%check
%tox
ln -sr %{buildroot}%{python3_sitelib}/certifi_system_store-%{version}.dist-info %{buildroot}%{python3_sitelib}/certifi-%{version}.dist-info
%files -n python3-certifi-system-store -f %{pyproject_files}
%doc README.md
%license LICENSE
%{python3_sitelib}/certifi-%{version}.dist-info
$ rpm -qvl python3-certifi-system-store-3021.3.13-1.fc35.noarch.rpm | grep dist-info
lrwxrwxrwx 1 root root 40 Mar 16 10:56 /usr/lib/python3.9/site-packages/certifi-3021.3.13.dist-info -> certifi_system_store-3021.3.13.dist-info
drwxr-xr-x 2 root root 0 Mar 16 10:56 /usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info
-rw-r--r-- 1 root root 4 Mar 16 10:56 /usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info/INSTALLER
-rw-r--r-- 1 root root 16726 Mar 16 10:56 /usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info/LICENSE
-rw-r--r-- 1 root root 6023 Mar 16 10:56 /usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info/METADATA
-rw-r--r-- 1 root root 92 Mar 16 10:56 /usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info/WHEEL
-rw-r--r-- 1 root root 8 Mar 16 10:56 /usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info/top_level.txt
-rw-r--r-- 1 root root 1 Mar 16 10:56 /usr/lib/python3.9/site-packages/certifi_system_store-3021.3.13.dist-info/zip-safe
But the RPM dist dep generator is not fooled by the linked dist-info:
$ rpm -qp --provides /var/lib/mock/fedora-rawhide-x86_64/result/python3-certifi-system-store-3021.3.13-1.fc35.noarch.rpm
python-certifi-system-store = 3021.3.13-1.fc35
python3-certifi-system-store = 3021.3.13-1.fc35
python3.9-certifi-system-store = 3021.3.13-1.fc35
python3.9dist(certifi-system-store) = 3021.3.13
python3dist(certifi-system-store) = 3021.3.13
Thanks for looking into this, Miro!
The symlink is a nasty/clever hack. For now Python packages cannot declare that they provide an alternative distribution name. Currently pip ignores provides and obsoletes metadata fields. Hopefully https://discuss.python.org/t/packaging-forks/7502 will make the hack obsolete in the future.
PR #9 adds better logging and error handling.
Tox tests should only create the symlink in tox's virtual env. As you mentioned in your last comment, you also have to create the symlink and ship the symlink in the packaged site-packages directory.