with github tag version tar.gz package it is not possible to create scramp versionned rpm (missing PKG-INFO)
bruno-at-bareos opened this issue · 11 comments
bruno-at-bareos commented
I've found in issue 10 the following comment, but I didn't success to update the build of scramp on openSUSE obs build.
https://build.opensuse.org/package/show/home:bruno_friedmann:branches:devel:languages:python/python-scramp
Scramp finish with 0.0.0 which is not good. Beware the build being done on an isolated vm with no remote access we can't download whatever from internet.
So is there a way to setup correctly the version with the rpm ${version} variable ?
Hi @robert-schmidtke, I think the problem may lie with the way Scramp is packaged for conda. Scramp now finds its version by doing:
try:
from importlib.metadata import version
except ImportError:
from importlib_metadata import version
__version__ = version("scramp")
so it uses Scramp's metadata to find the version. The version metadata is created using versioningit. So I think my first thought is that the problem lies with the Conda packaging, but let me know if you find that we need to change something in Scramp.
Originally posted by @tlocke in #10 (comment)
bruno-at-bareos commented
build changelog extract
[ 13s] -----------------------------------------------------------------
[ 13s] + exec rpmbuild -ba --define '_srcdefattr (-,root,root)' --nosignature --define '_build_create_debug 1' /home/abuild/rpmbuild/SOURCES/python-scramp.spec
[ 13s] setting SOURCE_DATE_EPOCH=1670976000
[ 13s] Executing(%prep): /usr/bin/bash -e /var/tmp/rpm-tmp.UNUDSW
[ 13s] + umask 022
[ 13s] + cd /home/abuild/rpmbuild/BUILD
[ 13s] + cd /home/abuild/rpmbuild/BUILD
[ 13s] + rm -rf scramp-1.4.4
[ 13s] + /usr/lib/rpm/rpmuncompress -x /home/abuild/rpmbuild/SOURCES/python-scramp-1.4.4.tar.gz
[ 13s] + STATUS=0
[ 13s] + '[' 0 -ne 0 ']'
[ 13s] + cd scramp-1.4.4
[ 13s] + /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
[ 13s] + RPM_EC=0
[ 13s] ++ jobs -p
[ 13s] + exit 0
[ 13s] Executing(%build): /usr/bin/bash -e /var/tmp/rpm-tmp.jo7abu
[ 13s] + umask 022
[ 13s] + cd /home/abuild/rpmbuild/BUILD
[ 13s] + /usr/bin/rm -rf /home/abuild/rpmbuild/BUILDROOT/python-scramp-1.4.4-0.x86_64
[ 13s] ++ dirname /home/abuild/rpmbuild/BUILDROOT/python-scramp-1.4.4-0.x86_64
[ 13s] + /usr/bin/mkdir -p /home/abuild/rpmbuild/BUILDROOT
[ 13s] + /usr/bin/mkdir /home/abuild/rpmbuild/BUILDROOT/python-scramp-1.4.4-0.x86_64
[ 13s] + cd scramp-1.4.4
[ 13s] ++ '[' -f _current_flavor ']'
[ 13s] ++ true
[ 13s] + last_flavor=
[ 13s] + '[' -z '' ']'
[ 13s] + last_flavor=tmp
[ 13s] + '[' tmp '!=' python38 ']'
[ 13s] + '[' -d build ']'
[ 13s] + '[' -d _build.python38 ']'
[ 13s] + echo python38
[ 13s] + python_flavor=python38
[ 13s] + myargs=
[ 13s] + '[' -f './dist/*-none-any.whl' ']'
[ 13s] + /usr/bin/python3.8 -mpip wheel --verbose --progress-bar off --disable-pip-version-check --use-pep517 --no-build-isolation --no-deps --wheel-dir ./build .
[ 14s] Processing /home/abuild/rpmbuild/BUILD/scramp-1.4.4
[ 14s] Preparing metadata (pyproject.toml): started
[ 14s] Running command Preparing metadata (pyproject.toml)
[ 14s] Traceback (most recent call last):
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/git.py", line 146, in ensure_is_repo
[ 14s] self.read(
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/git.py", line 178, in read
[ 14s] return readcmd("git", *args, cwd=str(self.path), **kwargs)
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/util.py", line 71, in readcmd
[ 14s] s = runcmd(*args, stdout=subprocess.PIPE, text=True, **kwargs).stdout
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/util.py", line 66, in runcmd
[ 14s] return subprocess.run(arglist, **kwargs)
[ 14s] File "/usr/lib64/python3.8/subprocess.py", line 493, in run
[ 14s] with Popen(*popenargs, **kwargs) as process:
[ 14s] File "/usr/lib64/python3.8/subprocess.py", line 858, in __init__
[ 14s] self._execute_child(args, executable, preexec_fn, close_fds,
[ 14s] File "/usr/lib64/python3.8/subprocess.py", line 1704, in _execute_child
[ 14s] raise child_exception_type(errno_num, err_msg, err_filename)
[ 14s] FileNotFoundError: [Errno 2] No such file or directory: 'git'
[ 14s]
[ 14s] During handling of the above exception, another exception occurred:
[ 14s]
[ 14s] Traceback (most recent call last):
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/core.py", line 257, in run
[ 14s] description = self.do_vcs()
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/core.py", line 321, in do_vcs
[ 14s] description = self.vcs(project_dir=self.project_dir)
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/methods.py", line 162, in __call__
[ 14s] return self.method(params=self.params, **kwargs)
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/git.py", line 225, in describe_git
[ 14s] repo.ensure_is_repo()
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/git.py", line 154, in ensure_is_repo
[ 14s] raise NotVCSError("Git not installed; assuming this isn't a Git repository")
[ 14s] versioningit.errors.NotVCSError: Git not installed; assuming this isn't a Git repository
[ 14s]
[ 14s] During handling of the above exception, another exception occurred:
[ 14s]
[ 14s] Traceback (most recent call last):
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/core.py", line 539, in get_version_from_pkg_info
[ 14s] Path(project_dir, "PKG-INFO").read_text(encoding="utf-8")
[ 14s] File "/usr/lib64/python3.8/pathlib.py", line 1236, in read_text
[ 14s] with self.open(mode='r', encoding=encoding, errors=errors) as f:
[ 14s] File "/usr/lib64/python3.8/pathlib.py", line 1222, in open
[ 14s] return io.open(self, mode, buffering, encoding, errors, newline,
[ 14s] File "/usr/lib64/python3.8/pathlib.py", line 1078, in _opener
[ 14s] return self._accessor.open(self, flags, mode)
[ 14s] FileNotFoundError: [Errno 2] No such file or directory: '/home/abuild/rpmbuild/BUILD/scramp-1.4.4/PKG-INFO'
[ 14s]
[ 14s] During handling of the above exception, another exception occurred:
[ 14s]
[ 14s] Traceback (most recent call last):
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/hook.py", line 23, in setuptools_finalizer
[ 14s] report = vgit.run(write=True, fallback=True)
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/core.py", line 280, in run
[ 14s] version=get_version_from_pkg_info(self.project_dir)
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/core.py", line 542, in get_version_from_pkg_info
[ 14s] raise NotSdistError(f"{project_dir} does not contain a PKG-INFO file")
[ 14s] versioningit.errors.NotSdistError: /home/abuild/rpmbuild/BUILD/scramp-1.4.4 does not contain a PKG-INFO file
[ 14s]
[ 14s] During handling of the above exception, another exception occurred:
[ 14s]
[ 14s] Traceback (most recent call last):
[ 14s] File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 351, in <module>
[ 14s] main()
[ 14s] File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 333, in main
[ 14s] json_out['return_val'] = hook(**hook_input['kwargs'])
[ 14s] File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
[ 14s] return hook(metadata_directory, config_settings)
[ 14s] File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 377, in prepare_metadata_for_build_wheel
[ 14s] self.run_setup()
[ 14s] File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 335, in run_setup
[ 14s] exec(code, locals())
[ 14s] File "<string>", line 1, in <module>
[ 14s] File "/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
[ 14s] return distutils.core.setup(**attrs)
[ 14s] File "/usr/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 147, in setup
[ 14s] _setup_distribution = dist = klass(attrs)
[ 14s] File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 475, in __init__
[ 14s] _Distribution.__init__(
[ 14s] File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 283, in __init__
[ 14s] self.finalize_options()
[ 14s] File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 899, in finalize_options
[ 14s] ep(self)
[ 14s] File "/usr/lib/python3.8/site-packages/versioningit/hook.py", line 28, in setuptools_finalizer
[ 14s] raise RuntimeError(
[ 14s] RuntimeError:
[ 14s] versioningit could not find a version for the project in /home/abuild/rpmbuild/BUILD/scramp-1.4.4!
[ 14s]
[ 14s] You may be installing from a shallow clone, in which case you need to unshallow it first.
[ 14s]
[ 14s] Alternatively, you may be installing from a Git archive, which is not supported by default. Install from a git+https://... URL instead.
[ 14s]
[ 14s]
[ 14s] error: subprocess-exited-with-error
[ 14s]
[ 14s] × Preparing metadata (pyproject.toml) did not run successfully.
[ 14s] │ exit code: 1
[ 14s] ╰─> See above for output.
[ 14s]
[ 14s] note: This error originates from a subprocess, and is likely not a problem with pip.
[ 14s] full command: /usr/bin/python3.8 /usr/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpntpjz1nq
[ 14s] cwd: /home/abuild/rpmbuild/BUILD/scramp-1.4.4
[ 14s] Preparing metadata (pyproject.toml): finished with status 'error'
[ 14s] error: metadata-generation-failed
tlocke commented
Hi @bruno-at-bareos, I'm not sure what to do about this. The modern method of Python packaging has the version held in the metadata of the wheel. But that means that the version isn't necessarily present in the tar.gz. Do you have any ideas? Also, I wonder what other Python packages do?
tlocke commented
I've looked into this a bit more, and actually versioningit should be able to read the version from the PKG-INFO file which is in the sdist. So I'm puzzled by the error you found:
FileNotFoundError: [Errno 2] No such file or directory: '/home/abuild/rpmbuild/BUILD/scramp-1.4.4/PKG-INFO
How is it that this file is missing, and yet it's in the .tar.gz?
tlocke commented
In fact I've just done a test where I downloaded scramp-1.4.4.tar.gz, unpacked it, and then was able to build it with python -m build.
bruno-at-bareos commented
I have to review what the dedicated macros does, and I hope none of them are removing the PKG-INFO.
Thanks a lot for your support, I'll post the result here.
bruno-at-bareos commented
I'm a bit puzzled by your report, as here the content of the tar.gz doesn't contain the PKG-INFO
tar -tvf 1.4.4.tar.gz
drwxrwxr-x root/root 0 2022-11-01 18:15 scramp-1.4.4/
drwxrwxr-x root/root 0 2022-11-01 18:15 scramp-1.4.4/.github/
drwxrwxr-x root/root 0 2022-11-01 18:15 scramp-1.4.4/.github/workflows/
-rw-rw-r-- root/root 1025 2022-11-01 18:15 scramp-1.4.4/.github/workflows/test.yml
-rw-rw-r-- root/root 1222 2022-11-01 18:15 scramp-1.4.4/.gitignore
-rw-rw-r-- root/root 906 2022-11-01 18:15 scramp-1.4.4/LICENSE
-rw-rw-r-- root/root 18790 2022-11-01 18:15 scramp-1.4.4/README.rst
-rw-rw-r-- root/root 1533 2022-11-01 18:15 scramp-1.4.4/pyproject.toml
drwxrwxr-x root/root 0 2022-11-01 18:15 scramp-1.4.4/scramp/
-rw-rw-r-- root/root 335 2022-11-01 18:15 scramp-1.4.4/scramp/__init__.py
-rw-rw-r-- root/root 22503 2022-11-01 18:15 scramp-1.4.4/scramp/core.py
-rw-rw-r-- root/root 655 2022-11-01 18:15 scramp-1.4.4/scramp/utils.py
-rw-rw-r-- root/root 158 2022-11-01 18:15 scramp-1.4.4/setup.cfg
drwxrwxr-x root/root 0 2022-11-01 18:15 scramp-1.4.4/test/
-rw-rw-r-- root/root 222 2022-11-01 18:15 scramp-1.4.4/test/test_readme.py
-rw-rw-r-- root/root 18116 2022-11-01 18:15 scramp-1.4.4/test/test_scramp.py
And when I extract this locally and try to build it I got this error message
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/versioningit/git.py", line 146, in ensure_is_repo
self.read(
File "/usr/lib/python3.10/site-packages/versioningit/git.py", line 178, in read
return readcmd("git", *args, cwd=str(self.path), **kwargs)
File "/usr/lib/python3.10/site-packages/versioningit/util.py", line 71, in readcmd
s = runcmd(*args, stdout=subprocess.PIPE, text=True, **kwargs).stdout
File "/usr/lib/python3.10/site-packages/versioningit/util.py", line 66, in runcmd
return subprocess.run(arglist, **kwargs)
File "/usr/lib64/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'rev-parse', '--is-inside-work-tree']' returned non-zero exit status 128.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/versioningit/core.py", line 257, in run
description = self.do_vcs()
File "/usr/lib/python3.10/site-packages/versioningit/core.py", line 321, in do_vcs
description = self.vcs(project_dir=self.project_dir)
File "/usr/lib/python3.10/site-packages/versioningit/methods.py", line 162, in __call__
return self.method(params=self.params, **kwargs)
File "/usr/lib/python3.10/site-packages/versioningit/git.py", line 225, in describe_git
repo.ensure_is_repo()
File "/usr/lib/python3.10/site-packages/versioningit/git.py", line 156, in ensure_is_repo
raise NotVCSError(f"{self.path} is not in a Git repository")
versioningit.errors.NotVCSError: /ioda/home/bruno/openSUSE/obs/home:bruno_friedmann:branches:devel:languages:python/python-scramp/scramp-1.4.4 is not in a Git repository
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/versioningit/core.py", line 539, in get_version_from_pkg_info
Path(project_dir, "PKG-INFO").read_text(encoding="utf-8")
File "/usr/lib64/python3.10/pathlib.py", line 1134, in read_text
with self.open(mode='r', encoding=encoding, errors=errors) as f:
File "/usr/lib64/python3.10/pathlib.py", line 1119, in open
return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: '/ioda/home/bruno/openSUSE/obs/home:bruno_friedmann:branches:devel:languages:python/python-scramp/scramp-1.4.4/PKG-INFO'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/versioningit/hook.py", line 23, in setuptools_finalizer
report = vgit.run(write=True, fallback=True)
File "/usr/lib/python3.10/site-packages/versioningit/core.py", line 280, in run
version=get_version_from_pkg_info(self.project_dir)
File "/usr/lib/python3.10/site-packages/versioningit/core.py", line 542, in get_version_from_pkg_info
raise NotSdistError(f"{project_dir} does not contain a PKG-INFO file")
versioningit.errors.NotSdistError: /ioda/home/bruno/openSUSE/obs/home:bruno_friedmann:branches:devel:languages:python/python-scramp/scramp-1.4.4 does not contain a PKG-INFO file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib64/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/lib/python3.10/site-packages/build/__main__.py", line 22, in <module>
from build.env import IsolatedEnvBuilder
File "/usr/lib/python3.10/site-packages/build/env.py", line 23, in <module>
import virtualenv
File "/usr/lib/python3.10/site-packages/virtualenv/__init__.py", line 1, in <module>
from .run import cli_run, session_via_cli
File "/usr/lib/python3.10/site-packages/virtualenv/run/__init__.py", line 12, in <module>
from .plugin.creators import CreatorSelector
File "/usr/lib/python3.10/site-packages/virtualenv/run/plugin/creators.py", line 4, in <module>
from virtualenv.create.via_global_ref.builtin.builtin_way import VirtualenvBuiltin
File "/usr/lib/python3.10/site-packages/virtualenv/create/via_global_ref/builtin/builtin_way.py", line 3, in <module>
from virtualenv.create.creator import Creator
File "/usr/lib/python3.10/site-packages/virtualenv/create/creator.py", line 11, in <module>
from virtualenv.discovery.cached_py_info import LogCmd
File "/usr/lib/python3.10/site-packages/virtualenv/discovery/cached_py_info.py", line 23, in <module>
_CACHE[Path(sys.executable)] = PythonInfo()
File "/usr/lib/python3.10/site-packages/virtualenv/discovery/py_info.py", line 103, in __init__
self.distutils_install = {u(k): u(v) for k, v in self._distutils_install().items()}
File "/usr/lib/python3.10/site-packages/virtualenv/discovery/py_info.py", line 186, in _distutils_install
d = dist.Distribution({"script_args": "--no-user-cfg"}) # conf files not parsed so they do not hijack paths
File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 475, in __init__
_Distribution.__init__(
File "/usr/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 283, in __init__
self.finalize_options()
File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 899, in finalize_options
ep(self)
File "/usr/lib/python3.10/site-packages/versioningit/hook.py", line 28, in setuptools_finalizer
raise RuntimeError(
RuntimeError:
versioningit could not find a version for the project in /ioda/home/bruno/openSUSE/obs/home:bruno_friedmann:branches:devel:languages:python/python-scramp/scramp-1.4.4!
You may be installing from a shallow clone, in which case you need to unshallow it first.
Alternatively, you may be installing from a Git archive, which is not supported by default. Install from a git+https://... URL instead.
So I wonder why there's a such difference compared to your experience.
tlocke commented
A mystery! So I copied the download link from https://pypi.org/project/scramp/#files and then did:
curl --remote-name https://files.pythonhosted.org/packages/de/26/64cb6f2564c79b8e4e8809e844c9fe464a948221c0b2d173cb9586d06e38/scramp-1.4.4.tar.gz
tar -ztvf scramp-1.4.4.tar.gz
drwxrwxr-x tlocke/tlocke 0 2022-11-01 17:17 scramp-1.4.4/
-rw-rw-r-- tlocke/tlocke 906 2022-10-22 14:23 scramp-1.4.4/LICENSE
-rw-rw-r-- tlocke/tlocke 19808 2022-11-01 17:17 scramp-1.4.4/PKG-INFO
-rw-rw-r-- tlocke/tlocke 18790 2022-11-01 17:14 scramp-1.4.4/README.rst
-rw-rw-r-- tlocke/tlocke 1533 2022-10-22 14:44 scramp-1.4.4/pyproject.toml
drwxrwxr-x tlocke/tlocke 0 2022-11-01 17:17 scramp-1.4.4/scramp/
-rw-rw-r-- tlocke/tlocke 335 2022-05-20 11:43 scramp-1.4.4/scramp/__init__.py
-rw-rw-r-- tlocke/tlocke 22503 2022-11-01 17:11 scramp-1.4.4/scramp/core.py
-rw-rw-r-- tlocke/tlocke 655 2021-07-26 10:16 scramp-1.4.4/scramp/utils.py
drwxrwxr-x tlocke/tlocke 0 2022-11-01 17:17 scramp-1.4.4/scramp.egg-info/
-rw-rw-r-- tlocke/tlocke 19808 2022-11-01 17:17 scramp-1.4.4/scramp.egg-info/PKG-INFO
-rw-rw-r-- tlocke/tlocke 282 2022-11-01 17:17 scramp-1.4.4/scramp.egg-info/SOURCES.txt
-rw-rw-r-- tlocke/tlocke 1 2022-11-01 17:17 scramp-1.4.4/scramp.egg-info/dependency_links.txt
-rw-rw-r-- tlocke/tlocke 69 2022-11-01 17:17 scramp-1.4.4/scramp.egg-info/requires.txt
-rw-rw-r-- tlocke/tlocke 7 2022-11-01 17:17 scramp-1.4.4/scramp.egg-info/top_level.txt
-rw-rw-r-- tlocke/tlocke 197 2022-11-01 17:17 scramp-1.4.4/setup.cfg
drwxrwxr-x tlocke/tlocke 0 2022-11-01 17:17 scramp-1.4.4/test/
-rw-rw-r-- tlocke/tlocke 222 2022-05-20 11:43 scramp-1.4.4/test/test_readme.py
-rw-rw-r-- tlocke/tlocke 18116 2022-11-01 17:11 scramp-1.4.4/test/test_scramp.py
so the PKG-INFO file is there in my listing. Does that help solve it?
bruno-at-bareos commented
My bad the title is misleading, the tar.gz I use was from github because several release in the past the pypi was missing the tests part. (I don't know why there's difference between both tar.gz) So I will give pypi a new try.
tlocke commented
So for the releases on PyPI (both the source distribution and the wheel) the version is in the package's metadata. For the Github snapshot of the code there is no metadata, and so build tools can't determine the version. When a release is created, Scramp uses a tool called versioningit to find the version from a git tag, and so versioningit requires a git repository rather than just a snapshot. Hope that explains the results that you're seeing.
bruno-at-bareos commented
Yes Finally make sense, we packager are often caught between a rock and a hard place, I'm sorry for the noise, after you add fixed the missing tests in pypi package I should have changed the source.
Maybe you can envisage to create a release tarball also for github in the future too.
Thanks a lot for you help.
tlocke commented
Great, glad it's sorted out!