python2.7, python3.6 and python3.8 still in ELN
hroncok opened this issue · 12 comments
Hey ELN people.
Despite my efforts to block python2.7, python3.6 and python3.8 from ELN, I still see that the packages are built there.
Could you please help me understand what needs to be done to prevent that from happening?
I've tried to communicate this in https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/U4TY26R3S65E474FUKGKB6H4JOCIEITY/ but there was no useful info except one private reply by @sgallagher who suggested that the proper communication channel for this are PRs in this repo.
The relevant PRs were:
Later moved around by #68.
Two answers.
First - marking something as unwanted does not automatically remove it. It marks it as unwanted.
You can then go to the content resolver and see the list of packages marked as unwanted that are still in the release. There are also ways to track down what is pulling in those unwanted dependencies.
In a few weeks, (when the vast majority of the wanted packages are set) we will start working our way through those unwanted packages. Opening pull requests and/or bugs.
Second - You have shown me I have a bug in my buildroot-generator script.
I re-wrote it specifically so that it wouldn't show duplicate packages (python3.5, 3.6, 3.7 and 3.8) for the same build dependencies. But, I wasn't using python3.* as my test case, I was using fedora-release* ... and a couple others, as my test case. And for those, this worked.
But, since all the different python's can be installed at the same time, they all get listed as build dependencies.
I'm going back to work on my script and get it fixed.
First - marking something as unwanted does not automatically remove it. It marks it as unwanted...
Yes, I am aware of that. That's why I've said (in the email thread):
While I understand that python2.7 might still be pulled in as dependency by
other ELN packages (presumably gimp), I don't understand why is python3.8 and
python3.6 built."
I re-wrote it specifically so that it wouldn't show duplicate packages (python3.5, 3.6, 3.7 and 3.8)...
I am not sure I follow what "duplicate packages" means here.
But, since all the different python's can be installed at the same time, they all get listed as build dependencies.
As build dependencies of what packages? https://tiny.distro.builders/view-unwanted--view-eln--x86_64.html doesn't show python3.6 and python3.8 as depended on.
First - marking something as unwanted does not automatically remove it. It marks it as unwanted...
Yes, I am aware of that. That's why I've said (in the email thread):
While I understand that python2.7 might still be pulled in as dependency by
other ELN packages (presumably gimp), I don't understand why is python3.8 and
python3.6 built."I re-wrote it specifically so that it wouldn't show duplicate packages (python3.5, 3.6, 3.7 and 3.8)...
I am not sure I follow what "duplicate packages" means here.
But, since all the different python's can be installed at the same time, they all get listed as build dependencies.
As build dependencies of what packages? https://tiny.distro.builders/view-unwanted--view-eln--x86_64.html doesn't show python3.6 and python3.8 as depended on.
That page shows:
Not present, but listed here as it's in:
Unwanted Python 3 packages exclusion list (maintainer: sst_cs_apps)
So that seems like it's reporting that we're no longer including it in the list. That said, clearly it's still getting picked up by the rebuild automation, so that's a bug.
Sorry, I guess I should have linked to the content with the buildroot.
https://tiny.distro.builders/view--view-eln-and-buildroot.html
It's the buildroot dependencies that are pulling them in.
What a package needs when it's built.
But none of the packages are saying that they need all those python's, just that they need python3-devel or python3-setuptools.
Two examples are python-ldap and pyproject-rpm-macros
I don't see how python3-devel or python3-setuptools is related. They are not provided by python3.6 / python3.8. What buildroot dependencies that are pulling them in?
For completeness:
$ repoquery --repo=rawhide{,-source} --whatrequires python3.6
marshalparser-0:0.2.0-1.fc33.src
$ repoquery --repo=rawhide{,-source} --whatrequires python3.8
blender-1:2.82a-5.fc33.x86_64
kdevelop-python-0:5.5.1-1.fc33.i686
kdevelop-python-0:5.5.1-1.fc33.x86_64
marshalparser-0:0.2.0-1.fc33.src
micropython-0:1.12-2.fc33.src
python3-csound-0:6.13.0-7.fc33.i686
python3-csound-0:6.13.0-7.fc33.x86_64
python3-gpaw-mpich-0:19.8.1-6.fc32.x86_64
python3-gpaw-openmpi-0:19.8.1-6.fc32.x86_64
python3-graph-tool-0:2.29-3.fc33.x86_64
python3.8-0:3.8.5-2.fc33.src
$ repoquery --repo=eln{,-source} --whatrequires python3.6
(nothing)
$ repoquery --repo=eln{,-source} --whatrequires python3.8
python3.8-0:3.8.5-2.eln103.src
If repoquery worked without problems, I'd be using it. It doesn't.
That's why my first scripts were using.
I'm not saying it's wrong in this case, it's probrubly right.
But I've had to re-write "dnf builddep" so that it will run without having to install.
And if you run
dnf builddep python-ldap
You'll see that it tries to install all the python3.*
So, it looks like it's a weak dependencies things, which is fixable, but since I thought I'd already done that ... need to get that part fixed.
And if you run
dnf builddep python-ldap
You'll see that it tries to install all the python3.*
Oh. It has to do with recommends. Tox recommends all the pythons.
So, it looks like somewhere in all my edits, I dropped my "...install_weak_deps = False" line.
I have put it back into the right place, and all of the extra python3.* packages go away.
I am putting the script into the pipeline, but it takes a while for everything to funnel through. So, hopefully tomorrow Content Resolver will have them dropped, and they will come out of ELN shortly after that.
So, apparently also python3.7, python3.5, pypy and pypy3 is built for ELN. I'll make sure to mark them as unwanted.
My code fix worked, and the resulting buildroot has gone through the Content Resolver..
We still need to untag all the packages that were previously tagged in, and then a new compose of the repo's and everything else needs to be done. I plan on getting the packages untagged today, and by tomorrow (or earlier) we should have all those extra python's out of the compose.
python2.7 is still in ELN, but it is pulled into it by some dependents. The other packages are not.