cms-sw/cmspkg

RPM seeding on cc7-base fails

Closed this issue · 11 comments

ericvaandering commented

I'm trying to install WMAgent on a cc7-base docker image and I've tracked the problem down as far as this:

+ rpm --define '_rpmlock_path /home/dmwm/deploy/1.1.7pre4/sw/slc7_amd64_gcc630/var/lib/rpm/__db.0' -U -r /home/dmwm/deploy/1.1.7pre4/sw --rcfile /home/dmwm/deploy/1.1.7pre4/sw/bootstraptmp/BOOTSTRAP/inst/slc7_amd64_gcc630/external/rpm/4.13.0.1/lib/rpm/rpmrc --dbpath /home/dmwm/deploy/1.1.7pre4/sw/slc7_amd64_gcc630/var/lib/rpm RPMS/system-base-import.rpm
error: open of RPMS/system-base-import.rpm failed: No such file or directory

Any ideas? The same type of script is working just fine on the Jenkins nodes...

ericvaandering commented

How to reproduce:

docker run  -it  --entrypoint /bin/bash gitlab-registry.cern.ch/cmsdocks/dmwm:wmcore_base

then inside container

git clone https://github.com/dmwm/deployment.git
$PWD/deployment/Deploy -R wmagent-dev@1.1.7-pre4 -r comp=comp -t 1.1.7-pre4 -A slc7_amd64_gcc630 -s 'prep sw post' /home/dmwm/unittestdeploy/wmagent admin/devtools wmagent

There you can see the problem is the bootstrap script and if you run that separately

cd ./unittestdeploy/wmagent/1.1.7-pre4/sw/
sh -x ./bootstrap.sh -architecture slc7_amd64_gcc630 -path /home/dmwm/unittestdeploy/wmagent/1.1.7-pre4/sw -repository comp -server cmsrep.cern.ch setup

You can see the failure. Any ideas @smuzaffar @amaltaro @ticoann or @h4d4?

davidlt commented

You probably want to review the whole bootstrap log. The base images are extremely bare and might not have all the commands required to run bootstrap script (happens). Alternatively something could be wrong with RPM itself (happens). Then sourcing it and manually running command sometimes reveals what's wrong.

amaltaro commented

Can it be the docker images are missing one of the following RPM dependencies (we had to manually deploy those on the CC7 agents):
libaio-0.3.109-13.el7.x86_64 unzip wget libXft-devel libXpm-devel libXext-devel mesa-libGLU-devel perl-Switch perl-Env perl-Thread-Queue
?

davidlt commented

It generated malformed system-base-import.spec because which is not installed:

Provides: perl(vmsish)
Provides: perl(warnings)
Provides: perl(warnings::register)
./bootstrap.sh: line 1499: which: command not found

%description
Seeds RPM repository from the base system.
ericvaandering commented

Interesting. I didn't know you could have a linux system without which. I put it in, let me see what happens.

davidlt commented

Adding which, libaio, curl or/and wget to http://cmsrep.cern.ch/cgi-bin/cmspkg/driver/cms/<SCRAM_ARCH> could be beneficial.

Hmm... which and libaio are installed if the system has HEP_OSlibs installed.

ericvaandering commented

I do install HEP_OSlibs. I'm not trying to install onto cc7-base itself, but an image derived from it with wget, oslibs, and a fair bit of other stuff. However, "which" is actually not in the final image. You can see how I'm generating this with https://github.com/dmwm/Docker/blob/master/cms_grid/Dockerfile and then follow-on file in wmcore_base

It's wmcore_tests that follows THAT and attempts the bootstrap.

davidlt commented

Container images are getting smaller (see: https://fedoraproject.org/wiki/Changes/ContainerMinimalImage it's just ~20MB in size ) Basically you get minimal shell and access to package manager.

davidlt commented

Hmm.. Looks like which was removed from HEP_OSlibs in CentOS 7.

ericvaandering commented

INFO: bootstrap successful

Ok, I hope that clears the way for the Agent install as well. Thanks guys!

davidlt commented

Here is proof that it's removed:

[davidlt@lxplus029]~% rpm -q --requires HEP_OSlibs_SL6 | grep -i which
which(x86-64)

[dmwm@96dee7d4c883 ~]$ rpm -q --requires HEP_OSlibs | grep which