openshift-eng/doozer

Error in state.yaml after failed rebase

Closed this issue · 7 comments

dorzel commented

OS: rhel7
Arch: s390x
Doozer version: v0.4.12

I'm seeing an error when attempting to build 3.11. After cloning all repos and then starting a rebase, golang-github-openshift-oauth-proxy fails to rebase, giving the following error (full output here: http://pastebin.test.redhat.com/759623):

The following non-critical images failed to rebase:
golang-github-openshift-oauth-proxy

doozer logs an error (see line 140 in above pastebin) but the error output itself doesn't appear in the output. This also leads to a malformed state file:

...
images:rebase:
  images:
    aos3-installation: true
    atomic-openshift-cluster-autoscaler: true
    atomic-openshift-descheduler: true
    atomic-openshift-metrics-server: true
    atomic-openshift-node-problem-detector: true
    automation-broker-apb: true
    cluster-monitoring-operator: true
    configmap-reload: true
    csi-attacher: true
    csi-driver-registrar: true
    csi-livenessprobe: true
    csi-provisioner: true
    efs-provisioner: true
    golang-github-openshift-oauth-proxy: ''
    golang-github-prometheus-alertmanager: true
...

Where the expected false is replaced by single quotes. This causes any build commands to fail with:

[dorzel@devel42: oit-BB_eWT.tmp]$ doozer --latest-parent-version --local -i openshift-enterprise-registry images:build
/home/nfs/dorzel/.config/doozer/settings.yaml
2019-04-29 14:51:14,505 INFO Using branch from group.yml: rhaos-3.11-rhel-7
2019-04-29 14:51:14,876 INFO [containers/openshift-enterprise-registry] Distgit directory already exists; skipping clone: /tmp/oit-BB_eWT.tmp/distgits/containers/openshift-enterprise-registry
Traceback (most recent call last):
  File "/usr/bin/doozer", line 1831, in <module>
    main()
  File "/usr/bin/doozer", line 1812, in main
    cli(obj={})
  File "/usr/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args[1:], **kwargs)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/bin/doozer", line 801, in images_build_image
    img_obj = runtime.image_map[img]
KeyError: 'golang-github-openshift-oauth-proxy'

manually editing the state.yaml value to be false resolves this issue.

adammhaile commented

That empty string is supposed to be the error it got when failing... in this case apparently it didn't get any specific error. All you did by changing the value was mask the error. Try rebasing just golang-github-openshift-oauth-proxy and then look in debug.log

dorzel commented

Doing that rebase leaves me with:

[dorzel@devel42: oit-BB_eWT.tmp]$ doozer --latest-parent-version --local -i golang-github-openshift-oauth-proxy images:rebase
/home/nfs/dorzel/.config/doozer/settings.yaml
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:838: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/security.html
  InsecureRequestWarning)
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:838: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/security.html
  InsecureRequestWarning)
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:838: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/security.html
  InsecureRequestWarning)
2019-04-30 18:28:06,183 INFO Using branch from group.yml: rhaos-3.11-rhel-7
2019-04-30 18:28:06,629 INFO [containers/golang-github-openshift-oauth-proxy] Distgit directory already exists; skipping clone: /tmp/oit-BB_eWT.tmp/distgits/containers/golang-github-openshift-oauth-proxy
2019-04-30 18:28:06,868 INFO Source 'golang-github-openshift-oauth-proxy_oauth-proxy' already exists in (skipping clone): /tmp/oit-BB_eWT.tmp/sources/golang-github-openshift-oauth-proxy_oauth-proxy
2019-04-30 18:28:06,880 ERROR [containers/golang-github-openshift-oauth-proxy] 

Doozer Failed With Error:
No required images were specified, but all images failed.

and in debug.log:

2019-04-30 18:28:06,183 INFO (4397998139120) Using branch from group.yml: rhaos-3.11-rhel-7
2019-04-30 18:28:06,608 DEBUG (4397998139120) Loading metadata from /tmp/oit-BB_eWT.tmp/ocp-build-data/images/golang-github-openshift-oauth-proxy.yml
2019-04-30 18:28:06,629 INFO (4397962205456) [containers/golang-github-openshift-oauth-proxy] Distgit directory already exists; skipping clone: /tmp/oit-BB_eWT.tmp/distgits/containers/golang-github-openshift-oauth-proxy
2019-04-30 18:28:06,849 DEBUG (4397998139120) Resolving local source directory for alias golang-github-openshift-oauth-proxy_oauth-proxy
2019-04-30 18:28:06,849 DEBUG (4397998139120) checking for source directory in source_dir: /tmp/oit-BB_eWT.tmp/sources/golang-github-openshift-oauth-proxy_oauth-proxy
2019-04-30 18:28:06,868 INFO (4397998139120) Source 'golang-github-openshift-oauth-proxy_oauth-proxy' already exists in (skipping clone): /tmp/oit-BB_eWT.tmp/sources/golang-github-openshift-oauth-proxy_oauth-proxy
2019-04-30 18:28:06,880 ERROR (4397998139120) [containers/golang-github-openshift-oauth-proxy]
adammhaile commented
dorzel commented

I'm using openshift-3.11, here's my settings.yaml:

#Persistent working directory to use
working_dir: /tmp/oit-BB_eWT.tmp

#Git URL or File Path to build data
#data_path: https://github.com/openshift/ocp-build-data
data_path: /tmp/oit-BB_eWT.tmp/ocp-build-data

#Sub-group directory or branch to pull build data
group: openshift-3.11

#Username for running rhpkg / brew / tito
user: dorzel

global_opts:
  distgit_threads: 20
adammhaile commented
dorzel commented

Yes it seems to fallback to the master branch, and the versions are done with git tags for the source repo instead of having a branch.

Messing with 3.10, it looks like it doesn't even use a source repo, so it can't run into this problem with that version at least.

thiagoalessio commented

Hi @dorzel i'm closing this issue since a lot has changed since May. But please, feel free to re-open it in case it is still unresolved in our latest release (https://pypi.org/project/rh-doozer/0.5.26/)

  • 👍1