coreos-koji-tagger: error re-authenticating: `Mutual authentication failed`
dustymabe opened this issue · 7 comments
dustymabe commented
So now that we properly catch the koji.AuthError exception we have moved to the next failure. We apparently hit a problem when trying to authenticate again.
2020-01-09 04:03:50,216 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-01-09 04:03:50,217 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-01-09 04:03:50,568 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-01-09 04:03:50,640 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-01-09 04:03:50,643 ERROR coreos_koji_tagger - Caught Exception!
2020-01-09 04:03:50,643 ERROR coreos_koji_tagger - ###################################
Traceback (most recent call last):
2020-01-09 04:03:50,643 ERROR coreos_koji_tagger - ###################################
2020-01-09 04:03:50,643 ERROR coreos_koji_tagger - continuing...
File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 195, in wrapper
return func(*args, **kwargs)
File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 300, in process_message
self.koji_login()
File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 461, in koji_login
self.koji_client.gssapi_login(principal, self.keytab_file)
File "/usr/lib/python3.7/site-packages/koji/__init__.py", line 2415, in gssapi_login
raise AuthError('unable to obtain a session')
koji.AuthError: unable to obtain a session
The interesting message in there is the requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed which comes from the requests-kerberos python library.
I'll dig into this later, for now I'll restart the tagger.
jlebon commented
Hmm, I wonder if we should try to just wrap the gssapi_login call in a retry loop. If we were logged out because Koji is restarting, it might not be fully up yet by the time we try to log back in.
dustymabe commented
now we are at:
2020-03-18 13:56:42,446 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 13:56:42,446 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 13:56:43,443 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 13:56:43,642 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 13:57:13,855 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 13:57:13,855 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 13:57:13,865 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 13:57:13,924 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 13:57:44,022 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 13:57:44,022 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 13:57:44,032 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 13:57:44,069 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 13:58:14,126 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 13:58:14,126 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 13:58:14,134 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 13:58:14,170 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 13:58:44,248 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 13:58:44,248 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 13:58:44,256 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 13:58:44,314 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 13:59:14,393 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 13:59:14,394 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 13:59:14,402 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 13:59:14,453 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 13:59:44,548 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 13:59:44,548 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 13:59:44,559 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 13:59:44,601 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:00:14,645 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:00:14,646 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:00:14,653 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:00:14,689 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:00:44,735 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:00:44,736 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:00:44,746 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:00:44,785 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:01:14,849 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:01:14,849 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:01:14,857 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:01:14,894 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:01:44,974 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:01:44,974 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:01:44,985 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:01:45,022 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:02:15,121 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:02:15,121 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:02:15,130 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:02:15,181 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:02:45,276 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:02:45,278 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:02:45,295 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:02:45,357 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:03:15,438 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:03:15,438 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:03:15,446 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:03:15,499 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:03:45,565 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:03:45,565 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:03:45,573 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:03:45,633 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:04:15,700 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:04:15,701 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:04:15,710 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:04:15,758 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:04:45,845 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:04:45,845 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:04:45,853 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:04:45,905 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:05:15,983 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:05:15,983 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:05:15,992 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:05:16,040 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:05:46,118 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:05:46,118 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:05:46,127 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:05:46,177 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:06:16,249 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:06:16,250 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:06:16,258 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:06:16,310 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:06:46,392 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:06:46,392 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:06:46,401 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:06:46,445 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:07:16,517 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:07:16,518 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:07:16,527 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:07:16,623 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:07:46,721 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:07:46,721 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:07:46,732 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:07:46,772 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:08:16,846 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:08:16,847 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:08:16,858 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:08:16,909 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:08:46,991 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:08:46,992 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:08:47,000 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:08:47,065 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:09:17,149 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:09:17,149 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:09:17,159 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:09:17,214 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:09:47,290 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:09:47,290 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:09:47,298 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:09:47,334 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:10:17,397 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:10:17,397 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:10:17,406 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:10:17,441 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:10:47,501 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:10:47,502 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:10:47,510 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:10:47,554 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:11:17,623 INFO coreos_koji_tagger - Received koji.AuthError from koji. Re-attempting login.
2020-03-18 14:11:17,624 INFO coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-03-18 14:11:17,632 INFO coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-03-18 14:11:17,668 ERROR requests_kerberos.kerberos_ - handle_other(): Mutual authentication failed
2020-03-18 14:11:17,670 ERROR coreos_koji_tagger - Caught Exception!
2020-03-18 14:11:17,670 ERROR coreos_koji_tagger - ###################################
Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/tenacity/__init__.py", line 361, in call
result = fn(*args, **kwargs)
File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 465, in koji_login
self.koji_client.gssapi_login(principal, self.keytab_file)
File "/usr/lib/python3.7/site-packages/koji/__init__.py", line 2467, in gssapi_login
raise AuthError('unable to obtain a session')
koji.AuthError: unable to obtain a session
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 196, in wrapper
return func(*args, **kwargs)
File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 301, in process_message
self.koji_login()
File "/usr/lib/python3.7/site-packages/tenacity/__init__.py", line 292, in wrapped_f
return self.call(f, *args, **kw)
File "/usr/lib/python3.7/site-packages/tenacity/__init__.py", line 358, in call
do = self.iter(retry_state=retry_state)
File "/usr/lib/python3.7/site-packages/tenacity/__init__.py", line 332, in iter
six.raise_from(retry_exc, fut.exception())
File "<string>", line 3, in raise_from
tenacity.RetryError: RetryError[<Future at 0x7f58db1cc9d0 state=finished raised AuthError>]
jlebon commented
Hmm weird. So either Koji was actually down for that amount of time (30m), or the koji python bindings is maintaining some state that isn't quite getting flushed out. I guess one thing we could also try is simply to exit if retrying fails to give the app a chance to restart clean. Though it'd be good to get to the bottom of what actually is causing the re-auth failure.
dustymabe commented
Only time will tell. Closing this for now and will re-open if it needs more attention.
dustymabe commented
It appears the fix in #118 is continuing to work here (so no need to do anything, but just noting the workaround is still needed). Saw this today:
2022-10-10 13:42:13,106 INFO builtins - Skipping message from unrelated repo: linux-system-roles/mssql
2022-10-10 13:42:43,246 INFO builtins - Skipping message from unrelated repo: fedora-infra/bodhi
2022-10-10 13:43:04,272 INFO builtins - Skipping message from unrelated repo: fedora-infra/bodhi
2022-10-10 13:43:36,994 INFO builtins - Skipping message from unrelated repo: fedora-infra/bodhi
2022-10-10 13:46:19,222 INFO builtins - Skipping message from unrelated repo: fedora-infra/bodhi
2022-10-10 13:50:19,733 INFO builtins - Skipping message from unrelated repo: fedora-infra/bodhi
2022-10-10 13:50:40,574 INFO builtins - Skipping message from unrelated repo: fedora-infra/bodhi
2022-10-10 14:02:54,041 INFO builtins - Skipping message from unrelated repo: packit/specfile
2022-10-10 14:05:12,010 ERROR builtins - Received koji.AuthError from koji. Exiting so a new pod will get created.
2022-10-10 14:05:12,010 ERROR builtins - See https://github.com/coreos/fedora-coreos-releng-automation/issues/70
2022-10-10 14:05:12,011 ERROR fedora_messaging.twisted.consumer - Received unexpected exception from consumer Consumer(queue=amq.gen-PrA1JzZUwUzd5qeDzkVKIQ, callback=<Consumer object at 0x7f22c2c2a1d0>)
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/koji/__init__.py", line 2839, in _callMethod
return self._sendCall(handler, headers, request)
File "/usr/lib/python3.10/site-packages/koji/__init__.py", line 2757, in _sendCall
raise e
File "/usr/lib/python3.10/site-packages/koji/__init__.py", line 2753, in _sendCall
return self._sendOneCall(handler, headers, request)
File "/usr/lib/python3.10/site-packages/koji/__init__.py", line 2803, in _sendOneCall
ret = self._read_xmlrpc_response(r)
File "/usr/lib/python3.10/site-packages/koji/__init__.py", line 2815, in _read_xmlrpc_response
result = u.close()
File "/usr/lib64/python3.10/xmlrpc/client.py", line 668, in close
raise Fault(**self._stack[0])
xmlrpc.client.Fault: <Fault 1002: 'Invalid session or bad credentials'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/fedora_messaging/twisted/consumer.py", line 207, in _read_one
yield threads.deferToThread(self.callback, message)
File "/usr/lib/python3.10/site-packages/twisted/python/threadpool.py", line 244, in inContext
result = inContext.theWork() # type: ignore[attr-defined]
File "/usr/lib/python3.10/site-packages/twisted/python/threadpool.py", line 260, in <lambda>
inContext.theWork = lambda: context.call( # type: ignore[attr-defined]
File "/usr/lib/python3.10/site-packages/twisted/python/context.py", line 117, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/lib/python3.10/site-packages/twisted/python/context.py", line 82, in callWithContext
return func(*args, **kw)
File "/work/coreos_koji_tagger.py", line 276, in __call__
self.process_message(message)
File "/work/coreos_koji_tagger.py", line 197, in wrapper
return func(*args, **kwargs)
File "/work/coreos_koji_tagger.py", line 310, in process_message
self.koji_client.getLoggedInUser()
File "/usr/lib/python3.10/site-packages/koji/__init__.py", line 2360, in __call__
return self.__func(self.__name, args, opts)
File "/usr/lib/python3.10/site-packages/koji/__init__.py", line 2858, in _callMethod
raise err
koji.AuthError: Invalid session or bad credentials
2022-10-10 14:05:12,014 ERROR fedora_messaging.cli - Unexpected error occurred in consumer Consumer(queue=amq.gen-PrA1JzZUwUzd5qeDzkVKIQ, callback=<Consumer object at 0x7f22c2c2a1d0>): <twisted.python.failure.Failure koji.AuthError: Invalid session or bad credentials>