canonical/notebook-operators

Integration tests failing intermittently in GH runners

Opened this issue · 0 comments

Bug Description

When executing integration tests bundle tests fail intermittently.

  • In one case failure was due to admission-webhook failure. See relevent logs secion for more details

Failed run:
https://github.com/canonical/notebook-operators/actions/runs/6221726919/job/16884245612

Successful run:
https://github.com/canonical/notebook-operators/actions/runs/6221726919

After re-run of on pull request CI integration test are passing.

To Reproduce

Trigger pull request workflow.

Environment

GH runners

Relevant log output

# admission-webhook failed to deploy
unit-admission-webhook-0: 11:45:47 ERROR unit.admission-webhook/0.juju-log Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-admission-webhook-0/charm/venv/charmed_kubeflow_chisme/pebble/_update_layer.py", line 31, in update_layer
    container.replan()
  File "/var/lib/juju/agents/unit-admission-webhook-0/charm/venv/ops/model.py", line 1915, in replan
    self._pebble.replan_services()
  File "/var/lib/juju/agents/unit-admission-webhook-0/charm/venv/ops/pebble.py", line 1680, in replan_services
    return self._services_action('replan', [], timeout, delay)
  File "/var/lib/juju/agents/unit-admission-webhook-0/charm/venv/ops/pebble.py", line 1761, in _services_action
    raise ChangeError(change.err, change)
ops.pebble.ChangeError: cannot perform the following tasks:
- Start service "admission-webhook" (cannot start service: exited quickly with code 255)
----- Logs from task 0 -----
2023-09-18T11:45:47Z INFO Most recent service output:
    F0918 11:45:47.428138      13 config.go:46] config=main.Config{CertFile:"/etc/webhook/certs/cert.pem", KeyFile:"/etc/webhook/certs/key.pem"} Error: open /etc/webhook/certs/cert.pem: no such file or directory
2023-09-18T11:45:47Z ERROR cannot start service: exited quickly with code 255
-----
unit-admission-webhook-0: 11:45:47 ERROR unit.admission-webhook/0.juju-log Failed to handle <ConfigChangedEvent via AdmissionWebhookCharm/on/config_changed[11]> with error: Failed to replan

Additional context

No response