RedHatInsights/bonfire

Some tests look like are broken but they still pass

Opened this issue · 2 comments

For example, here's an excerpt of a couple of test outputs:

tests/test_bonfire.py::test_ns_reserve_flag_timeout[user-7-ns-7-700] ERROR: namespace reserve: hit unexpected error: 

  RAN: /usr/local/bin/oc api-resources --verbs=list
  STDOUT:

  STDERR:
error: Missing or incomplete configuration info.  Please point to an existing, complete config file:

  1. Via the command-line flag --kubeconfig
  2. Via the KUBECONFIG environment variable
  3. In your home directory as ~/.kube/config

To view or setup config directly use the 'config' command.

PASSED

And

tests/test_bonfire.py::test_ns_reserve_flag_name[ns-7] ERROR: namespace reserve: hit unexpected error: Object of type MagicMock is not JSON serializable
PASSED

These errors can be spotted in the "Unit tests" step of the pipeline job.

The first error is probably caused by some function calling ocviapy's 'get_api_resources' function -- whatever is calling it would need to be mocked.

Unclear on where exactly the second error is occurring ... probably need to temporarily edit the code handling that exception (where 'hit unexpected error:' is being printed) to make it print out the traceback

It also just occurred to me, we can probably just mock or monkeypatch the whole 'get_api_resources' function entirely -- we can just look at that function's output on a real openshift cluster and make the mock return that.