minvws/nl-kat-coordination

Object reference not found in Octopoes

Closed this issue · 4 comments

Describe the bug
When scanning mispo.es I get an object reference not found error in the boefje logs for the finding 'KAT-No-certificate'.

To Reproduce
Steps to reproduce the behavior:

  1. Go through onboarding for mispo.es.
  2. Enable nmap tcp + the SSLCertificates
  3. Wait
  4. See error

Expected behavior
No error messages in the logs when boefjes are enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

OpenKAT version
main

Additional context

boefje-1  | {"event": "Set status to TaskStatus.COMPLETED in the scheduler for task[id=e44582f0-8a77-43ec-986b-e087e1b4b093]", "level": "info", "timestamp": "2024-09-30T11:40:59.953143"}
boefje-1  | HTTP Request: PATCH http://scheduler:8000/tasks/eba7db28-2f4c-4815-a0f5-a7d234c7a28d "HTTP/1.1 200 OK"
boefje-1  | {"event": "Handling boefje kat-finding-types[task_id=eba7db28-2f4c-4815-a0f5-a7d234c7a28d]", "level": "info", "timestamp": "2024-09-30T11:40:59.961040"}
boefje-1  | HTTP Request: GET http://octopoes_api/aa/object?reference=KATFindingType%7CKAT-NO-CERTIFICATE&valid_time=2024-09-30%2011%3A41%3A00.090250%2B00%3A00 "HTTP/1.1 404 Not Found"
boefje-1  | {"exc_info": true, "event": "An error occurred handling scheduler item[id=eba7db28-2f4c-4815-a0f5-a7d234c7a28d]", "level": "error", "timestamp": "2024-09-30T11:41:00.099727"}
boefje-1  | Traceback (most recent call last):
boefje-1  |   File "/app/boefjes/octopoes/connector/octopoes.py", line 51, in _verify_response
boefje-1  |     response.raise_for_status()
boefje-1  |   File "/usr/local/lib/python3.11/site-packages/httpx/_models.py", line 763, in raise_for_status
boefje-1  |     raise HTTPStatusError(message, request=request, response=self)
boefje-1  | httpx.HTTPStatusError: Client error '404 Not Found' for url 'http://octopoes_api/aa/object?reference=KATFindingType%7CKAT-NO-CERTIFICATE&valid_time=2024-09-30%2011%3A41%3A00.090250%2B00%3A00'
boefje-1  | For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404
boefje-1  | 
boefje-1  | During handling of the above exception, another exception occurred:
boefje-1  | 
boefje-1  | Traceback (most recent call last):
boefje-1  |   File "/app/boefjes/boefjes/job_handler.py", line 111, in handle
boefje-1  |     ooi = get_octopoes_api_connector(boefje_meta.organization).get(
boefje-1  |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
boefje-1  |   File "/app/boefjes/octopoes/connector/octopoes.py", line 97, in get
boefje-1  |     res = self.session.get(
boefje-1  |           ^^^^^^^^^^^^^^^^^
boefje-1  |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1066, in get
boefje-1  |     return self.request(
boefje-1  |            ^^^^^^^^^^^^^
boefje-1  |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 837, in request
boefje-1  |     return self.send(request, auth=auth, follow_redirects=follow_redirects)
boefje-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
boefje-1  |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 926, in send
boefje-1  |     response = self._send_handling_auth(
boefje-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^
boefje-1  |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 954, in _send_handling_auth
boefje-1  |     response = self._send_handling_redirects(
boefje-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
boefje-1  |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1011, in _send_handling_redirects
boefje-1  |     raise exc
boefje-1  |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 994, in _send_handling_redirects
boefje-1  |     hook(response)
boefje-1  |   File "/app/boefjes/octopoes/connector/octopoes.py", line 55, in _verify_response
boefje-1  |     raise ObjectNotFoundException(data["detail"])
boefje-1  | octopoes.models.exception.ObjectNotFoundException: KATFindingType|KAT-NO-CERTIFICATE
boefje-1  | 
boefje-1  | The above exception was the direct cause of the following exception:
boefje-1  | 
boefje-1  | Traceback (most recent call last):
boefje-1  |   File "/app/boefjes/boefjes/app.py", line 251, in _start_working
boefje-1  |     handler.handle(p_item.data)
boefje-1  |   File "/app/boefjes/boefjes/job_handler.py", line 115, in handle
boefje-1  |     raise ObjectNotFoundException(f"Object {reference} not found in Octopoes") from e
boefje-1  | octopoes.models.exception.ObjectNotFoundException: Object KATFindingType|KAT-NO-CERTIFICATE not found in Octopoes
boefje-1  | HTTP Request: GET http://scheduler:8000/tasks/eba7db28-2f4c-4815-a0f5-a7d234c7a28d "HTTP/1.1 200 OK"

Very likely related related to #3585 (or one of its former attempt at resolutions)

As far as I understand the situation is:

  1. Boefje is scheduled
  2. The input OOI is deleted
  3. Boefje is picked up by the boefje runner, but it can't find the input OOI anymore.

I think that we can fix this by just logging an info level message that the input OOI doesn't exist anymore instead of raising an exception. I think this is something that will just happen and is hard to avoid, it also isn't a big problem if we just don't run the boefje if it happens.

Probably good to check whether it's an affirmation before not throwing.

Proposing #3686