Splitting spec files still throws schema and file not found errors (json_schema exceptions) but still works with Connexion <= 3.0.2

Question

Splitting spec files still throws schema and file not found errors (json_schema exceptions) but still works with Connexion <= 3.0.2

eharvey71 opened this issue 9 months ago · 2 comments

I have my yml files divided into subdirectories for paths, parameters, schemas, etc. This worked fine (and still works fine) on Connexion ~~3.0.4~~ 3.0.2.
Connexion ~~3.0.5 and 3.0.6~~ 3.0.3 - 3.0.6 are still throwing errors. The prior fixes for $refs don't seem to have fixed issues. I thought this was just a Windows filesystem or jsonschema issue but it's happening all over - in my MacOS, Windows and docker container deployments running Debian.

If I downgrade to Connexion ~~3.0.4~~ 3.0.2 or prior, the error goes away, in all cases.

I am working with a separate swagger UI deployment, so I can do customizations.

In my configuration, I'm initializing my app like this:

basedir = pathlib.Path(__file__).parent.resolve()
swagoptions = SwaggerUIOptions(swagger_ui = True, swagger_ui_template_dir = basedir / 'swagger-ui')
connex_app = FlaskApp(__name__, specification_dir=basedir / "apispecs")

Paths are like this to specs, where my swagger.yml contains $refs to the proper paths of each of the divided spec files:

project/
      │── apispecs/
               |── swagger.yml
               |── parameters/
                         |── _index.html
               │── securitySchemas/
                         |── _index.html
               │── schemas/
                         |── _index.html

swagger.yml sample:

components:
  schemas:
    $ref: "./schemas/_index.yml"
  parameters:
    $ref: "./parameters/_index.yml"
  securitySchemes:
    $ref: "./security/_index.yml"

Full contents of error:

_RefResolutionError(_cause=FileNotFoundError(2, 'No such file or directory'))
Traceback (most recent call last):
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/connexion/json_schema.py", line 88, in _do_resolve
    retrieved = deep_get(spec, path)
                ^^^^^^^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/connexion/utils.py", line 112, in deep_get
    return deep_get(obj[keys[0]], keys[1:])
                    ~~~^^^^^^^^^
KeyError: 'schemas'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/jsonschema/validators.py", line 1102, in resolve_from_url
    document = self.store[url]
               ~~~~~~~~~~^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/jsonschema/_utils.py", line 20, in __getitem__
    return self.store[self.normalize(uri)]
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
KeyError: './schemas/_index.yml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/jsonschema/validators.py", line 1105, in resolve_from_url
    document = self.resolve_remote(url)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/jsonschema/validators.py", line 1202, in resolve_remote
    result = self.handlers[scheme](uri)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/connexion/json_schema.py", line 41, in __call__
    with open(filepath) as fh:
         ^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/schemas/_index.yml'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/connexion/middleware/swagger_ui.py", line 110, in _get_openapi_json
    content=jsonifier.dumps(self._spec_for_prefix(request)),
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/connexion/middleware/swagger_ui.py", line 73, in _spec_for_prefix
    return self.specification.with_base_path(base_path).raw
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/connexion/spec.py", line 209, in with_base_path
    new_spec = self.clone()
               ^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/connexion/spec.py", line 199, in clone
    return type(self)(copy.deepcopy(self._raw_spec))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/connexion/spec.py", line 83, in __init__
    self._spec = resolve_refs(raw_spec, base_uri=base_uri)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/connexion/json_schema.py", line 106, in resolve_refs
    res = _do_resolve(spec)
          ^^^^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/connexion/json_schema.py", line 100, in _do_resolve
    node[k] = _do_resolve(v)
              ^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/connexion/json_schema.py", line 100, in _do_resolve
    node[k] = _do_resolve(v)
              ^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/connexion/json_schema.py", line 96, in _do_resolve
    with resolver.resolving(node["$ref"]) as resolved:
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/jsonschema/validators.py", line 1044, in resolving
    url, resolved = self.resolve(ref)
                    ^^^^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/jsonschema/validators.py", line 1091, in resolve
    return url, self._remote_cache(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eharvey71/dev/integration-bridge/venv/lib/python3.12/site-packages/jsonschema/validators.py", line 1107, in resolve_from_url
    raise exceptions._RefResolutionError(exc) from exc
jsonschema.exceptions._RefResolutionError: [Errno 2] No such file or directory: '/schemas/_index.yml'
INFO:     127.0.0.1:55767 - "GET /api/openapi.json HTTP/1.1" 500 Internal Server Error

Output of the commands:

Python is 3.12

Answer 1 · 2024-04-08T22:59:12.000Z

Actually, when I test with 3.04 or 3.03, I have a different issue with specs being invalid:

Failed validating 'oneOf' in schema['properties']['paths']['patternProperties']['^\\/']['patternProperties']['^(get|put|post|delete|options|head|patch|trace)$']['properties']['requestBody']:
    {'oneOf': [{'$ref': '#/definitions/RequestBody'},
               {'$ref': '#/definitions/Reference'}]}

3.0.2 seems to be the most stable for my needs and works perfectly, but it would be great to run the latest and greatest.

Answer 2 · 2024-06-07T00:15:37.000Z

3.1 has fixed my issue. Thank you!