Cornice @view decorator creates a reference cycle

[romuald]

When debugging an issue with our application, we found out that cornice creates a reference cycle with the request object

In cornice/service.py:

request.cornice_args = (args, ob)

ob here is (can be?) an application object that was initialized with the request object

So request is referencing an application object which is referencing the request, which creates a reference cycle.

Those are not inherently bad but this makes it so the request will not be cleaned up at the end of its life cycle, and some related resources will also not be freed up until then

A practical example is a large JSON body sent to the webserver: webob will create a temporary file, that will only be deleted on the next garbage collection (that will raise a ResourceWarning, but only during tests)

Here is a minimal example exposing the problem, disabling the gc will lead to a ever-increasing number of requests

import gc
from wsgiref.simple_server import make_server

from cornice.resource import resource, view
from pyramid.config import Configurator
from pyramid.request import Request


@resource(collection_path="/submit", path="/submit/{id}")
class HelloWorldResource:
    def __init__(self, request, context=None):
        self.request = request
        self.context = context

    @view()
    def collection_post(self):
        # find number of in-memory pyramid requests
        count = sum(1 for x in gc.get_objects() if isinstance(x, Request))
        return {"message": f"I have {count} request(s) in memory."}


def main():
    with Configurator() as config:
        config.include("cornice")
        config.scan()
        app = config.make_wsgi_app()
        return app


if __name__ == "__main__":
    app = main()
    server = make_server("0.0.0.0", 6543, app)
    print("Server running on http://localhost:6543")

    gc.collect()

    # disable gc for demonstration
    gc.disable()
    server.serve_forever()

In our case we'll be fixing the issue by using

class XXXResource:
    def __init__(self, request, context=None):
        self.request = weakref.proxy(request)

The issue could be fixed by using params=dict(request=weakref.proxy(request)) in the original source, but it may have side effect I'm not foreseeing (this does not break the tests however)