eli-b/pytest-memory-usage

support pypy - add gc.collect()

Opened this issue · 3 comments

Thanks for this code

PyPy requires forcing a collection before measuring memory usage after a test. For me it was enough to modify the runtest code:

is_pypy = '__pypy__' in sys.builtin_module_names

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_call(item):
    if configuration.getoption('memory_usage') or configuration.getini('memory_usage'):
        if not is_pypy:
            gc.disable()
        state.before_call = get_memory(state.process)
        yield
        if is_pypy:
            gc.collect();
        state.after_call = get_memory(state.process)
        if not is_pypy:
            gc.enable()
    else:
        yield

would you rather get this as a pull request?

eli-b commented

Hi Matti! Thanks for your interest.

I'll be glad to work this into the code.

Could you explain why PyPy requires forcing a collection before measuring memory usage? I'm curious.

First CPython: its refcount garbage collection is triggered on exiting a frame, in a predictable and consistent way
Now PyPy: it uses a modified mark-and-sweep strategy, triggered by the amount of spare room in a memory "nursery" available for new allocations. In many situations, the nursery is large enough to run a complete program without ever triggering a garbage collection, especially in small bits of code like tests

So if you want to clean out all the old objects, on CPython it is sufficient to exit a function, but on PyPy you need to explicitly ask for a GC collection cycle

More about this can be found on the PyPy doc site http://pypy.readthedocs.io/en/latest/gc_info.html or on the RPython doc site http://rpython.readthedocs.io/en/latest/garbage_collection.html

eli-b commented

Wow! Thanks for the thorough answer.