Poor collection performance due to expensive `iscoroutinefunction` check
etripier opened this issue · 1 comments
Hello pytest-asyncio developers! We've been using your plugin for a while. So much so that we now have tens of thousands of tests and hundreds of thousands of fixture arguments and ... it takes over three minutes to collect our tests.
Of that time, roughly two and a half minutes is taken up by pytest-asyncio. Specifically, this piece of code:
pytest-asyncio/pytest_asyncio/plugin.py
Line 213 in edb9ae0
That ends up calling https://github.com/python/cpython/blob/3aea6c4823e90172c9bc36cd20dc51b295d8a3c4/Lib/inspect.py#L384, which unwraps each fixture argument, makes sure it's a function, and then finally checks its co_flags to see if it's been marked as a coroutine function. The thing is, because you're feeding fixtures through, they tend to all be functions.
We ended up forking the plugin and inlining iscoroutinefunction and isasyncgenfunction - just the bit masking lines. Then we removed the code that was attaching the scoped_event_loop fixture because we don't use it and it was compounding the issue described above. That took us down to one minute.
I think it would help a lot to make the same changes here upstream. You could also swap the ordering of the comparison with
pytest-asyncio/pytest_asyncio/plugin.py
Line 217 in edb9ae0
scoped_event_loop optional would bring your plugin even closer to baseline performance.Thanks for the investigation! I suspected that pytest-asyncio will have to check its performance someday.
There's currently no setup to do any performance testing, though. I promise to look into it and I'm all for providing the functionality (and quality) so you don't have to maintain a custom fork, but I cannot give any time frame. The v0.23 release is plagued with functional issues, which are of higher priority at the moment.
Switching around the if statement as you suggested can be done immediately, though.