Add an api coverage feature
anis-campos opened this issue · 3 comments
I feel like we could greatly benefit from a coverage tool. The idea is to:
- have a percentage of "covered" path, meaning that all possibles requests/responses were validated.
- have a percentage of "covered" components, meaning that the object was seen at least once in the responses.
The goal would be to give a good idea of how much of the API is tested and therefore make it possible to identify and address weak spots.
How would you implement this?
I'm not sure. We need some pytest-cov equivalent of https://github.com/viclovsky/swagger-coverage, but now that I'm thinking about it, it might be completely unrelated to the schema validation done here.
The only extra information coming from this tester is the validity of the response, whereas coverage is simply a matter of identifying not tested code.
Yeah this sounds like a cool idea, but maybe one that would be better implemented as it's own complementary or standalone package. There would be very little overlap with the existing package internals here, unfortunately.
If you want to experiment and open a PR, I'm not necessarily opposed to a feature like this, depending on how it turns out - but I suspect you would need to essentially write the equivalent of a standalone package, inside drf-openapi-tester, which sounds suboptimal.