Proposing a Code Coverage approach for PGTap

[dvasdekis]

Hi Theory! Thanks for maintaining a great repo.

I'm using PGTap an increasing amount, and its gotten me inspired to try having a rough go at developing a method to calculate the code coverage of a set of migrations against a given set of tests. My thoughts are, running from the Wikipedia definition of 'Function Coverage', that an initial set of tests would look for coverage of the below:

• SELECTS, INSERTS, UPDATES and DELETES on tables and writeable views
• SELECTS on non-writeable views
• EXECUTIONS on functions and procedures

Any objects created by the user should see the above behaviour being tested, except where permissions for non-Superuser users to do so is disallowed. I'm obviously missing some object types from the above list, but lets start small.

My challenge is that I think PGTap users use all sorts of functions in order to write tests (I use results_eq() almost exclusively myself), and I'm not sure how I'd, for example extract the prepared statements run as part of a plan out of the database at the end of a set of tests. Do you have any thoughts as to how I could approach this?

Secondly, if I was to contribute a code coverage approach that worked, what's your personal appetite to have that functionality in the main PGTap repo? I imagine that it would add (hopefully only a small amount) to your maintenance workload for future releases.

Thanks again!!

[theory]

I don't have any thoughts on such an approach; I suspect you'd ned to look at some internal information maintained by the database, or else a log of all queries. As for distinguishing from calls to the pgTAP functions themselves, it would work best if you put them into their own schema without anything else and then excluded anything from that schema. I think the best place to start would be pgsql-hackers.

If you do develop something useful, I think it'd be ideal if it could be distributed separately somehow. Only if you have to get into the guts of pgTAP itself to make it work would it make sense to include in pgTAP. But I'm not the only maintainer anymore, and suspect your work would be super interesting to integrate with in any event.

Gonna close this issue, since it was mostly a question. Please re-open if you run into an issue with pgTAP, or submit a PR if you need changes to pgTAP itself.

[wolfgangwalther]

@dvasdekis You might want to have a look at https://github.com/okbob/plpgsql_check#coverage-metrics. plpgsql_check provides basic statement and branch coverage tools for plpgsql functions.

[theory]

Thank you @wolfgangwalther, I couldn't find that the other day when I looked!