data-apis/dataframe-interchange-tests

Fold in existing tests?

honno opened this issue · 1 comments

honno commented

Efforts have already been made to test the df interchange protocol

Do we fold them in to this test suite?

  • With single-method tests, it depends how #2 goes. We see in array-api-tests that Hypothesis-powered tests per method end up covering a lot of manual testing anywho, in a much more maintainable/generalised way that also covers test cases we never think of.
  • With tests related to interesting interactions between methods, these won't be covered by #2 per-say:
    • A "complete" test per method covers basic interactions, e.g. you don't need to test whether you can get a column name and then access those columns, if you correctly test each step separately.
    • There are possibly interactions which aren't like that... a TODO is to identify what they are.

So a TODO for me is to properly review each of the listed test suites to see where inspiration can be taken from. Probably if #1 and/or #2 doesn't end up being that worth, just copying tests to have the coverage is nice, even if the test cases in dataframe-interchange-tests don't have clearly distinct test cases like I think array-api-tests does.

honno commented

Do we fold them in to this test suite?

So no test cases have been lifted as-is, but I think we've captured all the interesting things out there now in the test suite, so closing.