ome/bioformats

Consider more thorough comparison of image data in automated tests

Opened this issue · 0 comments

#4205 (review) highlights that existing automated tests do not thoroughly check for consistency of openBytes results across all planes. https://github.com/ome/bioformats/blob/develop/components/test-suite/src/loci/tests/testng/OpenBytesPerformanceTest.java was one previous attempt along similar lines, but doesn't compare all plane hashes between two commits as in the review for #4205.

One option would be to "just" expand the MD5s stored in configuration files to include all planes, and then update https://github.com/ome/bioformats/blob/develop/components/test-suite/src/loci/tests/testng/FormatReaderTest.java#L2292 and related tests to check all planes. That would require a substantial amount of configuration regeneration and will almost certainly have a significant impact on runtime of the tests, so requires some more thought.