Matlab/xUnit unit test framework clone
apjanke opened this issue · 6 comments
Maybe this package should implement a clone of Matlab's xUnit-based unit test framework after all?
If you're a potential user and interested in having this, add a comment on this issue to indicate your interest.
+1
@apjanke Octave#56325 was recently filed to add the Matlab runtests function to Octave. I have zero familiarity with the Matlab unit test framework, but it seems like they may be asking for part of the framework, possibly without knowing what they are asking for. If you know more, maybe add some insight there?
Sure sounds like they are. I have commented on Savannah.
Okay, I think I'm going to try this. It'll be a kinda large effort – the Matlab Unit Testing framework is large-ish, like fifty classes – but this would be a distinctive feature that could help get users interested in adopting Testify.
Luckily, the Matlab Unit Testing API is very granular in its class/interface definitions, so it won't be hard to figure out what to do just from the public class documentation.
The hard part will be test discovery: Matlab uses labels like methods (Test) ... and properties (TestParameter) ... to indicate which methods are tests and which properties are parameterizers for tests. Octave won't choke on that syntax, but it doesn't seem to respect it in any way: the Test and TestParameters aren't reflected in the meta.class objects for class definitions. So we'll instead need to rely on some combination of naming conventions (for the defined tests) and parsing the test class definition files ourselves to discover the tests. I think this is doable, and doable in a way that you can write Test class definition files that are compatible with both Matlab and Octave/Testify. Say, something like:
- A Test definition file is one that contains at least one
methods (Test)section - All test methods must be named
test<Xxx>, where the capitalization of the first letter ofXxxis significant. - All test parameter properties must be named
testParam<Xxx>orparam<Xxx>
Without some tough parsing, I don't think we'll be able to reliably detect which functions and properties are actually inside the special methods (Test) or properties (TestParameter) sections. Unless we make some additional assumptions about "correct" indentation etc of the source files.
Note to self: I talked to jwe in the #octave Freenode channel, and he says that Octave does not support classdef Events. (It parses them, but they're ignored and there's no functionality for them.) The Matlab Unit Testing API is partially defined in terms of classdef Events, so we won't be able to mimic it exactly.
No big deal. It won't be hard to rig up event-handling functionality using regular classes (just like you do in most programming languages). Just means that advanced functionality (like user-defined custom TestRunner plugins) won't be portable between Octave and Matlab. I think this is a minor issue.
The BIDS-Matlab folks are interested in maybe using this. Bumping up the priority on it.