eliminate ordering
Closed this issue · 3 comments
Generally, test suites avoid constructs that require ordering because they can lead to unreliable reports if the order is short-circuited. I think we need to remove all support for ordering (the priority feature) and replace it with reliable setup/fixtures.
I'm not really married to the idea and am happy to remove it but ordering features wasn't intended as a way of fulfilling dependencies. It made sense to me, for instance, that the core protocol tests would run before secondary or non-core protocols even though each test is completely independent from the others.
One of my secondary goals for the test suite was displaying the flow of messages, if you choose to, as a way to get familiar with what is expected behavior for a protocol. Ordering the protocols makes more sense with this goal in mind and less when all you want out of it is the report at the end.