Improve test coverage
Opened this issue · 4 comments
Add tests for any parts that are nontrivial and/or can be tested with reasonable effort.
- Quasiquote system
- Corner cases
- Multiple macro invocations in the same
with - Decorator macro that edits the decorator list
- Multiple macro invocations in the same
May need a small utility to clear .pyc caches, to make the test reproduceable.
Too bad we can't use unpythonic.test.fixtures to test this.
We could also run the demos as part of the tests, to check that mcpyrate doesn't crash on them - so that we can guarantee them to be compatible with the mcpyrate version being tested.
However, we also need more real unit tests for the mcpyrate codebase itself, in order for the test run of the demos not to exaggerate its coverage. Tests should do more than just assert that there are no crashing statements ;)
Also, at the very least, we need a test runner script to run all demos and see if any of them crashed.
As of 9bf1461 (release 3.3.0 and later), runtests.py runs both tests and demos.
As of 42d93d4 (release 3.2.0 and later), macropython -c . clears .pyc caches in the current directory. The test runner calls this functionality programmatically, and runs everything twice - once from a clean slate, and once with existing bytecode caches.
Both tests and demos are discovered automatically, so new ones are easy to add. See CONTRIBUTING.md (and runtests.py itself) for details.
I'm leaving the issue open for now, because as of release 3.5.0, we still need a lot more tests. Porting our sister proejct unpythonic to use mcpyrate has already lead to discovering (and fixing!) some bugs in mcpyrate that would have otherwise gone unnoticed.
Now that we have a proper CI process (see #5), what remains is to bump up the test coverage significantly. Issue title changed.