manytask/checker

Separate tester and course logic

kalabukdima opened this issue · 0 comments

Right now course tester is highly dependent on the course repository structure and config files (e.g. CMakeLists). It's very hard to make your own course repo without copying a working example, and there are no examples provided in this repo.

I think everything would be much simpler if the testing script would be defined in the course repo and the checker would just run a specified command to check the solution. The MakeTester aims to provide that option but it is not flexible enough because it doesn't allow access to repo files outside the current task directory.

I propose to add a new tester that just reads the command from the .tester.json file and runs that command to test the task and get the score. That would probably require #34 to be closed first.