Support multiple evaluation tasks
Opened this issue · 0 comments
ahumenberger commented
As of now, we only have a single evaluation task, that is, "provide tests for this code". In a future release we want to run other tasks as well, like "repair this compilation error". For this, we need structural changes.
Tasks:
- Introduce string enum
TaskIdentifier
- Change
Model
interface such that it has aRunTask(TaskIdentifier,...)
method- Remove
GenerateTestsForFile
fromModel
interface as this becomes a task
- Remove
- Add a method
IsTaskSupported(TaskIdentifier)
to theModel
interface to let models specify if they support a specific task. - Introduce proper
Repository
type- Each repository can allow/disallow certain tasks
- This should not be hardcoded. Therefore a
repository.json
file can be created in the repository's root directory. If no such file exists, all tasks are allowed.
- This should not be hardcoded. Therefore a
- Each repository can allow/disallow certain tasks
- Create
Task
interface and suitable implementations-
TaskPostRunTests
which callsModel.RunTask
and then runs the repository tests to validate the result
-