symflower/eval-dev-quality

Code repair should only consider #(passing tests) and never coverage

Closed this issue · 0 comments

For the transpilation task we changed it such that the coverage metric does not contribute to scoring, but instead the number of passing test from the test suite do.

The same thing should have happened for the code repair task. Please do the same thing there. No coverage, just #(passing tests).