mrigankpawagi/HinglishEval

Issue in 32 GroundTruth ( no doctest of form assert f(x) == y )

Closed this issue · 2 comments

image

Since no easily extractable doctests - hence this is ignored from the data set

def check(candidate):
    import math
    import random
    rng = random.Random(42)
    import copy
    for _ in range(100):
        ncoeff = 2 * rng.randint(1, 4)
        coeffs = []
        for _ in range(ncoeff):
            coeff = rng.randint(-10, 10)
            if coeff == 0:
                coeff = 1
            coeffs.append(coeff)
        solution = candidate(copy.deepcopy(coeffs))
        # assert math.fabs(poly(coeffs, solution)) < 1e-4


check(find_zero)

Can you check the test-cases in HumanEval's JSON file. This groundtruth has some more issues if I am not wrong (incomplete contracts)

Closing as we changed our approach to semi-automatic translation.