nuprl/MultiPL-E

C# test sequence equality

PootieT opened this issue · 1 comments

Issue 1: comparing lists

example program: HumanEval_9_rolling_max

// current comparison, returns False
Debug.Assert((new List<long>(new long[]{(long)1L, (long)2L, (long)3L, (long)4L})).Equals((new List<long>(new long[]{(long)1L, (long)2L, (long)3L, (long)4L}))));
// proposed comparison, returns True
Debug.Assert((new List<long>(new long[]{(long)1L, (long)2L, (long)3L, (long)4L})).SequenceEqual((new List<long>(new long[]{(long)1L, (long)2L, (long)3L, (long)4L}))));

should be changed to .SequenceEqual, as Equals checks for reference equality, where as the second checks for equality element-wise (source)

Issue 2: comparing dictionaries

example program HumanEval_111_histogram

Dictionary<string,long> dic1 = new Dictionary<string,long>(){{"a", 2L}, {"b", 2L}};
Dictionary<string,long> dic2 = new Dictionary<string,long>(){{"a", 2L}, {"b", 2L}};
// current comparison, returns False
Console.WriteLine((dic1).Equals(dic2));
// proposed comparison, returns True
Console.WriteLine(dic1.Count == dic2.Count && !dic1.Except(dic2).Any());

source

Proposed change

currently in humaneval_to_cs.py, we have:

    def deep_equality(self, left: str, right: str) -> str:
        """
        All tests are assertions that compare deep equality between left and right.
        Use ==  for primitive types and Equals for objects
        """
        #Empty the union declarations
        self.union_decls = {}
        if self.is_primitive_type(self.translated_return_type):
            return f"    Debug.Assert({left} == {right});"
        else:
            return f"    Debug.Assert({left}.Equals({right}));"

instead, we can change to this:

    def deep_equality(self, left: str, right: str) -> str:
        """
        All tests are assertions that compare deep equality between left and right.
        Use ==  for primitive types and Equals for objects
        """
        #Empty the union declarations
        self.union_decls = {}
        if self.is_primitive_type(self.translated_return_type):
            return f"    Debug.Assert({left} == {right});"
        elif self.list_type in self.translated_return_type:
            return f"    Debug.Assert({left}.SequenceEqual({right}));"
        elif self.dict_type in self.translated_return_type:
            return f"    Debug.Assert({left}.Count == {right}.Count && !{left}.Except({right}).Any()));"
        else:
            return f"    Debug.Assert({left}.Equals({right}));"

@abhijangda any opinions?