Nondeterministic Tests in MappingRedisConverterUnitTests

[EdwinIngJ]

Description

I noticed some nondeterminism in the following tests in MappingRedisConverterUnitTests.java:

• org.springframework.data.redis.core.convert.MappingRedisConverterUnitTests.writeShouldWritePartialUpdatePathWithRegisteredCustomConversionCorrectly
• org.springframework.data.redis.core.convert.MappingRedisConverterUnitTests.writeShouldHonorCustomConversionOnNestedType

The tests above may non-deterministically pass or fail without any changes made to the source code when ran on different JVMs. The problem arises from the fact that JSON properties in the JSON string returned in assertThat(...).containsEntry(...) are not guaranteed to have a defined order but the tests assume a specific order.

Steps to Reproduce

Here, I show the steps to reproduce the failure in org.springframework.data.redis.core.convert.MappingRedisConverterUnitTests.writeShouldWritePartialUpdatePathWithRegisteredCustomConversionCorrectly.

Run the following commands:

# Compile the module where the test is, including all its dependency
mvn clean install -pl . -am -DskipTests

# Run writeShouldWritePartialUpdatePathWithRegisteredCustomConversionCorrectly test using NonDex
mvn -pl . edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest=org.springframework.data.redis.core.convert.MappingRedisConverterUnitTests#writeShouldWritePartialUpdatePathWithRegisteredCustomConversionCorrectly -DnondexRuns=5

Running the commands above outputted in the following results:

[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   MappingRedisConverterUnitTests.writeShouldWritePartialUpdatePathWithRegisteredCustomConversionCorrectly:1778                                                                            
Expecting map:                                                                                   
  {"address"="{"country":"Tel'aran'rhiod","city":"unknown"}"}                                    
to contain entries:                                                                              
  ["address"="{"city":"unknown","country":"Tel'aran'rhiod"}"]                                    
but the following map entries had different values:                                              
  ["address"="{"country":"Tel'aran'rhiod","city":"unknown"}" (expected: "{"city":"unknown","country":"Tel'aran'rhiod"}")]                                                                         
[INFO] 
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

Reason for Failure: As shown above, the assertion in the test failed because the ordering of the JSON properties in the JSON string changed but the assertion assumed a specific predefined ordering for the JSON string.

Here, I show the steps to reproduce the failure in org.springframework.data.redis.core.convert.MappingRedisConverterUnitTests.writeShouldHonorCustomConversionOnNestedType.

Run the following commands:

# Compile the module where the test is, including all its dependency
mvn clean install -pl . -am -DskipTests

# Run writeShouldHonorCustomConversionOnNestedType test using NonDex
mvn -pl . edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest=org.springframework.data.redis.core.convert.MappingRedisConverterUnitTests#writeShouldHonorCustomConversionOnNestedType

Running the commands above outputted in the following results:

[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   MappingRedisConverterUnitTests.writeShouldHonorCustomConversionOnNestedType:1083 
Expecting map:                                                                                   
  {"_class"="org.springframework.data.redis.core.convert.ConversionTestEntities$Person", "address"="{"country":"Tel'aran'rhiod","city":"unknown"}"}                                               
to contain entries:                                                                              
  ["address"="{"city":"unknown","country":"Tel'aran'rhiod"}"]                                    
but the following map entries had different values:                                              
  ["address"="{"country":"Tel'aran'rhiod","city":"unknown"}" (expected: "{"city":"unknown","country":"Tel'aran'rhiod"}")]                                                                         
[INFO] 
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

Reason for Failure: Like the previous test, the assertion in the test failed because the ordering of the JSON properties in the JSON string changed but the assertion assumed a specific predefined ordering for the JSON string.

Proposed Changes: A ObjectMapper can be used to parse the JSON string and expected JSON string, enabling the assertion to test for equality while being insensitive to property ordering.

[mp911de]

Thanks for reaching out. Ordering issues may happen due to the field order during runtime, it makes sense to address these issues but there's a simpler way than introducing ObjectMapper: Just asserting parts of the substring.