This repository includes lab materials to support the Spring Semester of the COM3523/6523 module "Software Reengineering" at the University of Sheffield.
In this week, we explore testing for reengineering.
This repository contains skeleton classes for checking safety violations of an Automated Driving System (ADS) by running a driving scenario using a simulator. The ADS under analysis and the simulator are external (i.e., not included in this code base). In other words, this code base is supposed to work with external ADSs and simulators.
Since our aim is to learn "testing for reengineering", you can only modify test cases (i.e., files under src/test).
You must not modify the source code under src/main to solve the problems below.
The only exception is Scenario.java which should be modified in Part 3.
By default, a CI/CD pipeline is set up to automatically check the following commands (i.e., the commands that must pass when you complete all the tasks below) when your commits are pushed to the main branch:
mvn test -Dtest=TestSimulator#testRun2mvn test -Dtest=TestADS#testPrintPathmvn test -Dtest=TestChecker#testCheckCollisionViolationsmvn test -Dtest=TestScenario#testEquality
This part is to set up your own repository and understand the code in the repository.
First, clone this repository to your local machine (e.g., the user drive in the University computer; u:/Teaching/COM3523/2022-23 in my case). Open Git Bash and enter the following commands:
cd u:/Teaching/COM3523/2022-23
git clone [YOUR_REPOSITORY_ADDRESS]Now, take a look at the code you just cloned. If you want, open the project in IntelliJ and browse the files in it. For those who wants to install IntelliJ 2022 on your shared drive (/U:), please refer to this guideline.
Try the following command in a terminal (Git Bash) to see the status of the testing results:
mvn testIt will return several errors because there are broken test cases you must update during this lab session.
This part is about test automation. You already learned CI/CD basics in Week 6, so this will be easy.
Open TestSimulator.java.
You can see that an example test case is written for you:
@Test
package example.project;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.fail;
import example.project.domain.Scenario;
import org.junit.jupiter.api.Test;
public class TestSimulator {
@Test
public void testRun() {
Simulator simulator = new Simulator("dummy");
Scenario scenario = new Scenario();
assertEquals(null, simulator.run(scenario));
}
}In TestSimulator.java, please fix testRun2() following the instructions in the code.
In TestADS.java, please fix testPrintPath() following the instructions in the code.
This is about using mocks in testing for reengineering.
In TestADS.java, we have a test case testPredict():
package example.project;
import static org.junit.jupiter.api.Assertions.assertEquals;
import example.project.domain.Scenario;
import org.junit.jupiter.api.Test;
import java.util.Arrays;
import java.util.List;
public class TestADS {
@Test
public void testPredict() {
ADS ads = new ADS("dummy");
Scenario scenario = new Scenario("special scenario leading to the prediction of [0, 0]");
List<Object> prediction = ads.predict(scenario); // the prediction will be null since the ADS is dummy.
assertEquals(Arrays.asList(0, 0), prediction);
}
}If you run this test case using the following command:
mvn test -Dtest=TestADS#testPredictYou will see a failure as follows:
...
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running example.project.TestADS
WARNING: ADS.predict() is not yet implemented.
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.029 s <<< FAILURE! - in example.project.TestADS
[ERROR] example.project.TestADS.testPredict Time elapsed: 0.018 s <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <[0, 0]> but was: <null>
...This is because, as noted above in the comment,
ads.predict() is dummy (in the source code,
you can see that the implementation of predict() returns null).
If a working ADS was given, we could have updated predict() to use the working ADS.
However, in many cases, the legacy software to reengineer might have external dependendies,
such as an ADS in this example, that are unavailable or difficult to get.
Then should we just give up testing relevant code? No, we can use mocks!
Instead of using an incomplete/external ADS, we can use a mock as follows:
ADS ads = mock(); // instead of ADS ads = new ADS("dummy");This allows us to modify the behaviour of ads.predict() as we want.
Based on this, we can fix the above testPredict() as follows:
package example.project;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.fail;
import static org.mockito.Mockito.*;
import example.project.domain.Scenario;
import org.junit.jupiter.api.Test;
import java.util.Arrays;
import java.util.List;
public class TestADS {
@Test
public void testPredict() {
ADS ads = mock();
Scenario scenario = new Scenario("special scenario leading to the prediction of [0, 0]");
when(ads.predict(scenario)).thenReturn(Arrays.asList(0, 0));
List<Object> prediction = ads.predict(scenario);
assertEquals(Arrays.asList(0, 0), prediction);
verify(ads, times(1)).predict(scenario); // verify if ads.predict(scenario) has been called only once
verify(ads, times(0)).printPath(); // verify if ads.printPath() has been never called
}
}Please update your test code using the above, so it can pass, and check if it really passes by running the same command:
mvn test -Dtest=TestADS#testPredictIn TestChecker.java, we have a failing test case testCheckCollisionViolations(). The reason is basically the same as testPredict() above. Now it's your turn to fix this testCheckCollisionViolations() using mocks.
HINT: Investigate why checker.checkCollisionViolations(scenario) internally raises a null pointer exception.
It's mainly because simulator.run(scenario) in Checker.java.
Now, which object should be mocked?
- Mockito framework
- API documentation for
verify()
This is about testing interface instead of implementation.
In TestSnapshot.java, please try to understand how testEquality() works.
@Test
public void testEquality()
{
assertEquals(s1.roadType, s2.roadType);
assertEquals(s1.weatherCondition, s2.weatherCondition);
assertTrue(Arrays.equals(s1.egoCarPos.toArray(), s2.egoCarPos.toArray()));
assertTrue(Arrays.equals(s1.carInFrontPos.toArray(), s2.carInFrontPos.toArray()));
}You can see that it has four assertions to check the equality between s1 and s2.
This is not good for maintainability; for example, what happens if we update Snapshot.java to have more attributes than the existing ones?
We must update the test case additionally whenever Snapshot.java is updated.
To keep the test case the same while updating the implementation details, we can improve Snapshot.java by adding the following lines:
@Override
public boolean equals(Object obj) {
if (obj == null)
return false;
Snapshot snapshot = (Snapshot) obj;
return roadType.equals(snapshot.roadType)
&& weatherCondition.equals(snapshot.weatherCondition)
&& Arrays.equals(egoCarPos.toArray(), snapshot.egoCarPos.toArray())
&& Arrays.equals(carInFrontPos.toArray(), snapshot.carInFrontPos.toArray());
}Adding the above lines to Snapshot.java will allow us to simplify the test case as follows:
@Test
public void testEqualitySimple()
{
assertEquals(s1, s2);
}Now, it's your turn to fix testEquality() in TestScenario.java according to the instructions provided in the code.
HINT: Override equals() in Scenario.java.
- JDK 11 documentation for
Object.equals(). - Overriding methods