STAMP-project/dspot

Hangs on Warning while testing

Smorka opened this issue · 17 comments

Smorka commented

Characteristics

  • Issue Type: test report
  • Reproducibility: always
  • Severity: block
  • Tool/Service/Component: dspot 2.2.0 jar
  • Execution Environment: ubuntu virtualbox
  • Reporter: Sverre Morka, sverre.morka@tellu.no

Description

I am testing D-spot as part of the STAMP project. We have a library module with 77 tests, and I want to amplify them with d-spot.
I follow the documentatio on GitHub. I have compiled a dspot.properties file with the following content:

project=.
src=src/main/java/
testSrc=src/test/java
javaVersion=8
outputDirectory=dspot-out/
filter=no.tellu.*

then i Execute

$java -jar dspot-2.2.0-jar-with-dependencies.jar --path-to-properties dspot.properties

it seem to run the suite, but then it gives a warning and hangs:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  6.504 s
[INFO] Finished at: 2019-09-26T12:24:05Z
[INFO] ------------------------------------------------------------------------
[INFO] 2019-09-26 14:24:08 Amplification - Amplification of no.tellu.lib.data.BooleanOpTest (4 test(s))
[INFO] 2019-09-26 14:24:08 Amplification - Assertion amplification of no.tellu.lib.data.BooleanOpTest (4 test(s))
[INFO] 2019-09-26 14:24:08 EhcacheManager - Cache 'frameworkCache' created in EhcacheManager.
[INFO] 2019-09-26 14:24:08 MavenAutomaticBuilder - Using /home/smorka/development/tellulib/./.dspot_pom.xml to run maven.
[INFO] 2019-09-26 14:24:08 MavenAutomaticBuilder - Using /usr/share/maven/ for maven home
[INFO] 2019-09-26 14:24:08 MavenAutomaticBuilder - run maven: /usr/share/maven//bin/mvn org.pitest:pitest-maven:mutationCoverage 
[WARN] Maven will be executed in interactive mode, but no input stream has been configured for this MavenInvoker instance.

I've tried specifying a specific test as well, same result

$java -jar dspot-2.2.0-jar-with-dependencies.jar --path-to-properties dspot.properties --test no.tellu.lib.data.BooleanOpTest

Is there something I need to configure?

Steps to reproduce

Properties file
project=.
src=src/main/java/
testSrc=src/test/java
javaVersion=8
outputDirectory=dspot-out/
filter=no.tellu.*
Command Line / Options
$java -jar dspot-2.2.0-jar-with-dependencies.jar --path-to-properties dspot.properties 
$java -jar dspot-2.2.0-jar-with-dependencies.jar --path-to-properties dspot.properties --test no.tellu.lib.data.BooleanOpTest

Other files and URLs

Relationships

Help on issue template

Preview to follow the link or open file .github/ISSUE_DOC.md

danglotb commented

Hello @Smorka

According to the last line of your traces:

[INFO] 2019-09-26 14:24:08 MavenAutomaticBuilder - run maven: /usr/share/maven//bin/mvn org.pitest:pitest-maven:mutationCoverage

DSpot launched PIT to compute the mutation score. This is why it might be seen as stuck.

You can enable the verbose nodem --verbose to see the logs of PIT.

Smorka commented

This is the output I get when including the --verbose parameter

[INFO] Mutating from /home/smorka/development/tellulib/target/classes
8:58:13 AM PIT >> INFO : Verbose logging is disabled. If you encounter an problem please enable it before reporting an issue.
8:58:13 AM PIT >> INFO : MINION : Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.VerifyError: Expecting a stackmap frame at branch target 73
Exception Details:
  Location:
    org/pitest/coverage/execute/CoverageMinion.rec
8:58:13 AM PIT >> INFO : MINION : eiveTestClassesFromParent(Lorg/pitest/util/SafeDataInputStream;)Ljava/util/List; @50: if_icmpeq
  Reason:
    Expected stackmap frame at this location.
  Bytecode:
    0x0000000: 2ab6 01bb 3cb2 0074 bb00 7659 b700 7713
    0x0000010: 01bd b600 861b b600 81
8:58:13 AM PIT >> INFO : MINION : 13 01bf b600 86b6
    0x0000020: 008a b601 c2bb 01c4 591b b701 c64d 033e
    0x0000030: 1d1b 9f00 172c 2ab6 01c9 b801 cdb9 01d0
    0x0000040: 0200 5784 0301 a7ff eab2 0074 1301 d2b6
    0x0000050: 01c2 2cb0                              

	at java.lang.Cla
8:58:13 AM PIT >> INFO : MINION : ss.getDeclaredMethods0(Native Method)
	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
	at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
	at java.lang.Class.getMethod0(Class.java:3018)
	at java.lang.Class.getMethod(Class.java:178
8:58:13 AM PIT >> INFO : MINION : 4)
	at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)

Have you seen it before? Is there any configuration I can do to proceed?

danglotb commented

Hello @Smorka

Never seen this error before.

Investigating for a solution.

In the meantime, could you provide me the project on which you are executing DSpot?

Thank you!

danglotb commented

Could you give more details about your installation such as the output of:

mvn --version
java -version
Smorka commented 
$mvn --version
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=2048m; support was removed in 8.0
Listening for transport dt_socket at address: 5005
Apache Maven 3.6.0
Maven home: /usr/share/maven
Java version: 1.8.0_222, vendor: Private Build, runtime: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.15.0-60-generic", arch: "amd64", family: "unix"
Smorka commented

Can I send you the project code per mail?

  • 👍1
danglotb commented

Hello,

I could run then whole amplification process locally, meaning that the problem comes from your installation (I guess).

@oscarlvp Do you know this error?

8:58:13 AM PIT >> INFO : MINION : Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.VerifyError: Expecting a stackmap frame at branch target 73
Exception Details:
  Location:
    org/pitest/coverage/execute/CoverageMinion.rec

I looked on the internet but found nothing related to PIT, while on the traces it's clearly related to PIT>

danglotb commented

Did you manage to launch PIT/Descartes outside of DSpot, e.g. typing something like mvn org.pitest:pitest-maven:1.4.7:mutationCoverage ?

Thank you.

Smorka commented

I get the same error as versbose when doing the mvn org.pitest:pitest-maven:1.4.7:mutationCoverage

However, I'll try to install this on a windows machine and see if I get better progress.
Thank you for investigating 👍

danglotb commented

Hello, could you show me your JAVA_HOME? Thank you.

Smorka commented 
$echo $JAVA_HOME
/usr/lib/jvm/java-8-openjdk-amd64

After setting MAVEN_OPTS to "" I am able to run the mvn org.pitest:pitest-maven:1.4.7:mutationCoverage

$ echo $MAVEN_OPTS
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005 -XX:MaxPermSize=2048m -Xmx2048m -Duser.timezone=UTC -javaagent:/home/smorka/devEnvironment/grails-2.5.5/lib/org.springframework/springloaded/jars/springloaded-1.2.5.RELEASE.jar -Xverify:none
$echo MAVEN_OPTS=""
danglotb commented

After setting MAVEN_OPTS to "" I am able to run the mvn org.pitest:pitest-maven:1.4.7:mutationCoverage

Interesting, did you retry to execute DSpot with empty MAVEN_OPTS?

I'm not sure but It might be related to:

-javaagent:/home/smorka/devEnvironment/grails-2.5.5/lib/org.springframework/springloaded/jars/springloaded-1.2.5.RELEASE.jar
Smorka commented

I'm still not able to run it using the jar file

$ java -jar dspot-2.2.0-jar-with-dependencies.jar --path-to-properties dspot.properties --test no.tellu.lib.data.BooleanOpTest

However, it does execute, with a build success when using the dspot-maven plugin

<plugin>
        <groupId>eu.stamp-project</groupId>
        <artifactId>dspot-maven</artifactId>
	<version>2.2.0</version>
	<configuration>
	</configuration>
</plugin>
$ mvn dspot:amplify-unit-tests

provides a BUILD SUCCESS after 03:46 min

danglotb commented

Hello, could you provide me both traces? The one with jar and the one with maven.

BTW, it is not mandatory to modify your pom.xml.
You can run directly something like:

mvn eu.stamp-project:dspot-maven:2.2.1-SNAPSHOT:amplify-unit-tests -Dtest=no.tellu.lib.data.BooleanOpTest

Thank you.

danglotb commented

Hello @Smorka

Thank you for the traces.

So, you did manage to amplify using maven (I obfuscated the name of test classes):

[INFO] Print Test1.java with 1 amplified test cases in target/dspot/output
[INFO] Print Test2.java with 2 amplified test cases in target/dspot/output
[INFO] Print Test3.java with 1 amplified test cases in target/dspot/output

Thus, there are positives results which is good.

What you can do know, is to have a look at your amplified test classes, they should be in target/dspot/output/.

You can also try to use input amplifiers (by default, dspot only generates new assertions, i.e. the assertion amplification).

To do so, you can use:

-Damplifiers=FastLiteralAmplifier,MethodDuplicationAmplifier,ReturnValueAmplifier -Diteration=1

Options:
-Damplifiers=<list> specify the input amplifiers (pre-defined operators to modify the input part the tests). Elements must be separated by a comma ,. See the README to know the accepted values.
-Diteration=<int> the number of DSpot's main loop, i.e input-amplification + assertion-amplification.
Values:
FastLiteralAmplifier modifies the literal (numbers, boolean and string).
MethodDuplicationAmplifier duplicates an existing method call.
ReturnValueAmplifier uses the return value of an existing method to collect its state.

For the maven execution, it seems that it is still stuck on the pit execution:

[INFO] 2019-09-30 14:56:27 MavenAutomaticBuilder - run maven: /usr/share/maven//bin/mvn org.pitest:pitest-maven:mutationCoverage 
[WARN] Maven will be executed in interactive mode, but no input stream has been configured for this MavenInvoker instance.https://github.com/STAMP-project/dspot/issues/889#issuecomment-536555042

I suggest maybe to stick to the execution using maven.

danglotb commented

Hello @Smorka

Any progress?

Smorka commented