STAMP-project/dspot

DSpot 2.2.0 not generating anything anymore

vmassol opened this issue · 10 comments

vmassol commented

Tested on xwiki-commons-core/xwiki-commons-xml:

        <plugin>
          <groupId>eu.stamp-project</groupId>
          <artifactId>dspot-maven</artifactId>
          <version>2.2.0</version>
          <configuration>
            <clean>true</clean>
            <descartes>true</descartes>
            <verbose>true</verbose>
            <withComment>true</withComment>
            <generateNewTestClass>true</generateNewTestClass>
            <pathToProperties>./dspot.properties</pathToProperties>
            <amplifiers>
              <!-- Uncomment the amplifier to use or pass them on the command line using -Damplifiers=... -->
              <!--amplifier>StringLiteralAmplifier</amplifier-->
              <!--amplifier>NumberLiteralAmplifier</amplifier-->
              <!--amplifier>CharLiteralAmplifier</amplifier-->
              <!--amplifier>BooleanLiteralAmplifier</amplifier-->
              <!--amplifier>AllLiteralAmplifiers</amplifier-->
              <amplifier>MethodAdd</amplifier>
              <amplifier>MethodRemove</amplifier>
              <amplifier>MethodGeneratorAmplifier</amplifier>
              <amplifier>ReturnValueAmplifier</amplifier>
              <!--amplifier>ReplacementAmplifier</amplifier-->
              <amplifier>NullifierAmplifier</amplifier>
            </amplifiers>
          </configuration>
        </plugin>

dspot.properties:

#relative path to the project root from dspot project
project=../../
targetModule=xwiki-commons-core/xwiki-commons-xml/
#relative path to the source project from the project properties
src=src/main/java/
#relative path to the test source project from the project properties
testSrc=src/test/java
#java version used
javaVersion=8
# (optional) path to the output folder, default to "output_diversify"
outputDirectory=target/dspot-output
# (optional) filter on the package name containing tests to be amplified ("example" => "example.*"
filter=org.xwiki.*
tmpDir=target/dspot
srcResources=src/main/resources/
#excludedTestCases=createDomDocument

Execution of mvn dspot:amplify-unit-tests -X:

...
[DEBUG] Configuring mojo eu.stamp-project:dspot-maven:2.2.0:amplify-unit-tests from plugin realm ClassRealm[plugin>eu.stamp-project:dspot-maven:2.2.0, parent: sun.misc.Launcher$AppClassLoader@70dea4e]
[DEBUG] Configuring mojo 'eu.stamp-project:dspot-maven:2.2.0:amplify-unit-tests' with basic configurator -->
[DEBUG]   (f) allowPathInAssertions = false
[DEBUG]   (f) amplifiers = [MethodAdd, MethodRemove, MethodGeneratorAmplifier, ReturnValueAmplifier, NullifierAmplifier]
[DEBUG]   (f) budgetizer = RandomBudgetizer
[DEBUG]   (f) clean = true
[DEBUG]   (f) descartes = true
[DEBUG]   (f) example = false
[DEBUG]   (f) executeTestParallelWithNumberProcessors = -1
[DEBUG]   (f) executeTestsInParallel = false
[DEBUG]   (f) generateNewTestClass = true
[DEBUG]   (f) gregor = false
[DEBUG]   (f) help = false
[DEBUG]   (f) iteration = 3
[DEBUG]   (f) keepOriginalTestMethods = false
[DEBUG]   (f) maxTestAmplified = 200
[DEBUG]   (f) noMinimize = false
[DEBUG]   (f) numberParallelExecutionProcessors = -1
[DEBUG]   (f) outputPath = target/dspot/output
[DEBUG]   (f) pathToProperties = ./dspot.properties
[DEBUG]   (f) pitOutputFormat = XML
[DEBUG]   (f) project = MavenProject: org.xwiki.commons:xwiki-commons-xml:11.8-SNAPSHOT @ /Users/vmassol/dev/xwiki/xwiki-commons/xwiki-commons-core/xwiki-commons-xml/pom.xml
[DEBUG]   (f) randomSeed = 23
[DEBUG]   (f) runOnAllModules = false
[DEBUG]   (f) targetOneTestClass = false
[DEBUG]   (f) test = [all]
[DEBUG]   (f) testCases = []
[DEBUG]   (f) testCriterion = PitMutantScoreSelector
[DEBUG]   (f) timeOut = 10000
[DEBUG]   (f) useMavenToExeTest = false
[DEBUG]   (f) verbose = true
[DEBUG]   (f) withComment = true
[DEBUG]   (f) workingDirectory = false
[DEBUG] -- end configuration --
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.101 s
[INFO] Finished at: 2019-09-19T16:30:41+02:00
[INFO] ------------------------------------------------------------------------
vmassol ~/dev/xwiki/xwiki-commons/xwiki-commons-core/xwiki-commons-xml (master *) $

So no error, just nothing happens and no target/dspot directory created...

Any idea?

danglotb commented

Hello @vmassol

The plugin is called dspot-maven and not dspot.

Thus, you should run dspot from the topest parent, and specify which module you want to amplify. In this way, you won't have any trouble with dependencies and stuff.

Keep me updated, please.

vmassol commented

ok this is weird. It means a dspot plugin exists too since maven didn't fail and I'm pretty sure this is what I was running before since I commented about it in our pom.xml:

        <!-- In order to make it simple to execute Dspot on XWiki with: mvn dspot:amplify-unit-tests -->
        <plugin>
          <groupId>eu.stamp-project</groupId>
          <artifactId>dspot-maven</artifactId>

Also in the doc it says to use "dspot" at https://github.com/STAMP-project/dspot/tree/master/dspot-maven:

After setting up your pom.xml and add your configuration with different options,run:

mvn dspot:amplify-unit-tests

Regarding execution of dspot, I've alwas executed it from the module directory. Has this changed in version 2.2.0?

danglotb commented

Also in the doc it says to use "dspot" at https://github.com/STAMP-project/dspot/tree/master/dspot-maven:

Thanks, must fix that.

Regarding execution of dspot, I've alwas executed it from the module directory. Has this changed in version 2.2.0?

You can still run it from the module, but DSpot might have some troubles to gather the required dependencies, and these troubles exist from the beginning.

Did you manage to run DSpot on your module ?

Thank you.

vmassol commented

I still don't understand what you meant by:

The plugin is called dspot-maven and not dspot.

Also

You can still run it from the module, but DSpot might have some troubles to gather the required dependencies, and these troubles exist from the beginning.

I've never, ever, ran Dspot from the top level and it has always worked so I don't understand what you mean. I wouldn't even know how to do that (I don't have the proper dspot.properties file for that)!

So there's definitely something that has changed. Again, I've used the exact same dspot.properties and exact same maven command from before.

Ok I've just proved it by reverting to DSpot 2.1.0 and it works fine.... And then put back DSpot 2.2.0 again and it fails (ie didn't do anything). So there's a regression in Dspot 2.2.0 for sure :) I was not dreaming.

Thanks!

danglotb commented

I neither understand.

Run some tests and seems like that there are two behaviors, depending on what it is in your pom.xml.

When you define the dspot-maven plugin inside your pom.xml, you have to type dspot:amplify-unit-tests, but if you type dspot-maven:amplify-unit-tests it does not work.

If you do not add this definition, you have to type dspot-maven:amplify-unit-tests and not dspot:amplify-unit-tests.

On my side, I avoid to modify the pom.xml, since it is less easy to custom your runs than on command lines, this is why I told you that the plugin was named dspot-maven.

The question is: why there are these two behaviors?

Ok I've just proved it by reverting to DSpot 2.1.0 and it works fine.... And then put back DSpot 2.2.0 again and it fails (ie didn't do anything). So there's a regression in Dspot 2.2.0 for sure :) I was not dreaming.

In fact, I identified the two PR responsible: #831 and #814.

I do not remember in which particular case running DSpot from a module encountered troubles but I remember facing it. Maybe the fact that if you do not run mvn install before, some dependencies, like other modules on which the current module under amplification relies will be missing.

I'm looking for a solution to make all the ways of applying DSpot work, keeping you updated.

vmassol commented

ok thanks Benjamin!

danglotb commented

Just opened #879 that should fix the problem.

However, I still do not understand the dspot and dspot-maven thing...

vmassol commented

However, I still do not understand the dspot and dspot-maven thing...

BTW normally you should have used an artifactId of dspot-maven-plugin to follow the best practices/conventions. This allows to use just "dspot", see https://maven.apache.org/guides/plugin/guide-java-plugin-development.html and especially this part:

You can assign a shortened prefix to your plugin, such as mvn hello:sayhi. This is done automatically if you follow the convention of using ${prefix}-maven-plugin (or maven-${prefix}-plugin if the plugin is part of the Apache Maven project).

Maybe Maven also supports the convention ${prefix}-plugin by default?

danglotb commented

Okay, thank you for the info.

vmassol commented

I confirm DSpot executes well now. Thanks! Testing to see if I can generate any test.