Parallel execution of Features is not working

Question

Parallel execution of Features is not working

sherish2021 opened this issue 2 years ago · 5 comments

sherish2021 commented 2 years ago

Hi,

I am currently working with the Serenity-cucumber-starter repo and have added a few features which I am looking to test the parallel execution functionality for. I am following the config steps detailed here:

https://johnfergusonsmart.com/parallel-test-execution-with-cucumber-and-serenity-bdd/

However the parallel execution feature does not seem to be working, it just fires off each test and feature sequentially? I would expect with this config working as expecting it is creating multiple browser instances in parallel to decrease execution times.

Is there some additional I am missing here or a current known issue?

4.0.0
com.serenity-bdd
bootstrap
1.0.0-SNAPSHOT
jar

<name>Sample Serenity BDD project using Cucumber</name>

<properties>

    <checkstyle.version>10.3</checkstyle.version>
    <checkstyle.plugin.version>3.1.2</checkstyle.plugin.version>
    <sevntu.checkstyle.plugin.version>1.42.0</sevntu.checkstyle.plugin.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <serenity.version>3.3.0</serenity.version>
    <!-- this is cucumber 6 version -->
    <!-- <serenity.version>3.2.1</serenity.version> -->
    <encoding>UTF-8</encoding>
    <tags></tags>
    <webdriver.base.url></webdriver.base.url>
    <failsafe.version>3.0.0-M5</failsafe.version>
    <surefire.version>3.0.0-M5</surefire.version>
    <pmd.version>3.13.0</pmd.version>
    <maven.compiler.plugin.version>3.8.1</maven.compiler.plugin.version>
</properties>
<dependencies>

    <!--<dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-core</artifactId>
        <version>${serenity.version}</version>
        <scope>test</scope>
    </dependency> -->
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-cucumber</artifactId>
        <version>${serenity.version}</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-cucumber6</artifactId>
        <version>2.3.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-screenplay</artifactId>
        <version>${serenity.version}</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-screenplay-webdriver</artifactId>
        <version>${serenity.version}</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-ensure</artifactId>
        <version>${serenity.version}</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.platform</groupId>
        <artifactId>junit-platform-launcher</artifactId>
        <version>1.8.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>5.8.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.vintage</groupId>
        <artifactId>junit-vintage-engine</artifactId>
        <version>5.8.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
        <version>1.2.10</version>
    </dependency>
    <dependency>
        <groupId>org.assertj</groupId>
        <artifactId>assertj-core</artifactId>
        <version>3.6.2</version>
        <scope>test</scope>
    </dependency>
</dependencies>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-pmd-plugin</artifactId>
            <version>${pmd.version}</version>
            <configuration>
                <includeTests>true</includeTests>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-checkstyle-plugin</artifactId>
            <version>${checkstyle.plugin.version}</version>
            <dependencies>
                <dependency>
                    <groupId>com.puppycrawl.tools</groupId>
                    <artifactId>checkstyle</artifactId>
                    <version>${checkstyle.version}</version>
                </dependency>
                <dependency>
                    <groupId>com.github.sevntu-checkstyle</groupId>
                    <artifactId>sevntu-checks</artifactId>
                    <version>${sevntu.checkstyle.plugin.version}</version>
                </dependency>
            </dependencies>

            <configuration>
                <includeTestSourceDirectory>true</includeTestSourceDirectory>
               <!-- <configLocation>checkstyle.xml</configLocation>-->
                <consoleOutput>true</consoleOutput>
                <failsOnError>false</failsOnError>
                <logViolationsToConsole>true</logViolationsToConsole>
                <linkXRef>false</linkXRef>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${surefire.version}</version>
            <configuration>
                <skip>true</skip>
            </configuration>
        </plugin>
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>${failsafe.version}</version>
            <configuration>
                <includes>
                    <include>**/*Test.java</include>
                 <!--   <include>**/Test*.java</include>
                    <include>**/*TestSuite.java</include>
                    <include>**/When*.java</include>-->
                    <include>**/*.java</include>
                </includes>
                <systemProperties>
                    <environment>${target_environment}</environment>
                    <marionette.actors.enabled>false</marionette.actors.enabled>
                    <acceptInsecureCerts>true</acceptInsecureCerts>
                    <webdriver.chrome.whitelistedIps></webdriver.chrome.whitelistedIps>
                </systemProperties>
                <systemPropertyVariables>
                    <webdriver.base.url>${webdriver.base.url}</webdriver.base.url>
                </systemPropertyVariables>
                <parallel>methods</parallel>
                <threadCount>5</threadCount>
                <!--<useUnlimitedThreads>true</useUnlimitedThreads>-->
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>${maven.compiler.plugin.version}</version>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>
        <plugin>
            <groupId>net.serenity-bdd.maven.plugins</groupId>
            <artifactId>serenity-maven-plugin</artifactId>
            <version>${serenity.version}</version>
            <configuration>
                <tags>${tags}</tags>
            </configuration>
            <executions>
                <execution>
                    <id>serenity-reports</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>aggregate</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>
<reporting>
    <plugins>
        <!-- that plugin is required to link violation with source code, without that plugin report is useless and hard to use -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-jxr-plugin</artifactId>
            <version>2.5</version>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-checkstyle-plugin</artifactId>
            <version>2.15</version>
            <!--  Specifying configuration here will take effect on the execution of "mvn site",
               but will not take effect on the execution of "mvn checkstyle:checkstyle"  -->
            <configuration>
                <configLocation>checkstyle.xml</configLocation>
                <failOnViolation>true</failOnViolation>
                <enableFilesSummary>false</enableFilesSummary>
            </configuration>
        </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-pmd-plugin</artifactId>
                <version>3.13.0</version>
                <configuration>
                    <failOnViolation>true</failOnViolation>
                </configuration>
            </plugin>

    </plugins>
</reporting>

Answer 1 · 2022-08-11T08:20:52.000Z

Parallel execution of classes via surefire or failsafe doesn't work with junit 5 - you need to use junit 4 for that.

Answer 2 · 2022-08-11T08:42:56.000Z

Thanks @wakaleo for the response, much appreciated - anywhere I can source the junit4 pom config required for Serenity to get the parallel execution running?

Are there future plans to get the parallel execution working with junit5?

Answer 3 · 2022-08-11T08:44:26.000Z

I'll update an example in the coming weeks. The issue is a junit thing, not a Serenity thing.

Answer 4 · 2022-09-21T18:49:07.000Z

hi @wakaleo, could you share any working parallel running pom example?
or could you update serenity-cucumber-starter pom.xml to run features in parallel?
thanks in advance! :)

Answer 5 · 2022-09-21T20:29:40.000Z

It can work running JUnit 4 separate runner files. Parallel execution doesn't seem to work in JUnit 5 as yet.