cross browser/platform parallel test execution

[itkhanz]

Hi,
i am doing cross platform testing where I am running the same cucumber set of cucumber feature files on two different platforms using testng.xml.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="Swag Labs Demo App Suite" parallel="tests" thread-count="2" configfailurepolicy="continue">

    <test name="Pixel_5">
        <parameter name="platformName" value="Android" />
        <parameter name="deviceName" value="Pixel 5" />
        <classes>
            <class name="runners.AndroidTestNGRunnerTest"/>
        </classes>
    </test> <!-- Test -->

    <test name="iPhone14">
        <parameter name="platformName" value="iOS" />
        <parameter name="deviceName" value="iPhone 14" />
        <classes>
            <class name="runners.IOSTestNGRunnerTest"/>
        </classes>
    </test> <!-- Test -->
</suite> <!-- Suite -->

I have modified the test runner class configuration to generate cucumber json and html report at separate path for each of my platform using plugin option in CucumberOptions. This results in html and json reports for each of my test getting created at separate path and without getting merged or over-written.

After the test execution is finished, net.masterthought.cucumber.ReportParser parseJsonFiles picks up the two json files from the path specified in the inputDirectory and jsonFiles tags and then create a single report combining the result of both of the test executions.

Jul 26, 2023 4:00:46 PM net.masterthought.cucumber.ReportParser parseJsonFiles
INFO: File '/************/target/cucumber/android-cucumber-report.json' contains 1 feature(s)
Jul 26, 2023 4:00:46 PM net.masterthought.cucumber.ReportParser parseJsonFiles
INFO: File '/***********/target/cucumber/ios-cucumber-report.json' contains 1 feature(s)

This behavior is as expected because the outputDirectory path is same for both of the tests and inputDirectoy has the path where it can pick up both of the json files.

<outputDirectory>${project.build.directory}</outputDirectory>
<inputDirectory>${project.build.directory}/cucumber</inputDirectory>
<jsonFiles>
    <param>**/*.json</param>
</jsonFiles>

What I am looking for is a way to create separate report for each of the json file generated from the cucumber.

The problem right now is that I am not able to see the platform specific information on the report. For example, in the image below Login scenarios feature displayed got executed by different testng test but since the feature and scenario name is same for each of the testng test, there is no information about the testng parameters or platform specific information in report that could help to know which test this feature belongs to.

Or perhaps there is a way to tag the features or scenarios dynamically at run-time that would help in visualising the platform specific information about the feature and scenario.

I could think of the following possible solutions:

1. Generate separate masterthought report from each cucumber json file
2. Add some tags or platform specific information to the features and scenarios

Could you please provide some information if this is possible with the existing setup?

I am using following version of libraries:
cucumber-java 7.13.0
cucumber-testng 7.13.0
maven-cucumber-reporting 5.7.5
JDK 17.0.2

[itkhanz]

I was able to resolve the above issue by using the cucumber-reporting plugin instead and add the configuration with different quantifiers for each json file programatically in onFinish() method of ISuiteListener from TestNG.
Now i can differentiate between the features on dashboard.