Have tests grouped by suite in summary

[guusdk]

My test result is a one-file XML structure, that consists of one test-suite, in which sub-suites are nested:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<testsuites failures="0" name="XMPP specification test run with ID uj4f1" skipped="0" tests="57" time="5.922" timestamp="2024-04-19T08:20:31.569834">
    <testsuite failures="0" name="XEP-0199: XMPP Ping" skipped="0" tests="2" time="0.054">
        <testcase classname="org.jivesoftware.smackx.ping.PingIntegrationTest" name="PingIntegrationTest.pingAsync (Normal)" time="0.054">
            <properties>
                <property name="attachment" value="PingIntegrationTest.pingAsync (Normal).log"/>
            </properties>
        </testcase>
        <testcase classname="org.jivesoftware.smackx.ping.PingIntegrationTest" name="PingIntegrationTest.pingServer (Normal)" time="0.0">
            <properties>
                <property name="attachment" value="PingIntegrationTest.pingServer (Normal).log"/>
            </properties>
        </testcase>
    </testsuite>
    <testsuite failures="0" name="XEP-0045: Multi-User Chat" skipped="0" tests="27" time="1.508">
        <testcase classname="org.jivesoftware.smackx.muc.MultiUserChatRolesAffiliationsPrivilegesIntegrationTest" name="MultiUserChatRolesAffiliationsPrivilegesIntegrationTest.mucRoleTestForReceivingModerator (Normal)" time="0.046">
            <properties>
                <property name="attachment" value="MultiUserChatRolesAffiliationsPrivilegesIntegrationTest.mucRoleTestForReceivingModerator (Normal).log"/>
            </properties>
        </testcase>
...

Although not shown here, a testcases from the same suite do not necessarily share the same classname.

The test summary, as generated by this Action, seems to aggregate tests from all suites, as shown below. For my use-case, having the tests grouped by suite would add significant value. Is there a way to have the test summary group results by suite?

caveat: the XML file that is producing these results is a hand-crafted file. Apologies if I'm asking a question that's already supported, but not working properly in my setup, because of that.

[Fishbowler]

Looking at

action-junit-report/src/testParser.ts

Lines 226 to 236 in 9379f0c

	let suiteName = ''
	if (suiteRegex) {
	if (parentName) {
	suiteName = `${parentName}/${testsuite._attributes.name}`
	} else if (suiteRegex !== '*') {
	suiteName = testsuite._attributes.name.match(suiteRegex)
	}
	if (!suiteName) {
	suiteName = testsuite._attributes.name
	}
	}

, try setting the suite_regex to '*' ?

[guusdk]

That didn't make much of a difference:

[Fishbowler]

MoodIntegrationTest.testNotificationAfterFilterChange (Normal)

to

MoodIntegrationTest.XEP-0107: User Mood/MoodIntegrationTest.testNotificationAfterFilterChange (Normal)

Which is more information, but unsure if it's actually nicer.

[guusdk]

Ah, I didn't notice that. It doesn't really improve things from a UX perspective. I'd prefer to be able to have grouping.

[mikepenz]

thank you very much for the ticket.

That's quite a good idea. Would you also have a proposal on how to format it. E.g. all in one table with headers in-between or individual table per group?

[guusdk]

I'm far from a UX expert, but maybe:

• one table that provides a summary for each suite
• a table per suite

The summary table could possibly be switched on/off by configuration.

Something super-funky would generate a table with a row per suite, that folds out when clicked on it maybe. That might be not one, but two bridges to far for now though.

[jenniferlianne]

This would be great. Agree with guusdk's suggested format.

[mikepenz]

While this does not yet support your requirement, I started refactoring the action to enable the new BREAD_CRUMB title template.

Furthermore does this refactor lay the foundation to enable a more structured report in the future. I don't have an ETA yet, however I do think that with the latest changes the possibility comes closer :D