Support ESLint or Jest junit reports

[villelahdenvuo]

Hey, this is a great action I started using with our ESLint workflow!

I notice that it creates annotations, but they don't show up in the PR, looks like the file, line & col values don't get parsed correctly.

Here's an example junit report ESLint generates:

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
<testsuite package="org.eslint" time="0" tests="1" errors="1" name="/home/runner/work/ecom-storefront/ecom-storefront/cypress/plugins/s3-email-client/s3-utils.ts">
<testcase time="0" name="org.eslint.@typescript-eslint/dot-notation" classname="/home/runner/work/ecom-storefront/ecom-storefront/cypress/plugins/s3-email-client/s3-utils"><failure message="[&quot;Bucket&quot;] is better written in dot notation."><![CDATA[line 7[6](https://github.com/granodigital/ecom-storefront/actions/runs/7671315746/job/20909395839#step:15:7), col 28, Error - [&quot;Bucket&quot;] is better written in dot notation. (@typescript-eslint/dot-notation)]]></failure></testcase>
</testsuite>
</testsuites>

And here's the annotation I get now:

Note the missing file extension and line/col information.

[mikepenz]

Unfortunately there is no real consistent approach how those are provided in exports.

We have some different approaches in the action which tries to identify file and line number, however these are unfortunately not perfect.

[villelahdenvuo]

@mikepenz Yeah, it's unfortunate. Perhaps there could be an option similar to VS Code's problem matchers? https://code.visualstudio.com/docs/editor/tasks#_defining-a-problem-matcher

That way you could add support manually and perhaps add presets/docs for common ones?

[villelahdenvuo]

Meanwhile if someone else needs this, here's a small script I asked ChatGPT to write to process the ESLint junit reports so that they work better with this action:

#!/usr/bin/env node
import { readFile, writeFile } from 'node:fs/promises';
import { Parser, Builder } from 'xml2js';

const filePath = process.argv[2];

const data = await readFile(filePath, 'utf8');
const parser = new Parser();
const builder = new Builder();

const xmlData = await parser.parseStringPromise(data);

for (const testsuite of xmlData?.testsuites?.testsuite ?? []) {
	testsuite.$.file = testsuite.$.name;
	testsuite.$.name = undefined;
	for (const testcase of testsuite.testcase ?? []) {
		if (testcase.failure) {
			testcase.$.name = testcase.$.name.replaceAll('org.eslint.', '');
			testcase.$.classname = undefined;
			const failureData = testcase.failure[0];
			const lineMatch = failureData._.match(/line (\d+)/);
			if (lineMatch) failureData.$.line = lineMatch[1];
			// TODO: See if we can get support for warnings in the github action.
			// if (failureData._.includes('Warning - ')) failureData.$.type = 'warn';
		}
	}
}
const output = builder.buildObject(xmlData);
console.log(output);
await writeFile(filePath, output, 'utf8');

Example:

      ...
      - name: Retrieve Changed Files
        id: files
        uses: masesgroup/retrieve-changed-files@v3.0.0
      - name: Lint
        run: npm run lint -- ${{ steps.files.outputs.added_modified }}
      - name: Process ESLint Report
        run: ./scripts/process-eslint-junit.mjs junit/eslint.xml
        if: always()
      - name: Publish Report
        uses: mikepenz/action-junit-report@v4
        if: always()
        with:
          report_paths: junit/*.xml
          check_title_template: '{{TEST_NAME}}'
          annotate_only: true
          annotations_limit: 50
          detailed_summary: true

P.S. Would be nice to support warning level annotations also. :)

[villelahdenvuo]

@mikepenz Any opinions on this? I was thinking about it a bit more and I thought maybe instead of simple regex there could be an option to pass XPath queries for the different things you want to parse and we could document some as examples e.g. replace(//failure/text(), '.*line (\d+).*', '$1') gets the line number from the ESLint report.

[villelahdenvuo]

As we are planning on adding this ESLint annotation capability to all our repos and we haven't heard back about any way of solving this issue, instead of copying the hack fix to all our repos we decided to make our own tool that is more flexible when it comes to different report flavors.

If you want ESLint or Jest junit report support check out:

Report Annotate 🏴

[mikepenz]

Great work 👏