[FEATURE] How to remove flaky tests from slack report?

[sianka]

Is your feature request related to a problem? Please describe.
I'd like to remove flaky tests from the report overview in slack.

Describe the solution you'd like
I've tried to experiment with statuses, but there is no flaky one, so I'd need a solution how to do it.
As you can see from the screenshots provided, in the main screen of the report I have 2 tests (flaky + failed), but I'd like to see the same result I have in the thread (only 1 failed test). Thanks

Additional context

[ryanrosello-og]

@sianka can you try this #100 (comment)

[sianka]

@ryanrosello-og yes, thanks, I tried this solution, but it didn't help to remove the titles of flaky tests.
Also, I tried this solution

for (const t of tests) {
    if ((t.status === 'failed' || t.status === 'timedOut') && t.retry == 0) {
      const testTitle = `:pushpin: ${t.name.split('@')[0].trim()}`;

but it didn't work either

Here is the screenshot how it looks after implementing your solution:

[ryanrosello-og]

@sianka what if you change your logic slightly:

from

for (const t of tests) {
    if ((t.status === 'failed' || t.status === 'timedOut') && t.retry == 0) {
      const testTitle = `:pushpin: ${t.name.split('@')[0].trim()}`;

to

for (const t of tests) {
    if ((t.status === 'failed' || t.status === 'timedOut') && t.retry == [retries as per your playwright config e.g. 2]) {
      const testTitle = `:pushpin: ${t.name.split('@')[0].trim()}`;

that way, it will only report on the tests that have failed their last attempt.

[sianka]

@ryanrosello-og it worked for us, thanks a lot!