maxNumberOfFailuresToShow doesn't seem to work as expected.
Closed this issue · 8 comments
Bug description
Looking at the example from the doc here : maxNumberOfFailuresToShow, it doesn't seem to work as expected when I set maxNumberOfFailuresToShow
to 0 in my cli_config.json
file like that : "maxNumberOfFailuresToShow": 0,
How to reproduce (please include a snippet of your Playwright Slack Reporter configuration)
- Set the config
maxNumberOfFailuresToShow
to '0
' - Run your test with some failing tests cases.
- See some failure reported in your Slack.
Not sure if it's useful, but I use :
"sendUsingWebhook": {
"webhookUrl": "my slack webhook URL"
}
Expected behavior
No failure shown in the Slack report.
Environment & setup
- OS: macOS
- Node.js version: v21.1.0
playwright-slack-report version
"playwright-slack-report": "^1.1.72"
yeah this is a legit bug. Seems this bit of the code https://github.com/ryanrosello-og/playwright-slack-report/blob/main/src/SlackReporter.ts#L77 is treating zero as falsy so its always falling back to showing 10 failures even though you specified 0 as its value.
should be straight-forward fix:
this.maxNumberOfFailuresToShow
= slackReporterConfig.maxNumberOfFailuresToShow !== undefined ? slackReporterConfig.maxNumberOfFailuresToShow : 10;
leave it with me. I can take a look
Okey ! Waiting for the fix then ! Thanks 🙏🏻
Hello, @ryanrosello-og FYI, I tested by replacing the value to 1, I still have multiple error shown in the slack reporting and still have the message : There are too many failures to display - 5 out of 28 failures shown
My cli_config.json looks like that :
{
"sendResults": "always",
"slackLogLevel": "error",
"sendUsingWebhook": {
"webhookUrl": "$SLACK_WEBHOOK_URL"
},
"meta": [
{ "key": "branch", "value" : "$CI_COMMIT_BRANCH"},
{ "key": "commit", "value" : "$CI_COMMIT_SHA"}
],
"maxNumberOfFailuresToShow": 1,
"disableUnfurl": true
}
@ryanrosello-og Hello, I updated to the latest version : "playwright-slack-report": "^1.1.80",
I still have multiple errors appearing when setting "maxNumberOfFailuresToShow": 0,
See my cli_config.json file :
{
"sendResults": "always",
"slackLogLevel": "error",
"sendUsingWebhook": {
"webhookUrl": "$SLACK_WEBHOOK_URL"
},
"meta": [
{ "key": "Pipeline", "value" : "$CI_PIPELINE_URL"}
],
"maxNumberOfFailuresToShow": 0,
"disableUnfurl": true
}
Here is a resume of the slack report :
:performing_arts: Playwright Results
:white_check_mark: 24 | :x: 59 | :large_yellow_circle: 0 | :fast_forward: 0
Pipeline : $CI_PIPELINE_URL
72 | // Set currency
> 74 | await this.page.getByText(currency).click();
| ^
TimeoutError: locator.click: Timeout 10000ms exceeded.
Call log:
- waiting for getByText('USD ($)')
:warning: There are too many failures to display - 5 out of 59 failures shown
@samixchoumi it turns out your config is using the wrong key, use "maxNumberOfFailures": 0,
instead of "maxNumberOfFailuresToShow": 0,
When you provided maxNumberOfFailuresToShow
, it was causing it to fallback to the default value (5) see ....
https://github.com/ryanrosello-og/playwright-slack-report/blob/main/src/cli/cli_schema.ts#L30
Oh, I though I needed to use this parameter : https://github.com/ryanrosello-og/playwright-slack-report?tab=readme-ov-file#maxnumberoffailurestoshow
I update my config file to see if it's fixed then. Thanks
It works !
Thanks (maybe updating the documentation to avoid conflict between both parameter ?)
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.