Send MatterMost message with test result from popular testing frameworks
⭐ If you find this project useful, please consider following the CTRF organisation and giving this repository a star ⭐
It means a lot to us and helps us grow this open source library.
- Send Test Results to MatterMost: Automatically send test results to a MatterMost channel.
- Send Flaky Test Details to MatterMost: Automatically send flaky test details to a MatterMost channel.
- Conditional Notifications: Use the
--onFailOnlyoption to send notifications only if tests fail.
You'll need a CTRF report generated by your testing framework. CTRF reporters are available for most testing frameworks and easy to install.
No CTRF reporter? No problem!
Use junit-to-ctrf to convert a JUnit report to CTRF.
Go to the MatterMost API: Incoming Webhooks page and follows instructions to setup an incoming webhook.
Set the webhook URL as an environment variable in your shell or CI environment:
export MATTERMOST_WEBHOOK_URL='https://your/webhook/url'Make sure to replace 'https://your/webhook/url' with your actual webhook URL.
You might want to store the webhook URL as a secret.
To send the test results summary to MatterMost:
npx mattermost-ctrf results /path/to/ctrf-report.json
To send flaky test report to MatterMost:
npx mattermost-ctrf flaky /path/to/ctrf-report.json
To send the test results summary to MatterMost only if there are failed tests, use the --onFailOnly option:
npx mattermost-ctrf results /path/to/ctrf-file.json --onFailOnly--onFailOnly, -f: Send notification only if there are failed tests.
You can merge reports if your chosen reporter generates multiple reports through design, parallelisation or otherwise.
The ctrf-cli package provides a method to merge multiple ctrf json files into a single file.
After executing your tests, use the following command:
npx ctrf merge <directory>Replace directory with the path to the directory containing the CTRF reports you want to merge.
CTRF is a universal JSON test report schema that addresses the lack of a standardized format for JSON test reports.
Consistency Across Tools: Different testing tools and frameworks often produce reports in varied formats. CTRF ensures a uniform structure, making it easier to understand and compare reports, regardless of the testing tool used.
Language and Framework Agnostic: It provides a universal reporting schema that works seamlessly with any programming language and testing framework.
Facilitates Better Analysis: With a standardized format, programatically analyzing test outcomes across multiple platforms becomes more straightforward.
If you find this project useful, consider giving it a GitHub star ⭐ It means a lot to us.