opensearch-project/alerting

[ACTION NEEDED] Fix flaky integration tests at distribution level

Closed this issue · 3 comments

gaiksaya commented

What is the bug?
It was observed in 2.13.0 and previous other releases that alerting manually signed off on the release for failing integration tests. See opensearch-project/opensearch-build#4433 (comment)
The flakiness of the test runs take a lot of time from the release team to collect go/no-go decision and significantly lower the confidence in the release bundles.

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Run integration testing for altering and see the failures.
  2. Issues can be reproduced using the steps declared in AUTOCUT issues for failed integration testing

What is the expected behavior?
Tests should be consistently passing.

Do you have any additional context?
Please note that this is a hard blocker for 2.14.0 release as per the discussion here

bbarani commented

@praveensameneni @eirsep Can you please provide your inputs?

gaiksaya commented

Adding 2.14.0 release manager @rishabh6788

AWSHurneyt commented

For reference, as mentioned in #1517 (comment), test SecureMonitorRestApiIT.test create monitor with an user without index read role appears to be flaky.