[GH Action] UI Tests sometimes fail with Selenoid
ElSnoMan opened this issue · 0 comments
Problem
When a test or pipeline passes sometimes and fails other times without any changes, we call that "flaky". Well, it seems our test-pylenium.yml
pipeline is flaky 😢 . Here's an example:
https://github.com/ElSnoMan/pyleniumio/actions/runs/5336984960/jobs/9672430764
Potential Solutions
When our pipeline is "flaky", it seems the last 2-3 tests fail because of a timeout
issue. The solution may be something as simple as configuring Selenoid to have a longer timeout. Look at their docs for some ideas:
https://github.com/marketplace/actions/start-selenoid-server?version=v2
Also, it would be nice to be able to run multiple tests as the same time to speed up the Test Run time. Right now we run all of our tests sequentially because the GitHub Action container isn't very powerful, but with some optimizations to Selenoid, we might be able to run 2-4 concurrently