Flank/flank

maximum `max-test-shards` incorrect for arm virtual devices

alistairsykes opened this issue · 1 comments

alistairsykes commented

Describe the bug
When selecting max-test-shards of -1, flank sets 500 for all virtual devices. According to test lab rest documentation when using arm virtual devices the limit is actually 100
https://firebase.google.com/docs/test-lab/reference/testing/rest/v1/projects.testMatrices#uniformsharding

I presume this is function which would need to be changed, but I'm not too familiar with the flank codebase.
https://github.com/Flank/flank/blob/master/test_runner/src/main/kotlin/ftl/args/PrepareAndroidCommonConfig.kt#L24

To Reproduce

  1. Set max-test-shards to -1
  2. Set device config to:
device:
        - model: SmallPhone.arm
          version: 30
          locale: en
          orientation: portrait
  1. Observe the outputted config shows:
max-test-shards: 500

Expected behavior
I expect flank to determine that arm devices are present and therefore select 100 as the max-test-shards.

Details (please complete the following information):
version: v23.06.2
revision: c7bd980

Additional context
None

alistairsykes commented

🙌 Thank you for turning this around so quickly.