Stalled polling

Question

Stalled polling

alexanderbezverhni opened this issue 2 years ago · 3 comments

alexanderbezverhni commented 2 years ago

Describe the bug
Flank log indicates stalled polling. 9 minutes pass between Tearing down Android test. and FINISHED.

This happens on ~50% of our CI builds. I'm not completely sure if the problem is in Flank polling or in FTL itself.

I'm curios if others are observing similar behavior, and if they were able to pinpoint the culprit. Asking for community help before I dig into building my own Flank with extensive logging/debugging.

To Reproduce
I unfortunately can't share deterministic repro steps, but hopefully the undermentioned Flank log might be helpful.

Expected behavior
Tearing down Android test. -> FINISHED takes ~1min, as observed for other shards.

Details (please complete the following information):

Have you tested on the latest Flank snapshot?

Negative

Post the output of flank --version.

Using fladle with flankVersion = "21.11.0"

Additional context
Flank log:

[2022-06-16T16:37:28.969Z]       robo-directives:

[2022-06-16T16:37:28.969Z]       robo-script: null

[2022-06-16T16:37:28.969Z]       device:

[2022-06-16T16:37:28.969Z]         - model: Pixel2

[2022-06-16T16:37:28.969Z]           version: 26

[2022-06-16T16:37:28.969Z]           locale: en_US

[2022-06-16T16:37:28.969Z]           orientation: portrait

[2022-06-16T16:37:28.969Z]       num-flaky-test-attempts: 1

[2022-06-16T16:37:28.969Z]       test-targets-for-shard:

[2022-06-16T16:37:28.969Z]       fail-fast: false

[2022-06-16T16:37:28.969Z]       parameterized-tests: default

[2022-06-16T16:37:28.969Z] 

[2022-06-16T16:37:28.969Z]     flank:

[2022-06-16T16:37:28.969Z]       max-test-shards: 5

[2022-06-16T16:37:28.969Z]       shard-time: -1

[2022-06-16T16:37:28.969Z]       num-test-runs: 1

[2022-06-16T16:37:28.969Z]       smart-flank-gcs-path: OMITTED

[2022-06-16T16:37:28.969Z]       smart-flank-disable-upload: false

[2022-06-16T16:37:28.969Z]       default-test-time: 120.0

[2022-06-16T16:37:28.969Z]       use-average-test-time-for-new-tests: false

[2022-06-16T16:37:28.969Z]       files-to-download:

[2022-06-16T16:37:28.969Z]         - .*\.ec

[2022-06-16T16:37:28.969Z]         - .*/screenshots/.*

[2022-06-16T16:37:28.969Z]       test-targets-always-run:

[2022-06-16T16:37:28.969Z]       disable-sharding: false

[2022-06-16T16:37:28.969Z]       project: OMITTED

[2022-06-16T16:37:28.969Z]       local-result-dir: results

[2022-06-16T16:37:28.969Z]       full-junit-result: true

[2022-06-16T16:37:28.969Z]       # Android Flank Yml

[2022-06-16T16:37:28.969Z]       keep-file-path: false

[2022-06-16T16:37:28.969Z]       additional-app-test-apks:

[2022-06-16T16:37:28.969Z]       run-timeout: 25m

[2022-06-16T16:37:28.969Z]       legacy-junit-result: false

[2022-06-16T16:37:28.969Z]       ignore-failed-tests: false

[2022-06-16T16:37:28.969Z]       output-style: verbose

[2022-06-16T16:37:28.969Z]       disable-results-upload: false

[2022-06-16T16:37:28.969Z]       default-class-test-time: 240.0

[2022-06-16T16:37:28.969Z]       disable-usage-statistics: false

[2022-06-16T16:37:28.969Z]       output-report: none

[2022-06-16T16:37:28.969Z]       skip-config-validation: false

[2022-06-16T16:37:28.969Z]       custom-sharding-json: 

[2022-06-16T16:37:28.969Z] 

[2022-06-16T16:37:28.969Z] RunTests

[2022-06-16T16:37:30.874Z] 

[2022-06-16T16:37:30.874Z]  Smart Flank cache hit: 100% (116 / 116)

[2022-06-16T16:37:30.874Z]   Shard times: 242s, 243s, 245s, 245s, 245s

[2022-06-16T16:37:30.874Z] 

[2022-06-16T16:37:30.874Z]   Saved 1 shards to OMITTED

[2022-06-16T16:37:31.175Z]   Uploading [android_shards.json] to https://console.developers.google.com/storage/browser/test-lab-m3jj5is8sy790-m167rsysxhj7i/2022-06-16_16-37-28.430800_ntIG/...

[2022-06-16T16:37:39.289Z]   Uploading [OMITTED.apk] to https://console.developers.google.com/storage/browser/test-lab-m3jj5is8sy790-m167rsysxhj7i/2022-06-16_16-37-28.430800_ntIG/...

[2022-06-16T16:37:40.993Z]   Uploading [OMITTED-androidTest.apk] to https://console.developers.google.com/storage/browser/test-lab-m3jj5is8sy790-m167rsysxhj7i/2022-06-16_16-37-28.430800_ntIG/...

[2022-06-16T16:37:44.803Z] 

[2022-06-16T16:37:44.803Z]   116 tests / 5 shards

[2022-06-16T16:37:44.803Z] 

[2022-06-16T16:37:45.004Z]   Uploading [session_id.txt] to https://console.developers.google.com/storage/browser/test-lab-m3jj5is8sy790-m167rsysxhj7i/2022-06-16_16-37-28.430800_ntIG/...

[2022-06-16T16:37:45.004Z]   1 matrix ids created in 0m 16s

[2022-06-16T16:37:45.004Z]   Raw results will be stored in your GCS bucket at [https://console.developers.google.com/storage/browser/test-lab-m3jj5is8sy790-m167rsysxhj7i/2022-06-16_16-37-28.430800_ntIG]

[2022-06-16T16:37:45.004Z] 

[2022-06-16T16:37:45.004Z] Matrices webLink

[2022-06-16T16:38:06.661Z]   matrix-k7u2s8h0ewhza https://console.firebase.google.com/project/OMITTED/testlab/histories/bh.48d96a53e7afed24/matrices/5716286014138517245

[2022-06-16T16:38:06.661Z] 

[2022-06-16T16:38:06.762Z]   0m  0s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Error: 

[2022-06-16T16:38:06.762Z]   0m  0s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y PENDING

[2022-06-16T16:38:06.762Z]   0m  0s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Error: 

[2022-06-16T16:38:06.762Z]   0m  0s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe PENDING

[2022-06-16T16:38:06.762Z]   0m  0s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Error: 

[2022-06-16T16:38:06.762Z]   0m  0s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 PENDING

[2022-06-16T16:38:06.762Z]   0m  0s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Error: 

[2022-06-16T16:38:06.762Z]   0m  0s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s PENDING

[2022-06-16T16:38:06.762Z]   0m  0s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Error: 

[2022-06-16T16:38:06.762Z]   0m  0s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 PENDING

[2022-06-16T16:38:32.033Z]   0m 25s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Starting attempt 1.

[2022-06-16T16:38:32.033Z]   0m 25s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y RUNNING

[2022-06-16T16:38:32.033Z]   0m 25s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Starting attempt 1.

[2022-06-16T16:38:32.033Z]   0m 25s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe RUNNING

[2022-06-16T16:38:32.033Z]   0m 25s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Starting attempt 1.

[2022-06-16T16:38:32.033Z]   0m 25s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 RUNNING

[2022-06-16T16:38:32.033Z]   0m 25s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 RUNNING

[2022-06-16T16:38:37.147Z]   0m 30s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Starting attempt 1.

[2022-06-16T16:38:37.147Z]   0m 30s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s RUNNING

[2022-06-16T16:38:37.147Z]   0m 30s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Starting attempt 1.

[2022-06-16T16:39:53.355Z]   1m 46s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Started logcat recording.

[2022-06-16T16:39:53.355Z]   1m 46s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Started crash monitoring.

[2022-06-16T16:39:53.355Z]   1m 46s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Preparing device.

[2022-06-16T16:40:03.382Z]   1m 56s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Started logcat recording.

[2022-06-16T16:40:03.382Z]   1m 56s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Started crash monitoring.

[2022-06-16T16:40:03.382Z]   1m 56s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Preparing device.

[2022-06-16T16:40:13.609Z]   2m  6s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Started logcat recording.

[2022-06-16T16:40:13.609Z]   2m  6s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Started crash monitoring.

[2022-06-16T16:40:13.609Z]   2m  6s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Preparing device.

[2022-06-16T16:40:13.609Z]   2m  6s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Started logcat recording.

[2022-06-16T16:40:13.609Z]   2m  6s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Started crash monitoring.

[2022-06-16T16:40:13.609Z]   2m  6s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Preparing device.

[2022-06-16T16:40:23.734Z]   2m 17s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Started logcat recording.

[2022-06-16T16:40:23.734Z]   2m 17s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Started crash monitoring.

[2022-06-16T16:40:23.734Z]   2m 17s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Preparing device.

[2022-06-16T16:40:38.877Z]   2m 32s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Installing apps.

[2022-06-16T16:40:54.219Z]   2m 47s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Installing apps.

[2022-06-16T16:40:59.234Z]   2m 52s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Installing apps.

[2022-06-16T16:41:14.476Z]   3m  7s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Retrieving Performance Environment information from the device.

[2022-06-16T16:41:14.476Z]   3m  7s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Setting up Android test.

[2022-06-16T16:41:14.476Z]   3m  7s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Started crash detection.

[2022-06-16T16:41:14.476Z]   3m  7s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Started Out of memory detection

[2022-06-16T16:41:14.476Z]   3m  7s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Started performance monitoring.

[2022-06-16T16:41:14.476Z]   3m  7s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Starting Android test.

[2022-06-16T16:41:19.491Z]   3m 12s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Installing apps.

[2022-06-16T16:41:29.721Z]   3m 23s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Retrieving Performance Environment information from the device.

[2022-06-16T16:41:29.721Z]   3m 23s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Setting up Android test.

[2022-06-16T16:41:29.721Z]   3m 23s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Started crash detection.

[2022-06-16T16:41:29.721Z]   3m 23s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Started Out of memory detection

[2022-06-16T16:41:29.721Z]   3m 23s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Started performance monitoring.

[2022-06-16T16:41:34.837Z]   3m 28s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Installing apps.

[2022-06-16T16:41:34.837Z]   3m 28s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Retrieving Performance Environment information from the device.

[2022-06-16T16:41:34.837Z]   3m 28s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Setting up Android test.

[2022-06-16T16:41:34.837Z]   3m 28s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Started crash detection.

[2022-06-16T16:41:34.837Z]   3m 28s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Started Out of memory detection

[2022-06-16T16:41:34.837Z]   3m 28s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Started performance monitoring.

[2022-06-16T16:41:34.837Z]   3m 28s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Starting Android test.

[2022-06-16T16:41:39.950Z]   3m 33s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Starting Android test.

[2022-06-16T16:41:55.093Z]   3m 48s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Retrieving Performance Environment information from the device.

[2022-06-16T16:41:55.093Z]   3m 48s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Setting up Android test.

[2022-06-16T16:41:55.093Z]   3m 48s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Started crash detection.

[2022-06-16T16:41:55.093Z]   3m 48s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Started Out of memory detection

[2022-06-16T16:41:55.093Z]   3m 48s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Started performance monitoring.

[2022-06-16T16:42:05.222Z]   3m 58s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Starting Android test.

[2022-06-16T16:42:20.461Z]   4m 13s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Retrieving Performance Environment information from the device.

[2022-06-16T16:42:20.461Z]   4m 13s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Setting up Android test.

[2022-06-16T16:42:20.461Z]   4m 13s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Started crash detection.

[2022-06-16T16:42:20.461Z]   4m 13s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Started Out of memory detection

[2022-06-16T16:42:20.461Z]   4m 13s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Started performance monitoring.

[2022-06-16T16:42:20.461Z]   4m 13s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Starting Android test.

[2022-06-16T16:45:17.840Z]   7m 11s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Completed Android test.

[2022-06-16T16:45:27.962Z]   7m 21s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Stopped performance monitoring.

[2022-06-16T16:45:27.963Z]   7m 21s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Tearing down Android test.

[2022-06-16T16:45:58.529Z]   7m 51s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Stopped crash monitoring.

[2022-06-16T16:45:58.529Z]   7m 51s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Stopped logcat recording.

[2022-06-16T16:45:58.529Z]   7m 51s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Done. Test time = 238 (secs)

[2022-06-16T16:45:58.529Z]   7m 51s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Starting results processing. Attempt: 1

[2022-06-16T16:45:58.529Z]   7m 51s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Completed Android test.

[2022-06-16T16:46:09.252Z]   8m  2s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Completed Android test.

[2022-06-16T16:46:09.252Z]   8m  2s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Stopped performance monitoring.

[2022-06-16T16:46:09.252Z]   8m  2s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Tearing down Android test.

[2022-06-16T16:46:14.665Z]   8m  8s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Stopped performance monitoring.

[2022-06-16T16:46:14.665Z]   8m  8s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Tearing down Android test.

[2022-06-16T16:46:19.976Z]   8m 13s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 Completed results processing. Time taken = 17 (secs)

[2022-06-16T16:46:19.976Z]   8m 13s matrix-k7u2s8h0ewhza Pixel2-26 shard-2 execution-2ykxcnenvc6o2 FINISHED

[2022-06-16T16:46:30.400Z]   8m 23s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Completed Android test.

[2022-06-16T16:46:40.723Z]   8m 34s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Stopped performance monitoring.

[2022-06-16T16:46:40.723Z]   8m 34s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Tearing down Android test.

[2022-06-16T16:46:56.058Z]   8m 49s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Stopped crash monitoring.

[2022-06-16T16:46:56.058Z]   8m 49s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Stopped logcat recording.

[2022-06-16T16:47:01.270Z]   8m 54s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Done. Test time = 259 (secs)

[2022-06-16T16:47:01.270Z]   8m 54s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Starting results processing. Attempt: 1

[2022-06-16T16:47:11.392Z]   9m  4s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 Completed results processing. Time taken = 12 (secs)

[2022-06-16T16:47:11.392Z]   9m  4s matrix-k7u2s8h0ewhza Pixel2-26 execution-36jnexeoctl73 FINISHED

[2022-06-16T16:47:16.503Z]   9m  9s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Completed Android test.

[2022-06-16T16:47:21.614Z]   9m 15s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Stopped crash monitoring.

[2022-06-16T16:47:21.614Z]   9m 15s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Stopped logcat recording.

[2022-06-16T16:47:26.825Z]   9m 20s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Done. Test time = 258 (secs)

[2022-06-16T16:47:26.825Z]   9m 20s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Starting results processing. Attempt: 1

[2022-06-16T16:47:26.825Z]   9m 20s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Stopped performance monitoring.

[2022-06-16T16:47:26.825Z]   9m 20s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Tearing down Android test.

[2022-06-16T16:47:37.149Z]   9m 30s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y Completed results processing. Time taken = 11 (secs)

[2022-06-16T16:47:37.149Z]   9m 30s matrix-k7u2s8h0ewhza Pixel2-26 shard-3 execution-1qtq8nsffza3y FINISHED

[2022-06-16T16:48:02.507Z]   9m 55s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Stopped crash monitoring.

[2022-06-16T16:48:02.507Z]   9m 55s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Stopped logcat recording.

[2022-06-16T16:48:02.507Z]   9m 55s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Done. Test time = 283 (secs)

[2022-06-16T16:48:02.507Z]   9m 55s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Starting results processing. Attempt: 1

[2022-06-16T16:48:17.742Z]   10m 11s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe Completed results processing. Time taken = 13 (secs)

[2022-06-16T16:48:17.742Z]   10m 11s matrix-k7u2s8h0ewhza Pixel2-26 shard-1 execution-2m47tr9sw8nxe FINISHED

[2022-06-16T16:55:19.186Z]   17m 12s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Stopped crash monitoring.

[2022-06-16T16:55:19.186Z]   17m 12s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Stopped logcat recording.

[2022-06-16T16:55:19.186Z]   17m 12s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Done. Test time = 248 (secs)

[2022-06-16T16:55:19.186Z]   17m 12s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Starting results processing. Attempt: 1

[2022-06-16T16:55:29.408Z]   17m 22s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s Completed results processing. Time taken = 11 (secs)

[2022-06-16T16:55:29.408Z]   17m 22s matrix-k7u2s8h0ewhza Pixel2-26 shard-4 execution-35itechqor73s FINISHED

[2022-06-16T16:55:44.641Z]   17m 38s matrix-k7u2s8h0ewhza FINISHED

Answer 1 · 2022-06-22T14:23:39.000Z

@alexanderbezverhni Please post these matrix ids in #test-lab on the https://firebase.community/ Slack. The FTL team can use the matrix ids to investigate if the issue is in the server or on the client. This looks like a server issue to me.

21.11.0 is also an old version of flank. I'd suggest updating to the latest version of Flank so that the client libraries that communicate with FTL are up to date.

Answer 2 · 2022-06-22T15:52:21.000Z

@alexanderbezverhni Please post these matrix ids in #test-lab on the https://firebase.community/ Slack. The FTL team can use the matrix ids to investigate if the issue is in the server or on the client. This looks like a server issue to me.

21.11.0 is also an old version of flank. I'd suggest updating to the latest version of Flank so that the client libraries that communicate with FTL are up to date.

I appreciate your timely reply and suggestion!