Assertions on HTML (webview) not working
testcenter opened this issue · 0 comments
testcenter commented
Is there an existing issue for this?
- I have searched the existing issues and didn't find mine.
Steps to reproduce
The problem is related to maestro tests on Android.
The Bitmovin player uses a Webview with HTML/Javascript to visualize the video player UI elements.
- Clone the Bitmovin Samples Repo
- Build and install the BasicPlaybackKotlin example app
- You need to provide a key in the AndroidManifest which can be retrieved via creating a free account on the Bitmovin page
- Execute following config and flow via maestro
- the assertion
- assertVisible: "en"
fails- Asserting on a physical device works as expected
- Asserting on an emulator with
google_apis_playstore
works as expected - Asserting on an emulator with
google_apis
does not work - Asserting on maestro cloud fails. I assume, maestro cloud uses
maestro start-device
to create a device. It usesgoogle_apis
, see DeviceConfig
config.yaml
flows:
- "DisplaysSubtitles.yaml"
DisplaysSubtitles.yaml
appId: com.bitmovin.player.samples.playback.basic
name: "Playback: Displays subtitles"
---
- launchApp
# toggle settings
- tapOn:
text: "settings"
retryTapIfNoChange: false
- extendedWaitUntil:
visible: "subtitles"
timeout: 10000
# open subtitle menu
- tapOn: "Select subtitle"
# select subtitle "en"
- tapOn: "en"
# !!! following assertion fails !!!
- assertVisible: "en"
# close settings menu
- tapOn:
text: "settings"
retryTapIfNoChange: false
# start playing
- tapOn:
text: "Play/Pause"
retryTapIfNoChange: false
# subtitle must be visible
- assertVisible: "Welcome to bitmovin's bitdash!"
- stopApp
Actual results
The assertion - assertVisible: "en"
fails although visible. (inside a WebView
)
Using google_apis_playstore
instead of google_apis
would work
Expected results
Assertions behave the same on google_apis
and google_apis_playstore
(might be a general issue when parsing WebView
hierarchy of google_apis
vs google_apis_playstore
)
About app
- Android App
- open source sample app (link provided in the steps to reproduce)
- native app
- native framework (android views, xml)
WebView
(html + javascript) is used to visualize the player UI elements
About environment
java -version
openjdk version "17.0.12" 2024-07-16
OpenJDK Runtime Environment (build 17.0.12+7-Ubuntu-1ubuntu222.04)
OpenJDK 64-Bit Server VM (build 17.0.12+7-Ubuntu-1ubuntu222.04, mixed mode, sharing)
uname -a
Linux ubuntu 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
please note, the problem also happens on maestro cloud!
Logs
Logs
Waiting for flows to complete...
[Failed] Playback: Displays subtitles (35s) (Assertion is false: "en" is visible)
1/1 Flow Failed
Maestro version
1.38.1
How did you install Maestro?
install script (https://get.maestro.mobile.dev)
Anything else?
No response