google/mobly-bundled-snippets

Document a test strategy for new snippets

yschimke opened this issue · 1 comments

yschimke commented

With this PR #158, there is a question "What are the breakages from bumping the targetSDK?"

It would be good to have a pattern for new snippets at least, to demonstrate that the snippet works on some particular device.

Useful when

  • changing the code of the snippet
  • changing the build such as targetSdk
  • testing against some new device
yschimke commented

Question from @xpconanfan

How you would confirm the API actually behaved though?
It sounds like end to end or integration tests? Who would provide the HW to run those on?

run on emulator (GMD) to test or FTL devices?

GMD here https://android-developers.googleblog.com/2021/10/whats-new-in-scalable-automated-testing.html

Similar integration with FTL https://developer.android.com/studio/preview/features#ftl-gmd

As for what to test, I think that depends on the snippet, it might be enough to test against known state of the emulator.