Android UI testing utils

A set of TestRules, ActivityScenarios and utils to facilitate UI & screenshot testing under certain configurations, independent of the UI testing framework you are using.

For screenshot testing, it supports Jetpack Compose, android Views (e.g. custom Views, ViewHolders, etc.), Activities and Fragments.

Currently, with this library you can easily change the following configurations in your instrumented tests:

Locale (also Pseudolocales en_XA & ar_XB)
1. App Locale (i.e. per-app language preference)
2. System Locale
Font size
Orientation
Custom themes
Dark mode /Day-Night mode
Display size

You can find out why verifying our design under such configurations is important in this blog post:

Design a pixel perfect Android app 🎨

For examples of usage of this library in combination with Shot and Dropshots, check the following repo:

Android screenshot testing playground

In the near future, there are plans to also support, among others:

framework-agnostic & shared screenshot testing i.e. same test running either on device or on JVM
Reduce snapshot testing flakiness
Folding features
Accessibility features

Integration

Add jitpack to your root build.gradle file:

allprojects {
    repositories {
        maven { url 'https://jitpack.io' }
    }
}

Set the comple SdkVersion and add a dependency to build.gradle

compileSdkVersion 33

dependencies {
    androidTestImplementation('com.github.sergio-sastre:AndroidUiTestingUtils:1.2.3') {
        // if necessary, add this to avoid compose version clashes
        exclude group: 'androidx.compose.ui'
    }
    // add this if excluding 'androidx.compose.ui' due to compose version clashes 
    androidTestImplementation "androidx.compose.ui:ui-test-junit4:your_compose_version"
}

Usage

Configuration

ActivityScenarios

Add the following permission and activities to your debug/manifest

<!-- Required for ActivityScenarios only -->
<application...
    <activity
        android:name="sergio.sastre.uitesting.utils.activityscenario.ActivityScenarioConfigurator$PortraitSnapshotConfiguredActivity" 
    />
    <activity
        android:name="sergio.sastre.uitesting.utils.activityscenario.ActivityScenarioConfigurator$LandscapeSnapshotConfiguredActivity"
        android:screenOrientation="landscape" 
    />
...
</application>

Pseudolocales

To enable pseudolocales en_XA & ar_XB for your screenshot tests, add this to your build.gradle.

android {
    ...
    buildTypes {
        ...
        debug {
            pseudoLocalesEnabled true
        }
    }
}

System Locale

To change the System Locale , you also need to add the following permission to your debug/manifest. For multi-module apps, do this in the app module.

<!-- Required to change the Locale via SystemLocaleTestRule (e.g. for snapshot testing Activities) -->
<uses-permission android:name="android.permission.CHANGE_CONFIGURATION"
    tools:ignore="ProtectedPermissions" />

App Locale

To change the App Locale, which is possible via LocaleTestRule, you need to add the following dependency in your app/build.gradle

androidTestImplementation 'androidx.appcompat:appcompat:1.6.0-alpha04' //or higher version!

Warning: LocaleTestRule does ONLY work with ActivityScenarioConfigurator.ForActivity(), i.e. it does not work with ActivityScenarioForActivityRule. However, for Fragments, Views and Composables call the setLocale("my_locale") of their corresponding Fragment/ActivityScenarioConfigurator or the ConfigItem(locale = "myLocale") of their corresponding TestRule e.g. to achieve it:

ActivityScenarioConfigurator.ForView().setLocale("my_locale")

@get:Rule
val rule =
    ActivityScenarioForViewRule(
        config = ViewConfigItem(locale = "my_locale")
    )

Screenshot testing examples

The examples use pedrovgs/Shot. It also works with any other on-device screenshot testing library, like Facebook screenshot-tests-for-android, Dropbox Dropshots or with a custom screenshot testing solution.

You can find more complete examples with Shot and Dropshots in the Android screenshot testing playground repo.

Activity

The simplest way is to use the ActivityScenarioForActivityRule, to avoid the need for closing the ActivityScenario.

@get:Rule
val rule =
    activityScenarioForActivityRule<MyActivity>(
        config = ActivityConfigItem(
            orientation = Orientation.LANDSCAPE,
            uiMode = UiMode.NIGHT,
            fontSize = FontSize.HUGE,
            systemLocale = "en",
            displaySize = DisplaySize.LARGEST,
        )
    )

@Test
fun snapActivityTest() {
    compareScreenshot(
        activity = rule.activity,
        name = "your_unique_test_name",
    )
}

In case you don't want to/cannot use the rule, you can use ActivityScenarioConfigurator.ForActivity() directly in the test. Currently, this is the only means to set

A TimeOut for the FontSize and DisplaySize TestRules
A LocaleTestRule for per-app language preferences

Apart from that, this would be equivalent:

// Sets the Locale of the app under test only, i.e. the per-app language preference feature
@get:Rule
val locale = LocaleTestRule("ar")

// Sets the Locale of the Android system
@get:Rule
val systemLocale = SystemLocaleTestRule("en")

@get:Rule
val fontSize = FontSizeTestRule(FontSize.HUGE).withTimeOut(inMillis = 15_000) // default is 10_000

@get:Rule
val displaySize = DisplaySizeTestRule(DisplaySize.LARGEST).withTimeOut(inMillis = 15_000)

@get:Rule
val uiMode = UiModeTestRule(UiMode.NIGHT)

@Test
fun snapActivityTest() {
    // Custom themes are not supported
    // AppLocale, SystemLocale, FontSize & DisplaySize are only supported via TestRules for Activities
    val activityScenario = ActivityScenarioConfigurator.ForActivity()
        .setOrientation(Orientation.LANDSCAPE)
        .launch(MyActivity::class.java)

    val activity = activityScenario.waitForActivity()

    compareScreenshot(activity = activity, name = "your_unique_test_name")

    activityScenario.close()
}

Android View

The simplest way is to use the ActivityScenarioForViewRule, to avoid the need for closing the ActivityScenario.

@get:Rule
val rule =
    ActivityScenarioForViewRule(
        config = ViewConfigItem(
            fontSize = FontSize.NORMAL,
            locale = "en",
            orientation = Orientation.PORTRAIT,
            uiMode = UiMode.DAY,
            theme = R.style.Custom_Theme,
            displaySize = DisplaySize.SMALL,
        )
    )

@Test
fun snapViewHolderTest() {
    // IMPORTANT: The rule inflates a layout inside the activity with its context to inherit the configuration 
    val layout = rule.inflateAndWaitForIdle(R.layout.your_view_holder_layout)

    // wait asynchronously for layout inflation 
    val viewHolder = waitForMeasuredViewHolder {
        YourViewHolder(layout).apply {
            // bind data to ViewHolder here
            ...
        }
    }

    compareScreenshot(
        holder = viewHolder,
        heightInPx = viewHolder.itemView.height,
        name = "your_unique_test_name",
    )
}

In case you don't want to/cannot use the rule, you can use ActivityScenarioConfigurator.ForView(). This would be its equivalent:

// example for ViewHolder
@Test
fun snapViewHolderTest() {
    val activityScenario =
        ActivityScenarioConfigurator.ForView()
            .setFontSize(FontSize.NORMAL)
            .setLocale("en")
            .setInitialOrientation(Orientation.PORTRAIT)
            .setUiMode(UiMode.DAY)
            .setTheme(R.style.Custom_Theme)
            .setDisplaySize(DisplaySize.SMALL)
            .launchConfiguredActivity()

    val activity = activityScenario.waitForActivity()

    // IMPORTANT: To inherit the configuration, inflate layout inside the activity with its context 
    val layout = activity.inflateAndWaitForIdle(R.layout.your_view_holder_layout)

    // wait asynchronously for layout inflation 
    val viewHolder = waitForMeasuredViewHolder {
        YourViewHolder(layout).apply {
            // bind data to ViewHolder here
            ...
        }
    }

    compareScreenshot(
        holder = viewHolder,
        heightInPx = viewHolder.itemView.height,
        name = "your_unique_test_name",
    )

    activityScenario.close()
}

If the View under test contains system Locale dependent code, like NumberFormat.getInstance(Locale.getDefault()), the Locale formatting you've set via ActivityScenarioConfigurator.ForView().setLocale("my_locale") will not work. That's because NumberFormat is using the Locale of the Android system, and not that of the Activity we've configured. Beware of using instrumenation.targetContext in your tests when using getString() for the very same reason: use Activity's context instead.
To solve that issue, you can do one of the following:

Use NumberFormat.getInstance(anyViewInsideActivity.context.locales[0]) in your production code.
Use SystemLocaleTestRule("my_locale") in your tests instead of ActivityScenarioConfigurator.ForView().setLocale("my_locale").

Jetpack Compose

The simplest way is to use the ActivityScenarioForComposableRule, to avoid the need for:

calling createEmptyComposeRule()
closing the ActivityScenario.

@get:Rule
val rule = ActivityScenarioForComposableRule(
        config = ComposableConfigItem(
            fontSize = FontSize.SMALL,
            locale = "de",
            uiMode = UiMode.DAY,
            displaySize = DisplaySize.LARGE,
            orientation = Orientation.PORTRAIT,
        )
    )

@Test
fun snapComposableTest() {
    rule.activityScenario
        .onActivity {
            it.setContent {
                AppTheme { // this theme must use isSystemInDarkTheme() internally
                    yourComposable()
                }
            }
        }

    compareScreenshot(
        rule = rule.composeRule,
        name = "your_unique_test_name",
    )
}

In case you don't want to/cannot use the rule, you can use ActivityScenarioConfigurator.ForComposable() together with createEmptyComposeRule(). This would be its equivalent:

// needs an EmptyComposeRule to be compatible with ActivityScenario
@get:Rule
val composeTestRule = createEmptyComposeRule()

@Test
fun snapComposableTest() {
    val activityScenario = ActivityScenarioConfigurator.ForComposable()
        .setFontSize(FontSize.SMALL)
        .setLocale("de")
        .setInitialOrientation(Orientation.PORTRAIT)
        .setUiMode(UiMode.DAY)
        .setDisplaySize(DisplaySize.LARGE)
        .launchConfiguredActivity()
        .onActivity {
            it.setContent {
                AppTheme { // this theme must use isSystemInDarkTheme() internally
                    yourComposable()
                }
            }
        }

    activityScenario.waitForActivity()

    compareScreenshot(rule = composeTestRule, name = "your_unique_test_name")

    activityScenario.close()
}

If you are using a screenshot library that cannot take a composeTestRule as argument (e.g. Dropshots), you can still screenshot the Composable as follows:

// with ActivityScenarioForComposableRule
dropshots.assertSnapshot(
   view = activityScenarioForComposableRule.activity.waitForComposeView(),
   name = "your_unique_test_name",
)

// withm ActivityScenarioConfigurator.ForComposable()
val activityScenario =
   ActivityScenarioConfigurator.ForComposable()
        ...
        .launchConfiguredActivity()
        .onActivity {
            ...
        }

dropshots.assertSnapshot(
   view = activityScenario.waitForActivity().waitForComposeView(),
   name = "your_unique_test_name",
)

If the Composable under test contains system Locale dependent code, like NumberFormat.getInstance(Locale.getDefault()), the Locale formatting you've set via ActivityScenarioConfigurator.ForComposable().setLocale("my_locale") will not work. That's because NumberFormat is using the Locale of the Android system, and not that of the Activity we've configured, which is applied to the LocaleContext of our Composables.
To solve that issue, you can do one of the following:

Use NumberFormat.getInstance(LocaleContext.current.locales[0]) in your production code.
Use SystemLocaleTestRule("my_locale") in your tests instead of ActivityScenarioConfigurator.ForComposable().setLocale("my_locale").

Fragment

The simplest way is to use the FragmentScenarioConfiguratorRule

@get:Rule
val rule = fragmentScenarioConfiguratorRule<MyFragment>(
        fragmentArgs = bundleOf("arg_key" to "arg_value"),
        config = FragmentConfigItem(
            orientation = Orientation.LANDSCAPE,
            uiMode = UiMode.DAY,
            locale = "de",
            fontSize = FontSize.SMALL,
            displaySize = DisplaySize.SMALL,
            theme = R.style.Custom_Theme,
        ),
    )

@Test
fun snapFragment() {
    compareScreenshot(
        fragment = rule.fragment,
        name = "your_unique_test_name",
    )
}

In case you don't want to/cannot use the rule, you can use the plain FragmentScenarioConfigurator. This would be its equivalent:

@Test
fun snapFragment() {
    val fragmentScenarioConfigurator =
        FragmentScenarioConfigurator
            .setInitialOrientation(Orientation.LANDSCAPE)
            .setUiMode(UiMode.DAY)
            .setLocale("de")
            .setFontSize(FontSize.SMALL)
            .setDisplaySize(DisplaySize.LARGE)
            .setTheme(R.style.Custom_Theme)
            .launchInContainer<MyFragment>(
                fragmentArgs = bundleOf("arg_key" to "arg_value"),
            )

    compareScreenshot(
        fragment = fragmentScenarioConfigurator.waitForFragment(),
        name = "your_unique_test_name",
    )

    fragmentScenarioConfigurator.close()
}

Bitmap

Most screenshot testing libraries use Canvas with Bitmap.Config.ARGB_8888 as default for generating bitmaps (i.e. the screenshots) from the Activities/Fragments/ViewHolders/Views/Dialogs/Composables... That's because Canvas is supported in all Android versions.
Nevertheless, such bitmaps generated using Canvas have some limitations, e.g. UI elements are rendered without considering elevation.

Fortunately, such libraries let you pass the bitmap (i.e.the screenshot) as argument in their record/verify methods. In doing so, we can draw the views with elevation to a bitmap with PixelCopy.

AndroidUiTestingUtils provides methods to easily generate bitmaps from the Activities/Fragments/ViewHolders/Views/Dialogs/Composables:

drawToBitmap(config = Bitmap.Config.ARGB_8888) -> uses Canvas under the hood
drawToBitmapWithElevation(config = Bitmap.Config.ARGB_8888) -> uses PixelCopy under the hood

Differences between both might be specially noticeable in API 31:

Note
If using PixelCopy with ViewHolders/Views/Dialogs/Composables, consider launching the container Activity with transparent background for a more realistic screenshot of the UI component.
ActivityScenarioConfigurator.ForView() // or .ForComposable()
      ...
      .launchConfiguredActivity(backgroundColor = Color.TRANSPARENT)
or
ActivityScenarioForViewRule( // or ActivityScenarioForComposableRule()
      viewConfig = ...,
      backgroundColor = Color.TRANSPARENT,
)
Otherwise it uses the default Dark/Light Theme background colors (e.g. white and dark grey).

Using PixelCopy instead of Canvas comes with its own drawbacks though. In general, don't use PixelCopy to draw views that don't fit on the screen.

Canvas	PixelCopy
✅ Can render elements beyond the screen, e.g. long ScrollViews	❌ Cannot render elements beyond the screen, resizing if that's the case
❌ Ignores elevation¹ of UI elements while drawing	✅ Considers elevation¹ of UI elements while drawing

¹ Elevation can be manifested in many ways: a UI layer on top of another or a shadow in a CardView.

And using PixelCopy in your screenshot tests is as simple as this (example with Shot):

// for UI Components like Activities/Fragments/ViewHolders/Views/Dialogs
compareScreenshot(
    bitmap = uiComponent.drawToBitmapWithElevation(),
    name = "your_unique_test_name",
)

// for Composables
compareScreenshot(
    bitmap = activity.waitForComposableView().drawToBitmapWithElevation(),
    name = "your_unique_test_name",
)

Utils

Wait

waitForActivity: Analog to the one defined in pedrovgs/Shot. It's also available in this library for compatibility with other screenshot testing frameworks like Facebook screenshot-tests-for-android .
waitForFragment: Analog to waitForActivity but for Fragment.
activity.waitForComposeView: Returns the root Composable in the activity as a ComposeView. You can call later drawToBitmap or drawToBitmapWithElevation on it to screenshot test its corresponding bitmap.
waitForMeasuredView/Dialog/ViewHolder(exactWidth, exactHeight): Inflates the layout in the main thread, sets its width and height to those given, and waits till the thread is idle, returning the inflated view. Comes in handy with libraries that do not support, to take a screenshot with a given width/height, like Dropshots.

Warning
Prefer waitForMeasuredView over waitForView (which is discouraged), specially if using Dropshots:

Inflate or measure
5. activity.inflate(R.layout_of_your_view): Use it to inflate android Views with the activity's context configuration. In doing so, the configuration becomes effective in the view. It also adds the view to the Activity's root.

6. activity.inflateAndWaitForIdle(R.layout_of_your_view): Like activity.inflate, but waits till the view is Idle to return it. Do not wrap it with waitForMeasuredView{} or it will throw an exception.

7. MeasureViewHelpers: Analog to the ViewHelpers defined in Facebook screenshot-tests-for-android. In most cases, you don't need to use it directly but via waitForMeasuredView(exactWidth, exactHeight), which calls MeasuredViewHelpers under the hood.

Reading on screenshot testing

Standard UI testing

For standard UI testing, you can use the same approach as for snapshot testing Activities. The following TestRules and methods are provided:

// Sets the Locale of the app under test only, i.e. the per-app language preference feature
@get:Rule
val locale = LocaleTestRule("en")

// Sets the Locale of the Android system
@get:Rule
val systemLocale = SystemLocaleTestRule("en")

@get:Rule
val fontSize = FontSizeTestRule(FontSize.HUGE).withTimeOut(inMillis = 15_000) // default is 10_000

@get:Rule
val displaySize = DisplaySizeTestRule(DisplaySize.LARGEST).withTimeOut(inMillis = 15_000)

@get:Rule
val uiMode = UiModeTestRule(UiMode.NIGHT)

activity.rotateTo(Orientation.LANDSCAPE)

Warning
When using DisplaySizeTestRule and FontSizeTesRule together in the same test, make sure your emulator has enough RAM and VM heap to avoid Exceptions when running the tests. The recommended configuration is the following:

RAM: 4GB

VM heap: 1GB

Code attributions

This library has been possible due to the work others have done previously. Most TestRules are based on code written by others:

SystemLocaleTestRule -> Screengrab
FontSizeTestRule -> Novoda/espresso-support
UiModeTestRule -> AdevintaSpain/Barista
Orientation change for activities -> Shopify/android-testify

Contributing

Create an issue in this repo
Fork the repo Android screenshot testing playground
In that repo, add an example and test where the bug is reproducible/ and showcasing the new feature.
Once pushed, add a link to the PR in the issue created in this repo and add @sergio-sastre as a reviewer.
Once reviewed and approved, create an issue in this repo.
Fork this repo and add the approved code from the other repo to this one (no example or test needed). Add @sergio-sastre as a reviewer.
Once approved, I will merge the code in both repos, and you will be added as a contributor to Android UI testing utils as well as Android screenshot testing playground .

I'll try to make the process easier in the future if I see many issues/feature requests incoming :)

Android UI testing utils logo modified from one by Freepik - Flaticon

tochukwumunonye/AndroidUiTestingUtilscompose