Screenshots are output in wrong width on Moto G7 Power

[dbjorge]

Describe the bug

On some devices (I saw this on a Moto G7 Power), the screenshot image data returned from the service includes some padding on the right side that shouldn't be there:

The root cause of this is our logic for copying data from the Image that we get back from the screenshot-grabbing routines into a Bitmap suitable for passing as output from the service. Specifically, our logic is a little off for cases where the Image we are given has a getRowStride() different from its getWidth().

Here's the code in question:

accessibility-insights-for-android-service/AccessibilityInsightsForAndroidService/app/src/main/java/com/microsoft/accessibilityinsightsforandroidservice/OnScreenshotAvailable.java

Lines 40 to 55 in cada52f

	private Bitmap getBitmapFromImage(Image image) {
	Image.Plane[] imagePlanes = image.getPlanes();
	int bitmapWidth = getBitmapWidth(image, imagePlanes);
	Bitmap screenshotBitmap =
	bitmapProvider.createBitmap(bitmapWidth, metrics.heightPixels, Bitmap.Config.ARGB_8888);
	ByteBuffer buffer = imagePlanes[0].getBuffer();
	screenshotBitmap.copyPixelsFromBuffer(buffer);
	return screenshotBitmap;
	}
	private int getBitmapWidth(Image image, Image.Plane[] imagePlanes) {
	int pixelStride = imagePlanes[0].getPixelStride();
	int rowStride = imagePlanes[0].getRowStride();
	int rowPadding = rowStride - pixelStride * metrics.widthPixels;
	return image.getWidth() + rowPadding / pixelStride;
	}

The issue occurs when rowStride is larger than the image width (ie, there is some empty padding in the raw pixel data at the end of each row). Our code is correctly accounting for the extra padding when it initially creates the the Bitmap, such that the Bitmap's backing data buffer will be sized appropriately to receive the original data including the extra padding. However, the resulting Bitmap doesn't understand that width != stride, so it treats the extra padding as part of the image; that empty padding is what you're seeing in the top screenshot (the padding is actually transparent pixels).

Unfortunately, Bitmap::copyPixelsFromBuffer doesn't have a variant that accepts a stride parameter. Bitmap::createBitmap and Bitmap::setPixels have variants that accept strides, but only in combination with other bitmaps or int[] data, and the buffer we get back from the original Image is a "direct" buffer that doesn't allow array-like to its underlying buffer array. We may need to either do an intermediate copy to our own array to use with createBitmap, or write our own stride-aware version copyPixelsFromBuffer.

To Reproduce
Steps to reproduce the behavior:

1. Install the service on a Moto G7 Power
   note: we aren't sure whether it's the specific device or the specific display metrics that trigger Android to output the Image properties required for the repro; if you don't have access to this specific device, you might try creating an emulator with the same OS level (9 / Pie) and same DisplayMetrics (720x1520 320dpi)
2. Use Accessibility Insights for Android (prod or dev are both fine) to run a scan
3. Observe the extra padding and wrongly-offset failure highlighting

Expected behavior

No extra padding, failure highlights line up with elements

Context (please complete the following information)

• Android Version: 9 (Pie)
• Service Version & Environment: Repros in 1.1.0 and the 1.2.0 release candidate
• Target Application: Any

[Deleted user]

This issue has been marked as ready for team triage; we will triage it in our weekly review and update the issue. Thank you for contributing to Accessibility Insights!