A Comparative Study on the Energy Consumption of Progressive Web Apps

Project setup

The project is using Python 3
Inside the project folder python -m venv venv or virtualenv venv creates a virtual environment
./venv/Scripts/activate activates the virtual environment (Windows 10)
Install all project dependencies with pip install -r requirements.txt within an active virtual environment, the project dependencies are mainly required for the analysis and result preparation
For running the battery tests real Android devices are required, which enable the on-device developer options
For connecting and controlling the Android devices the Android Debug Bridge (adb) must be installed on the controlling computer

Repository content

sample app implementations: Source code of the apps under test and instrumentation
- instrumentation: UI Automator Android app
- contactapp-native: Android native implementation of app
- contactapp-react-native: React native implementation of app
- contactapp-stencil: Stencil/Ionic/Capacitor implementation of app
- contactapp-flutter: Flutter implementation of app
- contactapp-webviewwrapper: PWA wrapped inside the default Android WebView
- contactapp-geckowrapper: PWA wrapped inside the Gecko WebView
data: raw data measured with the batterystats service, the sample_apps subfolder contains the sample app measurements and the real_world_apps supfolder contains the real-world app measurements
figures: generated figures for paper
helper: python helper functions, e.g. for using adb or batterystats
Analysis.ipynb: Notebook with statistical analysis and generation of figures
connect.py: Script to try to establish Wi-Fi connection with USB-connected device
prepare.py: Setup script for preparing a connected device
start_battery_test.py: Script to start the complete battery test for sample apps on a connected device
start_single_battery_test.py: Script to start a single app battery test for a sample app on a connected device
start_scroll_battery_test.py: Script to start a single app battery test for a real-world app on a connected device

Setup for battery tests

Connect device via Wi-Fi

connect mobile device via USB cable to computer
get IP-address form mobile device: adb shell ip address and search for wlan0
enable port 5555 for tcp/ip connection mode adb tcpip 5555 on mobile device
connect to mobile device via Wi-Fi: adb connect "IP-ADDRESS":5555 (adb connect 192.168.2.100:5555)
unplug USB cable from mobile device

The connect.py script automatizes this procedure

Prepare device for testing sample apps

Sample apps

The sample apps and instrumentation project are compiled, signed and put into the apk folder. To install all apps inside the apk folder run the prepare.py script. This requires an adb connected device. The .apk files can be downloaded compiled and signed from the GitHub releases.

> python prepare.py

In order to test the apps, test entries must be created inside the sample apps.

Sample PWA

The PWA is deployed here and must be installed with Chrome and Firefox on the respective device.

Run battery test for sample apps

This requires that the device is connected via Wi-Fi and the IP-ADDRESS is known. See Connect adb via Wi-Fi above for connecting to device. To start the battery test for the sample apps the following command can be used (IP-ADDRESS must be substituted with the respective ip address of the device, -n states the number of test runs, which should be executed).

> python start_battery_test.py -i IP-ADDRESS -n 30

Prepare device for testing real-world apps

Real-world apps

The following real-world apps must be installed on the respective test device:

App	Development
NewPipe	Android Native
F-Droid	Android Native
Tiny Weather Forcast	Android Native
Ulangi	React Native
Bus Timetable	React Native
YumMeals	React Native
Flutter Ebook App	Flutter
Metal Releases	Flutter
No Mans sky Recipes	Flutter
Aliexpress	PWA
Versus	PWA
Swiggy	PWA
Sworkit	Capacitor
JustWatch	Capacitor
Untappd	Apache Cordova

App screens

For each app the respective app screen must be opened, before the test run is started:

Untappd: Top rated beers without filters
MetalReleases: Upcoming
JustWatch: Popular
Assistant for No Man's Sky: All Recipies
Flutter Ebook App: Explore > Short Stories
Swiggy: View all Restaurants
Swörkit: Exercise Library
letgo: Electronics
Ulangi: Spanish - English, Animals
Bus Timetable: Lines
AliExpress: Search something
YumMeals: Open in Expo Go, Search San Francisco
F-Droid: Category Internet
NewPipe: Trending
Tiny Weather Forcast: Just open

Real-world app sources

The following sources were used to collect the real-world apps:

For decompiling an app, the .apk file was downloaded from APKMirror or a similar website and decompiled with the Apktool.

Run battery test for real-world apps

> python start_scroll_battery_test.py -i IP-ADDRESS -n 30 -a aliexpress

Result analysis

All analysis steps and the result preparation of the energy measurements can be found inside the accompanying Analysis.ipynb Jupyter notebook.

Inside an active virtual environment the notebook can be run with jupyter notebook. Inside Jupyter overview open the Analysis.ipynb notebook.

stefanhuber/pwa-energy-comparison