/DroidBot-GPT

Automating Android apps with ChatGPT-like LLM.

Primary LanguagePythonMIT LicenseMIT

DroidBot-GPT demo

DroidBot-GPT

About

DroidBot-GPT is a GPT-powered UI automator for Android. It can control an app automatically based on a natural language task description.

It is built upon DroidBot with ChatGPT-style APIs.

Technical report:

DroidBot-GPT: Hao Wen, Hongming Wang, Jiaxuan Liu, Yuanchun Li. "DroidBot-GPT: GPT-powered UI Automation for Android"

DroidBot: Li, Yuanchun, et al. "DroidBot: a lightweight UI-guided test input generator for Android." In Proceedings of the 39th International Conference on Software Engineering Companion (ICSE-C '17). Buenos Aires, Argentina, 2017.

How to install

Make sure you have:

  1. Python (both 2 and 3 are supported)
  2. Java
  3. Android SDK
  4. Added platform_tools directory in Android SDK to PATH

Then clone this repo and install with pip:

git clone https://github.com/honeynet/droidbot.git
cd droidbot/
pip install -e .

If successfully installed, you should be able to execute droidbot -h.

How to use

  1. Prepare:

    • An app to use. Download the .apk file to your host machine.
    • A device or an emulator connected to your host machine via adb.

  2. Start DroidBot:

    droidbot -a <path_to_apk> -o output_dir -task <your_task>

    That's it! The options are mostly the same as DroidBot except for the new -task option, where you can specify any task you want DroidBot-GPT to complete. For example,

    • Create a contact named Yuanchun with phone number 1234567.
    • Book a table for 4 people on Saterday.
    • Send a message to OpenAI to open their AI.
    • ...

Note that DroidBot-GPT is currently for research purpose only. It may perform unintended actions. Please use at your own risk.

Enjoy!