This repository contains experimental code showcasing the usage of GPT-4V (a powerful language model) in combination with PyAutoGUI (a library for controlling the mouse and keyboard) to automate tasks. It is important to note that this code is purely experimental and should be used with caution. This project is part of the research conducted in Simular AI, aiming to understand the frontier of existing foundation models in computer control.
Using automation tools can be potentially dangerous if not used responsibly. Please be aware of the following risks and guidelines before attempting to use this code:
- Unauthorized access: Automating tasks can involve interacting with sensitive information or systems. Ensure that you have proper authorization and permissions before using this code on any platform or application.
- Data loss or corruption: Incorrect usage of automation tools can result in data loss or corruption. Carefully review and test the code before running it on any important files or data.
- Unintended consequences: Automating tasks can sometimes lead to unintended consequences. Always verify the behavior of the code and ensure that it performs the intended actions.
- System instability: Depending on the complexity of the automated tasks, there is a risk of causing system instability or crashes. This can lead to loss of work or damage to the system. Use this code in controlled environments and be prepared to handle any unforeseen issues.
- Legal implications: Some actions performed through automation may violate the terms of service or laws of certain platforms or jurisdictions. Ensure that you comply with all applicable rules and regulations before running this code.
To use this experimental code, follow the steps below:
- Clone the repository to your local machine.
- Install the required dependencies using
pip install -r requirements.txt. - Export OpenAI API key
export OPENAI_API_KEY=<openai-api-key> - Run the code using
python app.py. - Write whatever concept you want to click in the terminal.
Please remember to review and modify the code according to your specific use case and requirements. It is highly recommended to thoroughly test the code in a controlled environment before using it in any production or critical systems.
Contributions to this experimental codebase are welcome. If you discover any issues, have suggestions, or would like to add new features, please open an issue or submit a pull request.
This repository is licensed under the Apache 2.0. Please review the license file for more details.
The code provided in this repository is experimental in nature and may not be suitable for all use cases. The authors and contributors of this code are not responsible for any damages or losses incurred through the use of this code. Use it at your own risk.
Always exercise caution and follow best practices when using automation tools.