What can be said can be solved.
Get early access to the PTA model | Scale on our shoulders
Intent-Pilot is an orchestration of two tools: AskUI's object detector with OpenAI's GPT-4v to achieve automation. It is designed to automate repetitive tasks, and to assist users in performing complex tasks with ease. This repository is our attempt to understand the GPT-4v's potential in automation and building an end-to-end automation tool.
We are inspired by and improve upon Self-Operating-Computer by a more accurate object detection model and an improved prompting strategy. We also provide a more user-friendly interface, and a more intuitive way to interact with the tool. For example, the notification feature to let users know what is happening and what to do next. Also, our tool works across all keyboard layouts - US, German, etc, which was one of the limitations of similar tools.
demo-eco.mp4
- Python 3.9 or later
- OpenAI Key
- AskUI token
- For staters, we provide a global AskUI key with 4000 free credits.
ASKUI_WORKSPACE_ID
: 'e26b43ea-a18f-4cd8-a6f1-e1e41dddec18' andASKUI_TOKEN
: 'gbwdeDjlWBrsr8nLjOVB' - After that, you can get your own AskUI key by signing up at AskUI
- For staters, we provide a global AskUI key with 4000 free credits.
- You can also copy the .env.example file to .env and fill in the required details OR You can enter the credentials in the terminal when you start the app.
- In case of linux, you may need to install the following packages:
sudo apt-get install xsel xclip python3-tk python3-dev
- In case of MacOS, you will have to grant permissions to the terminal to access the clipboard. You can do this by going to
System Preferences
->Security & Privacy
->Privacy
->Accessibility
and then adding the terminal to the list of apps that can control your computer.
We are currently working on the Windows version of the tool. It will be available soon.
Quick Fix: The package also works on Windows but the Windows Defender is deleting the src/intent_pilot/utils/screenshot.py
file. You have to restore the file from the quarantine and add it to the exclusion list.
pip install intent-pilot
After installation, simply run intent
in your terminal:
intent
In case, you are unable to run the command, try running the following command:
python -m intent_pilot
For real-time discussions and community support, join our Discord server:
- Join our Discord Server and then navigate to the #intent-pilot channel.
Thank you for your interest in contributing! We welcome involvement from the community.
We are still deciding on the contribution guidelines. Please stay tuned for updates.
We are currently in the process of building PTA (Prompt-to-Automation) model, a Multi-Modal Model that can understand and execute commands in natural language, in real-time and faster than any VPA (Virtual Personal Assistant) in the market.