👀🧠GPT-4 Vision x 💪⌨️Vimium = Autonomous Web Agent
Demo Video: GPT-4V finds a cute cat image through Chrome.
This project leverages GPT4V to create an autonomous / interactive web agent. The action space are discretized by Vimium.
It's a minimal proof-of-concept project at the current stage and performance can be furhter improved with more engineering.
Install
# For Mac users
brew install chromedriver
# run `brew upgrade chromedriver` if you already have it
pip install git+https://github.com/Jiayi-Pan/GPT-V-on-Web.gitRun
webai
# or if you want start at a specific website
webai --start_link "https://www.google.com"