Fine tuning Donut transformers for UI Referring Expressions task
This repo contains fine tuning code and a playground for the UI Referring Expression task - using natural language to reference UI components in a screenshot.
- Fine tuning notebook: Fine_tune_Donut_on_UI_RefExp.ipynb.
- Inference playground: Inference_Playground_Donut_UI_RefExp_Gradio.ipynb