sterzhang/image-textualization

Image Textualization: An Automatic Framework for Generating Rich and Detailed Image Descriptions (NeurIPS 2024)

Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions

News🔥

• Fine-grained annotation can be found in 🤗Huggingface.
• Dataset can be found in 🤗Huggingface, which contain 219,437 image descriptions. Link to our paper: arxiv.

Contents

• Install
• Datasets
• Use
• Visualization

Install

See detailed instructions in install.md.

Datasets

Images

• COCO: Download here train2017.
• SAM: Click here SAM (sa_000000.tar ~ sa_000024.tar).
• VG: Click here VG.

After downloading, organize the image datasets as follows in ./dataset/:

├── coco
│   └── train2017
├── sam
    └── images
├── vg

Use

After install all the requirements, you can follow use.md to generate description on your datasets.

Visualization

Acknowledgement

If you find our work useful for your research or applications, please cite using this BibTeX:

@misc{pi2024image,
      title={Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions}, 
      author={Renjie Pi and Jianshu Zhang and Jipeng Zhang and Rui Pan and Zhekai Chen and Tong Zhang},
      year={2024},
      eprint={2406.07502},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}