[Project Page] [Paper] [Code] [HuggingFace] [OpenDataLab]
JourneyDB is a large-scale generated image understanding dataset that contains 4,429,295 high-resolution Midjourney images, annotated with corresponding text prompt, image caption and visual question answering.
JourneyDB supports 4 downstream tasks, i.e. Prompt Inversion, Style Retrieval, Image Caption, and Visual Question Answering. We evaluate many existing methods on these tasks and provide a comprehensive benchmark. Please see our Paper for more details.
For each image instance, we acquire the corresponding text prompts used to generate the images with Midjourney. Furhtermore, we employ the GPT3.5 to generate the caption and VAQ groundtruth.
We provide several examples to show the contents of each instance of the dataset.
We provide detailed statistics for each split subset in the following table. We randomly split the whole dataset into roughly 20 : 1 to obtain the training and validation set. The training set contains 4,189,737 labeled images and 1,385,317 labeled prompts. The validation set contains 235,156 images and 82,093 prompts. And we additionally sample a testing set for manual filtering. The testing set contains 5,402 images and 5,171 prompts.
Image | Prompt | Labeled Image | Labeled Prompt | Style QA | Content QA | |
---|---|---|---|---|---|---|
Training Set | 4,453,193 | 1,643,375 | 4,189,737 | 1,385,317 | 7,056,394 | 8,775,971 |
Validation Set | 234,156 | 82,093 | 234,156 | 82,093 | 311,569 | 374,310 |
Testing Set | 5,402 | 5,171 | 5,402 | 5,171 | 10,040 | 11,369 |
Total | 4,692,751 | 1,730,639 | 4,429,295 | 1,472,581 | 7,378,003 | 9,161,650 |
Please fill in the form to acquire the download link.
The JourneyDB dataset is available under the customized Terms of Usage.
@misc{pan2023journeydb,
title={JourneyDB: A Benchmark for Generative Image Understanding},
author={Junting Pan and Keqiang Sun and Yuying Ge and Hao Li and Haodong Duan and Xiaoshi Wu and Renrui Zhang and Aojun Zhou and Zipeng Qin and Yi Wang and Jifeng Dai and Yu Qiao and Hongsheng Li},
year={2023},
eprint={2307.00716},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Junting Pan*, Keqiang Sun*, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, Jifeng Dai, Yu Qiao, Hongsheng Li+
(* equal contribution, + corresponding author)
If you have any problem or suggestion, please feel free to open an issue or send emails to the contributors.
- Project Page
- Technical Report
- Release Dataset
- Release Dataset Tools
- Release Evaluation Scripts
- Release Pretrained Model
- Stable Diffusion Version