/FMNERG

Primary LanguagePython

FMNERG

Here are codes and dataset for our ACM MM2023 paper: Fine-Grained Multimodal Named Entity Recognition and Grounding with a Generative Framework

  • Dataset

    Our dataset is built on the GMNER dataset.

    • The preprocessed CoNLL format files are provided in this repo. For each tweet, the first line is its image id, and the following lines are its textual contents.
    • Download each tweet's associated images via this link (https://drive.google.com/file/d/1PpvvncnQkgDNeBMKVgG2zFYuRhbL873g/view)
    • Use VinVL to identify all the candidate objects, and put them under the folder named "twitterFMNERG_vinvl_extract36". We have uploaded the features extracted by VinVL to Google Drive and Baidu Netdisk (code: TwVi).

    Usage

    Prepare data

    python T5_data/format_data.py

    Training for TIGER

    sh run.sh

    Evaluation

    sh eval.sh

    Acknowledgements

    • Using the dataset means you have read and accepted the copyrights set by Twitter and original dataset providers.
    • Some codes are based on the codes of VL-T5, thanks a lot!