Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge

This repository contains the data and code for the baseline described in the following paper:

Entity Cloze By Date: What LMs Know About Unseen Entities
Yasumasa Onoe, Michael J.Q. Zhang, Shankar Padmanabhan, Greg Durrett, Eunsol Choi
ACL 2023

Getting Started

This codebase uses Python 3.7.9. Other versions may work as well.

Dependencies:

$ conda create -n ekp -y python=3.7.9
$ conda activate ekp
(ekp) $ pip install -r requirements.txt

Data

Entity Inferences: data/entity_inferences
ECBD: data/ecbd

Running experiments

From the root dir, run an experiment python file.

Example:

(ekp) $ python experiments/gpt_ft.py

Experiment	Base Model	Editing Method	Data
`gpt_ft_ecbd.py`	GPT2-XL or GPT-Neo 1.3B	Finetuning	ECBD
`gpt_ft_entity_inferences.py`	GPT2-XL or GPT-Neo 1.3B	Finetuning	Entity Inferences
`gpt_mend_ecbd.py`	GPT2-XL	MEND	ECBD
`gpt_mend_entity_inferences.py`	GPT2-XL	MEND	Entity Inferences
`t5_ft_ecbd.py`	T5-Large	Finetuning	ECBD
`t5_ft_entity_inferences.py`	T5-Large	Finetuning	Entity Inferences
`t5_mend_ecbd.py`	T5-Large	MEND	ECBD
`t5_mend_entity_inferences.py`	T5-Large	MEND	Entity Inferences

NOTE: ROME with GPT2-XL will be added soon...

Citing the paper

@inproceedings{onoe-etal-2023-lms,
    title = {{Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge}},
    author = "Onoe, Yasumasa  and
      Zhang, Michael  and
      Padmanabhan, Shankar  and
      Durrett, Greg  and
      Choi, Eunsol",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.300",
    pages = "5469--5485",
}

Contact

Please contact at yasumasa@utexas.edu or yasumasaonoe@google.com if you have any questions.