Framenet test set for comparison
McKracken opened this issue · 1 comments
Hi, I saw that in your paper about open-sesame you said that you used the same test set as Das and Smith 2011, which is a subset of FrameNet 1.5 fulltext annotations. However, in this code it seems that you're dealing with the 1.7 version.
Is there any way to get the test set annotation IDs from 1.5 that you (and Das and Smith 2011 used) to run some comparable experiments, without converting everything to formats that might be not needed?
Thanks
You could recreate FN 1.5 validation and test sets by considering the files specified in this repository here: https://github.com/swabhs/open-sesame/blob/master/sesame/globalconfig.py#L43-L78 You can find the 1.5 version on the FrameNet website: https://framenet.icsi.berkeley.edu/fndrupal/framenet_request_data
For FN 1.7, I have used the same files as Das & Smith (2011) to create the test and validation sets. Note that since 1.7 is a larger version, there are many more annotations of the same sentences from the same files, resulting in larger train, validation and test sets. Hence 1.7 and 1.5 are not comparable.