psseval.py : the function of .goldid and .autoid

[nelson-liu]

Probably a trivial question, but when would you use .goldid or .autoid in the predictions file for psseval.py? What do they each mean?

Thanks!

[nschneid]

"id" refers to target identification, i.e. determining which spans deserve a preposition supersense. That's not as easy as it sounds due to various kinds of multiword expressions and other special phenomena. So .goldid is for files generated in the oracle ID condition, and measures accuracy. The .autoid files are evaluated with precision/recall/F-score.

[nelson-liu]

Fantastic, thanks for the response!