Code for our paper AttesTable at SemEval-2021 Task 9: Extending Statement Verification with Tables for Unknown Class, and Semantic Evidence Finding
- Task-A (Statement Verification) : Given a table and a statement, we need to determine whether the table entails the statement, refutes the statement, or has insufficient information about it ("unknown")
- Task-B (Evidence Finding) : When the table entails or refutes the statement, we need to select which cells in the table provide the evidence for it
Data and other competition related details can be found at the SEM-TAB-FACTS website and on the corresponding Codalab competition
models/
Task-A.ipynb
: Notebook used for training/validating the models for Task-ATask-B.ipynb
: Notebook used for training/validating the models for Task-B
utils/
Task-A/
augment_df.py
: Synthesizes the unknown statements as proposed in our paperaugment_df_2.py
: Augments the DF directly and inserts unknown statements randomly in each table from other tables in the same file, if no other tables, then it randomly selects the statements from other files. Remove some tables of autogenerated folder so that number of tables from manual and autogenerated folders are same.transform_data.py
: Transforms the data from the givenXML
format toCSV
formattransform_data_2.py
: Given a(table, statement)
DF, generates a(table, statements)
DF directlyunaugment_df.py
: Removes the unknown statements from a DF
Task-B/
create_data.py
: Creates the DF used while training the models for task-B in a(cell, statement)
formatcreate_data_list.py
: Creates the DF used while training the models for task-B in a(statement, [cells])
formatcreate_data_undersampled.py
: Creates the DF used while training the models for task-B in a(statement, [cells])
format, while simultaneously undersampling the "irrelevant" class
merge_df.py
: Merges/stacks two given CSV files
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
If you find this work useful, please cite:
@inproceedings{varma-etal-2021-attestable,
title = "{A}ttes{T}able at {S}em{E}val-2021 Task 9: Extending Statement Verification with Tables for Unknown Class, and Semantic Evidence Finding",
author = "Varma, Harshit and
Jain, Aadish and
Ratadiya, Pratik and
Rathi, Abhishek",
booktitle = "Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)",
month = aug,
year = "2021",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.semeval-1.182",
doi = "10.18653/v1/2021.semeval-1.182",
pages = "1276--1282"
}
Feel free to contact Pratik Ratadiya (pratik.r@vcreatek.com) for any queries