Tools and utilities for supporting AIDA evaluations.
Tools and utilities for Phase-II evaluation
Name |
Description |
Location |
Latest release branch |
AIDA-Evaluation Docker |
Tool for applying the entire evaluation pipeline to task1 submissions |
tools/aida-evaluation |
AIDAED-v2020.1.0 |
Tools and utilities for Phase-I evaluation conducted at the end of 2019
Name |
Description |
Location |
Latest release branch |
SPAQRL Query Application Docker |
The tool for applying SPARQL queries to KBs |
tools/sparql-query-application |
AIDAQA-v2019.2.5 |
AIDA-ValidateResponses |
Tool for validating responses obtained after applying SPARQL queries to KBs |
tools/validate-responses |
AIDAVR-v2019.0.3 |
AIDA-ConfidenceAggregation |
Tool (and docker) for aggregating confidences |
tools/confidence-aggregation |
AIDACA-v2019.0.2 |
AIDA-ScoreResponses |
Tool for scoring responses |
tools/scorer |
AIDASR-v2019.2.1 |
AIF Generator |
Tool for converting annotations into AIF |
python/generate_aif.py |
AIDAAG-v2019.0.1 |
AIDA-Evaluation Docker |
Tool for applying the entire evaluation pipeline |
tools/aida-evaluation |
AIDAED-v2019.0.1 |
Tools and utilities for 2018 (M9) evaluation
Name |
Description |
utils/translate-annotations/seedling |
Utility to translate LDC's seedling annotation to turtle-RDF format |
tools/resolve-queries |
Tool for applying SPARQL queries to a knowledge base in AIF format in order to produce output in the simplified xml response format |
tools/validate-responses |
Tool for validating responses for AIDA M9 evaluation |
Please refer to tool/utility specific README file(s) for more details.