Mark "higher order" annotators
Opened this issue · 0 comments
Description
The experiment types have been structured using a hierarchy of tasks in which higher order experiment types include the lower order hierarchy types, e.g., an A2KB annotator can be used for a D2KB experiment. However, this may lead to results which are not fully comparable. While the practice of using higher order annotators in the absence of a special configuration for a lower order experiment is not a problem, it should be reported to the user. At the moment, the GUI is not giving any hint regarding this issue.
Solution
The GUI as well as the experiment results should offer an information about such cases. While the latter can handle this indirectly by providing meta data about the system, the GUI should have this information
- on the configuration screen as well as
- in the visualization of experiment results
Steps
(incomplete)
- Introduce the possibility for the program to identify higher order annotators.
- In the GUI, add a
⚠️ or ℹ️ symbol to higher order annotators. Include an explanation which is shown when the symbol is hovered.