g8a9/ferret

Support salience annotation formats from open-source platforms

g8a9 opened this issue · 0 comments

g8a9 commented

Part of our API accepts human rationale annotations to evaluate explanations on plausibility metrics. It is reasonable to assume that human workers will annotate text with such rationales using open-source tools such as Label Studio.

In this regard, we might want to support off-the-shelf some of the output formats these platforms have. E.g:

  • Label Studio: list of spans, where each span is a tuple with (starting index, ending index)