Add "pip install" support
Opened this issue · 3 comments
This could be a first step towards the inclusion in evaluate.
Interesting: The current scorer still relies on many preprocessing steps that are idiosyncratically bound to the HIPE format and evaluation scenario. In a way it could still be seen as an evaluation space. https://huggingface.co/evaluate-metric (similar to GLUE).
@simon-clematide (hoping I did not misunderstood) I don’t see it as such a problem that the metric depends on the annotation style (domain-dependent). I was raising the issue mainly because it could be easier to integrate on my side for the training of different models and of course, it could be easier to integrate in a metric such as seqeval.
One can see the metric as instead of multitask (CoNLL with columns for each task eg NER, chunking), some type of multilevel (columns in HIPE) - multilevel-seqeval 🙂