sign-language-processing/signwriting-evaluation

Qualitative evaluation

Closed this issue · 1 comments

AmitMY commented

Like the README says:

To qualitatively demonstrate the efficacy of these evaluation metrics, we implement a nearest-neighbor search for selected signs from the SignBank corpus. The rationale is straightforward: the closer the sign is to its nearest neighbor in the corpus, the more effective the evaluation metric is in capturing the nuances of sign language transcription and translation.

We need to implement this most-similar signs metric to visualize the quality of our metrics

It might also be good to show the similarity distribution (is everything basically 0 or 1? or 0.5? or more uniform)

AmitMY commented

Done.