melody.raw_chroma_accuracy wrong scores on unvoiced estimates

[kukas]

If the input estimate is unvoiced, the chroma accuracy metric should return 0. Instead it counts reference frames that are the multiples of 1200 cents as correctly estimated frames. Consider this example:

import mir_eval
import numpy as np

ref_time = np.array([0, 1, 2])
est_time = np.array([0, 1, 2])
ref_freq = np.array([40, 80, 160])
est_freq = np.array([0, 0, 0])

arrays = mir_eval.melody.to_cent_voicing(ref_time, ref_freq, est_time, est_freq)
# ref_cent = [2400., 3600., 4800.]
chroma = mir_eval.melody.raw_chroma_accuracy(*arrays)

print(chroma)
# should be 0.0 but actually is 1.0

Just for fun I averaged the raw chroma score of unvoiced estimates on MedleyDB, the result is 7% accuracy.

results = []
for t in mdb.load_melody_multitracks():
    ref_time, ref_freq = np.array(t.melody2_annotation).T
    est_time = ref_time
    est_freq = np.zeros_like(ref_freq)
    arrays = mir_eval.melody.to_cent_voicing(ref_time, ref_freq, est_time, est_freq)
    results.append(mir_eval.melody.raw_chroma_accuracy(*arrays))
print(np.mean(results))
# 0.07011304858972267

[kukas]

I created a pull request to fix this issue: #312

[bmcfee]

Wow, that seems.. not great! Thanks for catching it.

Tagging @rabitt and @justinsalamon for input.

[justinsalamon]

Just had a quick look, seems like there's indeed a bug (thanks @kukas for catching this!). Will give it (and the PR) a second look hopefully tomorrow to confirm fix.