NanoBeirEvaluator returns a mix of floats and numpy.float64 in the results.

[JINO-ROHIT]

I was writing test cases and noticed that the results were returned as a mix of floats and np floats. Should we aim to convert these to floats as well?

{'NanoQuoraRetrieval_cosine_accuracy@1': 0.1,
 'NanoQuoraRetrieval_cosine_accuracy@3': 0.18,
 'NanoQuoraRetrieval_cosine_accuracy@5': 0.24,
 'NanoQuoraRetrieval_cosine_accuracy@10': 0.28,
 'NanoQuoraRetrieval_cosine_precision@1': np.float64(0.1),
 'NanoQuoraRetrieval_cosine_precision@3': np.float64(0.06666666666666667),
 'NanoQuoraRetrieval_cosine_precision@5': np.float64(0.052000000000000005),
 'NanoQuoraRetrieval_cosine_precision@10': np.float64(0.034),
 'NanoQuoraRetrieval_cosine_recall@1': np.float64(0.09),
 'NanoQuoraRetrieval_cosine_recall@3': np.float64(0.18),
 'NanoQuoraRetrieval_cosine_recall@5': np.float64(0.21666666666666665),
 'NanoQuoraRetrieval_cosine_recall@10': np.float64(0.27),
 'NanoQuoraRetrieval_cosine_ndcg@10': np.float64(0.1759841616710832),
 'NanoQuoraRetrieval_cosine_mrr@10': 0.14733333333333334,
 'NanoQuoraRetrieval_cosine_map@100': np.float64(0.15211020997551106),
 'NanoMSMARCO_cosine_accuracy@1': 0.0,
 'NanoMSMARCO_cosine_accuracy@3': 0.02,
 'NanoMSMARCO_cosine_accuracy@5': 0.04,
 'NanoMSMARCO_cosine_accuracy@10': 0.06,
 'NanoMSMARCO_cosine_precision@1': np.float64(0.0),
 'NanoMSMARCO_cosine_precision@3': np.float64(0.006666666666666666),
 'NanoMSMARCO_cosine_precision@5': np.float64(0.008),
 'NanoMSMARCO_cosine_precision@10': np.float64(0.006000000000000001),
 'NanoMSMARCO_cosine_recall@1': np.float64(0.0),
 'NanoMSMARCO_cosine_recall@3': np.float64(0.02),
 'NanoMSMARCO_cosine_recall@5': np.float64(0.04),
 'NanoMSMARCO_cosine_recall@10': np.float64(0.06),
 'NanoMSMARCO_cosine_ndcg@10': np.float64(0.02702231788278665),
 'NanoMSMARCO_cosine_mrr@10': 0.016857142857142855,
 'NanoMSMARCO_cosine_map@100': np.float64(0.019069983940167533),
 'NanoBEIR_mean_cosine_accuracy@1': np.float64(0.05),
 'NanoBEIR_mean_cosine_accuracy@3': np.float64(0.09999999999999999),
 'NanoBEIR_mean_cosine_accuracy@5': np.float64(0.13999999999999999),
 'NanoBEIR_mean_cosine_accuracy@10': np.float64(0.17),
 'NanoBEIR_mean_cosine_precision@1': np.float64(0.05),
 'NanoBEIR_mean_cosine_precision@3': np.float64(0.03666666666666667),
 'NanoBEIR_mean_cosine_precision@5': np.float64(0.030000000000000002),
 'NanoBEIR_mean_cosine_precision@10': np.float64(0.02),
 'NanoBEIR_mean_cosine_recall@1': np.float64(0.045),
 'NanoBEIR_mean_cosine_recall@3': np.float64(0.09999999999999999),
 'NanoBEIR_mean_cosine_recall@5': np.float64(0.12833333333333333),
 'NanoBEIR_mean_cosine_recall@10': np.float64(0.165),
 'NanoBEIR_mean_cosine_ndcg@10': np.float64(0.10150323977693491),
 'NanoBEIR_mean_cosine_mrr@10': np.float64(0.0820952380952381),
 'NanoBEIR_mean_cosine_map@100': np.float64(0.0855900969578393)}

Who can help?
@tomaarsen

[tomaarsen]

Hello!

Yes, it'd be ideal to normalize this to floats. I think this might mean that the same happens in InformationRetrievalEvaluator.

• Tom Aarsen

[JINO-ROHIT]

cool, ill run across the other ones too and check if the same happens and raise a PR at one go