embeddings-benchmark/mteb

Got unsupported ScalarType BFloat16 in RerankingEvaluator.py

xujunrt opened this issue · 2 comments

xujunrt commented

In the RerankingEvaluator.py, np.asarray does not support BFloat16. A forced conversion should be added to avoid similar errors. For example, you can first convert the Tensor to float16 and then call numpy_asarray.

isaac-chung commented

CC-ing @orionw and @KennethEnevoldsen on this.

@xujunrt it would be great if you could also share an example of how you're using the Reranking evaluator. Thanks!

  • 👍1
KennethEnevoldsen commented

This sounds like it is related to #941

Agree with @xujunrt. This sounds like a problem we could have in more places as well.