RetrivalEvaluator overrides encode kwargs

[Samoed]

When developing the wrappers, I tried to remove the explicit conversion from tensor to NumPy, but I encountered errors due to the convert_to_tensor parameter in encode_kwargs. I later found that RetrievalEvaluator overrides these parameters and sets batch_size. Maybe this should be changed?

mteb/mteb/evaluation/evaluators/RetrievalEvaluator.py

Lines 73 to 78 in 650e8b8

	if "batch_size" not in encode_kwargs:
	encode_kwargs["batch_size"] = 128
	if "show_progress_bar" not in encode_kwargs:
	encode_kwargs["show_progress_bar"] = True
	if "convert_to_tensor" not in encode_kwargs:
	encode_kwargs["convert_to_tensor"] = True

[KennethEnevoldsen]

ahh, yes, these were added for backward compatibility when converting to encode_kwargs (#1030).

As you see in the diff, there are multiple of these, especially for batch size. For now, it is easy to just remove convert_to_tensor. However, it seems like the other two are being referred to in the code in multiple places. Not sure what the best choice is for these.

[Samoed]

I think that batch_size and convert_to_tensor can be removed and show_progress_bar can be passed directly to model.encode. I don't think that removing these kwargs will break something

[KennethEnevoldsen]

Removing batch size completely could lead to sudden OOM for many users, and it is also used in a few places in the evaluators.

[Samoed]

The current implementation often results in OOM errors because it sets the batch size too high. By default (if not specified), SentenceTransformer uses a batch size of 32. Another solution would be to explicitly set a default batch size in MTEB.run

[KennethEnevoldsen]

I am all for removing it completely (it is really a model concern, not an evaluation concern)