How to get msls-challenge?

Question

How to get msls-challenge?

Closed this issue 7 days ago · 3 comments

STSTERANDMOMO commented 4 months ago

In many benchmarks,msls-challenge dataset is needed.How to get it?

Answer 1 · 2024-06-17T04:59:29.000Z

I think it is an online challenge, you can check here for more info.

Answer 2 · 2024-06-19T02:51:24.000Z

I think it is an online challenge, you can check here for more info.

But the benchmark don't provide how to handle the msls-challenge dataset. I don't konw how to congnize the dataset.

Answer 3 · 2024-09-28T04:46:14.000Z

Hi, the challenge dataset does not have public labels and therefore we can't process it in the same way that we do with any other dataset, so we preferred not to include it in this list. Furthermore, when testing to submit the solution on the challenge's server, the test should be perform city by city (i.e. do retrieval for Athens's queries only within Athens's database), which would make it even more complex (and less consistent with the rest of the benchmark)