embeddings-benchmark/mteb

Add benchmark overview table

Closed this issue · 3 comments

KennethEnevoldsen commented

Add an overview of available benchmarks (similar to what we have for tasks).

isaac-chung commented

Are you envisioning a doc update to list all available benchmarks, a CLI call (like available tasks but for benchmarks), both, or something else?

KennethEnevoldsen commented

Documentation update, we could update the CLI as well, but I would probably start with the docs

  • 👍1
isaac-chung commented

I'm starting to look into adding this table. What do we want in it? I'm envisioning these columns that aggregate all tasks from each benchmark:

  • Name (str)
  • Task type count (dict) (e.g. Retrieval: 4, Classification: 10)
  • Domains (list)
  • Languages (list)