embeddings-benchmark/mteb

Issues

• Scrollable dropdown

  #1304 opened 2 months ago
  0

• Leaderboard - Overview Issue

  #1303 opened 2 months ago
  2

• Arxiv paper submission

  #1299 opened 2 months ago
  3

• Standardize language codes for Chinese

  #1298 opened 2 months ago
  4

• Got unsupported ScalarType BFloat16 in RerankingEvaluator.py

  #1297 opened 2 months ago
  2

• Serialization and validation are slow

  #1296 opened 2 months ago
  17

• Add older MMTEB baselines

  #1295 opened 2 months ago
  1

• Remove encode_corpus and encode_queries and implement a "document" class

  #1284 opened 3 months ago
  0

• Transitioning retrieval datasets to retrieval format recommend in #1090

  #1282 opened 3 months ago
  1

• Hacktoberfest 🎉

  #1279 opened 2 months ago
  0

• Add mteb(eng), mteb(europe), mteb(indic) and mteb(multilingual)

  #1273 opened 2 months ago
  2

• New benchmark interface

  #1272 opened 3 months ago
  1

• Add FinMTEB

  #1267 opened 3 months ago
  3

• Add documentation for custom rerankers

  #1266 opened 3 months ago
  3

• Add PIRB

  #1265 opened 3 months ago
  5

• Allow Numpy 2.0

  #1263 opened 2 months ago
  1

• Only skip benchmarking if split results are the same too

  #1260 opened 24 days ago
  1

• Sort benchmarks by name

  #1257 opened 3 months ago
  2

• Add benchmark CLI options

  #1250 opened 2 months ago
  0

• Massive Multimodal Extension of MTEB

  #1249 opened 3 months ago
  1

• SWE-Bench Retrieval

  #1238 opened 3 months ago
  0

• Add proper metadata to models

  #1234 opened in 2 hours
  6

• Allow aggregated tasks within benchmarks

  #1231 opened 3 months ago
  4

• rernaker eval

  #1230 opened 3 months ago
  2

• Unable to reproduce MTEB leaderboard results on TRECCOVID using `Snowflake/snowflake-arctic-embed-m-v1.5`

  #1227 opened 3 months ago
  2

• PreTrainedTokenizerFast._batch_encode_plus() got an unexpected keyword argument 'prompt_name'

  #1224 opened 3 months ago
  3

• Mismatch of the category of AmazonPolarityClassification

  #1219 opened 3 months ago
  2

• Error on MTOPDomainClassification task

  #1216 opened 3 months ago
  5

• Cross-Encoder Support for Reranking Abstract Task

  #1214 opened 3 months ago
  0

• Figure out an approach for adding experiments to the leaderboard

  #1211 opened 3 months ago
  4

• Add benchmark overview table

  #1209 opened 3 months ago
  3

• BUG: if the model defines a similarity score, STS pearson and spearman are written as arrays, not as scores.

  #1206 opened 3 months ago
  2

• Evaluating only on English

  #1205 opened 3 months ago
  3

• Main Example in the README fails

  #1204 opened 4 months ago
  8

• Sequence Length information in the leaderboard

  #1202 opened 3 months ago
  13

• BadRequestError with OpenAI models for some datasets

  #1200 opened 3 months ago
  1

• mteb.load_results raises pydantic error when trying to load all tasks

  #1197 opened 3 months ago
  3

• CrossEncoder object has no attribute model_card_data

  #1192 opened 3 months ago
  1

• MTEB Implementation of e5-mistral-7b-instruct does not match original implementation

  #1185 opened 4 months ago
  4

• [paper writing]: code in appendix is very low resolution

  #1183 opened 4 months ago
  4

• Error while downloading

  #1176 opened 4 months ago
  2

• Potential bug in datasets suggested by uniform score

  #1172 opened 4 months ago
  3

• Re-add CQADupstack merge script / find other way to make merging easier

  #1171 opened 4 months ago
  0

• Add updated Touche-2020

  #1170 opened 2 months ago
  6

• Standardize license in task metadata

  #1165 opened 3 months ago
  2

• How to output the wrong retrieval data?

  #1162 opened 4 months ago
  5

• SummarizationEvaluator mean score issue

  #1156 opened 4 months ago
  6

• Add CRAG

  #1151 opened 4 months ago
  0

• [New dataset request] Please add MKQA

  #1149 opened 4 months ago
  8

• Missing import for SadeemQuestionRetrieval

  #1143 opened 4 months ago
  3