huggingface/evaluate

Issues

• need evaluator code reference for object detector & zero shot object dector task

  #552 opened a year ago
  0

• Add geometric mean of per-token-Perplexities

  #551 opened a year ago
  0

• MSE not accepting (n_samples, n_ouptuts) despite docs stating so

  #550 opened a year ago
  2

• Try to import evaluate but get "Library not loaded: @rpath/libwebp.7.dylib" error

  #548 opened a year ago
  2

• `DatasetColumn` and `DatasetColumnPair` should have `__repr__()` methods defined

  #547 opened a year ago
  0

• Token classification bootstrap crashing with custom dataset

  #546 opened a year ago
  0

• Documentation claims that `label_mapping` is an accepted input to `TokenClassificationEvaluator.compute()`; it is not

  #545 opened a year ago
  0

• several breakages due to recent `datasets`

  #542 opened 10 months ago
  17

• Is there a plan to translate the documentation for this repository?

  #541 opened a year ago
  0

• Evaluate fails to acquire lock interrupting single-node multi-GPU training

  #540 opened 10 months ago
  3

• Is TextGenerationEvaluator incomplete?

  #537 opened a year ago
  5

• How should i cite this repository?

  #534 opened a year ago
  2

• `perplexity` yields `IndexError: index out of range in self`

  #532 opened a year ago
  1

• Bug in BLEU calculation - BLEU of identical strings should be 1.0, not 0.0

  #529 opened a year ago
  2

• Cannot use evaluate.TextClassificationEvaluator for multi-label classification

  #527 opened 8 months ago
  2

• Using huggingface evaluate for distributed evaluation

  #526 opened a year ago
  0

• Getting error <lambda>() takes 0 positional arguments but 1 was given when trying to import evaluator

  #525 opened a year ago
  0

• os.unlink(entry.name, dir_fd=topfd) TypeError: 'NoneType' object is not callable

  #523 opened a year ago
  0

• DefaultCPUAllocator: can't allocate memory during perplexity.compute for Llama 2

  #522 opened a year ago
  0

• `evaluate-cli create` fails with `subprocess.CalledProcessError`

  #519 opened a year ago
  1

• Importing Error for evaluator module

  #518 opened a year ago
  0

• Cannot list evaluation modules types

  #517 opened a year ago
  0

• Can't calculate combined metric (WER + CER)

  #516 opened a year ago
  2

• Allow BLEURT optimisation parameters

  #515 opened a year ago
  0

• HuggingFace evaluate-cli create command results Error: "datasets[0]" must be a string

  #514 opened a year ago
  2

• add_batch taking very long time (SegFormer semantic segmentation).

  #513 opened a year ago
  0

• about evaluate/perplexity

  #512 opened a year ago
  2

• Loading metrics that are shipped with the evaluate package is slow

  #511 opened a year ago
  2

• Compatibility Issue with datasets library

  #504 opened a year ago
  3

• Move away from `Repository` to `HfApi` upload files

  #503 opened a year ago
  0

• Perplexity Space Broken

  #502 opened a year ago
  0

• wrong rouge-Lsum results due to lack of parameter control

  #501 opened a year ago
  0

• Wrong accuracy on the validation set!!!

  #500 opened a year ago
  1

• Multi-label with f1

  #497 opened a year ago
  2

• CER.py error in class CERConfig

  #495 opened a year ago
  1

• is it possible to evaluate a proprietary model without uploading it to HF

  #494 opened a year ago
  7

• Feature request for metric result keys in `MetricInfo`

  #493 opened a year ago
  0

• Matthew's Correlation Coefficient (MCC) always zero

  #492 opened a year ago
  1

• perplexity.compute - Add args to pass model and tokenizer objects instead of `model_id`

  #491 opened a year ago
  1

• Concurrent Programs Sharing Evaluate Lib Would raise "ValueError: Error in finalize: another evaluation module instance is already using the local cache file."

  #489 opened a year ago
  6

• Evaluate a quantize model

  #487 opened a year ago
  2

• evaluate.load("wer") is returning Filenotfound error

  #486 opened a year ago
  3

• How to use `SubTask` with metrics that require valid `config_name`

  #485 opened a year ago
  0

• Metric returned instead of the actual COMET scores

  #484 opened a year ago
  7

• `.compute()` not compatible with `perplexity` for `text-generation`

  #483 opened 2 years ago
  1

• [Metrics] Can't find / acquire lock files in distributed multi-node shared file system

  #481 opened 2 years ago
  4

• meteor don't works with latest datasets

  #480 opened a year ago
  13

• Trainer Custom_loss - Empty metrics in Evaluate

  #478 opened 2 years ago
  2

• Question: Computing utterance-level and corpus level metrics

  #477 opened 2 years ago
  0

• Change package name from "evaluate" to something less generic

  #476 opened 2 years ago
  0