bigscience-workshop/evaluation

Issues

• translate validation prompts into all training languages

  #50 opened 3 years ago by epavlick
  9

• Add MasaskhaNER to Full Benchmark

  #20 opened 3 years ago by epavlick
  5

• Add DiaBLa to Full Benchmark

  #5 opened 3 years ago by epavlick
  5

• Add HuffPo Text Classification to Full Benchmark

  #31 opened 3 years ago by epavlick
  3

• Add GEM ToTTo to Full Benchmark

  #16 opened 3 years ago by epavlick
  3

• Add GEM XSum to Full Benchmark

  #12 opened 2 years ago by epavlick
  4

• Add WinoMT to Full Benchmark

  #35 opened 3 years ago by epavlick
  2

• Add LinCE Testbed to Full Benchmark

  #25 opened 3 years ago by epavlick
  2

• Add GEM WebNLG to Full Benchmark

  #17 opened 3 years ago by epavlick
  2

• Add CrowS-Pairs to Full Benchmark

  #37 opened 3 years ago by epavlick
  5

• Add BLiMP to Full Benchmark

  #22 opened 3 years ago by epavlick
  5

• Add LAMA to Full Benchmark

  #26 opened 3 years ago by epavlick
  6

• Add GEM MLSum to Full Benchmark

  #11 opened 3 years ago by epavlick
  2

• Add GEM WikiAuto to Full Benchmark

  #14 opened 3 years ago by epavlick
  1

• Add BioASQ to Full Benchmark

  #29 opened 3 years ago by epavlick
  2

• Add QASPER to Full Benchmark

  #28 opened 3 years ago by epavlick
  2

• Add WikiANN to Full Benchmark

  #21 opened 3 years ago by epavlick
  7

• Add TyDiQA to Full Benchmark

  #7 opened 3 years ago by epavlick
  3

• Add MNLI to Full Benchmark

  #33 opened 3 years ago by epavlick
  3

• Add HANS to Full Benchmark

  #34 opened 3 years ago by epavlick
  3

• Add SuperGLUE Tasks to Full Benchmark

  #6 opened 3 years ago by epavlick
  2

• Add ANLI to Full Benchmark

  #32 opened 3 years ago by epavlick
  2

• Add Jigsaw Toxicity Classification to Full Benchmark

  #36 opened 3 years ago by epavlick
  4

• Add MKQA to Full Benchmark

  #73 opened 2 years ago by shayne-longpre
  0

• Add QA-SRL to Full Benchmark

  #23 opened 3 years ago by epavlick
  2

• Add XQuAD to Full Benchmark

  #9 opened 3 years ago by epavlick
  1

• Add PIAF to Full Benchmark

  #8 opened 3 years ago by epavlick
  1

• Add GEM Wikilingua to Full Benchmark

  #13 opened 3 years ago by epavlick
  1

• Add WMT to Full Benchmark

  #4 opened 3 years ago by epavlick
  2

• Refactor task template to merge multilingual.json and english.json

  #64 opened 3 years ago by marinecarpuat
  0

• Add lambada to validation set

  #47 opened 3 years ago by epavlick
  0

• Create toy tasks/dummy code for prompt-based evals

  #39 opened 3 years ago by epavlick
  1

• Wrap evaluation benchmark using HF-trainer

  #61 opened 3 years ago by sbmaruf
  2

• benchmark mt5 on tydiqa prompting setup

  #49 opened 3 years ago by epavlick
  1

• Create Targeted Minimal Pair "Stress-Tests" for Sensitivity to Social Groups

  #38 opened 3 years ago by epavlick
  2

• Create toy tasks/dummy code for fine-tuning evals

  #40 opened 3 years ago by epavlick
  1

• Start overleaf for benchmark tech report

  #54 opened 3 years ago by epavlick
  1

• Add PIQA to validation set

  #46 opened 3 years ago by epavlick
  0

• Setup testing

  #57 opened 3 years ago by jaketae
  2

• Set code conventions

  #43 opened 3 years ago by jaketae
  3

• Add GEM E2E to Full Benchmark

  #18 opened 3 years ago by epavlick
  2

• Convert validation code to work with Megatron as well as huggingface

  #48 opened 3 years ago by epavlick
  0

• Add POS Tagging with UD to Full Benchmark

  #24 opened 3 years ago by epavlick
  1

• Add Flores 101 to to Full Benchmark (as LM tasks, not MT tasks)

  #10 opened 3 years ago by epavlick
  1

• Add XSum to Full Benchmark

  #41 opened 3 years ago by aobaruwa
  5

• Adding TyDi QA to simple_benchmark

  #3 opened 3 years ago by wilsonyhlee
  0

• Add CRD3 to Full Benchmark

  #19 opened 3 years ago by epavlick
  2

• Add TyDiQA for non-training languages to Full Benchmark

  #30 opened 3 years ago by epavlick
  0

• Add Edge Probing Suite to Full Benchmark

  #27 opened 3 years ago by epavlick
  0

• Add GEM Response Generation to Full Benchmark

  #15 opened 3 years ago by epavlick
  0