declare-lab/instruct-eval

Issues

• Multi GPU Support is required

  #33 opened 6 months ago by chintan-ushur
  0

• Evaluate EncoderDecoderModels

  #32 opened 9 months ago by Bachstelze
  0

• Colab notebook

  #31 opened 9 months ago by Bachstelze
  0

• CRASS

  #30 opened 9 months ago by Bachstelze
  0

• Evaluate on a single 24GB/32GB GPU

  #29 opened 10 months ago by lemyx
  1

• How to submit own model to leaderboard?

  #28 opened 10 months ago by timothylimyl
  1

• [Prompt Template] Silent bug - Performance Killer

  #27 opened 10 months ago by timothylimyl
  0

• What are the metrics for the evaluation results?

  #26 opened a year ago by zhimin-z
  0

• Can not reproduce results on the table

  #3 opened 2 years ago by simplelifetime
  7

• Reproduce the accuracy of chavinlo/alpaca-native on MMLU

  #25 opened a year ago by sglucas
  0

• Support for larger batch_size

  #18 opened a year ago by soumyasanyal
  1

• HHH Benchmark evaluation question: why using base prompt and (A - A_base) > (B - B_base)?

  #20 opened a year ago by t170815518
  1

• What to do about broken Evals?

  #21 opened a year ago by damhack
  1

• Fail to Evaluate Model on human_eval

  #22 opened a year ago by yjw1029
  1

• C-Eval

  #17 opened a year ago by duanqiyuan
  0

• 请问能加入对baichuan大模型的支持吗

  #16 opened a year ago by linghongli
  1

• add multiple gpu support

  #15 opened a year ago by lxy444
  0

• [Feature Request] Saving Prediction Results

  #14 opened a year ago by guanqun-yang
  0

• Is there any parallel processing methods?

  #13 opened a year ago by wwngh1233
  0

• Add config to save eval results

  #12 opened a year ago by arthurtobler
  0

• Future directions

  #11 opened a year ago by tju01
  1

• 执行过程中报错，信息如下

  #9 opened a year ago by linghongli
  1

• Regarding the comparison to lm-evaluation-harness

  #10 opened a year ago by gakada
  0

• Integrate the evaluation in the Transformers trainer with transformers.TrainerCallback

  #7 opened a year ago by BaohaoLiao
  1

• AutoModelForCausalLM supports llama models now

  #6 opened a year ago by passaglia
  1

• Add License

  #5 opened a year ago by passaglia
  2

• Add zero-shot evaluation results

  #4 opened 2 years ago by LeeShiyang
  1

• Prompt format for LLaMa

  #2 opened 2 years ago by LeeShiyang
  2