CLUEbenchmark/SuperCLUE

Issues

• 请问一月榜单呢

  #42 opened a year ago by kindle939393
  2

• 断更了？？？

  #48 opened 2 months ago by yiliangfang
  1

• What are expected to submit for the leaderboard integration?

  #43 opened 8 months ago by zhimin-z
  1

• 请问如何对自己做的大模型使用该指标进行测评呢？

  #44 opened 9 months ago by AWangji
  2

• 请问评测的原理是什么呀，是人工打分的吗，如果是客观题是直接比较返回的答案的字符串，主观题是人工评判答案吗

  #47 opened 5 months ago by starplatinum3
  1

• 工具的评测是什么含义？ 是Function calling么，如果没有请添加此能力的评测。

  #46 opened 8 months ago by goqw
  1

• 公开评测集和评测标准

  #13 opened 2 years ago by plmsmile
  2

• Claude3有评测吗？

  #45 opened 9 months ago by Pancat007
  0

• Does it indicate using 5 shots for evaluation?

  #39 opened a year ago by zhimin-z
  1

• 阿里的通义千问没有吗？

  #22 opened a year ago by Pancat009
  2

• 数据集开源吗？可以在哪里下载呢

  #26 opened a year ago by vanshaw2017
  3

• c-eval是真的离谱，希望superclue能更新的稍微快一点，比如1-2周更新一次

  #31 opened a year ago by iammeizu
  2

• Where to locate the SuperCLUE-LYB leaderboards?

  #34 opened a year ago by zhimin-z
  0

• GPT4-Turbo is missing from the general leaderboard

  #41 opened a year ago by zhimin-z
  1

• anthropic拼错了

  #30 opened a year ago by JerryJiang12923
  0

• 咨询一下，从测评报告来看，SuperCLUE是采用自动化方式的客观评估，是否可提供针对某一模型的可实际运行的自动化评测的python样例代码(api调用或者web)？

  #40 opened a year ago by Romanzhang2024
  0

• Where to download the benchmark dataset?

  #38 opened a year ago by zhimin-z
  0

• How to calculate the metrics from the table in the paper to the leaderboard?

  #37 opened a year ago by zhimin-z
  1

• 大模型升级方式

  #36 opened a year ago by lukeup
  0

• 想问下 角色扮演 benchmark是怎么进行的

  #35 opened a year ago by xealml
  0

• 任务规划和工具使用的评价标准是什么样的？

  #32 opened a year ago by heibaidaolx123
  1

• 能否增加翻译的评估排名

  #33 opened a year ago by lx0126z
  0

• 求教一下 逻辑与推理 具体指哪方面？ 比如 "郭德纲2岁会看报,xxxx" ，请问郭德纲3岁会看书吗？ 这个属于推理还是语义理解能力？？

  #29 opened a year ago by ArtificialZeng
  0

• 请问可以把vicuna-33B模型加入评测吗？

  #28 opened a year ago by Mr-wang2016
  0

• 排名变化的原因是什么？

  #24 opened 2 years ago by zhaojiawen-coding
  1

• 测评时如何与标准答案进行匹配

  #27 opened a year ago by Starry-Hu
  0

• test the 智源大模型吧

  #23 opened 2 years ago by forkyguo
  3

• 没有文心一言吗

  #20 opened 2 years ago by p81sunshine
  1

• 关于prompt设计的问题

  #25 opened 2 years ago by lrs1353281004
  1

• 可以在superclue上测试自己的模型吗？

  #18 opened 2 years ago by guozhiyao
  2

• 开始搞手机测评榜那一套了？GPT4对应苹果，国产大模型对应华米OV

  #12 opened 2 years ago by ZhuGeRoastedFish
  3

• 这里"idea-jiangzhiya"应该是"idea-jiangziya"吧？

  #21 opened 2 years ago by ilongshan
  1

• 什么时候回公开测试数据集？

  #17 opened 2 years ago by wangrui6
  1

• Clarify which "Claude" is benchmarked?

  #19 opened 2 years ago by jekbradbury
  1

• 我个人使用后的感受，星火大模型是真的不如文心一言。。​

  #3 opened 2 years ago by MysteryMulberry
  8

• 单项能力有多少道题目啊

  #7 opened 2 years ago by leonall
  2

• 感谢徐亮老师团队的工作~关于评测细节 有一些疑问咨询下

  #1 opened 2 years ago by lrs1353281004
  5

• 超200人了，求拉群

  #2 opened 2 years ago by dinglei8908
  1

• 该如何引用你们的工作？

  #4 opened 2 years ago by MikeGu721
  1

• 这个superCLUE 有毒性和偏见等方面的评测吗

  #6 opened 2 years ago by devinbai
  3

• 评测数据客观公正很重要

  #8 opened 2 years ago by shichengustc
  3

• 作为一个测评榜，建议参考Chinese-LLaMA-Alpaca进行适度的测评说明和公开

  #9 opened 2 years ago by shm007g
  1

• 这个评测的参考价值

  #10 opened 2 years ago by liuyajun52
  2

• 置信度

  #14 opened 2 years ago by littlepan0413
  1

• 人类的数值怎么来的？

  #15 opened 2 years ago by So0ni
  3

• 建议补全人类的“专业能力”数据

  #16 opened 2 years ago by Triang-jyed-driung
  1

• 生成与创作如何用选择题的形式测试的？

  #5 opened 2 years ago by Howardqlz
  4

• 测评结果为什么全是整数？

  #11 opened 2 years ago by ltz0120
  1