AILab-CVC/SEED-Bench

Issues

• Wrong Question Types in SEED-Bench1

  #28 opened 23 days ago by littlepenguin89106
  1

• Question about evaluation input format

  #27 opened a month ago by yellow-binary-tree
  1

• Evaluation Method for Closed-Source Models like GPT4V

  #26 opened a month ago by JUNJIE99
  1

• any plan to release the original images for SEED-Bench-2-Plus?

  #25 opened a month ago by hjeun
  1

• [bugs] LLaVA-Evaluation : line81, '[img]' is supposed to be '<img>'?

  #23 opened 2 months ago by LeoWootsi
  1

• Question on multi-image input

  #24 opened 3 months ago by auhowielau
  2

• [bugs] LLaVA-Evaluation : RuntimeError: Expected all tensors to be on the same device

  #21 opened 4 months ago by JJJYmmm
  0

• Muti-GPUs Evaluation

  #20 opened 4 months ago by JJJYmmm
  0

• Question on how task27 generates images

  #19 opened 5 months ago by JunZhan2000
  2

• Request for the interface of minigpt4 and llava

  #10 opened 9 months ago by Richar-Du
  2

• What is the correct way to download the video

  #17 opened 6 months ago by teasgen
  6

• Easy way to probe result examples?

  #11 opened 9 months ago by chancharikmitra
  1

• a lot of data with more questions than pictures in SEED-Bench-2 level L2, is this reasonable?

  #15 opened 6 months ago by nemonameless
  5

• VLMs vs LLMs evaluation

  #12 opened 8 months ago by idan-tankel
  1

• Request for the removing duplicate results

  #16 opened 6 months ago by khanrc
  2

• Reproduce the Qwen-VL SOTAs results

  #9 opened 9 months ago by jinze1994
  2

• In-Context Example Selection Process

  #14 opened 6 months ago by mustafaadogan
  1

• How to download the images?

  #13 opened 7 months ago by dyahadila
  0

• Support for evaluation of other VLM models like MiniGPT-4, mPLUG-Owl, Llava, and VPGTrans

  #8 opened 9 months ago by WesleyHsieh0806
  2

• [Data] Could you provide a list including the files of something-something v2 which should be downloaded?

  #6 opened 9 months ago by aopolin-lv
  2

• This benchmark could lead to wrong conclusion.

  #7 opened 10 months ago by dannyhung1128
  3

• Evaluating latest version of OpenFlamingo

  #2 opened 10 months ago by anas-awadalla
  1

• Update for Otter-Image-MPT7B and Otter-Video

  #1 opened 10 months ago by Luodian
  1

• Update for mPLUG-Owl

  #3 opened 10 months ago by MAGAer13
  1

• Add InstructBlip Flan-T5-xl and InstructBlip Flan-T5-xxl

  #5 opened 10 months ago by brianjking
  1

• Ask for original captions from which you generate questions

  #4 opened 10 months ago by YangYangGirl
  1