MMMU-Benchmark/MMMU

Issues

• process_single_sample function's question

  #16 opened a year ago by bruceisme
  3

• Answer not present in the model prediction

  #41 opened 2 months ago by insafim
  2

• Qwen2-VL-7B Inference Code

  #42 opened 2 months ago by insafim
  1

• MMMU-Pro 4 Choices

  #40 opened 2 months ago by insafim
  8

• Temperature setting

  #38 opened 3 months ago by starriver030515
  2

• BIG BUG!

  #37 opened 3 months ago by warm345
  2

• Enquiry about the usage about your dataset

  #33 opened 4 months ago by tunantu
  1

• Add validation set to EvalAI

  #30 opened 5 months ago by dchichkov
  3

• ls

  #29 opened 5 months ago by yuanze-lin
  0

• .tsv file

  #28 opened 6 months ago by beichenzbc
  0

• How to convert images and prompt to the HF parquet?

  #25 opened 7 months ago by Gumpest
  2

• validation_Materials_25 answer seems wrong?

  #27 opened 7 months ago by Zarjagen
  2

• RuntimeError: The size of tensor a (162) must match the size of tensor b (7) at non-singleton dimension 1

  #18 opened 8 months ago by nrikoh
  17

• GPT4o

  #24 opened 8 months ago by dirtycomputer
  3

• There's an error in one of the Correct Examples for genetics

  #22 opened 8 months ago by eabase
  3

• Link for the open source methods in Leaderboard

  #14 opened 8 months ago by XiongweiWu
  1

• Can you release more result files from the validation leaderboard?

  #23 opened 8 months ago by kyleliang919
  1

• Fail to connect to the homepage: https://mmmu-benchmark.github.io/

  #21 opened 9 months ago by shannany0606
  2

• No (supported) data files found in /MMMU/MMMU

  #20 opened 10 months ago by Xiaolong-RRL
  3

• gpt4v refuse to answer/ insist on "I'm sorry, but I'm unable to view images" these kind of things

  #19 opened 10 months ago by SweetGUOguo
  2

• PNG files does not convert to RGB

  #17 opened a year ago by y-vectorfield
  4

• Request for answer_dict.json for test and dev

  #15 opened a year ago by boxin-wbx
  1

• Image and JSON dataset.

  #13 opened a year ago by sxj1215
  6

• How was "prompt engineering" performed?

  #12 opened a year ago by mckinziebrandon
  2

• Representing LLaVa-1.5-13b

  #10 opened a year ago by teasgen
  7

• Model Evaluatation

  #9 opened a year ago by Rubics-Xuan
  11

• Mismatch of the data label in Eval code

  #11 opened a year ago by XiongweiWu
  1

• Prompts

  #5 opened a year ago by teasgen
  3

• model evaluation

  #3 opened a year ago by mactavish91
  2

• How are the image types defined and labeled?

  #6 opened a year ago by CCYChongyanChen
  2

• Question about "Text as Input"

  #8 opened a year ago by fxmeng
  2

• Error reports when loading the dataset

  #4 opened a year ago by XiongweiWu
  4

• Evaluation Prompt for mPLUG-Owl2

  #1 opened a year ago by vateye
  8

• Why is every answer in Structural Engineering just "?"

  #7 opened a year ago by mckinziebrandon
  1