microsoft/promptbench

Issues

• The change in the 'api_key' parameter has not been reflected in the documentation and tutorials.

  #46 opened 5 months ago by dario-github
  2

• How to reproduce the transferability results?

  #55 opened 3 months ago by snbhanja
  0

• Enhancement Proposal for Dynamic Evaluation Framework Integration

  #43 opened 3 months ago by yihong1120
  2

• Parsing the output of a model

  #45 opened 5 months ago by jylong91
  2

• Can you provide a general data format for constructing test data independently?

  #54 opened 3 months ago by ShiYu05
  1

• The ‘temperature’ parameter in pb.LLMModel needs to be of type float

  #53 opened 3 months ago by ShiYu05
  1

• The `prompt engineering` leaderboard is cut off in half...

  #51 opened 4 months ago by zhimin-z
  4

• Is ` DyVal` included in `PromptBench` yet?

  #52 opened 4 months ago by zhimin-z
  2

• Where is defined inference_total_dataset()

  #49 opened 4 months ago by dmitrysarov
  1

• How do you get the evaluation results of the `prompt enginering` leaderboard?

  #50 opened 4 months ago by zhimin-z
  1

• The link is broken in https://llm-eval.github.io/pages/code.html

  #47 opened 4 months ago by zhimin-z
  1

• How to use visualize.py to llama-2-7b?

  #16 opened 4 months ago by Deep1994
  10

• Is this an ongoing project?

  #48 opened 4 months ago by tarikaltuncu
  2

• LLaMa 2 inference

  #15 opened 5 months ago by jubick1337
  3

• Is it possible to add phi-2?

  #32 opened 5 months ago by tr5tn
  6

• How to add a new Model

  #44 opened 5 months ago by minhopark-neubla
  2

• Interpretation of attack results

  #39 opened 6 months ago by xipq
  2

• Can you please explain how to interpret the output on the main/examples/basic.ipynb

  #38 opened 6 months ago by jacobweiss2305
  5

• Some datasets are not available

  #28 opened 6 months ago by koshyviv
  5

• Numerous datasets exhibit a label of -1

  #31 opened 6 months ago by camilochs
  1

• MMLU subset

  #9 opened 6 months ago by BinWang28
  4

• Potentially a bug in `LabelConstraint`

  #11 opened 6 months ago by SagiPolaczek
  2

• Chatgpt

  #10 opened 6 months ago by shayanshafquat
  3

• For the few shot setting, how many are used exactly?

  #26 opened 6 months ago by zhimin-z
  5

• Parameter size of this model?

  #21 opened 6 months ago by zhimin-z
  1

• Typo in the doc

  #20 opened 6 months ago by zhimin-z
  2

• Wrong call for OpenAI API

  #14 opened 7 months ago by Django-Jiang
  1

• transferibility app error

  #12 opened 9 months ago by ogencoglu
  3

• About Semantic Attacks Against Vicuna

  #8 opened 10 months ago by tianshuocong
  3

• About Runtime

  #7 opened 10 months ago by Harry-mic
  7

• About Checklist attack method

  #6 opened 10 months ago by Alex357853
  1

• About black-box attacks

  #5 opened 10 months ago by zhouzihao501
  1

• ChatGPT version

  #4 opened a year ago by titaniumrain
  2

• Running time

  #3 opened a year ago by BinWang28
  2