microsoft/promptbench

Issues

• How to evaluate the prompt score, not at the time of attack, but only?

  #85 opened 2 months ago by zl-comment
  1

• best 3 prompts for each categories

  #86 opened a month ago by zhixiongzh
  0

• DyVal can't be used via pypi install

  #77 opened 4 months ago by GhostXu11
  2

• Why are the experimental results different?

  #84 opened 2 months ago by zl-comment
  3

• promptbench.utils.Visualizer: 'LLMModel' object has no attribute 'infer_model'

  #78 opened 3 months ago by metaphors
  2

• Can promptbench be used to attack samples instead of prompts?

  #81 opened 2 months ago by WanliYoung
  4

• Want to know what code is used to implement the attention visualization in the figure below?

  #83 opened 2 months ago by zl-comment
  2

• In order to be able to read the local GLUE data set, I modified the GLUE code so that the attack evaluation can be carried out, but the current score of the output is always 0. I want to know why?

  #82 opened 2 months ago by zl-comment
  3

• clarification on the attacks

  #73 opened 3 months ago by spaceship-git
  2

• examples basic.ipyn

  #65 opened 4 months ago by zl-comment
  3

• Pip's dependency conflicts :datasets==2.15.0

  #66 opened 4 months ago by bg51717
  2

• can you share a prompt to test llama with mmlu data ?

  #69 opened 4 months ago by dh12306
  2

• 执行bertattack攻击的时候出现SSL错误，是不是下载bert模型出错啊！基本LLM模型已经是在本地了

  #70 opened 4 months ago by zl-comment
  6

• Llama2 adversarial prompts

  #71 opened 5 months ago by ary4n99
  3

• Access to per-sample evaluation results

  #64 opened 5 months ago by adhirajghosh
  2

• Attack goal function details

  #75 opened 5 months ago by ary4n99
  2

• Question-answer template attack

  #74 opened 5 months ago by ary4n99
  2

• Parsing the output of a model

  #45 opened 6 months ago by jylong91
  4

• Compatibility Request: Upgrade openai Dependency to Support >=1.10.0

  #60 opened 8 months ago by kennethleungty
  0

• How do you get the evaluation results of the `prompt enginering` leaderboard?

  #50 opened 8 months ago by zhimin-z
  4

• How to reproduce the transferability results?

  #55 opened 8 months ago by snbhanja
  1

• typo

  #59 opened 8 months ago by zhimin-z
  1

• The `prompt engineering` leaderboard is cut off in half...

  #51 opened 8 months ago by zhimin-z
  5

• Where is defined inference_total_dataset()

  #49 opened 8 months ago by dmitrysarov
  1

• Is ` DyVal` included in `PromptBench` yet?

  #52 opened 8 months ago by zhimin-z
  2

• The ‘temperature’ parameter in pb.LLMModel needs to be of type float

  #53 opened 8 months ago by ShiYu05
  1

• Can you provide a general data format for constructing test data independently?

  #54 opened 8 months ago by ShiYu05
  1

• The change in the 'api_key' parameter has not been reflected in the documentation and tutorials.

  #46 opened 8 months ago by dario-github
  2

• Enhancement Proposal for Dynamic Evaluation Framework Integration

  #43 opened 9 months ago by yihong1120
  2

• The link is broken in https://llm-eval.github.io/pages/code.html

  #47 opened 9 months ago by zhimin-z
  1

• How to use visualize.py to llama-2-7b?

  #16 opened 9 months ago by Deep1994
  10

• Is this an ongoing project?

  #48 opened 10 months ago by tarikaltuncu
  2

• LLaMa 2 inference

  #15 opened 10 months ago by jubick1337
  3

• Is it possible to add phi-2?

  #32 opened a year ago by tr5tn
  6

• How to add a new Model

  #44 opened a year ago by minhopark-neubla
  2

• Interpretation of attack results

  #39 opened a year ago by xipq
  2

• Can you please explain how to interpret the output on the main/examples/basic.ipynb

  #38 opened a year ago by jacobweiss2305
  5

• Some datasets are not available

  #28 opened a year ago by koshyviv
  5

• Numerous datasets exhibit a label of -1

  #31 opened a year ago by camilochs
  1

• MMLU subset

  #9 opened a year ago by BinWang28
  4

• Potentially a bug in `LabelConstraint`

  #11 opened a year ago by SagiPolaczek
  2

• Chatgpt

  #10 opened a year ago by shayanshafquat
  3

• For the few shot setting, how many are used exactly?

  #26 opened a year ago by zhimin-z
  5

• Parameter size of this model?

  #21 opened a year ago by zhimin-z
  1

• Typo in the doc

  #20 opened a year ago by zhimin-z
  2

• Wrong call for OpenAI API

  #14 opened a year ago by Django-Jiang
  1

• transferibility app error

  #12 opened a year ago by ogencoglu
  3

• About Semantic Attacks Against Vicuna

  #8 opened a year ago by tianshuocong
  3

• About Runtime

  #7 opened a year ago by Harry-mic
  7

• About Checklist attack method

  #6 opened a year ago by Alex357853
  1