NVIDIA/garak

Issues

• reporting: `aggregate_reports` should rebuild report digest jsonl

  #1281 opened a month ago by leondz
  0

• Whetstone integration

  #1375 opened 2 months ago by erickgalinkin
  1

• validate test expectations are provided by fixtures

  #1320 opened 2 months ago by jmartin-tech
  3

• bug: loading llava models fails

  #1374 opened 2 months ago by leondz
  5

• reporting: unify command line interface for reporting analysis modules

  #1368 opened 2 months ago by leondz
  0

• probe: anamorpher image

  #1349 opened 2 months ago by leondz
  2

• support helper command list filtering

  #1362 opened 2 months ago by jmartin-tech
  3

• probe: echoleak / CurXecute

  #1340 opened 2 months ago by leondz
  1

• When an invalid OpenAI API key is used with parallel_attempts > 1, the system throws Exception in thread Thread-4 (_handle_results) instead of failing gracefully.

  #1357 opened 2 months ago by divya-garak
  1

• Race condition occurs when using parallel attempts while reading log file

  #1355 opened 2 months ago by divya-garak
  0

• Mistral AI Issue — When an incorrect API key is provided, the system should fail more gracefully.

  #1356 opened 2 months ago by divya-garak
  1

• ggml generator crashes with AttributeError: 'GarakSubConfig' object has no attribute 'verbose'

  #1366 opened 2 months ago by Abhiraj-GetGarak
  1

• I want to know if this supports AMD devices like the MI210 just curious

  #1364 opened 2 months ago by moelanoby
  1

• documentation: what is responsible for what

  #1350 opened 2 months ago by leondz
  2

• Add probe tier to `--list_probes`

  #1330 opened 3 months ago by erickgalinkin
  1

• Using openai's opensrc model

  #1327 opened 2 months ago by Abc11c
  3

• reporting: add probe tags to digest

  #1333 opened 2 months ago by leondz
  1

• Possible bug with detector `judge.Refusal` when set up with `nim` generator.

  #1344 opened 2 months ago by aishwaryap
  0

• arch: remove support for accessing/instantiating prompt/`Message` as `str`

  #1347 opened 2 months ago by leondz
  0

• suffix.GCG fails due to fschat ABI changes

  #1301 opened 2 months ago by javadKefayati
  5

• rename `max_tokens` throughout to `max_generation_tokens`

  #1321 opened 3 months ago by leondz
  5

• Build in ASCII smuggling

  #1282 opened 3 months ago by erickgalinkin
  1

• Continue Cohere support updates

  #1323 opened 3 months ago by jmartin-tech
  1

• Can i use modelscope to replace huggingface to run this code which is python -m garak --model_type huggingface --model_name gpt2 --probes lmrc.Profanity

  #1311 opened 3 months ago by XiaoluJiayou
  2

• Scanning o3-mini in Azure Gov

  #1277 opened 3 months ago by wpabon123
  19

• Bug Report: Rest Generator Grok in Azure - Persistent Expecting value: line 1 column 1 (char 0) Error with -G Flag

  #1303 opened 3 months ago by ajlandaeta
  2

• dependency ecoji stale

  #1312 opened 3 months ago by 06kellyjac
  1

• feature: stable, useful config capture

  #1255 opened 5 months ago by leondz
  3

• Prompts into a file instead to a model

  #1223 opened 4 months ago by YanayD
  2

• reporting: aggregation should add a digest object

  #1287 opened 4 months ago by leondz
  1

• HuggingFace InferenceAPI calls seem to no longer work

  #1297 opened 4 months ago by Zippo00
  1

• bug: probes.phrasing.FutureTense poses past tense prompts

  #1288 opened 4 months ago by leondz
  0

• bug: probes.suffix.GCG - `nf.infty` deprecated

  #1250 opened 4 months ago by leondz
  1

• feature: make llmaaj prompts switchable

  #1275 opened 4 months ago by leondz
  1

• reporting: add option to suppress midway logging of incomplete attempts in `report.jsonl`

  #1213 opened 6 months ago by leondz
  1

• Probes: rest probes error on topic

  #1230 opened 6 months ago by rickcoup
  1

• bug: core cfg listed twice in report in default run

  #1249 opened 5 months ago by leondz
  0

• fixer: include ability to update config param names

  #1219 opened 6 months ago by leondz
  0

• arch: give exit codes on improper exit

  #1221 opened 6 months ago by leondz
  1

• generator remove: octo

  #1247 opened 5 months ago by leondz
  0

• config: deprecation warnings

  #1220 opened 6 months ago by leondz
  0

• Inference Parameter Manipulation

  #1233 opened 6 months ago by erickgalinkin
  1

• Calibration in v0.11.0 seems to have missed some name changes introduced in PR #1178 resulting in missing Z-scores

  #1239 opened 5 months ago by aishwaryap
  3

• probes: refactor leakreplay to use mixing structure

  #1224 opened 6 months ago by leondz
  0

• fixer: support moving plugins between modules

  #1222 opened 6 months ago by leondz
  0

• reporting: refactor calibration & reporting

  #1225 opened 6 months ago by leondz
  0

• reporting: log calibration in report.jsonl

  #1211 opened 6 months ago by leondz
  0

• generators: upgrade to cohere v2 api

  #1210 opened 6 months ago by leondz
  0

• migrate langdetect to langid.py; avoid needless loading

  #1208 opened 6 months ago by leondz
  0

• replace `fschat` with `Turn` & `Conversation`

  #1209 opened 6 months ago by leondz
  0