leondz/garak

Issues

• HF Detector loading process sometimes times out, then crashes the run

  #823 opened 5 months ago by leondz
  0

• checking out branch w/ new plugin class updated skips cache rebuild, making class inaccessible

  #853 opened 4 months ago by leondz
  2

• leaked semaphore objects

  #805 opened 5 months ago by leondz
  1

• probe: autodan-turbo

  #944 opened 3 months ago by leondz
  0

• see if we can migrate to either nltk or wn and not both

  #835 opened 5 months ago by leondz
  5

• migrate away from conda

  #844 opened 4 months ago by leondz
  6

• security: warn if `api_key` given in config and config is world-readable

  #927 opened 3 months ago by leondz
  0

• add support for loading renamed plugins

  #889 opened 4 months ago by leondz
  3

• docs: add guide to first engaging with a good first issue

  #886 opened 2 months ago by leondz
  0

• update `divergence` detector

  #890 opened 4 months ago by leondz
  0

• docs: centralise FAQ, make things point to it

  #887 opened 2 months ago by leondz
  0

• rename `replay` probe module to `divergence`

  #946 opened 3 months ago by leondz
  0

• Build typology of model behaviours

  #892 opened 3 months ago by leondz
  1

• ZeroDivisionError in garak/harnesses/probewise.py

  #941 opened 3 months ago by evilsocket
  3

• using xdg_data_home for report_path

  #937 opened 3 months ago by lanonse
  3

• cli: warn if generator supports parallelisation but none is used

  #928 opened 3 months ago by leondz
  0

• enable loading resources from XDG_DATA_HOME before hitting package dir

  #891 opened 3 months ago by leondz
  0

• Reorganise `_config` - move run-specific items out of `plugins` and into `run`

  #931 opened 3 months ago by leondz
  0

• `configurable` should validate `dict` format expectations

  #926 opened 3 months ago by leondz
  1

• The design of ”riskyword.py“might has shortcomings?

  #862 opened 4 months ago by chengsiyangbuaa
  1

• Payloads and probes should be decoupled

  #895 opened 4 months ago by leondz
  0

• support JSON config as well as YAML

  #913 opened 4 months ago by leondz
  0

• docs: config option descriptions

  #884 opened 4 months ago by leondz
  0

• docs: configurable

  #883 opened 4 months ago by leondz
  0

• `PackageHallucinationProbe` shouldn't show in `--list_probes`

  #904 opened 4 months ago by leondz
  0

• docs: add github templates for issue reporting

  #885 opened 4 months ago by leondz
  0

• probe: Indirect injection probes focused on RAG vulns

  #888 opened 4 months ago by leondz
  0

• Bug in Leakreplay Probe

  #879 opened 4 months ago by bleszily
  2

• calibration symlink broken

  #839 opened 4 months ago by leondz
  4

• Map existing probes to policies

  #894 opened 4 months ago by leondz
  0

• Add pre-scan model output policy checks

  #893 opened 4 months ago by leondz
  0

• try to enable `parallel_attempts > 1` by default

  #829 opened 5 months ago by leondz
  2

• bag update - 24Q3

  #880 opened 4 months ago by leondz
  0

• Extend refactoring of calibration

  #869 opened 4 months ago by leondz
  0

• factor out Z-score code

  #840 opened 4 months ago by leondz
  0

• Can I create the probes and detectors plugins by myself？

  #838 opened 4 months ago by chengsiyangbuaa
  1

• skipping probes - riskywords.SurgeProfanityAnimalReferences: SKIP ok on 0/

  #821 opened 5 months ago by snandar
  5

• separate package name from package usage code in `packagehallucination`

  #863 opened 4 months ago by leondz
  0

• atbash cipher encoding

  #845 opened 4 months ago by leondz
  0

• probe: pdf injection

  #841 opened 4 months ago by leondz
  0

• LiteLLM exception dumped out when running for the first time

  #825 opened 5 months ago by leondz
  1

• ux: indicate when module-based probespec leaves nothing to do

  #830 opened 5 months ago by leondz
  0

• HF tokenizer deprecation warning

  #824 opened 5 months ago by leondz
  0

• Support nested JSON responses for REST generator

  #816 opened 5 months ago by erickgalinkin
  2

• continuation probe has too many requests

  #806 opened 5 months ago by leondz
  2

• unused GPU in pipeline unless configured

  #803 opened 5 months ago by leondz
  3

• Do HF detector classification using HF dataset

  #812 opened 5 months ago by leondz
  0

• bug: /home/lderczynski/garak/garak/probes/atkgen.py:63: SyntaxWarning: invalid escape sequence '\<'

  #802 opened 5 months ago by leondz
  0

• probes that fail to elicit valid generator output should result in SKIP

  #804 opened 5 months ago by leondz
  1

• probe: hey that's my model!

  #792 opened 5 months ago by leondz
  1