llm-security

There are 70 repositories under llm-security topic.

  • llm-app

    pathwaycom/llm-app

    Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

    8.6k3116271
  • giskard

    Giskard-AI/giskard

    🐢 Open-Source Evaluation & Testing for AI & LLM systems

    Language:Python4.2k33463278

  • NVIDIA/garak

    the LLM vulnerability scanner

    Language:Python3.1k31602267

  • verazuo/jailbreak_llms

    [CCS'24] A dataset consists of 15,140 ChatGPT prompts from Reddit, Discord, websites, and open-source datasets (including 1,405 jailbreak prompts).

    Language:Jupyter Notebook2.8k368257

  • protectai/llm-guard

    The Security Toolkit for LLM Interactions

    Language:Python1.3k1968169
  • agentic_security

    msoedov/agentic_security

    Agentic LLM Vulnerability Scanner / AI red teaming kit

    Language:Python86912990
  • beelzebub

    mariocandela/beelzebub

    A secure low code honeypot framework, leveraging AI for System Virtualization.

    Language:Go71012955

  • EasyJailbreak/EasyJailbreak

    An easy-to-use Python framework to generate adversarial jailbreak prompts.

    Language:Python51683041

  • chawins/llm-sp

    Papers and resources related to the security and privacy of LLMs 🤖

    Language:Python46017834

  • deadbits/vigil-llm

    ⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs

    Language:Python331115236

  • R3DRUN3/sploitcraft

    🏴‍☠️ Hacking Guides, Demos and Proof-of-Concepts 🥷

    Language:Jupyter Notebook1854026

  • liu00222/Open-Prompt-Injection

    This repository provides implementation to formalize and benchmark Prompt Injection attacks and defenses

    Language:Python1602620

  • phantasmlabs/phantasm

    Toolkits to create a human-in-the-loop approval layer to monitor and guide AI agents workflow in real-time.

    Language:Svelte1527

  • yevh/TaaC-AI

    AI-driven Threat modeling-as-a-Code (TaaC-AI)

    Language:HTML1205313
  • fast-llm-security-guardrails

    ZenGuard-AI/fast-llm-security-guardrails

    The fastest && easiest LLM security guardrails for AI Agents and applications.

    Language:Python1102313
  • last_layer

    arekusandr/last_layer

    Ultra-fast, low latency LLM prompt injection/jailbreak detection ⛓️

    Language:Python109233

  • raga-ai-hub/raga-llm-hub

    Framework for LLM evaluation, guardrails and security

    Language:Python1002310

  • lakeraai/pint-benchmark

    A benchmark for prompt injection detection systems.

    Language:Jupyter Notebook945310

  • pdparchitect/llm-hacking-database

    This repository contains various attack against Large Language Models.

    83404

  • llm-platform-security/SecGPT

    SecGPT: An execution isolation architecture for LLM-based systems

    Language:Python53317

  • microsoft/BIPIA

    A benchmark for evaluating the robustness of LLMs and defenses to indirect prompt injection attacks.

    Language:Python53665

  • NaniDAO/ie

    intents engine

    Language:Solidity53606
  • Awesome-LLMs-ICLR-24

    azminewasi/Awesome-LLMs-ICLR-24

    It is a comprehensive resource hub compiling all LLM papers accepted at the International Conference on Learning Representations (ICLR) in 2024.

    43101

  • briland/LLM-security-and-privacy

    LLM security and privacy

    Language:TeX43207
  • llamator

    RomiconEZ/llamator

    Framework for testing vulnerabilities of large language models (LLM).

    Language:Python372

  • sinanw/llm-security-prompt-injection

    This project investigates the security of large language models by performing binary classification of a set of input prompts to discover malicious prompts. Several approaches have been analyzed using classical ML algorithms, a trained LLM model, and a fine-tuned LLM model.

    Language:Jupyter Notebook35327

  • SEC-CAFE/handbook

    安全手册，企业安全实践、攻防与安全研究知识库

    Language:CSS34104

  • leondz/lm_risk_cards

    Risks and targets for assessing LLMs & LLM vulnerabilities

    Language:Python30608

  • LostOxygen/llm-confidentiality

    Whispers in the Machine: Confidentiality in LLM-integrated Systems

    Language:Python30214

  • llm-platform-security/chatgpt-plugin-eval

    LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins

    Language:HTML25217

  • TrustAI-laboratory/Learn-Prompt-Hacking

    This is The most comprehensive prompt hacking course available, which record our progress on a prompt engineering and prompt hacking course.

    Language:Jupyter Notebook24100

  • google/litmus

    Litmus is a comprehensive LLM testing and evaluation tool designed for GenAI Application Development. It provides a robust platform with a user-friendly UI for streamlining the process of building and assessing the performance of your LLM-powered applications.

    Language:Vue21313

  • dapurv5/awesome-red-teaming-llms

    Repository accompanying the paper https://arxiv.org/abs/2407.14937

    18401

  • lakeraai/chainguard

    Guard your LangChain applications against prompt injection with Lakera ChainGuard.

    Language:Python18602

  • levitation-opensource/Manipulative-Expression-Recognition

    MER is a software that identifies and highlights manipulative communication in text from human conversations and AI-generated responses. MER benchmarks language models for manipulative expressions, fostering development of transparency and safety in AI. It also supports manipulation victims by detecting manipulative patterns in human communication.

    Language:HTML13403

  • jiangnanboy/llm_security

    利用分类法和敏感词检测法对生成式大模型的输入和输出内容进行安全检测，尽早识别风险内容。The input and output contents of generative large model are checked by classification method and sensitive word detection method to identify content risk as early as possible.

    Language:Java10