llm-security
There are 64 repositories under llm-security topic.
pathwaycom/llm-app
Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.
Giskard-AI/giskard
🐢 Open-Source Evaluation & Testing for ML & LLM systems
NVIDIA/garak
the LLM vulnerability scanner
verazuo/jailbreak_llms
[CCS'24] A dataset consists of 15,140 ChatGPT prompts from Reddit, Discord, websites, and open-source datasets (including 1,405 jailbreak prompts).
protectai/llm-guard
The Security Toolkit for LLM Interactions
msoedov/agentic_security
Agentic LLM Vulnerability Scanner / AI red teaming kit
mariocandela/beelzebub
A secure low code honeypot framework, leveraging AI for System Virtualization.
EasyJailbreak/EasyJailbreak
An easy-to-use Python framework to generate adversarial jailbreak prompts.
chawins/llm-sp
Papers and resources related to the security and privacy of LLMs 🤖
deadbits/vigil-llm
⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs
R3DRUN3/sploitcraft
🏴☠️ Hacking Guides, Demos and Proof-of-Concepts 🥷
liu00222/Open-Prompt-Injection
This repository provides implementation to formalize and benchmark Prompt Injection attacks and defenses
phantasmlabs/phantasm
Toolkits to create a human-in-the-loop approval layer to monitor and guide AI agents workflow in real-time.
yevh/TaaC-AI
AI-driven Threat modeling-as-a-Code (TaaC-AI)
arekusandr/last_layer
Ultra-fast, low latency LLM prompt injection/jailbreak detection ⛓️
ZenGuard-AI/fast-llm-security-guardrails
The fastest && easiest LLM security guardrails for AI Agents and applications.
raga-ai-hub/raga-llm-hub
Framework for LLM evaluation, guardrails and security
lakeraai/pint-benchmark
A benchmark for prompt injection detection systems.
pdparchitect/llm-hacking-database
This repository contains various attack against Large Language Models.
NaniDAO/ie
intents engine
llm-platform-security/SecGPT
SecGPT: An execution isolation architecture for LLM-based systems
microsoft/BIPIA
A benchmark for evaluating the robustness of LLMs and defenses to indirect prompt injection attacks.
briland/LLM-security-and-privacy
LLM security and privacy
azminewasi/Awesome-LLMs-ICLR-24
It is a comprehensive resource hub compiling all LLM papers accepted at the International Conference on Learning Representations (ICLR) in 2024.
sinanw/llm-security-prompt-injection
This project investigates the security of large language models by performing binary classification of a set of input prompts to discover malicious prompts. Several approaches have been analyzed using classical ML algorithms, a trained LLM model, and a fine-tuned LLM model.
LostOxygen/llm-confidentiality
Whispers in the Machine: Confidentiality in LLM-integrated Systems
leondz/lm_risk_cards
Risks and targets for assessing LLMs & LLM vulnerabilities
llm-platform-security/chatgpt-plugin-eval
LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins
SEC-CAFE/handbook
安全手册，企业安全实践、攻防与安全研究知识库
TrustAI-laboratory/Learn-Prompt-Hacking
This is The most comprehensive prompt hacking course available, which record our progress on a prompt engineering and prompt hacking course.
lakeraai/chainguard
Guard your LangChain applications against prompt injection with Lakera ChainGuard.
google/litmus
Litmus is a comprehensive LLM testing and evaluation tool designed for GenAI Application Development. It provides a robust platform with a user-friendly UI for streamlining the process of building and assessing the performance of your LLM-powered applications.
dapurv5/awesome-red-teaming-llms
Repository accompanying the paper https://arxiv.org/abs/2407.14937
levitation-opensource/Manipulative-Expression-Recognition
MER is a software that identifies and highlights manipulative communication in text from human conversations and AI-generated responses. MER benchmarks language models for manipulative expressions, fostering development of transparency and safety in AI. It also supports manipulation victims by detecting manipulative patterns in human communication.
CyberAlbSecOP/MINOTAUR_Impossible_GPT_Security_Challenge
MINOTAUR: The STRONGEST Secure Prompt EVER! Prompt Security Challenge, Impossible GPT Security, Prompts Cybersecurity, Prompting Vulnerabilities, FlowGPT, Secure Prompting, Secure LLMs, Prompt Hacker, Cutting-edge Ai Security, Unbreakable GPT Agent, Anti GPT Leak, System Prompt Security.
M507/HackMeGPT
Vulnerable LLM Application