prompt-injection

There are 77 repositories under prompt-injection topic.

  • CyberAlbSecOP/Awesome_GPT_Super_Prompting

    ChatGPT Jailbreaks, GPT Assistants Prompt Leaks, GPTs Prompt Injection, LLM Prompt Security, Super Prompts, Prompt Hack, Prompt Security, Ai Prompt Engineering, Adversarial Machine Learning.

    1.4k312173

  • protectai/llm-guard

    The Security Toolkit for LLM Interactions

    Language:Python1.3k1968168

  • protectai/rebuff

    LLM Prompt Injection Detector

    Language:TypeScript1.1k155682

  • whylabs/langkit

    🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability. 👀

    Language:Jupyter Notebook862165869

  • abilzerian/LLM-Prompt-Library

    Advanced Code and Text Manipulation Prompts for Various LLMs. Suitable for Siri, GPT-4o, Claude, Llama3, Gemini, and other high-performance open-source LLMs.

    79619078
  • promptmap

    utkusen/promptmap

    automatically tests prompt injection attacks on ChatGPT instances

    Language:Python66812059
  • brilliant-CV

    mintyfrankie/brilliant-CV

    💼 another CV template for your job application, yet powered by Typst and more

    Language:Typst48074733

  • tldrsec/prompt-injection-defenses

    Every practical and proposed defense against prompt injection.

    3614126

  • zacfrulloni/Prompt-Engineering-Holy-Grail

    # Prompt Engineering Hub ⭐️ If you find this helpful, give it a star to show your support! This repository is a one-stop resource for prompt engineering. Also available on: https://promptengineeringhub.dev/

    Language:HTML3358230

  • deadbits/vigil-llm

    ⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs

    Language:Python330115236

  • automorphic-ai/aegis

    Self-hardening firewall for large language models

    Language:Python259306

  • langgptai/Awesome-Multimodal-Prompts

    Prompts of GPT-4V & DALL-E3 to full utilize the multi-modal ability. GPT4V Prompts, DALL-E3 Prompts.

    2282016

  • dropbox/llm-security

    Dropbox LLM Security research code and results

    Language:Python2197023

  • liu00222/Open-Prompt-Injection

    This repository provides implementation to formalize and benchmark Prompt Injection attacks and defenses

    Language:Python1602620

  • yunwei37/prompt-hacker-collections

    prompt attack-defense, prompt Injection, reverse engineering notes and examples | 提示词对抗、破解例子与笔记

    1395017

  • lakeraai/pint-benchmark

    A benchmark for prompt injection detection systems.

    Language:Jupyter Notebook945310

  • pasquini-dario/project_mantis

    Project Mantis: Hacking Back the AI-Hacker; Prompt Injection as a Defense Against LLM-driven Cyberattacks

    Language:Python597

  • HumanCompatibleAI/tensor-trust

    A prompt injection game to collect data for robust ML research

    Language:Python4961645

  • gdalmau/lakera-gandalf-solutions

    My inputs for the LLM Gandalf made by Lakera

    40325
  • PromptGuard

    GPTSafe/PromptGuard

    Build production ready apps for GPT using Node.js & TypeScript

    Language:TypeScript391141

  • sinanw/llm-security-prompt-injection

    This project investigates the security of large language models by performing binary classification of a set of input prompts to discover malicious prompts. Several approaches have been analyzed using classical ML algorithms, a trained LLM model, and a fine-tuned LLM model.

    Language:Jupyter Notebook35327

  • LostOxygen/llm-confidentiality

    Whispers in the Machine: Confidentiality in LLM-integrated Systems

    Language:Python30214
  • gandalf_vs_gandalf

    microsoft/gandalf_vs_gandalf

    Turning Gandalf against itself. Use LLMs to automate playing Lakera Gandalf challenge without needing to set up an account with a platform provider.

    Language:Jupyter Notebook27611

  • MaxMLang/pytector

    A Python package designed to detect prompt injection in text inputs utilizing state-of-the-art machine learning models from Hugging Face. The main focus is on ease of use, enabling developers to integrate security features into their applications with minimal effort.

    Language:Python241020

  • TrustAI-laboratory/Learn-Prompt-Hacking

    This is The most comprehensive prompt hacking course available, which record our progress on a prompt engineering and prompt hacking course.

    Language:Jupyter Notebook24100

  • probonodev/jailbreak

    jailbreakme.xyz is an open-source decentralized app (dApp) where users are challenged to try and jailbreak pre-existing LLMs in order to find weaknesses and be rewarded. 🏆

    Language:JavaScript22

  • grepstrength/WideOpenAI

    Short list of indirect prompt injection attacks for OpenAI-based models.

    20102

  • SemanticBrainCorp/SemanticShield

    The Security Toolkit for managing Generative AI(especially LLMs) and Supervised Learning processes(Learning and Inference).

    Language:Python20304

  • wearetyomsmnv/Awesome-LLMSecOps

    LLM | Security | Operations in one github repo with good links and pictures.

    20302

  • ZapDos7/lakera-gandalf

    My solutions for Lakera's Gandalf

    20114

  • lakeraai/chainguard

    Guard your LangChain applications against prompt injection with Lakera ChainGuard.

    Language:Python18602

  • shaialon/ai-security-demos

    🤯 AI Security EXPOSED! Live Demos Showing Hidden Risks of 🤖 Agentic AI Flows: 💉Prompt Injection, ☣️ Data Poisoning. Watch the recorded session:

    Language:JavaScript16103

  • qroa/QROA

    QROA: A Black-Box Query-Response Optimization Attack on LLMs

    Language:Python1410

  • levitation-opensource/Manipulative-Expression-Recognition

    MER is a software that identifies and highlights manipulative communication in text from human conversations and AI-generated responses. MER benchmarks language models for manipulative expressions, fostering development of transparency and safety in AI. It also supports manipulation victims by detecting manipulative patterns in human communication.

    Language:HTML13403

  • peluche/deck-of-many-prompts

    Manual Prompt Injection / Red Teaming Tool

    Language:Python11101

  • Safetorun/PromptDefender

    A prompt defence is a multi-layer defence that can be used to protect your applications against prompt injection attacks.

    Language:Go111270