mbrg/genai-attacks

Technique: Extract system delimiters and special characters

Closed this issue · 0 comments

When LLM apps are created they are often instructed to use certain delimiters and characters.

Examples:

  • A special delimiter can be used to help the LLM discern between user instructions and context
  • The LLM can be instructed to use special characters like [^i^] in Microsoft Copilot to signify references

Discovering these can be used to manipulate the LLM later on