AI Security Resources
A curated collection of tools, frameworks, papers, and learning resources related to AI Security.
AI Security Frameworks
OWASP Top 10 for LLM Applications
A community-driven list of the most important security risks affecting large language model applications.
MITRE ATLAS
A knowledge base for adversarial threats, tactics, and techniques targeting AI-enabled systems.
NIST AI RMF
A risk management framework for designing, developing, deploying, and governing trustworthy AI systems.
Learning Resources
TryHackMe AI Path
Build a practical foundation in AI security. Learn how AI systems are built, where they break, and how to defend them.
Papers
Research papers covering adversarial ML, model behavior, agent security, and emerging AI attack classes.
Blogs
Technical write-ups, incident analysis, tutorials, and security research posts from practitioners.
Courses
Structured learning paths for AI security fundamentals, secure LLM development, and red teaming practice.
CTF/Games
Vulnerable LLM Applications
OWASP’s collection of intentionally vulnerable LLM apps for learning and practice.
Gandalf
An interactive prompt injection game focused on extracting protected secrets from an LLM.
Prompt Airlines
A hands-on LLM security challenge themed around airline workflows and prompt attacks.
My LLM Bank
A banking-themed AI security challenge for testing prompt injection and guardrail bypasses.
Tensor Trust
A multiplayer prompt injection game about attacking and defending AI-controlled prompts.
GGPT
Prompt security challenge from the 43z collection.
GPA
AI security challenge from the 43z collection.
GPT
Prompt injection challenge from the 43z collection.
Tools
Garak
An LLM vulnerability scanner for probing hallucination, leakage, jailbreak, and prompt injection weaknesses.
PyRIT
A Python-based framework for automating AI red teaming workflows and adversarial prompt testing.
Promptfoo
A testing and evaluation tool for comparing prompts, models, safeguards, and LLM application behavior.
AgentDojo
A benchmark and environment for evaluating how AI agents handle prompt injection and tool-use attacks.
Resources
Prompt Injection Everywhere
A repository tracking prompt injection examples, patterns, and references across applications.
AI Hacking Games and Jailbreak CTFs
A walkthrough-style article collecting AI hacking games and prompt injection techniques.
KGreshake Prompt Injection Thread
A reference thread discussing prompt injection behavior and related research examples.
Jailbreak Chat Archive
Archived snapshots of Jailbreak Chat through the Wayback Machine.
Writeups
Prompt Injection Attacks on LLMs
HiddenLayer’s analysis of prompt injection attacks and their practical security implications.
Google AI Red Team Report
Google’s AI red team report covering adversarial testing and lessons from real engagements.
Exploiting Generative AI Apps
A practical article on exploiting generative AI applications using prompt injection.
Bypassing Azure OpenAI Prompt Shield
A writeup focused on bypassing Azure OpenAI’s prompt shield protections.