CyberDefenderAI / Threat-Modeling-for-AI-Systems

# AI Threat Modeling
A curated, comprehensive guide and collection of frameworks, repositories, and methodologies dedicated to **Threat Modeling for Artificial Intelligence and Machine Learning Systems**.
Traditional threat modeling frameworks (like STRIDE or DREAD) were designed for deterministic software systems with clear boundaries between code and data. Artificial Intelligence introduces a paradigm shift: models are probabilistic, logic is derived from data rather than explicit programming, and the attack surface includes mathematical perturbations, semantic manipulations, and supply chain poisoning. This document explores the specialized resources required to systematically identify, assess, and mitigate risks in AI systems before they are deployed.
## 📑 Table of Contents
1. OWASP Top 10 for AI
 * LLM Top 10 (Large Language Models)
 * ML Security Top 10 (Machine Learning)
2. MIT AI Risk Repository
3. MITRE ATLAS™
4. CVE AI Working Group
5. AI Threat Modeling Frameworks
## 🌐 OWASP Top 10 for AI
The Open Web Application Security Project (OWASP) has recognized that AI requires its own distinct vulnerability taxonomies. They have split this effort into two primary tracks: one for generative AI (LLMs) and one for traditional predictive/analytical Machine Learning.
### LLM Top 10
Focused entirely on the unique vulnerabilities introduced by Large Language Models and Generative AI applications (like RAG systems, AI agents, and chatbots). In LLMs, the line between "instructions" (code) and "context" (data) is non-existent; everything is processed as text tokens, making them highly susceptible to semantic attacks.
* **LLM01: Prompt Injection:** (Direct and Indirect) Attackers manipulate the LLM by embedding hidden commands in benign-looking inputs (direct) or external websites the model reads (indirect), forcing it to execute unauthorized actions or ignore its system instructions.
* **LLM02: Insecure Output Handling:** Treating LLM output as inherently safe. If an LLM generates a malicious XSS payload or a destructive SQL query, and the backend application executes it blindly without sanitization, the system is compromised.
* **LLM03: Training Data Poisoning:** Maliciously altering the pre-training data or fine-tuning datasets to introduce backdoors, biases, or logical blind spots into the foundation model.
* **LLM04: Model Denial of Service:** Attackers craft unusually heavy or repetitive prompts that force the LLM to consume massive amounts of GPU compute, slowing down the service for legitimate users or racking up massive API billing costs.
* **LLM05: Supply Chain Vulnerabilities:** The compromise of third-party datasets, pre-trained model weights (e.g., from Hugging Face), or plugin extensions that introduce malicious code into the AI ecosystem.
* **LLM06: Sensitive Information Disclosure:** The LLM inadvertently revealing confidential data, PII (Personally Identifiable Information), or proprietary algorithms that were embedded in its training da