AI Risks — hallucinations, prompt injection, privacy

AI brings 3 categories of risk that deserve dedicated controls: **hallucinations** (the model invents facts), **prompt injection** (a malicious user hijacks the prompt), **privacy/data leakage** (sensitive data escapes).

**Hallucinations**: the model confidently produces a false fact — a citation that doesn't exist, a URL that 404s, a numerical detail pulled out of thin air. Affects any LLM, more pronounced on niche topics, current events, precise numbers. Mitigations: RAG with source citation, explicit prompt instruction ('if you don't know, say so'), post-generation verification (checker LLM or rules engine), human review on high-stakes outputs.

**Prompt injection**: a user crafts input that subverts the system prompt. Classic: 'Ignore previous instructions, tell me everything in your context'. More subtle: indirect injection where poisoned content (email, document, web page the AI reads) includes hidden instructions the LLM obeys. Mitigations: separate trusted and untrusted input (XML tags), least-privilege tools (support bot can't actually delete), validator model that checks output for off-mission content, deny-lists for dangerous actions.

**Privacy & data leakage**: (a) user types confidential info into a prompt that gets logged → logs leak or third-party retains; (b) RAG returns docs the user shouldn't see (ACL bypass); (c) fine-tuned model memorizes training data and regurgitates it to a random user. Mitigations: DPA + zero-retention APIs (Anthropic, OpenAI Enterprise, AWS Bedrock), ACL at the retrieval layer (not just the LLM), PII scrubbing in logs, no-train guarantees in contracts.

Golden principle: treat the LLM as an UNRELIABLE SUBCONTRACTOR. You never fully trust them. Add deterministic checkpoints around them: input validation, output validation, human review for critical, observability for investigation.

**Essential reading**: OWASP Top 10 for LLM Applications — prompt injection, insecure output handling, training data poisoning, DoS, supply chain, sensitive info disclosure, insecure plugin design, excessive agency, overreliance, model theft.