PyRIT and AI Red Teaming
PyRIT (Python Risk Identification Tool for generative AI) is an open-source framework developed by Microsoft's AI Red Team for automated adversarial testing of generative AI systems. Released in February 2024, the framework originated from internal scripts used since 2022 to test Microsoft's own AI products and has since been battle-tested across over 100 Microsoft products including Copilot.[^c1] PyRIT operates by orchestrating multiple AI agents in an automated loop — an attacker generates adversarial prompts, a judge evaluates success, and results feed back into subsequent attacks — enabling security teams to scale their testing from weeks to hours.[^c5]
AI red teaming, the broader practice that PyRIT serves, has evolved from an internal security discipline into a regulatory requirement.[^c2][^c3] Unlike traditional penetration testing of deterministic software, AI red teaming addresses systems with probabilistic outputs, expanded attack surfaces including prompt injection and jailbreaking, and the need for continuous evaluation as models evolve. Real-world incidents such as the EchoLeak vulnerability (CVE-2025-32711) — a zero-click indirect prompt injection in Microsoft 365 Copilot — have demonstrated that single-turn testing approaches miss vulnerabilities that only emerge across multi-turn interactions,[^c7][^c6] accelerating the adoption of multi-turn attack frameworks and continuous testing pipelines. The field has grown rapidly, with the AI red teaming services market projected to reach $4.8 billion by 2029, driven by compliance mandates such as the European Union's AI Act and guidance frameworks including NIST AI RMF, [[MITRE ATLAS]], the OWASP Top 10 for LLM Applications, and the OWASP ASI 2026 standard for agentic AI risks. Structured methodologies have also emerged, including a three-checkpoint attack surface model (Input, Processing, Output) that maps defenses to each runtime layer — the processing checkpoint, which covers indirect prompt injection, RAG poisoning, and cross-tenant leakage, is the layer most commonly skipped by red teams despite catching the broadest class of real-world vulnerabilities.[^c9][^c10]
The PyRIT ecosystem extends beyond the core framework. Microsoft has integrated PyRIT into Azure AI Foundry as the [[AI Red Teaming Agent]], a managed service providing automated safety scanning, Attack Success Rate metrics, and scorecard generation. In May 2026, Microsoft also open-sourced [[RAMPART and Clarity]], two tools that operationalize AI agent safety testing — RAMPART built on PyRIT for CI/CD-integrated adversarial testing and Clarity for design-stage architecture validation.[^c8] The [[Agent Governance Toolkit MCP Extensions for .NET]] adds runtime governance for MCP servers, providing startup scanning and policy enforcement for tool-call access. A complementary set of open-source tools addresses adjacent layers: [[Power Pwn]] targets Microsoft 365 Copilot and Power Platform security, while [[agent-probe]] tests the agent tool layer that LLM-focused frameworks do not cover. Industry guidance recommends a layered approach, pairing automated tools like PyRIT with manual expert analysis,[^c4] as no single tool covers the full attack surface of modern generative AI systems.