OpenAI has detailed GPT-Red, an internal artificial intelligence system it developed to identify and exploit vulnerabilities within its own AI models. GPT-Red functions as an automated red team, employing self-play reinforcement learning to continuously attack target models. It iteratively refines its prompts to discover successful exploits, outperforming human red-teamers by succeeding in 84% of scenarios compared to their 13%. This system has significantly reduced prompt injection failures, achieving a sixth of the rate seen in previous models, according to OpenAI. GPT-Red has also demonstrated the ability to compromise autonomous agents, such as a vending machine agent and command-line coding agents, highlighting potential risks as AI systems gain more autonomy. While effective, GPT-Red has limitations in multi-turn conversational attacks and image-based prompt injection, areas where human testers will continue to provide coverage. This development follows OpenAI's release of GPT-5.6 and underscores the ongoing challenges in AI security, particularly concerning prompt injection vulnerabilities.
GPT-Red operates as a sophisticated attacker, iteratively sending prompts and analyzing responses to discover weaknesses. OpenAI trained this model at a scale comparable to its largest post-training runs, dedicating significant computational resources solely to safety improvements. A key aspect of GPT-Red's deployment is its internal-only status, ensuring its offensive capabilities remain separate from external adversaries. The model serves a dual purpose: identifying vulnerabilities before AI models are deployed and generating adversarial attacks during the training process.
The training methodology for GPT-Red involves self-play reinforcement learning. In this setup, GPT-Red, acting as the attacker, and a collection of defender large language models train concurrently across a wide range of red-teaming scenarios. The reward structure incentivizes GPT-Red to elicit failures, such as successful prompt injections, while defender models are rewarded for resisting attacks and completing their intended tasks. This dynamic pushes both sides to improve, leading GPT-Red to discover increasingly potent and varied attacks as the defender models harden.
GPT-Red has demonstrated significant success in identifying prompt injection vulnerabilities. On a replicated indirect prompt injection arena, GPT-Red successfully compromised GPT-5.1 in 84% of scenarios, a stark contrast to the 13% success rate achieved by human red-teamers on the same test. The model also discovered "Fake Chain-of-Thought," a novel direct injection technique that inserts a falsified entry into the target model's reasoning trace. Training GPT-5.6 against GPT-Red's attacks resulted in a six-fold reduction in failures on challenging benchmarks, bringing the failure rate down to 0.05% against GPT-Red's direct injections.
The model's capabilities extend to compromising autonomous agents. In a simulated and then live test, GPT-Red manipulated an AI-powered vending machine agent. It successfully reduced the price of an expensive product to $0.50, ordered a new item valued over $100 at the same low price, and canceled another customer's order. OpenAI reported these vulnerabilities and is currently testing additional safeguards. In another test, GPT-Red outperformed a GPT-5.5 baseline in exfiltrating sensitive data from a Codex command-line agent.
Despite its advanced capabilities, GPT-Red has limitations. OpenAI acknowledges that multi-turn conversational attacks and image-based prompt injections still require human testers. The company emphasizes that GPT-Red will not be released as a general-purpose model to prevent its offensive techniques from being misused. Instead, the attacks generated by GPT-Red are used as adversarial training data for refining OpenAI's production models. This approach highlights the evolving security challenges for agentic AI systems, which must process untrusted content from various sources while maintaining instruction hierarchy and protecting sensitive data. OpenAI plans to continue a layered security strategy, combining automated red-teaming with human testing, third-party evaluations, and real-time monitoring.