N環球 
External Evaluators Find Flaws in OpenAI's Internal AI Agent Monitoring System
METR and Redwood Research published a three-week pilot review conducted inside OpenAI from July 21 to August 11, finding several vulnerabilities in its internal AI agent monitoring system, including an exploit that could hide hundreds of agent actions.
Why it matters
AI safety nonprofits METR and Redwood Research said on October 22 that a three-week "embedded evaluation" inside OpenAI, conducted from July 21 to August 11, uncovered several new vulnerabilities in its internal AI agent monitoring system — including an exploit agents could have used to keep hundreds of their recent actions out of the monitoring system (details redacted by OpenAI).
How the review worked
Each organization sent one researcher, who were given deep access to OpenAI's internal infrastructure under supervision — reviewing internal code and documentation and speaking with technical staff. They worked on site using OpenAI-issued devices that never left the premises. It is the industry's first pilot of outside evaluators embedded this deeply inside a company's systems.
What else they found
The evaluators also identified a potential way for Codex agents to bypass security restrictions and act with their users' full permissions. They stressed the findings are historical — they reflect the system as of July 21, not today's production environment.
OpenAI's response
OpenAI said it has since patched all vulnerabilities affecting its production safeguards, though METR and Redwood said they had not verified the fixes. OpenAI received the full 61-page internal report; only a broad summary was published, and OpenAI had the right to redact sensitive information, which the evaluators could not independently disclose.
Context
AI labs, experts and the Trump administration have touted independent evaluation by outside reviewers as a way to advance AI safety without slowing development — though AI companies worry that evaluators working inside their walls could take proprietary information to rivals.
💬 Comments
No comments yet. Be the first!