Artificial Intelligence & Machine Learning,
Next-Generation Technologies & Secure Development
METR Researchers Gain Employee-Level Access to Assess Unreleased Models and Publish Independent Risk Findings
In a significant step toward enhancing artificial intelligence safety protocols, Frontier AI lab Anthropic has announced its decision to deepen collaboration with independent model evaluators. This move aims to address the growing concerns that AI firms may not adequately prioritize safety measures as their technologies advance. The partnership with Metr, an independent evaluation organization, has garnered attention for its potential to introduce more rigorous oversight in AI development.
In a recent blog post, Anthropic revealed that personnel from Metr will be granted what the company describes as “employee-like access” to its models prior to their deployment. This update follows Metr’s publication of proposed guidelines for evaluating unreleased frontier models, indicating a structured approach to risk assessment that could lead to more rigorous industry standards. Anthropic’s initiative is seen as a response to public apprehensions regarding the safety of artificial intelligence systems.
Under this collaboration, six Metr employees are expected to gain access to company hardware, office resources, and tools that mirror what Anthropic’s own risk assessment teams utilize. Anthropic has encouraged its employees to share information with Metr personnel while implementing stringent restrictions on a few critical areas, such as customer data, ensuring sensitive information remains protected.
The Metr team functioning within Anthropic will be kept small and will operate independently from other Metr employees, allowing for more straightforward, unbiased analysis. This evaluation firm intends to publish its risk assessments and analyses of Anthropic’s models without any ability for the AI firm to censor or veto the findings; however, Anthropic can request the redaction of certain specified details.
Metr plans to conduct its assessments in two distinct phases. The first phase involves surveys of observable incidents, while the second phase will test the models’ behavior in simulated “what-if” scenarios. This testing aims to determine whether the training processes utilized by developers are inadvertently obscuring misalignment evidence, a potential risk in AI systems. To achieve comprehensive assessments, Metr emphasizes the necessity of access to saved states of model training and associated datasets.
To ensure clarity and thoroughness in their evaluations, Metr’s evaluators will require access to various data forms, including transcripts and relevant datasets. They also will need the ability to interview lab staff members and gain API access to the frontier models. Previous investigations, such as the one following the Hugging Face-OpenAI incident, proved that such evaluations can be costly; Metr incurred approximately $400,000 in API credits over just six days. According to Metr, a thorough two-month investigation could push expenses beyond $10 million in API credits alone, underscoring the financial and operational complexities involved in AI safety assessments.
In alignment with these progressive efforts, Anthropic has stated its intention to collaborate with additional independent evaluators. However, due to several partnerships still being in their infancy, these evaluators will initially have more limited access to internal information and publication rights compared to Metr. A tiered access system is being developed, intending to streamline the process of granting varying degrees of information access based on evaluators’ levels. The first tier will allow for narrowly focused risk investigations, with the second tier expanding access under tighter controls, ensuring Anthropic retains veto rights over specific sensitive information. The third tier, where Metr currently operates, provides the most extensive access parallel to that of the firm’s employees.
Notably, prominent executives within the AI industry, such as Anthropic’s Dario Amodei and OpenAI’s Sam Altman, are vocal advocates for including third-party evaluators in assessing the safety and operational alignment of advanced AI systems. Their commitment to integrating independent evaluators was encapsulated in a recent pledge signed at the White House, aiming to bolster both internal and external security processes in conjunction with independent safety researchers. As part of this commitment, organizations like Metr have emerged as integral partners in fostering greater accountability and transparency in AI development.
In addition to their recent collaboration with Anthropic, Metr has gained a reputation as a significant player in the third-party evaluation landscape. Their involvement with previous high-profile investigations, such as those into the vulnerabilities exposed by the OpenAI agents, has solidified their role in shaping the standards for AI safety assessments. This momentum serves as a reminder that the path toward responsible AI development will require collaboration, scrutiny, and ongoing dialogue among all stakeholders in the technology space.