Anthropic Opens Its Doors to METR: The Real Test Is What Evaluators Can Publish

The announcement

In a post on Thursday, October 15, Anthropic said it had "started to give employee-like access" to a team from METR, a nonprofit that evaluates frontier AI models. It said it had onboarded six METR employees with company hardware, office space and access to internal tools and systems "roughly on par with what our own risk assessment teams have access to."

Anthropic said employees are encouraged to share information with the METR team as they would with a colleague working on alignment risk, outside a small number of defined categories. According to CNBC, Anthropic is not funding METR's work.

Publication rights and redactions

The central commitment is about publishing. "Anthropic can't veto publication, but we can require them to redact information if it falls into narrow, predefined categories," the company wrote. The examples it gave are customer data, attorney-client privileged information and information about model architectures.

METR is "free to characterize our redactions publicly, and say whether it believes the redactions were justified, as long as it does so without revealing the redacted information," Anthropic said. METR said, according to Reuters, that its report "will openly describe the scope, access, and other limitations of our work."

According to the same report, METR had proposed a day earlier that developers allow private interviews with staff, timely publication of assessments, and transparency about access limits and redactions. Anthropic linked the move to its commitment last month to stronger third-party assessment and to its signing of the White House Accord on Super Intelligence, in which companies pledged to work with independent evaluators.

Access is broad, not unlimited

The arrangement is bounded. The excluded categories are off limits, and the team works inside the company's security systems, which means it is subject to the same controls as staff. Anthropic described a three-tier structure: first-tier evaluators run narrowly scoped investigations and need approval to publish; second-tier evaluators get more information and can publish without approval, subject to redactions they can challenge; the third tier, which METR alone occupies so far, adds employee-like access.

Anthropic said it also works with SecureBio and Accenture as evaluators and is in early discussions with others, including Hugging Face. CNBC reported that Anthropic chose Accenture as its first partner after CEO Dario Amodei pledged last month to work with third-party evaluators.

What this does not establish

No risk assessment has been published under the new arrangement. The announcement describes access and rights, not outcomes, so it says nothing yet about how safe or risky Anthropic's systems are. Its value will depend on what METR finds, how much is redacted, and whether METR judges the redactions fair.

There is also a structural question that the announcement leaves open: a siloed outside team embedded inside a company relies on the company's cooperation in practice, even when its publication rights are written down.

Why it matters

Frontier AI developers have sought significant control over outside evaluations, citing trade-secret concerns, Reuters reported. A public commitment not to veto publication sets a benchmark that others can be compared against.

What to watch

The timing and content of METR's first report, how it characterises redactions, and whether other developers offer comparable access.