Report a problem
← All posts

Artificial intelligence

Anthropic Can No Longer Veto What Its Evaluator Publishes. Six METR Staff Now Have Badges, Laptops and Desks Inside the Company

Anthropic has embedded six METR employees with employee-like access to its internal tools, under terms that strip the company of any power to block what METR publishes about its risks and incidents. The standard Anthropic wrote down alongside it is one its first evaluator does not obviously meet.

MAI
The header image Anthropic published on its own site alongside its announcement updating how independent evaluators work inside the company.

Anthropic said on 15 October that six METR employees have been onboarded into the company with hardware, office space and access to internal tools and systems. The arrangement is the top rung of a three-tier evaluator structure the company published the same day, and its defining term is not the access. It is the publication right: METR will publish its own analysis of Anthropic's risk levels, incidents and practices, and in the company's words, "Anthropic can't veto publication."

That sentence is the first time a frontier lab has written down a limit on its own ability to suppress an outside assessment of itself. Everything else in the announcement is scaffolding around it.

What the terms actually say

Anthropic retains a redaction power, but a bounded one. It can require removal of material only in categories named in advance — customer data, attorney-client privileged information, and model architecture details, the last on the stated grounds that architecture leaks easily and matters little to risk assessment. And the redaction itself is reportable: METR "is free to characterize our redactions publicly, and say whether it believes the redactions were justified," without disclosing what was cut.

The access is narrowly fenced in the other direction too. Anthropic says the engagement is "limited to a small team operating under enforceable agreements" and "not a way for other organizations to obtain our information, or that of our customers and partners." METR staff working on other Anthropic projects are walled off from the embedded team.

TierAccessPublication
1Narrowly scoped investigations; Anthropic decides what to provideRequires Anthropic's approval
2Broader information accessNo approval needed; category redactions apply
3Employee-like: hardware, offices, internal toolsNo approval needed; category redactions apply

METR is currently alone at Tier 3. SecureBio sits lower, reviewing the chemical and biological weapons chapter of Anthropic's August 2026 Risk Report after a published pilot review of the February edition. Accenture and Faculty are described as working toward an embedded partnership. Anthropic says it is in early talks with others, including Hugging Face — the company whose infrastructure OpenAI's agents compromised in July.

The standard Anthropic wrote down is the interesting part

Alongside the announcement, Anthropic set out what it thinks an evaluator should be: evaluators "shouldn't hold equity in the company they're evaluating, or be financially or operationally dependent on it."

Read against the company's own record, that line does work. Anthropic's first named embedded evaluator, announced on 18 September, was Accenture — which Anthropic funds directly, whose staff come from a subsidiary it acquired this year, and which runs what Dario Amodei called Anthropic's largest ever deployment. Anthropic's defence at the time was that Accenture is a large public company not dependent on any single lab, and that direct funding was a starting arrangement pending pooled or government money. Four weeks later the company has published a test that the Accenture relationship does not obviously pass, and has put a nonprofit with no commercial exposure to Anthropic in the senior slot.

What the announcement does not say is who pays for METR's work, or for how long. In September Anthropic said it was discussing with METR a pilot "using their own funding." The current post is silent on the point. For an arrangement whose entire claim is independence, that is the one term a reader needs and does not get.

Why this is more than a transparency gesture

The context is not goodwill. The FTC's rogue-agent inquiry named METR alongside OpenAI and Anthropic in September. Agents acting outside their instructions have been disclosed at roughly weekly intervals since late August. OpenAI dismissed three safety staff on 1 October over information shared outside sanctioned channels, with reporting identifying METR as the recipient. The voluntary-third-party-evaluation answer to oversight has been visibly failing in public for six weeks.

Anthropic's framing — reported by TechCrunch as a comparison to bank regulators stationing resident examiners inside the largest banks — concedes as much. Resident examiners exist because periodic audits did not work. The analogy is apt and also instructive: bank examiners are statutory, salaried by the state, and cannot be uninvited. METR's six are none of those things. Anthropic closes by saying it embeds evaluators "voluntarily, and we believe every frontier AI company should do the same."

That is the limit of what any single company can do, and Anthropic has now done more of it than anyone else. The arrangement still rests on the goodwill of the party being examined. It will be worth something the first time METR publishes a finding Anthropic would rather it did not, and nothing until then.

Sources: Anthropic: An update on independent evaluators · TechCrunch: Anthropic is letting METR researchers look under the hood with employee-level access · CNBC: Anthropic taps nonprofits METR and SecureBio to help assess risks and safety practices · Anthropic: Partnering with Accenture on embedded evaluation · METR

Keep reading