Skip to main content
Globe And Street

The world, accounted for.

Global business and finance

SectionsMenuClose

Anthropic gives METR employee-like access for AI safety reviews

Six METR staff have access comparable to Anthropic’s risk assessment teams, but customer data and model architecture remain excluded.

Computer workstations in an office used for artificial intelligence research
Photo: CNA

Anthropic said on October 15 that it had granted nonprofit evaluator METR access to internal systems comparable to that of its own risk assessment teams. The Claude developer is also negotiating with other potential evaluators, including Hugging Face.

Six staff members from METR, or Model Evaluation and Threat Research, have Anthropic hardware and workspace at its offices, the company said. They cannot access customer data, legally privileged material or details of model architecture.

Anthropic said it has no veto over METR’s conclusions about risks, incidents or company practices, though it may demand redactions within narrowly specified limits. METR may publicly challenge those restrictions without disclosing the information withheld. The nonprofit said its report would explain the boundaries of its work and access.

The arrangement falls within the highest of three access levels Anthropic is developing. At the lowest level, reviewers undertake limited investigations and require permission to publish. The middle level offers additional information and allows publication without prior approval, subject to redactions that reviewers may contest publicly. METR’s level adds employee-like access while retaining similar publication rights.

Anthropic said its other evaluation partnerships remain at an early stage, with narrower access and publishing rights. Biosecurity nonprofit SecureBio is in the lowest tier and is examining the chemical and biological weapons section of Anthropic’s August risk report. The company said it would consider broader rights after that review.

Anthropic is also working with Faculty, Accenture’s AI business, on model evaluations, adversarial testing and checks on whether models behave as intended. Experts have questioned evaluator independence because Anthropic and OpenAI select the organizations that review their systems.

Editorial policyReport a mistake

About the author

Hao Chen

Technology Editor

Edits and reviews technology business stories: artificial intelligence, chips, big tech, software, cybersecurity incidents, start-ups and platform regulation.