Skip to main content
Globe And Street

The world, accounted for.

Global business and finance

SectionsMenuClose

OpenAI to host six Apollo evaluators for AI safety pilot

Apollo says the project will examine whether AI models undermine safeguards, but access to information, employees and public disclosure will have limits.

Illustration of an AI model evaluation dashboard with safety checks and access controls
Photo: TechCrunch

OpenAI will bring six safety evaluators from Apollo Research into its organization for a pilot examining whether AI models try to undermine mechanisms intended to keep them under human control, Apollo said. Apollo did not announce a start date.

The project will investigate attempts by models to weaken safeguards governing themselves or their successors. Apollo said researchers will examine whether training encourages such behavior, whether tests can uncover it, whether it emerges during internal use and whether related reasoning can be detected.

According to Apollo, OpenAI will provide information needed for those assessments, including relevant code, training datasets, reinforcement-learning records and access to intermediate versions of models. Access will cover specified models, periods and claims agreed upon for the evaluation.

OpenAI can withhold information on grounds including technical feasibility, security and intellectual property protection, Apollo said. It has agreed that unfavorable findings alone will not justify withholding material. Apollo maintains that fully independent assessments require access equivalent to that of employees.

Researchers will also be restricted to employees authorized by OpenAI, although Apollo can propose people to contact. If another employee shares confidential information relevant to the work, Apollo must stop communicating with that person and pursue further contact through OpenAI.

Apollo will be permitted to publish its own conclusions and describe the conditions and limitations of its work. However, OpenAI may restrict the supporting evidence made public. Apollo said it is seeking broader access and disclosure rights and plans to pursue full embedded evaluations with multiple AI developers.

Editorial policyReport a mistake

About the author

Hao Chen

Technology Editor

Edits and reviews technology business stories: artificial intelligence, chips, big tech, software, cybersecurity incidents, start-ups and platform regulation.