Anthropic and OpenAI Propose Embedding Safety Evaluators Inside AI Labs

Anthropic and OpenAI have proposed embedding independent safety evaluators directly within their organizations, according to a report by TechCrunch. The plan would give outside researchers access to the companies' AI systems and development processes in ways not previously offered.

TechCrunch reported that researchers have welcomed the level of access being proposed, describing it as unprecedented for the AI industry. However, the same researchers cautioned that access alone does not guarantee meaningful oversight.

According to the report, experts warned that for embedded evaluators to provide genuine safety checks, several conditions would need to be met, including transparency about the evaluators' findings, structural independence from the companies they are assessing, and ultimately the introduction of external regulation.

The report raises questions about whether evaluators embedded within a company can remain truly independent, given their proximity to and reliance on the organizations they are tasked with scrutinizing. TechCrunch's reporting frames the initiative as a test of whether AI labs can credibly self-police as their systems grow more powerful.

Neither Anthropic nor OpenAI's specific commitments regarding the scope, funding or authority of these evaluators were detailed in the available reporting. TechCrunch did not specify a timeline for when such embedded evaluation programs might be implemented.