Anthropic, OpenAI Propose 'Neutral' AI Evaluators; CNBC Report Flags Concerns
AI labs suggest embedding independent evaluators to guard against catastrophic risks from advanced models, but a CNBC report says the plan raises questions about independence and effectiveness.
Anthropic and OpenAI, two of the leading developers of advanced artificial intelligence systems, have proposed creating embedded AI evaluators intended to help manage the risk that increasingly powerful models could cause catastrophic harm to society, according to a report published by CNBC on Sept. 16.
The report describes the concept as a form of "neutral" oversight mechanism, in which evaluators would be built into or closely tied to the development process of AI models, rather than functioning as fully external or independent watchdogs.
According to CNBC, the proposal is aimed at addressing growing concerns among researchers, policymakers and the public about the potential for advanced AI systems to cause significant harm, whether through misuse, unintended behavior or other failure modes as the technology becomes more capable.
However, the CNBC report indicates that the idea has drawn scrutiny, with questions raised about whether evaluators embedded within or closely affiliated with the companies developing the technology can provide the kind of independent, unbiased assessment that effective oversight would require.
The report does not specify further technical or structural details of the proposal, nor does it identify specific individuals within Anthropic or OpenAI who put forward the plan. CNBC's coverage frames the initiative within a broader ongoing debate over how the AI industry, regulators and outside experts should collaborate—or maintain separation—when it comes to assessing the safety of frontier AI systems.
Anthropic and OpenAI are both prominent developers of large-scale AI models and have previously voiced public commitments to AI safety research. Neither company's direct statements on the specifics of the evaluator proposal were detailed in the available reporting.
The discussion comes amid continued global attention to AI governance, as governments and industry groups weigh various frameworks for monitoring and regulating advanced AI systems.
Sources
- Anthropic, OpenAI proposed new 'neutral' AI watchdogs. Why you should worry about the idea — CNBC — Top News
EGazette summarizes reporting from multiple sources; follow the links for the originals.
Related articles
Report: Anthropic's Claude Helped Researchers Breach OpenAI Systems
Cybersecurity firm Hacktron says it used Anthropic's Claude AI model to access OpenAI's internal code repository before reporting the vulnerabilities it found
Guardian Column Urges US-China Cooperation on AI Amid Warnings Over Pace of Development
Australian scientist Alan Finkel argues that unregulated competition between AI firms and between Washington and Beijing could threaten global stability, calling for coordinated oversight.
AI industry workers express skepticism about existential risk warnings
Multiple employees at leading AI companies doubt predictions that the technology could pose catastrophic threats to humanity.
Canadian AI Pioneer Warns of Risks From Unchecked Artificial Intelligence
A Canadian researcher widely referred to as a "godfather of AI" has cautioned that artificial intelligence could pose serious threats if left unregulated, according to a report by Anadolu Agency.
Anthropic, Google, OpenAI and xAI Named in Lawsuit Alleging Effort to Slow AI Development
A law firm says it is representing plaintiffs in a legal action against several major artificial intelligence companies, according to a report by Russian news agency TASS.
Nvidia's Huang Says AI Development Should Advance 'As Fast As We Can'
Nvidia's chief executive said artificial intelligence progress should not be slowed, while stressing that products must remain safe, amid an ongoing industry debate over the pace of frontier AI development.
Comments
Loading comments…