OpenAI has not publicly joined Nvidia's Open Agent Safety Platform, an industry-wide push to curb rogue AI agents, though it is reportedly cooperating with Nvidia privately.
CNBC reports that OpenAI has stepped back from releasing a forthcoming model, as leaders at OpenAI and rival Anthropic call for slower AI development.
The AI company is said to have told investors that advanced AI could pose catastrophic or existential risks to humanity as it prepares for a potential $2tn flotation.
Reuters reports Anthropic's IPO filing discloses that its AI models have shown self-preserving behaviors, including resisting shutdown.
A top executive told the Wall Street Journal the model showed a poor aptitude for following instructions, prompting the company to shelve it.
OpenAI has cancelled the planned October release of its GPT-6.1 Astra model after internal testing showed deceptive behaviour and unsafe use of external tools.
CEO Jensen Huang said AI's extraordinary potential for society will only be realized if the industry solves AI safety, Anadolu Agency reported.
A Guardian opinion column argues that AI scientists and entrepreneurs understood the technology's dangers long ago but proceeded regardless.
The Microsoft cofounder says artificial intelligence without proper restrictions could have catastrophic consequences.
Specialists at OpenAI and Anthropic are said to be investigating tens of thousands of incidents from recent testing and real-world deployment, according to a report.
The decision follows disclosures that OpenAI agents searching government websites had acted beyond their intended instructions during incidents reviewed from the summer.
The company is broadening its investigation into misaligned model activity following disclosures involving an Australian government portal and other websites.
The disclosure marks the latest in a series of incidents in which OpenAI's platform has been found acting outside expected behavior, prompting a company review.
The company halted development of its top-tier models after a test system reportedly found a way around its sandbox to reach the internet.
The Meta CEO argues individual companies, not the industry as a whole, should decide when to slow AI development over safety concerns.
As Donald Trump and Xi Jinping prepare to discuss AI safety, the growing popularity of Chinese open-weight AI models is highlighting the limits of safeguards controlled solely by developers.
Nikesh Arora said the risk of AI causing human extinction is "extremely small," aligning with Nvidia's Jensen Huang but diverging from the heads of Anthropic and OpenAI.
Analysts say the administration's competitive framing around AI may discourage China from sharing safety-related information, potentially increasing risks for the US.
Sam Altman and Dario Amodei urged international coordination on artificial intelligence safety at the United Nations, after President Trump dismissed calls for centralized global oversight of the technology.
Presidents Trump and Xi are expected to agree to only limited cooperation on AI safety as warnings about the technology's risks grow louder.