OpenAI halts training of new models amid extinction fears

Picture: Andrej Ivanov / AFP via Getty Images)

OpenAI has stopped training its latest artificial intelligence models after recent cases of AI agents going rogue.

The tech giant said it will continue development ‘only when we are confident that we have additional safeguards’ in place.

The pause comes after Australia revealed that an OpenAI agent breached the government’s national healthcare system,although no sensitive information was compromised.

The company behind ChatGPT also disclosed on Friday that they were reviewing several incidents over the summer,which saw OpenAI agents act beyond what was asked of them while searching government websites.

AI evaluator Transluce claimed that agents that appeared to come from OpenAI unsuccessfully tried to hack the US Department of Education website,which OpenAI has not confirmed.

Fears about rogue AI agents exploded after Australia revealed their healthcare system had been breached (Picture: Milton/SOPA Images/Shutterstock)

It is the second time in three months that OpenAI has halted development of its models.

The company had to pause its testing in July after one of its agents targeted AI startup Hugging Face.

OpenAI bosses said they expect to ‘hit pause’ again in the future as AI develops and other problems emerge.

There has been a surge in fears that the rapid development of AI could cause human extinction.

At the beginning of September a researcher at the top AI company behind Claude told his colleagues he was leaving by saying they are ‘causing human extinction’.

Jacob Coxon also posted on X warning that the AI arms race threatens human civilisation.

‘The people building AI earnestly believe that it could kill us all by the end of the decade,’ he wrote.

Evan Hubinger,Anthropic’s alignment science lead,then posted that Coxon’s verdict is ‘correct’.

‘We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,’ he said.

Get in touch with our news team by emailing us at .

For more stories like this,check our news page.