OpenAI And Anthropic Lawyers Warn Of Risks From Open-Source AI Models

Lawyers at leading artificial intelligence companies warned on Thursday that malicious actors could exploit the capabilities of open-source AI models, raising concerns about cybersecurity threats and the need for stronger safeguards.

 

Alex Iftimie, deputy general counsel at OpenAI, said the potential use of open-source models by ransomware groups, hostile nation-state actors and intelligence services was an urgent concern.

 

“That is a problem that will manifest in serious ways, likely over the next few months,” he said at an American Bar Association conference on national security law in Washington, DC.

 

Will Hudson, associate general counsel at Anthropic PBC, said the open-source nature of AI models made it significantly easier to remove their safeguards. If models without such protections were available to anyone, cyber-defenders should also have access to the most capable systems, he added.

 

“Certainly cyber-defenders should have access to the most capable models,” Hudson said.

 

The discussion comes as AI has assumed a prominent role in Washington, with the US midterm election season under way and government leaders grappling with how best to regulate and manage the technology.

 

Concerns about the potential threat AI poses to humanity intensified last month after an Anthropic researcher resigned, warning that Anthropic and OpenAI “are racing straight to self-improving superintelligence and gambling with our lives”.

 

Several executives who met US President Donald Trump last week pledged to introduce “robust internal controls” to monitor AI models during training and deployment.

 

Phil Carter, senior counsel at Alphabet Inc.'s Google, said at Thursday's conference that the government needed to invest in the infrastructure required to identify and address vulnerabilities in open-source systems.

 

“Without that, I think there’s just enormous latent vulnerability in the global information system,” he said.

 

Carter also called on Congress to develop a single, large appropriations measure to “supercharge the research enterprise” in the US.

 

Hudson said the largest AI laboratories should be required to test their systems for “catastrophic risk”. They should publish the results, he said, with an independent evaluator subsequently assessing their findings.