AI could pose ‘extinction-level’ threat to humans; US must intervene, State Dept.-commissioned report warns

Mar 12, 2024, 2:24 PM

AI could pose ‘extinction-level’ threat to humans and the US must intervene, a report warns. (P...

AI could pose ‘extinction-level’ threat to humans and the US must intervene, a report warns. (PeopleImages/iStockphoto/Getty Images via CNN Newsource)

(PeopleImages/iStockphoto/Getty Images via CNN Newsource)

New York (CNN)A new report commissioned by the US State Department paints an alarming picture of the “catastrophic” national security risks posed by rapidly evolving artificial intelligence, warning that time is running out for the federal government to avert disaster.

The findings were based on interviews with more than 200 people over more than a year – including top executives from leading AI companies, cybersecurity researchers, weapons of mass destruction experts and national security officials inside the government.

The report, released this week by Gladstone AI, flatly states that the most advanced AI systems could, in a worst case, “pose an extinction-level threat to the human species.”

A US State Department official confirmed to CNN that the agency commissioned the report as it constantly assesses how AI is aligned with its goal to protect US interests at home and abroad. However, the official stressed the report does not represent the views of the US government.

The warning in the report is another reminder that although the potential of AI continues to captivate investors and the public, there are real dangers too.

“AI is already an economically transformative technology. It could allow us to cure diseases, make scientific discoveries, and overcome challenges we once thought were insurmountable,” Jeremie Harris, CEO and co-founder of Gladstone AI, told CNN on Tuesday.

“But it could also bring serious risks, including catastrophic risks, that we need to be aware of,” Harris said. “And a growing body of evidence — including empirical research and analysis published in the world’s top AI conferences — suggests that above a certain threshold of capability, AIs could potentially become uncontrollable.”

White House spokesperson Robyn Patterson said President Joe Biden’s executive order on AI is the “most significant action any government in the world has taken to seize the promise and manage the risks of artificial intelligence.”

“The President and Vice President will continue to work with our international partners and urge Congress to pass bipartisan legislation to manage the risks associated with these emerging technologies,” Patterson said.

‘Clear and urgent need’ to intervene

Researchers warn of two central dangers broadly posed by AI.

First, Gladstone AI said, the most advanced AI systems could be weaponized to inflict potentially irreversible damage. Second, the report said there are private concerns within AI labs that at some point they could “lose control” of the very systems they’re developing, with “potentially devastating consequences to global security.”

“The rise of AI and AGI [artificial general intelligence] has the potential to destabilize global security in ways reminiscent of the introduction of nuclear weapons,” the report said, adding there is a risk of an AI “arms race,” conflict and “WMD-scale fatal accidents.”

Gladstone AI’s report calls for dramatic new steps aimed at confronting this threat, including launching a new AI agency, imposing “emergency” regulatory safeguards and limits on how much computer power can be used to train AI models.

“There is a clear and urgent need for the US government to intervene,” the authors wrote in the report.

Safety concerns

Harris, the Gladstone AI executive, said the “unprecedented level of access” his team had to officials in the public and private sector led to the startling conclusions. Gladstone AI said it spoke to technical and leadership teams from ChatGPT owner OpenAI, Google DeepMind, Facebook parent Meta and Anthropic.

“Along the way, we learned some sobering things,” Harris said in a video posted on Gladstone AI’s website announcing the report. “Behind the scenes, the safety and security situation in advanced AI seems pretty inadequate relative to the national security risks that AI may introduce fairly soon.”

Gladstone AI’s report said that competitive pressures are pushing companies to accelerate development of AI “at the expense of safety and security,” raising the prospect that the most advanced AI systems could be “stolen” and “weaponized” against the United States.

The conclusions add to a growing list of warnings about the existential risks posed by AI – including even from some of the industry’s most powerful figures.

Nearly a year ago, Geoffrey Hinton, known as the “Godfather of AI,” quit his job at Google and blew the whistle on the technology he helped develop. Hinton has said there is a 10% chance that AI will lead to human extinction within the next three decades.

Hinton and dozens of other AI industry leaders, academics and others signed a statement last June that said “mitigating the risk of extinction from AI should be a global priority.”

Business leaders are increasingly concerned about these dangers – even as they pour billions of dollars into investing in AI. Last year, 42% of CEOs surveyed at the Yale CEO Summit last year said AI has the potential to destroy humanity five to ten years from now.

Human-like abilities to learn

In its report, Gladstone AI noted some of the prominent individuals who have warned of the existential risks posed by AI, including Elon Musk, Federal Trade Commission Chair Lina Khan and a former top executive at OpenAI.

Some employees at AI companies are sharing similar concerns in private, according to Gladstone AI.

“One individual at a well-known AI lab expressed the view that, if a specific next-generation AI model were ever released as open-access, this would be ‘horribly bad,’” the report said, “because the model’s potential persuasive capabilities could ‘break democracy’ if they were ever leveraged in areas such as election interference or voter manipulation.”

Gladstone said it asked AI experts at frontier labs to privately share their personal estimates of the chance that an AI incident could lead to “global and irreversible effects” in 2024. The estimates ranged between 4% and as high as 20%, according to the report, which noes the estimates were informal and likely subject to significant bias.

One of the biggest wildcards is how fast AI evolves – specifically AGI, which is a hypothetical form of AI with human-like or even superhuman-like ability to learn.

The report says AGI is viewed as the “primary driver of catastrophic risk from loss of control” and notes that OpenAI, Google DeepMind, Anthropic and Nvidia have all publicly stated AGI could be reached by 2028 – although others think it’s much, much further away.

Gladstone AI notes that disagreements over AGI timelines make it hard to develop policies and safeguards and there is a risk that if the technology develops slower-than-expected regulation could “prove harmful.”

KSL 5 TV Live

Science & Technology

About 576,000 Roku accounts were compromised in a cyberattack, the company said on April 12, the se...

John Towfighi, CNN

Roku says 576,000 accounts breached in cyberattack

About 576,000 Roku accounts were compromised in Roku's second cyberattack of the year.

4 hours ago

Phone with Instagram logo...

Kelvin Chan, AP Business Writer

Instagram begins blurring nudity in messages to protect teens and fight sexual extortion

Instagram says it’s deploying new tools to protect young people and combat sexual extortion.

1 day ago

MARTIN, OHIO - APRIL 08: The moon passes in front of the sun during a solar eclipse on April 08, 20...

Adithi Ramakrishnan, AP Science Writer

Can’t get enough of the total solar eclipse or got clouded out? Here are the next ones to watch for

If you missed North America's total solar eclipse or if you caught the eclipse bug, there are more chances to see the sun disappear.

3 days ago

The moon partially covers the sun behind the Statue of Liberty during the a solar eclipse on the Li...

Marcia Dunn, AP Aerospace Writer

Total solar eclipse wows North America. Clouds part just in time for most

A chilly, midday darkness fell across North America as a total solar eclipse raced across the continent. Monday's spectacle was witnessed by millions of spectators in the U.S., Mexico and Canada. It was North America's biggest eclipse crowd ever

4 days ago

MAZATLAN, MEXICO - APRIL 08: The sun disappears behind the moon during the Great North American Ecl...

Marcia Dunn, AP Aerospace Writer

A total solar eclipse races across North America. Clouds spoil the view for some

Millions of spectators along a narrow corridor stretching from Mexico to the U.S. to Canada eagerly awaited Monday's celestial sensation — a total eclipse of the sun — even as forecasters called for clouds.

4 days ago

Roger and Allyssa Sarkis collected all the leftover solar eclipse glasses after Utah's eclipse in 2...

Shelby Lofton

Provo family projected to sell half a million glasses for total solar eclipse

A Provo family made a business out of the leftover eclipse glasses of Utah's 2017 solar eclipse.

8 days ago

Sponsored Articles

Women hold card for scanning key card to access Photocopier Security system concept...

Les Olson

Why Printer Security Should Be Top of Mind for Your Business

Connected printers have vulnerable endpoints that are an easy target for cyber thieves. Protect your business with these tips.

Modern chandelier hanging from a white slanted ceiling with windows in the backgruond...

Lighting Design

Light Up Your Home With These Top Lighting Trends for 2024

Check out the latest lighting design trends for 2024 and tips on how you can incorporate them into your home.

Technician woman fixing hardware of desktop computer. Close up....

PC Laptops

Tips for Hassle-Free Computer Repairs

Experiencing a glitch in your computer can be frustrating, but with these tips you can have your computer repaired without the stress.

Close up of finger on keyboard button with number 11 logo...

PC Laptops

7 Reasons Why You Should Upgrade Your Laptop to Windows 11

Explore the benefits of upgrading to Windows 11 for a smoother, more secure, and feature-packed computing experience.

Stylish room interior with beautiful Christmas tree and decorative fireplace...

Lighting Design

Create a Festive Home with Our Easy-to-Follow Holiday Prep Guide

Get ready for festive celebrations! Discover expert tips to prepare your home for the holidays, creating a warm and welcoming atmosphere for unforgettable moments.

Battery low message on mobile device screen. Internet and technology concept...

PC Laptops

9 Tips to Get More Power Out of Your Laptop Battery

Get more power out of your laptop battery and help it last longer by implementing some of these tips from our guide.

AI could pose ‘extinction-level’ threat to humans; US must intervene, State Dept.-commissioned report warns