Pentagon's 'Responsible AI' Guidelines Are Not Always Followed, Congressional Watchdog Report Finds Gaps

The Pentagon is building weapons and projects with artificial intelligence capabilities, but officials themselves say they do not always closely adhere to the department's own guidelines for responsible AI, according to a congressional watchdog report published in October.

AI researchers and international organizations have long advocated for strict regulation of AI when used in weapons that could become unpredictable or function with little human control, arguing that those tools could endanger the lives of soldiers and civilians and risk quickly escalating wars.

By contrast, Defense Secretary Pete Hegseth has repeatedly dismissed previous caution over AI tools and encouraged rapid autonomous innovation at the Pentagon. A Pentagon memo in January said the department would become an 'AI-first' warfighting force across all components, from front to back. Last month, Hegseth announced a new warfare command focused on autonomous weapon development, including drones.

The watchdog report, published by the U.S. Government Accountability Office on October 14, grants a rare look inside how those kinds of projects are being developed. The GAO surveyed 10 officials involved with building 10 weapons and tools for the Pentagon that use AI to automatically identify targets on the battlefield, process surveillance, improve tactical decision making and perform other tasks. The projects are among more than 300 being built with AI in the Pentagon.

When responding to the GAO survey, six of the 10 officials said their teams had identified risks with using the projects in an operational environment. To address some of those sorts of risks, in 2021 the Pentagon sent a memo outlining six basic tenets to ensure that weapons built with AI are created responsibly.

Those six guidelines mainly stress the importance of Pentagon-wide oversight, verification and 'real-time monitoring' of the technology, and cooperation with allies 'to advance global norms grounded in shared values.'

Alissa Czyz, who directed the GAO review, said the tenets evolved from earlier ethical principles, and while they were not rules, they were understood to be important by people involved with building AI tools at the Pentagon. 'It's a memo, right, at the end of the day, so it's not a department instruction,' Czyz said. 'It's something that the department has signaled that should be implemented.'

Most of the people involved with making the 10 weapons and projects reviewed by GAO had taken 'at least initial steps' to follow the six tenets outlined in the memo, according to the report, and some of the projects were in early stages, Czyz clarified. But some responses indicated officials were not always closely following the guidelines.

Four officials told the GAO that requirements for the weapons and projects they were working on were not 'aligned with Responsible AI tenets.' And three of the 10 officials told the GAO they 'don't know' if the projects will be continuously monitored after they are deployed, though the six tenets encourage monitoring.

Officials told the GAO that more access to expertise about the agency's guidelines or staff who know about them would increase their ability to incorporate AI responsibly. But the group that helps manage education on the Pentagon's guidelines, the Responsible AI Division, is understaffed. Although the group was authorized to hire 10 people in 2023, as of February only three of those positions were filled, and the Pentagon 'is unlikely to fill vacant positions or increase staffing,' the report states. 'That's a very small number of folks tasked with a large responsibility of responsible AI throughout the department,' Czyz said.

The GAO issued two recommendations: the Pentagon should more clearly outline the responsibilities and meeting schedules of the office charged with overseeing AI regulation, and address understaffing. Although the Pentagon agreed with those recommendations, as of October 22 the agency had not implemented those changes, a GAO representative confirmed. NPR asked the Pentagon when it planned to make those changes, but did not receive a response.

When delivering a speech at SpaceX in January, Hegseth indicated he believed AI development at the Pentagon should not be slowed by those sorts of steps. 'Our enterprise currently operates on staffing and committee cycles measured in months and years, and that's unacceptable,' Hegseth said. 'Military AI is going to be a race for the foreseeable future where the risks to U.S. national security of moving too slowly outweigh the impacts of imperfect alignment.' What remains to be seen is how far the Pentagon now addresses the oversight and staffing gaps after the GAO recommendations.