US government wants AI to write rules governing plane safety, sparking concern among its own employees: 'It seems incredibly irresponsible'

The United States Department of Transportation appears to think it's a good idea to use artificial intelligence (AI) to write regulations affecting the safety of planes, cars and pipelines, a ProPublica investigation revealed Monday, as cited by Ars Technica.

Investigative journalists point out that the DoT (Department of Transportation) is becoming the first agency to use AI to draft rules, and that this raises major concerns as it is well-known that artificial intelligence even confidently gives wrong answers and in some cases “hallucinates” completely fabricated information.

Even DoT employees fear that any failure to spot AI errors could lead to flawed laws, resulting in lawsuits, injuries or even deaths in the transportation system.

But Gregory Zerzan, the department's chief legal officer, dismissed those concerns, according to transcripts of a December meeting. He argued at the hearing that the goal is not for AI to produce perfect results, but to help speed up the rulemaking process so that rules that would normally take weeks or months to complete can instead be drafted within 30 days.

Zerzan pointed out that Google's Gemini chatbot, the DoT's tool of choice, can draft rules in less than 30 minutes. “We don't need the perfect x, y, z rule,” Zerzan told employees at the meeting. “We don't even need a really good x, y, z rule. We want something good enough,” he explained.

DoT employees 'deeply skeptical' of Gemini use

ProPublica spoke with experts and granted anonymity to six DoT employees who were willing to speak about their concerns about the department's use of Google Gemini for rulemaking.

Some experts who monitor the use of AI in public administration told ProPublica that the DoT could save time by using Gemini as a research assistant, “with a lot of oversight and transparency.” For example, at one presentation, DOT employees were told that “most of what goes into the preambles of DOT regulatory documents is just “word salad,'' and “Gemini can make word salad.”

However, department employees told ProPublica they feel “deeply skeptical” that Gemini is up to the task. They pointed out that the DoT's regulatory process is “painful work” that sometimes requires decades of “expertise in the field as well as existing statutes, regulations and case law.”

Perhaps adding to employee unease, ProPublica noted that a demonstration of Gemini's ability to write rules produced a document missing key passages, which an employee was then supposed to fill in. What's more, the DoT initiative comes after a year of AI “hallucinations” that rocked US courts, with many lawyers being fined and even judges admitting they can be fooled by fabricated information.

Any errors in the rules could have serious consequences. These rules “touch virtually every facet of transportation safety,” keeping “planes in the air,” preventing “gas pipelines from exploding,” and stopping “freight trains carrying toxic chemicals from derailing,” ProPublica reported.

“It seems incredibly irresponsible,” a DoT employee told the investigative website.

The goal is for AI to eventually generate all regulatory documents

ProPublica also notes that despite concerns raised by employees within the department, the DoT appears to be moving ahead with the plan at an accelerated pace. The department has already used Gemini to draft “an as-yet-unpublished Federal Aviation Administration rule, according to a DOT employee briefed on the matter.”

Donald Trump has urged federal agencies to adopt AI at a rapid pace, but nowhere in his orders has the president explicitly called for artificial intelligence to be used to draft laws or regulations, ProPublica points out.

However, lawyer Zerzan told staff that Trump is “very excited” about the department's initiative to use Gemini and suggested that the president sees the DoT as “the spearhead” and expects other agencies to follow suit.

Currently, the DoT expects that Gemini can be used to “handle 80–90% of the regulatory drafting work,” ProPublica reports. But the goal is that eventually all federal employees who rely on AI tools like Gemini to write rules will end up having only an oversight role, monitoring “AI-AI interactions,” according to ProPublica.

Google talks about “creative solutions” to problems with the help of artificial intelligence

Google did not respond to Ars Technica's request for comment on this Gemini use case, which could expand across the government under Trump.

Instead, the tech giant posted a message on its blog Monday promoting Gemini for public administration at large and promising federal officials that AI will help them “creatively solve problems in the most critical aspects of their work.”

Google is competing with other AI rivals for US government contracts, undercutting OpenAI and Anthropic's $1 bids by offering a year of access to Gemini for $0.47.

“We are committed to supporting DoT's digital transformation and stand ready to help other federal leaders across government adopt this model for the success of their own missions,” Google's blog post said.