A Judicial Wake-Up Call on Government by AI | TechPolicy.PressPerspective<br>A Judicial Wake-Up Call on Government by AI<br>Jordan Ascher / Jul 2, 2026Protesters against the Trump administration gather in front of the US Capitol in Washington, D.C., on April 5, 2025. (The Yomiuri Shimbun via AP Images )
Republish
Share
The Trump administration was all-in on using AI to slash federal regulations before the president’s second term had even begun. In November 2024, Elon Musk and Vivek Ramaswamy laid out a vision for AI-powered deregulation, and since January 2025, DOGE (the Department of Government Efficiency) and other federal agencies have sought to use large language models to undo swaths of regulations with barely any human oversight. This maximalist strategy would sacrifice the careful work of federal regulators for the sake of speed.<br>In May, a federal court dispensed the administration a dose of reality. The case, American Council of Learned Societies v. National Endowment for the Humanities, concerned an initiative closely related to DOGE’s deregulatory push: its mass cancellation of NEH grants in 2025.<br>In litigation, an unsettling story emerged. Shortly after President Donald Trump’s inauguration and his mandates to terminate funding for “diversity, equity, inclusion, and accessibility” and “gender ideology,” two young DOGE staffers arrived at NEH. They fed descriptions of hundreds of NEH grants into ChatGPT with the prompt, “Does the following relate at all to DEI?”<br>The chatbot produced bizarre outputs. For example, it flagged a grant to a whaling museum in the Northeast. The offense: the museum sought to “create an inclusive and impactful experience, which is aligned with DEI principles.” So too an academic study of how the plastics industry influenced municipal policy across the country. ChatGPT’s substantive analysis was limited to: “#DEI.”<br>Beyond this incoherence was something far more disturbing. In numerous cases, ChatGPT’s “DEI” classification was based solely on a grant description’s reference to a particular race or gender. For instance, it flagged funding for a documentary about the Colfax Massacre because the film “explores a historical event that significantly impacted Black civil rights.” Other termination rationales included, “This proposal aims to expand understanding of Black life and geography during slavery,” and, “The book explores a Black-led project.”<br>DOGE endorsed this crude approach. One of the DOGE members testified that funding for a documentary surfacing the slave labor of Jewish women during the Holocaust was properly canceled because it was “specifically focused on Jewish cultures” and the “voices of the females in that culture.” In the end, DOGE rubber-stamped ChatGPT’s outputs, adopting them wholesale as the basis for terminating more than 1,400 grants.<br>The court found the DOGE cuts unconstitutional multiple times over. Terminating funding based on its purported connection to “disfavored ideas” violated the First Amendment, and canceling grants because they made reference to a particular race, gender, or other protected class defied the Constitution’s promise of equal protection.<br>Our content delivered to your inbox.<br>Join our newsletter on issues and ideas at the intersection of tech & democracy
Subscribe<br>Loading...
Thank you!<br>You have successfully joined our subscriber list.
DOGE’s use of ChatGPT was relevant to the court’s analysis. The bluntness—indeed, candor—of the dialogue between DOGE and ChatGPT made the government’s discriminatory motivation clear. The user explicitly asked the chatbot to discriminate; the chatbot obliged. Recall, for example, the grant canceled because ChatGPT noted that it “explore[d] a Black-led project.” This kind of smoking gun is extremely rare in constitutional litigation.<br>For its part, the government tried to distance itself from ChatGPT’s outputs. It insisted that the exchange with ChatGPT did not taint the ultimate termination decisions, and instead merely provided “context.” The court understood the government to be saying, “It wasn’t me; it was ChatGPT,” and rejected the defense. “Here,” the court found, “DOGE selected the AI tool, formulated the prompt, and defined the operative viewpoint-based criterion: whether a grant relates at all to DEI.” DOGE “adopted the classifications and supporting rationales that were generated by ChatGPT to justify terminating grant after grant,” and the court ruled that “those classifications are the Government’s own for constitutional purposes.”<br>This is the key insight of Learned Societies: LLMs are tools. Like any other tool, they can be used prudently or irresponsibly. To be sure, unlike, say, legal research databases, word processors, or pens, LLMs are generative—of prose, of answers to questions, of rationales. Consequently, agencies might be tempted, both in the funding and regulatory contexts, to treat LLMs as though they were themselves decision-makers. That could mean reflexively ratifying or...