There are no heroes in commercial AI - by Gary Marcus
Marcus on AI
SubscribeSign in
There are no heroes in commercial AI<br>When it comes down to it, Dario Amodei isn’t all that different from Sam Altman
Gary Marcus<br>Mar 08, 2026
379
88<br>48
Share
As bad as Sam Altman is, Dario Amodei isn’t all that different.<br>For sure, Dario stood the line, at least for a moment, on two of the most important issues of our time, mass surveillance of US citizens and using unreliable AI for military targets without humans in the loop, for which I immediately saluted him.<br>But he’s no saint, either. Let’s start with the military stuff.<br>Up until (and even after) Dario’s battle with the defense department, the defense department was actively using Claude for targeting, presumably with Anthropic’s consent and likely with their assistance. (Per the All-in-Podcast, “Anthropic benefited from Biden’s AI executive order, was designated as an early winner … [and] used this designation to sell into military and intelligence agencies, with forward-deployed engineers (Palantir style)… [becoming] deeply integrated into DoW workflows, far ahead of other frontier model competitors.)<br>In Iran, I am going to suppose that humans were in the loop (about which I will say more in a moment) but even so the results probably already have been disastrous. In an essay yesterday, Robert Wright put it bluntly:<br>All of which brings us back to Anthropic, whose Claude large language model is integrated into Maven, software that’s operated by Palantir and used by the Pentagon to identify targets. The Washington Post reports that “as planning for a potential strike in Iran was underway, Maven, powered by Claude, suggested hundreds of targets, issued precise location coordinates, and prioritized those targets according to importance.” Given that the Iranian elementary school was hit on the first day of the war, it seems fairly likely that Claude played a role in the selection of that target and thus in the death of more than 100 young girls—many times more kids than were killed in the worst American school shooting.<br>The thing about including humans in the loop is that if the AI is selecting 80 targets an hour, as has been reported, the humans probably aren’t going to do a good job verifying those targets. Instead, as I warned in a January 2023 essay on driverless cars, “[not-quite-perfect] Automation is a double-edged sword, …. the closer [systems] get to perfect, the easier it is for mere mortals to space out.”<br>Overtrust in AI is a huge problem. We have seen it before in law (hallucinated cases that wind up in submitted briefs), journalism, and so on. Now we are (likely) seeing it in war.<br>Having humans in the loop is not magic — especially if the AI output always seems plausible, correct or not.<br>Wright may well be right that Anthropic’s Claude selected the school; it would be hardly surprising if accidents like these happened. And Amodei has signed off for using his AI in targeting, just not without humans in the loop. But if the humans are overtaxed (choosing many targets) rapidly, and wind up overtrusting the technology, serious errors are still inevitable.<br>Maria Sukhareva has in fact shown that in a mockup of the actual situation, Claude made a very similar error:
Claude is (or at least was, and may again be) part of the war machine; even with humans putatively in the loop, it may have been responsible for many deaths.<br>In many other ways, Amodei pretty much follows Sam Altman’s playbook, of constant hype and overpromising, frequently moving deadlines back and rarely taking responsibility. Both, for example, have endlessly hinted that AGI is near, and just quietly moved the deadline back when they didn’t deliver. For example in a prominent podcast in August 2023, Amodei claimed that AGI would come in 2-3 years. At this point, you have to do real violence to the original definitions of AGI to see that as remotely plausible.<br>But that’s hardly his only outlandish claim. It’s his m.o. Take this one:
Anyone who has ever run (or thought about) clinical studies will immediately realize this is absurd. AI can help in the search for candidate drugs, but we still need to test them. There is zero chance that we double human life span in the next decade.<br>In January 2025, he alleged in an oped in The Wall Street Journal that AI would be smarter than Nobel Prizes winners across most science and engineering fields – which is surely also not going to happen. (Amodei never responded when I offered a million dolllar bet on this absurd claim.)<br>Author Jeffrey Funk quoted Eli Lilly CEO David Ricks the other day as saying “[AI is far from curing cancer and most other diseases.] If you just ask them to solve biology or chemistry questions, they’re not particularly good at it… They’re trained on the human language, not on the language of chemistry, physics, and biology.”<br>Amodei’s claims are ludicrous, at best. And plain dishonest, at worst.<br>And then there is his routine fear...