People prefer A.I. art because people prefer bad art

SubscribeSign in

People prefer A.I. art because people prefer bad art<br>Understanding the "AI Art Turing Test"

Max Read<br>Nov 22, 2024

741

90<br>100

Share

Greetings from Read Max HQ! In this week’s edition, we discuss two recent experiments (one non-scientific, one scientific) comparing A.I.-generated and human fashioned art.<br>A reminder: This newsletter is 99.9 percent funded by paying subscribers, whose support allows me to spend full-time hours reading, researching, writing, deleting, writing again, deleting again, panicking, etc. in the hopes of producing entertaining and even sometimes enlightening work. If you’re reading this newsletter for free, it’s thanks to the generosity of those paying subscribers; if you feel like you’ve gotten something of value from what I write, consider that it only costs about the price of one beer a month to receive 15,000-2,000 words a month from me.

Subscribe

On Wednesday, the prolific and popular blogger Scott Alexander published the preliminary results of a kind of poll he’d set up called “The AI Art Turing Test,” in which he asked readers to distinguish between A.I.-generated images and human-fashioned art. (Samples below.)

Open the schools!!!<br>As it turned out, the average score was 60.6 percent, meaning it was relatively difficult for most Astral Codex Ten readers to tell whether A.I. had been involved in the creation of a given image. Alexander also asked participants to choose their favorite picture; significantly, to him, the picture most-often chosen as the favorite was an impressionist-style A.I.-generated image of a café, prompted by a man named Jack Galler:

“What does this tell us about AI?” Alexander writes. “Seems like they’re1 good at art.” On Twitter, people are making even stronger claims: “Scott Alexander's simple ‘AI Art Turing Test’ proves AI is creative,” says podcaster Liron Shapira. “people who categorically dislike AI art are literally wrong,” says A.I. researcher David Dalrymple.<br>I would say, gently, that I don’t think any such conclusions can really be drawn from the available data. What can we say with any confidence? One obvious problem with the experiment is that Alexander has stacked the deck. The test is effectively designed to fool people, as Alexander admits--the “human” and “A.I.” works are in each case being chosen as to not demonstrate any of the features distinctive of human or A.I. authorship, among them “text… complicated wrestling-like poses… and pop art,” as well as anything in “the DALL-E ‘house style’… or in other similar styles that humans would have trouble replicating.” In other words, he’s asking his subjects to determine authorship of the A.I. images that most resemble human art, and the human art that most resembles A.I. images.<br>And, of course, we’re not technically comparing these A.I. images against “human art,” but against (in most instances) JPEGs of photographs of paintings. Not to get too undergrad about it but the materiality of painting is not some accident of its being; its form, its texture, its size, etc. all carry with them meaning and effect. Compare e.g., the heavily compressed and blown-out JPEG of Ingres’ The Apotheosis of Homer published in Alexander’s post with the actual painting, which measures something like 12’ by 16’, in situ in the Louvre, and suddenly questions of origin and preference are very different. And this isn’t even a painting I like very much!

Right image by Steven Zucker<br>So if you want to be really tediously clear about what’s being tested here, it’s the ability of generative A.I. to mimic a compressed reproduction of an actual painting in a manner that is more immediately pleasing to a Astral Codex Ten subscriber.<br>However. Having registered my objections to the design and interpretation of Alexander’s experiment, I want to note that I don’t actually dispute his conclusions. I saw too much of the answer key to be able to fairly take the test myself, but I’m not at all confident I would have done significantly better than the average had I taken it cold. Generative A.I. apps have gotten very good at creating satisfactory imitations of human product! Under the right circumstances it is indeed difficult to discern A.I.-generated images; indeed, this fact seems so obvious to me I’m not sure it even requires testing.<br>What was more interesting to me is that it seemed pretty easy, going over the images, to pick out original work by esteemed human artists, but much harder to distinguish between human- and A.I.-generated from the mass of images left over. Put another way, I rarely wondered if the good art was generated by A.I. prompting, but I was often uncertain if the bad art (of which there was a lot) was made by human or LLM. Galler’s image of the riverside cafe above is slop any way you cut it--art for a dentist’s office--but at a glance, on a computer screen, I have no definitive way of telling if it’s slop painted by a hack or...