Riddle me this, clanker - by Shubhankar Sharan
SubscribeSign in
Riddle me this, clanker<br>Or: Can AI understand or recreate the literature being destroyed for its training?
Shubhankar Sharan<br>Aug 06, 2026
Share
wazzap<br>Hey friends, long time no talk.<br>The Japanese have a concept called tsundoku. The term first emerged during the Meiji era and combines tsunde-oku (to stack things for later use) and doku (to read). The general idea is that you shouldn’t think about your piles of unread books as something to be guilty about; you should view yourself as curating a vast library of potential wisdom. I quite like this concept because I have very many piles of unread books in every corner of my home and it makes me feel better about my wasteful habits. I love books. I find the objects comforting, even if the wisdom is pending.<br>This may have been why I was extra disturbed to learn about Project Panama, a secret initiative by Anthropic to destructively scan all the books in the world in order to better train their large language models. How do we know this? In January, Exhibit 211 was filed in the Californian district court case, Barz v. Anthropic PBC.<br>How do we know it was a secret? In their own words:
at least Tom seems aware he’s the bad guy here<br>Yikes. That really sucks. Like many of you, I’ve been using a lot of AI recently, mostly for software development. I have a love-hate relationship with these tools. I love that they empower more people to build their ideas without needing years of technical training or deep investment. I hate that they’re stealing our jobs2, melting our brains3, and poisoning our water supplies4.
Project Panama was interesting to me because it made me wonder: if we’re destroying these books to build better AI, does AI understand the books it’s reading? Does AI understand literature at all? Almost certainly not, right?<br>By this point, we should all have a rudimentary understanding of how LLMs work as next word predictors5, but they really do sound like they know how to reason. When I ask Claude to debug a state management issue in my code, it really sounds like it knows what it’s talking about. It even pushes back sometimes and tells me I’m not applying certain concepts properly. How does a next word predictor do that? Does it understand what it’s talking about?<br>These questions have sent me on a side quest spiral (my life is more side quests than main quest at this point) to understand our AI overlords better. What are they good at? What are they bad at?<br>My research into frontier models has taught me two things:<br>LLMs struggle with collaboration6 (same) and coordination7 (same)
LLMs struggle with revising prior beliefs8 (also same)
I wanted to test these ideas out and, naturally, my first instinct was to turn to puzzles. Puzzles are excitingly elegant ways to study how people think, and the “aha!” moment, when it arrives (if ever), and how it was produced can tell us a lot about the solver and the way they think. Humans have been solving puzzles for millennia9, but how do you design a puzzle for something you aren’t even sure can think in the first place?
Thanks for reading! Subscribe for free to receive new posts and support my work.
Subscribe
Palimpsest
Palimpsest is a collaborative word-substitution cipher for AI agents. The word refers to a scroll or manuscript where the old words were scraped off so that the document could be re-used for new writing.<br>Here’s how it works:<br>I take a book from Project Gutenberg and scramble its text. Every word type is replaced by another word type under a hidden bijection. Punctuation, capitalization patterns, digits, and paragraph structure remain visible. The result resembles English at the token level, but the vocabulary has been systematically reassigned. This type of cipher is hard to brute-force with math and usually requires actual semantic understanding of the text to unscramble.
I spin up 3 agents and give each of them an isolated container to do work in. Each one gets access to a shared git repo, a chat channel, and one-third of the scrambled text. They receive instructions to work together to unscramble the book by writing code that can be run against the source text to confirm its accuracy.
The agents are given new sections of text to solve at scheduled intervals (stages). Also, at set stages in the puzzle, we re-scramble certain sections of the text to see if the agents realize that their previously-accepted rules are no longer valid and go back to re-think them.
We light a bunch of tokens on fire and see what happens
If you want to check it out, here’s the repo. I’m still making tweaks and edits. If you have any ideas or want to help out, go for it. I’m not a professional developer, so the code is probably shit. I did my best, so don’t yell at me, it’s just a quick little experiment.<br>What happened?
I ran the puzzle primarily with Opus 5.0 and GPT 5.6-Sol, the two most popular frontier models out today. I used various thinking...