Reinventing Daydreaming Machines - by Zbigniew Łukasiak

AI Adventures: A Programmer’s Journey

SubscribeSign in

Reinventing Daydreaming Machines<br>A Search‑Space–First Experiment

Zbigniew Łukasiak<br>Oct 13, 2025

Share

Can LLMs Be Creative?

Can we build systems using LLMs that are genuinely creative—that discover ideas not present in their training data?<br>An LLM by itself seems limited: it’s trained on existing text, so any “new” idea might just be recombination. But the infinite monkey thought experiment shows that random generation or exhaustive enumeration could theoretically produce anything—the problem is they drown in noise and make evaluation intractable.<br>Gwern’s essay on LLM daydreaming proposes a more practical approach: build a system that orchestrates LLMs in a generate-and-verify loop. Combine small sets of familiar concepts to generate candidate ideas, then use critique to filter for novelty. This keeps the search space manageable while enabling surprising connections between disparate fields.<br>I ran an experiment to test whether this approach can work, with a recursive twist: use Gwern’s method to reinvent Gwern’s essay itself. Using only models from before the essay was published, can we find concept combinations that make LLMs rediscover the essay’s novel ideas?<br>Why This Target?

“Can an LLM rediscover special relativity?” It’s a question people ask, and it sounds dramatic. But it’s nearly impossible to evaluate cleanly. We can’t isolate a true 19th-century training slice, and leakage is hard to rule out.<br>To test whether LLMs can be creative, we need a target that’s:<br>Demonstrably novel (not in training data)

Verifiable (we can check if it was rediscovered)

Realistic (achievable without requiring Einstein-level breakthroughs)

Gwern’s 2024 essay on LLM daydreaming fits perfectly. To ensure genuine reinvention, I use only models released before the essay was published. I also verified the essay contained plausibly novel ideas at publication:<br>OpenAI Deep Research

Google Deep Search

This makes “reinvention” a testable claim inside a controlled boundary.<br>What I Built and Found

I built a generation system based on search over concept combinations. I didn’t implement the search algorithm—I manually selected combinations and fed them into generic prompt templates. Each prompt combines 4-5 short, commonly understood concepts from diverse areas—neuroscience (Default Mode Network), economics (Innovation Models, Economic Moat), AI capabilities (Generator-Verifier Gap, Dearth of AI-driven Discoveries)—asking the LLM to synthesize something new. The surprising links come from connecting disparate fields.<br>The result : Using only pre-2024 models, these concept combinations consistently produced essays that reinvented the target ideas. The system keeps rediscovering the novel concepts identified by the Deep Research analyses. Not every generated essay captures all elements, but some get nearly all of them.<br>Browse the resulting essays: https://github.com/zby/DayDreamingDayDreaming/tree/main/data/results<br>Next Steps: Building the Complete System

This proof-of-concept demonstrates that productive concept combinations exist. Two major pieces remain for a complete creative discovery system. The first is an engineering challenge with known techniques; the second is the fundamental research bottleneck:<br>1) Implementing the Search Algorithm (Engineering Challenge)

I manually selected concept combinations. A full system needs an algorithm that explores the concept-composition space systematically. The space is large but manageable—far smaller than enumerating all possible strings. The prompt templates are generic and designed to work with different concept combinations and different target ideas.<br>Key design choices : Limit the search space (use only short, commonly understood concepts), track explored combinations to avoid redundancy.<br>Some promising directions : Guided exploration using evaluation scores, diversity-aware sampling to avoid clustering, quality-diversity search maintaining a Pareto frontier of novelty × usefulness, adaptive allocation of search budget, and learned evaluators trained on top/bottom examples. Many other search heuristics from optimization literature could apply.<br>2) General Novelty Verification (Research Bottleneck)

The current evaluator only checks against a known target. The bottleneck for general creative discovery is building a domain-agnostic novelty verifier that can judge “is this new?” without a pre-declared reference set. This is hard—it requires either:<br>Extensive domain knowledge and literature search

A way to check if an idea follows from established principles

Human-in-the-loop verification at scale

Want to Test Your Own Idea?

The approach generalizes: pick a recent, well-documented insight and see if you can manually find concept combinations that make LLMs rediscover it.<br>Good candidates include:<br>Technical insights with clear before/after boundaries (algorithm...