TS
Menu
Home
Archives
Tags
About Me
RSS
May 25, 2026
Stacking On Quicksand
LLM Within<br>What if we allowed dynamic evolution of programs mid-way through their execution? A question I had often mused about but could never figure out how to do. Given no prior knowledge in PLT, digging deeper into this seemed like a herculean possibility. With the advent of LLMs and the general clamor for ambitious projects, I decided to give this a try. Having an unused weekly Claude quota helped. I ended up building a small project and describe some of the interesting observations below. The complete code is at llm-clojure .<br>The Problem Statement<br>The intent is to have programs that can dynamically evolve from within, based on some arbitrary instructions. A few questions arise<br>How can you instruct while coding, is it comments or decorators etc.<br>What is the context required for the LLM to be effective ?<br>How does it interplay with rest of the language ecosystem i.e. function calling , basic forms like if/loop etc.<br>How does the execution stack look like when calling LLM vs calling a regular code path<br>On initial exploration, it looked like I would be creating a new language and answering the questions from the ground up, with all the associated complexities of building a grammar, lexer, and maybe even a VM. Then I remembered reading about Clojure/Lisp and its ability to change at runtime using macros1. That got me thinking — if I had a macro that could transform code at runtime, that would alleviate the need for building my own stuff2. The flip side was that I have only a sophomoric knowledge of Clojure, from what I remember from doing a couple of tutorials almost a decade ago and going to a few meetups. This means I would be relying on an LLM to generate code in a language that is not usually held up as a poster child for LLM-generated code. But given this is a weekend project and not a professional undertaking, it might be fun to explore riskier paths.<br>Solution Space<br>There were a couple of key decision points on how to build a working system.<br>Form<br>How developers will interact with the system is something of a sticking point. The intent is to make it as seamless as possible and fit within the existing host language paradigm. So, having a macro system that allows seamless transformation while keeping the rest of the programming model similar3, I decided to add a macro with the form ^, which allows for writing interwoven code, e.g.<br>(defn summarize [document max-words]<br>^{:intent "summarize document in plain language, stay under max-words"<br>:max-iterations 2}<br>(summarize-text-placeholder document max-words))
(summarize "Some long winded text..." 20)<br>or below which shows how it can be easily interwoven with a regular switch (cond / conditional) style code<br>(defn process-payment [payment]<br>(let [status ^{:intent "classify payment as :approved :declined :fraud or :pending"}<br>(classify-payment payment)]<br>(cond<br>(= status :approved) (complete-payment payment)<br>(= status :declined) (notify-declined payment)<br>(= status :fraud) (flag-and-block payment)<br>:else (queue-for-review payment))))<br>The main intent is to keep both the host and the LLM as interoperable as possible so that developers don't have to do mental context switching when programming. The system has a symbol lookup map for built-in functions like map, +, and filter, and uses apply to execute them. This currently has issues when importing/requiring from other namespaces, but works with a baseline set of functions pre-loaded in the base environment<br>In addition, the REPL usability is not great and will need more debugging tooling support for daily use.<br>Context<br>Providing the correct context is important for LLMs to be able to do their job. This comes in two parts: current context and historical memory. The current context is a must-have, while the historical memory is good to have to provide guidance to LLMs.<br>Current Context<br>Provided as part of the form itself and passed to the LLM which helps it steer<br>^{:intent "extract product name"<br>:language "detect automatically, normalize output to English"}<br>(extract-name (:description raw))<br>The keys are not enforced but serve as guidance for the LLM. In this case, extract-name is just a placeholder. The LLM responses are sent to a judge LLM, which adds error information before retrying in case of an error. This helps the next call get better guidance. The LLM evaluator keeps trying to resolve up to a fixed number of iterations4.<br>In addition, the system needs to identify when to expand/apply the forms inside ^ and when to pass them as literals. This is similar to template and backtick concerns in bash. A few examples:<br>before: (map (:name input) :to category-ref)<br>after: (map "iPhone 15 Pro" :to {:electronics 1 :clothing 2 :books 3 ...})<br>The long form flow chart of when forms are resolved and when passed as is<br>node is a nested ^/^^ form?<br>→ leave intact (will be resolved in its own llm-eval call)<br>node is a seq?<br>→ first element is a keyword?<br>→...