Code is the Byproduct

Sign in<br>Subscribe

Recently, the Jacobian Conjecture was disproven by a counterexample discovered by an LLM. Shortly thereafter, a ChatGPT session from mathematician Terence Tao made the rounds online. In his chat, Tao uses ChatGPT to help him wrap his head around the implications of the result.<br>If you use LLMs in your day-to-day, it is worth taking a look at his chat, even if (or especially if) you have no interest in higher level mathematics.<br>The "Understanding Bubble"<br>Most of us interact with LLMs within our own domain bubble. We spot inaccuracies where we have deep experience and validate their output when necessary. On the other hand, we get burned around the edges of our understanding when we know enough to understand the answer, but not enough to confidently refute an LLM when it is wrong.<br>Reading Tao's chat is something else entirely. We rarely get to see a world-class expert interact with an LLM in a domain obscure to most humans.<br>Terence Tao immediately begins communicating with ChatGPT about high level concepts and the LLM responds in kind. If this was chat was posted 5 years ago, we would assume it is a chat between two eminent mathematicians, not a human and a stochastic parrot.<br>It is remarkable that the jacobian is constant, that is an exceptional amount of cancellation. Does this polynomial map have any symmetry or other structure that makes this cancelation less miraculous?<br>Because the math itself is beyond most readers, what stands out is the shape of the interaction. Tao starts his inquiry by noting a single, precise detail that intrigues him. He then drills into the responses, expanding or narrowing his focus based on his own train of thought. This is an expert engaging with an LLM on his own terms.<br>Narrow Focus, Great Depth<br>What happens when a normal user tries to understand the Jacobian Conjecture counterexample by chatting with an LLM? Our questions are broad, our terms are vague and and the LLM mirrors our surface-level phrasing and understanding.<br>In other words, LLM have been talking down to us.<br>We know a single word can reframe what a prompt produces, which has been used as a sign of "LLM randomness". But on a deeper level, recognize that an LLM extracts every nanogram of understanding from a prompt that it can. For example, as a programmer if I say "make this code clean" the LLM will do one thing, but saying "revise according to DRY principles" gives the LLM a much clearer domain of understanding from which to make changes.<br>Your words do more than convey a request. They anchor the depth of an LLM's data retrieval before it even begins to assemble a response.<br>Short Prompts, Specific Details<br>Note what Tao is not doing. He is not building a rich contextual background for the LLM to unpack. He does not list his credentials or say, "you are a mathematical genius, make no mistakes." Instead, he is terse and precise. He asks narrow questions and gets narrow responses. He does not say "Generate a 20 page report that analyzes and describes this finding." That produces a wall of text that looks impressive but conveys little.<br>Keep your requests narrow and specific. Let the output compound through repeated inquiries. Engage with the goal of improving your personal understanding.<br>The Goal is Understanding<br>Many users treat LLMs as asset factories write code, generate images, or draft emails. Because we focus on the final asset, we skip the process of building deep comprehension.<br>Much of the time, a "roughly correct" output is good enough for our needs. But for a request that needs specificity, establishing a shared understanding through the LLM is a necessary waypoint.<br>How do you construct shared understanding? By narrowing the LLM's focus through domain-specific wording and keeping requests specific and cumulative.<br>This is how ChatGPT is able to respond to Tao's mathematical concerns without needing heaps of extraneous detail. Tao pins ChatGPT to a "deep mathematics" headspace through terminology and the specificity of his question.<br>If you don't understand something, set the stage for the LLM and it will capably fill in the missing 20%.<br>Curiosity Beats Capability<br>Curiosity is the new superpower, not raw capability. LLMs are capable workers, but they need precision to return accurate results. Without domain understanding, we fall back to broad requests and fuzzy answers. LLMs mirror our understanding.<br>Product or Byproduct?<br>As a software engineer, I have shifted my focus from asking for code to asking for clarity. Code is written only after understanding is constructed. Here are some ways I use LLMs in my day to day:<br>When planning a feature:Can this feature reuse any existing code? Is there an existing pattern the design of this feature should follow? Describe the architecture of this feature. Where are the bottlenecks? Is this an extensible approach if I want to add Feature B down the road? Is there a way to implement this without affecting this other part of the...