The Anatomy of a Learning Stall – Tagide
Preloader Close
Contact Us
);">
LLM
June 5, 2026
By
0 Comments
The Anatomy of a Learning Stall
The Anatomy of a Learning Stall
Or How LLM Hallucinations Become Human Hallucinations
I recently had an experience with an undergraduate student that really made me pause on how we use AI. I made a couple of posts on LinkedIn with very short snippets of what happened, but I want to write down this experience in detail, because the details shed a spotlight on the mechanics and the consequences of the often-heard idea that, in the near future, we, humans, will not need to know anything about the internals of the systems we build; that we just need to specify what we want and, once the specification is kosher, the AI agent will do it and it will all be correct.<br>Maybe it will all be correct. And maybe it will also be wrong. Correctness is not the same as "the right thing." And this distinction is crucial to separate hallucinations from reality.<br>The Independent Project, Week 1<br>This all started in early April with an undergraduate student, let’s call him Joe, approached me for an independent study — these are research project courses that students can take for credit, and are typically done by students who are considering graduate school or are simply curious about research. Joe, a junior, told me he wants to apply to PhD programs. So on week 1 of the quarter, we talked about his interests and my current interests, and we zoomed in on a possible project for him. Here’s a summary of that conversation.<br>Lately, I am generally interested in methods for automatic verification of software artifacts using LLMs/agents/neural networks (we had a first paper on that back in 2023/2024). Joe told me he is interested in the models themselves, and wanted the experience of fine-tuning a model. So I suggested a project around the idea of automatic verification of protocol specifications. I selected a specific protocol — MQTT— for him to do an experiment involving the development of an AI agent that could, potentially, automatically verify the protocol. I also mentioned to him two possible routes for this general idea: (1) we could have the AI agent verify the consistency of the protocol specification itself, that there are no inconsistencies between the many clauses; or (2) we could grab concrete implementations of MQTT out there and verify whether they comply or not with the protocol specification.<br>The meeting ended on a good note of mutual understanding, with Joe’s next goal being deciding which of the two problems to tackle, and then take it from there. In my mind, it was pretty clear that this project was feasible in a quarter, especially with the help of Claude/Gemini/etc.<br>Week 8<br>The way I work with students is to have one-on-one weekly meetings, possibly augmented with additional meetings when the students go through fast progress that needs my steering. I don’t make any of this mandatory, as I’ve worked with enough students to know that every one of them is different and needs different amounts of supervision; also because I treat these students as younger peers, not employees. Joe never showed up for the weekly meetings — fine.<br>He showed up on Week 8 with the complete execution of the project. He was excited to show it me. He told me he developed an agent that could, indeed, verify that MQTT was correct and that the agent was based on a fine tuning of Qwen. Glancing at his Visual Studio, the thing looked pretty impressive and well done: a proper folder structure, files that had meaningful names, a data folder that clearly contained training data for fine tuning, lots of references to MQTT , etc. Looked plausible and impressive! After this 2 minute oral abstract of the project, he asked if we could write a paper about this project and submit it to a conference. I was excited, too, so I asked him to explain what exactly did he do.<br>*Joe’s slight pause* I build an agent that verified the MQTT protocol<br>*Me* OK, but which of the two problems did you tackle? Is it the specification consistency problem or the implementation correctness problem?<br>*Joe hesitating* uh… I think… the specification correctness?<br>*Me, looking at his training data and seeing lots of Python code in it* Really? Are you sure?<br>*Joe hesitating even more but trying to sound confident* uh, yes, the specification correctness<br>*Me pointing at the training data* So what is this training data all about? What are all these Python functions that look like an MQTT implementation?<br>*Joe now visibly nervous* Sorry, I meant I did the implementation correctness. I verified that a given Python implementation is correct<br>*Me, now a bit worried* OK. And how did you do it?<br>*Joe with a blank stare of confusion* I build an agent.<br>*Me, even more worried* And how does the agent verify that the implementation is correct?<br>*Joe really nervous* uhh…<br>*Me jumping in to help him* OK let me take a closer look at your code.<br>I looked at the code...