Fine-Tuning from First Principles: LoRA, QLoRA, and Serverless Fine-Tuning on Crusoe – My Blog

Introduction

In my previous posts, we spent a lot of time learning how to train large models across many GPUs. We started with the fundamentals of distributed training, where we did the memory math that explains why a single GPU runs out of breath, and then took a deep dive into FSDP, where we sharded parameters, gradients, and optimizer states across a cluster.

But somewhere along that journey, a natural question comes up: when we fine-tune a model, do we really need to update all of its weights ? The model already knows English, grammar, world knowledge, and how to write code. We are usually just teaching it a new behavior, follow this format, redact this document, speak in this tone. Do we really need to move 8 billion parameters to do that ?

In this post, we’ll answer that question from first principles. We will build our way up from raw bits to a production fine-tuning workflow , and by the end, you’ll have a crystal-clear mental model of how modern parameter-efficient fine-tuning actually works. Here is the journey:

First, we’ll do the memory accounting for full fine-tuning and see why it’s a multi-GPU problem (and connect it back to what we learned in the distributed training post).

Then we’ll shrink the model itself with quantization , writing an 8-bit quantizer by hand before we trust a library to do it on our behalf.

Next, we’ll build LoRA (Low-Rank Adaptation) from scratch in PyTorch, train it, and prove mathematically with SVD (Singular Value Decomposition) that the update really is low-rank.

We’ll combine the two ideas into QLoRA and fine-tune a real model end to end with Hugging Face peft.

And finally, we’ll put all of this understanding to work on a real project: fine-tuning Qwen3-8B into a PII redaction engine using Crusoe’s Serverless Fine-Tuning , deploying it, and we will compare it to a general-purpose 70B model on the same task.

All the code in this post lives in this repo: here. You can run the first two notebooks on any single NVIDIA GPU, and the final project runs on Crusoe’s serverless platform (we’ll see how later).

NotePrerequisites

This post is designed to be self-contained, but it helps if you are familiar with:

The basics of how a neural network trains (forward pass, loss, backward pass, optimizer step)

What parameters, gradients, and optimizer states are, and roughly how Transformers are structured

A little bit of matrix multiplication (we will go over some basics)

If you’d like a refresher on the memory side of training, the static and dynamic memory sections of my distributed training post cover it in detail.

Why Fine-Tune at All ?

Before we touch a single matrix, let’s ask the obvious question: with models as capable as they are today, why would we fine-tune at all ? We have prompting, and we have RAG. Aren’t those enough ?

They often are! A good way to think about it:

Approach<br>What it changes<br>Best for

Prompting<br>Nothing, we just ask nicely<br>Quick experiments, general tasks

RAG<br>What the model sees (context)<br>Injecting fresh or private knowledge

Fine-tuning<br>What the model is (weights)<br>Teaching a consistent behavior, format, or style

The rule of thumb I find most useful: RAG changes what the model knows, fine-tuning changes how the model behaves. If our problem is “the model doesn’t know about our internal documents”, that’s RAG. If our problem is “the model won’t reliably produce this exact structured output, in this exact style, every single time”, that’s fine-tuning. And as we’ll see at the very end of this post, a small fine-tuned model can beat a model ~9x its size on exactly this kind of behavioral task.

Fine-tuning itself comes in a few flavors, depending on what signal we train on:

%%{init: {"theme": "base", "themeVariables": {"fontFamily": "sans-serif", "lineColor": "#868e96"}}}%%<br>flowchart TD<br>A(["Pre-trainingtrillions of tokens"]) --> B("Base model")<br>B --> C("Supervised / instructionfine-tuning (SFT)this post!")<br>B -.-> D("Continued pre-training(domain adaptation)")<br>C --> E("Preference tuning(DPO / RLHF)")<br>C ==> F(["Assistant / task model"])<br>E ==> F<br>classDef blue fill:#e7f5ff,stroke:#1971c2,stroke-width:2px,color:#1971c2<br>classDef green fill:#ebfbee,stroke:#2f9e44,stroke-width:3px,color:#2f9e44<br>classDef gray fill:#f8f9fa,stroke:#adb5bd,stroke-width:2px,color:#868e96<br>classDef violet fill:#f3f0ff,stroke:#6741d9,stroke-width:2px,color:#6741d9<br>classDef teal fill:#e3fafc,stroke:#0c8599,stroke-width:3px,color:#0c8599<br>class A,B blue<br>class C green<br>class D gray<br>class E violet<br>class F teal

In this post we’ll focus on supervised fine-tuning (SFT) : we show the model pairs of (prompt, desired response) and nudge its weights so that the desired response becomes more likely. The training loop is exactly the one we know, next-token prediction with cross-entropy loss, just on our curated examples instead of the open internet:

The algorithm is nothing new. The...