LiminalML — Deep Mastery Review for ML & SWE<br>Skip to contentSign in
Study to the depth you'll be tested at.<br>Six-stage sessions on hard ML and SWE topics: the intuition, the derivation, code that runs, and a retrieval check you have to answer. Add a resume and that check asks about work you have actually shipped, not a generic example. The tutor stops after every stage, and every thread saves so you can resume days later.<br>Start a sessionWatch a session run<br>Two full sessions with no account, no card, no email.<br>157 topics · 10 domains · 2 tracks · 6 stages per session.
Attention weights, causal maskquery 4 / 12
softmax( QK^T / sqrt(d_k) ) Vstage 03 of Attention Mechanisms
Why this beats asking a chatbot to explain it<br>six differences<br>01<br>It stops and waits<br>A chat window hands you a wall of text and moves on. Here the session halts after every stage until you continue, ask, or take a card. That pacing is what turns reading into review.
3The mathcomplete
Waiting for you. Nothing arrives until you choose.<br>ContinueAsk a follow-upRevision card
02<br>The derivation is motivated<br>Every term gets a reason to exist. You learn what breaks when a term is removed, which is the question an interviewer actually asks.
3Scaled dot-product attention<br>Attention(Q,K,V)=softmax (QK⊤dk)V\mathrm{Attention}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)VAttention(Q,K,V)=softmax(dkQK⊤)V<br>Why the scale factor. Drop it and the dot products grow with dimension:<br>Var(q⋅k)≈dk⟹∂L∂w→0\mathrm{Var}(q \cdot k) \approx d_k \quad\Longrightarrow\quad \frac{\partial L}{\partial w} \to 0Var(q⋅k)≈dk⟹∂w∂L→0
03<br>The code runs<br>Reference PyTorch next to the math, with the mask in the right place and a test at the end. Theory and implementation on one screen.
4Implementationpython<br>def attn(q, k, v, mask=None):<br># scale before the softmax<br>s = q @ k.transpose(-2, -1) / q.size(-1) ** 0.5<br>if mask is not None:<br>s = s.masked_fill(mask == 0, float('-inf'))<br>return torch.softmax(s, dim=-1) @ vmask applied before softmax, not after
04<br>You have to retrieve it<br>Stage 6 makes you reproduce the concept unaided, then grades the gaps. Recognition feels like knowing; retrieval is knowing.
6Retrieval check<br>State the attention equation and say what the scale factor does.<br>You scale by dk\sqrt{d_k}dk so the values stay small before the softmax.<br>+Correct. The equation and the scale factor are both right.<br>!Missing. Why saturation kills the gradient, not just that it keeps values small.
05<br>You write it, then it gets reviewed<br>The practice lab is an editor where you implement the thing from scratch in PyTorch or NumPy. What comes back is a code review — what is wrong and on which line, what you got right, and one question about a design choice you made.
▸Practice labyour code<br>def attention(q, k, v):<br>s = q @ k.T / d_k ** 0.5<br>return torch.softmax(s, -1) @ v!Correctness. k.T transposes a 2-D tensor only — batched heads need transpose(-2, -1).<br>+What's right. You scaled before the softmax, not after.<br>?One deep question. You masked after computing every score. At a 32k context, what does that cost you?
06<br>It asks about your projects<br>Add a resume and the retrieval check stops quizzing you on a generic model. It asks you to defend the decisions you actually made and shipped — which is the version of the question an interviewer will ask.
6Retrieval checkfrom your resume<br>without a resume<br>State the attention equation and say what the scale factor does.
with your resume<br>Your two-tower retrieval service runs at a 30ms budget. Which attention layer would you drop first to hold it, and what degrades when you do?
Watch one run<br>A full session, exactly as it renders.
A recorded replay of all six stages — the same math, code, and diagrams a live session produces. No account, no model calls.
Demo session · Attention Mechanisms<br>Scripted replay · no model calls<br>Try it right here<br>Watch a full session run.<br>A recorded replay of a real six-stage session — the exact pauses where the tutor stops and waits, plus two follow-ups you can ask. No account, no model calls.<br>Run the demo session6 stages · the same math, code, and diagrams a live session renders · ~3 minutes
Six stages, in order, never compressed — the pause after each one is the point.
The format<br>Every topic, in the same six stages.
Never compressed, never reordered. The pause between stages is the product.
01<br>Big Picture<br>The concept in context: what problem it solves, where it appears in real systems, and the mental frame to keep before details arrive.<br>context
02<br>Intuition + Visual<br>Core idea in plain language, then a structured diagram with tensor dimensions and data flow annotated. Mandatory for all DL architectures.<br>diagram
03<br>The Math<br>Step-by-step derivation with every term motivated. Not just what each symbol is, but what breaks if you remove it.<br>derivation
04<br>Implementation<br>Production-quality PyTorch with type annotations, every non-obvious line commented, and an explicit test snippet at the...