Deep Dive: Anthropic's Performance Take-Home (The One Claude Beat Humans At)

Deep Dive: Anthropic's Performance Take-Home (The One Claude Beat Humans At) | Tristan TrouwenToday, Anthropic open-sourced their original performance engineering take-home. The task: optimize a kernel running on a custom VLIW SIMD processor simulator. The baseline takes 147,734 cycles . Claude Opus 4.5 got it down to 1,487 cycles - a 99x speedup that beat most humans. I’m Tristan (@trirpi), and I work on AI kernels. Let’s break down how this whole system works. The Architecture at a Glance# This is a VLIW (Very Long Instruction Word) SIMD (Single Instruction Multiple Data) processor with a single core (older versions of the take-home had multiple cores). Let me break down what that means. VLIW: Compiler-Scheduled Parallelism# In a traditional processor, hardware figures out at runtime which instructions can run in parallel. In a VLIW processor, that job shifts to the compiler (or in this case, you). The single core has multiple functional units that can all execute simultaneously: UnitCountOperationsALU12 Scalar: +, -, *, /, ^, &, |, , >>, %, , ==VALU6 Vector (8 elements): same ops as ALULOAD2 load, vload (8 words), constSTORE2 store, vstore (8 words)FLOW1 select, jump, cond_jump, haltYou pack operations into instruction bundles . Each cycle, the processor executes one bundle, dispatching operations to all the units in parallel. If you only put one operation in a bundle, the other units sit idle. That’s why the baseline is so slow. Example bundle (executes in 1 cycle): {"alu": [op1, op2, op3], "valu": [vop1, vop2], "load": [ld1, ld2]}

With 12 ALUs and 6 VALUs (each processing 8 elements), this single core can theoretically do 12 + 6×8 = 60 arithmetic operations per cycle. Memory Hierarchy# flowchart LR subgraph mem["💾 MAIN MEMORY"] DATA["Problem Data (tree, indices, values)"] end

subgraph scratch["📦 SCRATCH SPACE (1536 words)"] REG["Works like registers All ALU ops read/write here"] end

mem |"LOAD/STORE ⚠️ 2 each per cycle"| scratch Main Memory : Where the problem data lives. ALU/VALU can’t access it directly. Scratch Space : 1536 words of fast storage. All compute operations read/write scratch addresses. Bottleneck : Only 2 loads and 2 stores per cycle. This is often the limiting factor, not compute. The Execution Engines# The processor has multiple engines , each capable of executing multiple slots per cycle. From problem.py: SLOT_LIMITS = { "alu": 12, # 12 scalar ALU operations per cycle "valu": 6, # 6 vector ALU operations per cycle "load": 2, # 2 load operations per cycle "store": 2, # 2 store operations per cycle "flow": 1, # 1 flow control operation per cycle "debug": 64, # Debug operations (not counted)

What an Instruction Bundle Looks Like# flowchart LR subgraph bundle["📦 Instruction Bundle (1 clock cycle)"] subgraph compute["Compute"] ALU["alu: ('+', dest, a, b) ('-', dest, a, b) ('*', dest, a, b) ...up to 12"] VALU["valu: ('*', vdest, va, vb) ('+', vdest, va, vb) ...up to 6"] end subgraph memory["Memory"] LOAD["load: ('load', dest, addr) ('vload', vdest, addr)"] STORE["store: ('store', addr, src) ('vstore', addr, vsrc)"] end subgraph control["Control"] FLOW["flow: ('select', d, c, a, b)"] DEBUG["debug: ('compare', loc, key) (not counted)"] end end An instruction is a Python dict mapping engine names to lists of operations. Here’s a real example: {"valu": [("*", 4, 0, 0), ("+", 8, 4, 0)], "load": [("load", 16, 17)]}

This executes three operations in one cycle : Vector multiply: scratch[4:12] = scratch[0:8] * scratch[0:8] Vector add: scratch[8:16] = scratch[4:12] + scratch[0:8] Scalar load: scratch[16] = memory[scratch[17]] The Problem: Batched Tree Traversal# The kernel implements a batched tree traversal with hashing. Here’s the flow: flowchart LR subgraph rounds["🔄 16 Rounds"] R0["Round 0"] --> R1["Round 1"] --> R2["Round 2"] --> RN["..."] end

subgraph batch["📊 Batch of 256 items"] B0["Item 0"] B1["Item 1"] B2["Item 2"] BN["..."] end

subgraph ALGO["⚙️ Per-item computation"] A1["idx = indices[i]"] --> A2["val = values[i]"] A2 --> A3["node_val = tree[idx]"] A3 --> A4["val = hash(val ^ node_val)"] A4 --> A5["idx = 2*idx + (1 if even else 2)"] A5 --> A6["if idx >= n_nodes: idx = 0"] end

rounds --> batch batch --> ALGO From the reference kernel: def reference_kernel(t: Tree, inp: Input): """ A parallel tree traversal where at each node we set cur_inp_val = myhash(cur_inp_val ^ node_val) and then choose the left branch if cur_inp_val is even. If we reach the bottom of the tree we wrap around to the top. """ for h in range(inp.rounds): for i in range(len(inp.indices)): idx = inp.indices[i] val = inp.values[i] val = myhash(val ^ t.values[idx]) idx = 2 * idx + (1 if val % 2 == 0 else 2) idx = 0 if idx >= len(t.values) else idx inp.values[i] = val inp.indices[i] = idx

Test configuration: Tree height : 10 (2047 nodes in a perfect binary tree) Batch size : 256 items processed Rounds : 16 iterations That’s 256 × 16 = 4096 traversal steps, each...

Deep Dive: Anthropic's Performance Take-Home (The One Claude Beat Humans At)

Related Articles

(no title)

Scientists reverse brain aging, with a nasal spray

AI has torched the market for junior programmers

Al Vigier: Canada's AI strategy shouldn't include secret Palantir bills

Google will expand age checks on Android worldwide till the end of the year