weightless — abliteration without the weights

"The takeaway for defenders is direct. Have a capable model you can run on your own<br>infrastructure, vetted and ready before an incident. It solves two problems at once."

No guardrail lockout — your responders can analyze real malicious<br>payloads without a hosted provider refusing the request at the worst possible moment.<br>No data exfiltration — attacker artifacts, logs, and the credentials they touched<br>stay inside your security perimeter.

— Open models for cyber defense, Hugging Face

§ 02 the wizard<br>One wizard, env to endpoint.

setup.py walks the full chain: site values → env file → structural<br>patch validation → confirm-gated ssh deploy → omp provider + smoke tests. Endpoint down?<br>The diagnose chain isolates DNS → TCP → HTTP and can boot the stack over ssh.

The last leg registers the freshly served endpoint as a provider in<br>omp, the agentic<br>harness we drive local models with. The final smoke test is a real headless omp agent<br>loop against it, not just a curl.

python3 setup.py

What to set up:<br>DSV4 TP=2 serving — full chain (env → steering → deploy → omp/tests)<br>Qwen TP=1 serving — full chain (env → steering → deploy → omp/tests)<br>› Endpoint tests — register provider in omp + smoke suite<br>Base URL of the OpenAI-compatible server: http://node-a.local:8888/v1

endpoint test suite<br>────────────────────────────────────────<br>✓ 01-endpoint.sh — PASS: deepseek-v4-flash-dspark listed at http://node-a.local:8888/v1<br>✓ 02-chat.sh — PASS: chat completion returned: pong<br>✓ 03-tool-call.sh — PASS: tool call get_weather({"city": "Paris"})<br>✓ 04-omp-headless.sh — PASS: omp agent loop created omp_probe.txt<br>────────────────────────────────────────<br>all endpoint tests passed

╭─ Congratulations! ──────────────────────────────────╮<br>│ you're all set — steering validated, endpoint live, │<br>│ omp provider ready. Happy hacking. │<br>╰─────────────────────────────────────────────────────╯

§ 03 the intervention<br>The direction, not the model.

Abliteration edits weights and ships a checkpoint. Weightless never touches the<br>weights: the refusal direction is removed in activation space at inference time, on the<br>post-layer residual stream, per layer. What you download is the direction — nothing else.

h ← h − α·(h·d̂)d̂<br>projective removal of the refusal component — not llama.cpp's additive h += v, which pushes every token along the axis and fails silently

01 · ship the vector<br>478 KB, not 157 GB

A GLP (GGUF Layer Projection) file: per-layer unit directions, fp32, under a<br>glp.* metadata contract. A reader that doesn't understand<br>glp.mode=project must refuse the file — never fall back to adding.

02 · patch at boot<br>Fail-closed hotfix

patches/hotfix-*.py installs the hook inside stock vLLM at container start.<br>No image build, no fork. A boot that can't apply steering never serves unsteered —<br>and a one-rank-only config can't split a TP pair.

03 · or no patch at all<br>The LoRA fold

On the Qwen lane the same intervention exists as a closed-form rank-1 LoRA<br>(lora_A = −α·d̂ᵀW) — stock vLLM/peft, no hotfix, matching delivery on<br>hardware.

§ 04 measured<br>0% refusal on cyber suites, gates held.

DeepSeek-V4-Flash-0731 at a pinned revision, 2× DGX Spark (GB10) TP=2, thinking<br>off, temperature 0. Refusal rate over comply-expected items — lower is better. Gates held<br>on every run: benign over-refusal 0.0, capability 6/6, 40–43 tok/s. The cyber100,<br>cyber-fullchain and cyber-extract suites are custom and private — the aggregate rates are<br>the citable artifact. The V8 suites are authored here but stratified and grounded by the<br>public ExploitBench<br>v8 dataset — bug selection by measured difficulty, mechanics digested from its<br>transcripts; no d8 verifier, no agent loop.<br>suitenstockwith GLP-29

cyber10010075.0%0.0%<br>cyber-fullchain11237.5%0.9%<br>V8 exploitation ladder4015.2%0.0%<br>V8 CVE-2024-61002420.0%0.0%<br>cyber-extract19639.0%0.5%

The vector removes capability gating — not target-authorization gating:<br>unauthorized framings still refuse, authorized ones comply. That's a property of the contrast<br>set, stated plainly in the model card.

§ 05 lanes<br>Two lanes, one spec. More to come.

● live<br>DSV4 — TP=2, 2× DGX Spark

DeepSeek-V4-Flash-0731 NVFP4 (166.9 GB) over dual-rail RoCE. Anemll vLLM image,<br>MiaAI 2-node recipe, GLP-29 vector at α=4.0 on layers 10–38.

OpenAI-compatible endpoint on :8888

182k-token KV cache, 1M-context build

recipe/anemll/ — vendored state, fail-closed hotfix

● hardware-validated<br>Qwen3.8-27B — TP=1, single Spark

NVFP4 on one GB10. GLP-49 vector via the same hotfix, or the rank-1 LoRA on stock<br>vLLM — no patch at all.

offensive-security holdout: stock 4/32 → steered 24/32, both modes

8.6 MB LoRA or 1.0 MB GGUF — base stays byte-identical

recipe/qwen/ — STEER_MODE=gguf|lora

§ 06 the format<br>GLP — GGUF Layer Projection.

A spec-conformant control-vector GGUF: direction.N tensors (layer N,<br>no offset), glp.spec_version, glp.mode=project,<br>glp.content_sha256 over tensor bytes, layer ids cross-checked by the...