GitHub - jrm-code-project/llambda: LLM hacks · GitHub
/" data-turbo-transient="true" />
Skip to content
Search or jump to...
Search code, repositories, users, issues, pull requests...
-->
Search
Clear
Search syntax tips
Provide feedback
--><br>We read every piece of feedback, and take your input very seriously.
Include my email address so I can be contacted
Cancel
Submit feedback
Saved searches
Use saved searches to filter your results more quickly
-->
Name
Query
To see all available qualifiers, see our documentation.
Cancel
Create saved search
Sign in
/;ref_cta:Sign up;ref_loc:header logged out"}"<br>Sign up
Appearance settings
Resetting focus
You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.
Dismiss alert
{{ message }}
jrm-code-project
llambda
Public
Notifications<br>You must be signed in to change notification settings
Fork
Star
main
BranchesTags
Go to file
CodeOpen more actions menu
Folders and files<br>NameNameLast commit message<br>Last commit date<br>Latest commit
History<br>9 Commits<br>9 Commits
.github
.github
.gitignore
.gitignore
LICENSE
LICENSE
README.md
README.md
llambda.asd
llambda.asd
llambda.lisp
llambda.lisp
package.lisp
package.lisp
tests.lisp
tests.lisp
View all files
Repository files navigation
llambda.lisp
A Bare-Metal, Multi-Threaded, AVX2-Accelerated LLM Inference Engine in Pure<br>Common Lisp.
llambda.lisp is an independent, zero-dependency (beyond sb-simd and<br>lparallel) inference engine for running quantized Large Language Models<br>directly from .gguf files.
It does not wrap llama.cpp. It does not call out to external C or C++<br>libraries. It reads the raw weights, unpacks the 4-bit/6-bit nibbles, constructs<br>the transformer architecture, and executes the forward pass natively within<br>SBCL.
Why?
Because the industry has succumbed to the dogma that C++ is the only path to<br>bare-metal AI inference. llambda.lisp exists to prove that properly<br>architected, aggressively typed, and hardware-aware Common Lisp can achieve<br>C-level throughput without sacrificing the interactive, REPL-driven elegance of<br>Lisp.
Features & Architecture
Native GGUF Parsing: Directly ingests and parses Q4_K_M and Q6_K quantized tensors from disk.
AVX2 / FMA Acceleration: The core GEMV (Matrix-Vector Multiplication) bottleneck is pulverized using SBCL's sb-simd. Unrolled f32.8 vectors, unaligned loads (VMOVUPS), and packed Fused Multiply-Add (VFMADD213PS) instructions are emitted natively by the Lisp compiler.
Multi-Threaded Execution: The outer loop of the GEMV processing is fully parallelized via lparallel, saturating modern multi-core memory buses (e.g., 24-core Ryzen processors) with isolated, lock-free writes.
Zero-Drift KV Cache: Safe, shared-KV reuse and perfectly aligned RoPE scaling. Exact-logit replay tests against fresh un-cached generations yield a max_diff of 0.0.
Advanced Sampling: Built-in Top-K, Top-P (Nucleus), and repetition penalties executing in-place with zero heap allocation in the hot path.
Gemma4 Support: Full support for Gemma4 architectures, including BPE tokenization, proper instruction-tuning chat templates (user...), and explicit tool-calling channel overrides.
Requirements
SBCL: You must run a modern SBCL compiled with SIMD support.
Hardware: An x86_64 CPU with AVX2 instruction set support. Multi-core processors heavily recommended to prevent memory-bus starvation.
Dependencies: sb-simd, lparallel.
Quickstart
(ql:quickload :llambda)
;; Load a model and run an end-to-end inference pass<br>(llambda:test-gguf-file-response<br>"D:/path/to/your/model/gemma-4-E4B-it-Q4_K_M.gguf"<br>"Write a haiku about a hacker drinking coffee."<br>:top-k 40<br>:top-p 0.90<br>:repetition-penalty 1.15)
Performance & Optimization
If you are modifying the core dot-product macros (expand-q4-k-body), heed<br>this warning: Do not allocate in the inner loop. The hot paths rely on<br>strict (declare (optimize (speed 3) (safety 0) (debug 0) (space 0))) policies<br>and zero-consing execution. If the compiler begins boxing floats or allocating<br>vectors on the heap, performance will catastrophically collapse.
Current Status & Roadmap
Gemma4 Base & Instruct (Verified)
Top-K / Top-P / Rep-Pen Sampler
AVX2/FMA Q4_K_M and Q6_K paths
Qwen3 / MoE Routing (In Progress)
LLaMA 3.1 Architecture (Planned)
Author Joe Marshall
License
MIT License. See LICENSE for details.
Contributions
Contributions are welcome! Please submit pull requests or open<br>issues for bug reports and feature requests.
About
LLM hacks
Resources
Readme
License
MIT license
Uh oh!
There was an error while loading. Please reload this page.
Activity
Stars
stars
Watchers
watching
Forks
forks
Report repository
Releases
No releases published
Packages
Uh oh!
There was an error while loading. Please reload this page.
Contributors
Uh oh!
There...