harrrshall on X: "introducing BarunLM (35m) 🤗

i pretrained the world's best small model under 100m parameters.

a 35m parameter language model that outperforms models over 6x larger, including LiquidAI's lfm2.5-230m, while training on a single H200 gpu.

the gains come from a better architecture. we took inspiration from deepseek, kimi, and other frontier open source models.

BarunLM combines hybrid local and global attention, grouped query attention to reduce memory, partial rope, gated attention, bounded swiglu, and residual selectors to improve how information flows through the network.

after pretraining, we further improved the model with rlvr (reinforcement learning with verifiable rewards) using deterministic verifiers and rloo, allowing the model to reinforce correct reasoning without relying on human preference labels." / X<br>Post

Log inSign up

Post

harrrshall

@HarshalsinghCN

introducing BarunLM (35m) 🤗

i pretrained the world's best small model under 100m parameters.

a 35m parameter language model that outperforms models over 6x larger, including LiquidAI's lfm2.5-230m, while training on a single H200 gpu.

the gains come from a better architecture. we took inspiration from deepseek, kimi, and other frontier open source models.

BarunLM combines hybrid local and global attention, grouped query attention to reduce memory, partial rope, gated attention, bounded swiglu, and residual selectors to improve how information flows through the network.

after pretraining, we further improved the model with rlvr (reinforcement learning with verifiable rewards) using deterministic verifiers and rloo, allowing the model to reinforce correct reasoning without relying on human preference labels.

span:not(:empty)~span:not(:empty)]:before:content-['·'] [&>span:not(:empty)~span:not(:empty)]:before:px-1 [&>span:not(:empty)~span:not(:empty)]:before:shrink-0">4:09 PM · Jul 31, 2026460Views

22

span:not(:empty)~span:not(:empty)]:before:content-['·'] [&>span:not(:empty)~span:not(:empty)]:before:px-1 [&>span:not(:empty)~span:not(:empty)]:before:shrink-0 min-w-0 overflow-hidden">harrrshall

@HarshalsinghCN

2h

model:huggingface.co/harrrshall/Bar…<br>receipe:github.com/harrrshall/bar…

harrrshall/BarunLM-35M · Hugging Face

From huggingface.co

svg]:size-5 text-body hover:bg-mix-current hover:bg-mix-amount-10 active:bg-mix-current active:bg-mix-amount-15 focus-visible:bg-mix-current focus-visible:bg-mix-amount-10 outline-current -m-2 shrink-0 cursor-pointer border-transparent p-0 text-body size-9 [&>svg]:size-[1.25em] [&>[data-engagement-icon]]:size-[1.25em] group-hover:bg-mix-current group-hover:bg-mix-amount-10" aria-label="View count" type="button" data-state="closed" href="/HarshalsinghCN/status/2083225256026923090/quotes">139

span:not(:empty)~span:not(:empty)]:before:content-['·'] [&>span:not(:empty)~span:not(:empty)]:before:px-1 [&>span:not(:empty)~span:not(:empty)]:before:shrink-0 min-w-0 overflow-hidden">Priyanshu (agentic arc)

@ubermensch_hb

2h

LFG🔥

svg]:size-5 text-body hover:bg-mix-current hover:bg-mix-amount-10 active:bg-mix-current active:bg-mix-amount-15 focus-visible:bg-mix-current focus-visible:bg-mix-amount-10 outline-current -m-2 shrink-0 cursor-pointer border-transparent p-0 text-body size-9 [&>svg]:size-[1.25em] [&>[data-engagement-icon]]:size-[1.25em] group-hover:bg-mix-current group-hover:bg-mix-amount-10" aria-label="View count" type="button" data-state="closed" href="/ubermensch_hb/status/2083228297513804262/quotes">55

Log in or sign up for X<br>See what’s happening and join the conversation<br>Continue with phoneContinue with AppleContinue with Google<br>or<br>Log in with username or email

Relevant people

harrrshall@HarshalsinghCNFollow<br>Believe the world can be a better place, and that it should be made by me.

Trending now