LFM2.5-DSpark: Up to 3.2x Faster Inference from H100 to MacBook — Blog — Liquid AI
Connect
h2]:clear-both [&>h3]:clear-both">Today, we release DSpark [1] draft model checkpoints for three models from our LFM2.5 family: LFM2.5-1.2B-Instruct, the recently released LFM2.5-2.6B, and LFM2.5-8B-A1B. These add a speculative decoding path that trades a minimal memory increase for a large decoding speedup without changing output quality. The draft models reach up to 3.18 throughput improvement on a GPU and up to 2.87x on-device.<br>This is the first public release of speculative decoding models for Liquid Foundation Models (LFMs). We believe co-designing the model architecture with speculation methods to model real-world inference characteristics will be a crucial part of model design going forward.<br>The DSpark draft models for LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B are available today on Hugging Face. Additionally, the LFM-compatible DSpark integration is open-sourced upstream in llama.cpp and SGLang. Check out the model cards for how to run them.<br>How DSpark works<br>The decode phase in LLM inference is traditionally memory-bound. Most of the latency comes from streaming weights from DRAM into SRAM, not from intense computation. This is true for both powerful GPUs, such as the NVIDIA H100, and for edge devices like MacBooks or iPhones.<br>Speculative decoding addresses this issue by using a lightweight draft model to produce candidate tokens. The target model then verifies them all in a single forward pass, sharing the cost of loading the weights across all tokens we are verifying.<br>Over the years, multiple approaches of speculation have been proposed, with the most prominent being EAGLE-3 [2], DFlash [3], and, most recently, DSpark, which combines three components:<br>DFlash-style parallel backbone conditioned on context features from the target model that runs a single forward pass over a block and produces hidden states and base logits for each of kkk draft tokens.<br>A lightweight sequential head , modeled as a Markov chain between neighboring tokens, that biases each position's logits toward continuations consistent with the token sampled just before it. This adds dependency between drafted tokens, which is absent in the DFlash approach, raising the acceptance rate at later positions.<br>A confidence-scheduled verifier : a separate head predicts each draft token's acceptance probability conditioned on all previous ones being accepted, and a hardware-aware scheduler prunes low-confidence suffixes whenever verifying them would cost more batch capacity than they're worth.<br>Training<br>We follow the DSpark recipe with an extended data mixture to train on larger and more diverse groups of datasets. Our final corpus mixes SFT, chat, code, and function-calling data.<br>To find the optimal setting, including the optimal number of layers, block size, and model architecture, we run ablations on a subset of the full training dataset. For the first version of the draft models, we used simplified attention-only draft models. Throughout the ablations, we settled on 5 layers with a block size of 9. For each draft model, we ran 15 epochs on the entire dataset.<br>For each epoch, we measured the validation loss and the acceptance rate across target benchmarks. Figure 1 shows that the three models do not behave alike: For LFM2.5-1.2B-Instruct, acceptance improves consistently across epochs, closely tracking the fall in validation loss. LFM2.5-2.6B improves for the first few epochs and then plateaus, its acceptance flattening well before the loss does. LFM2.5-8B-A1B is the least stable, with validation loss decreasing with more tokens but not yielding gains in acceptance.<br>Figure 1: Mean acceptance length and validation loss over training tokensThe takeaway is that validation loss is a useful in-flight signal for the LFM2.5-1.2B-Instruct draft, but for the larger drafts it keeps improving after the benchmark acceptance we care about has stopped moving. For the final published checkpoints, we therefore select the epoch with the highest acceptance rate rather than based on loss.<br>The resulting draft models are relatively small, with each around ~300M parameters, as shown in Table 1. We trained each model and ran all ablation studies exclusively on AMD hardware using Liquid AI’s training framework.<br>Component<br>LFM2.5-1.2B-Instruct<br>LFM2.5-8B-A1B<br>LFM2.5-2.6B<br>Decoder stack (5 layers)<br>241.2M<br>241.2M<br>241.2M<br>Hidden-state projection<br>21.0M<br>21.0M<br>21.0M<br>Markov head<br>33.6M<br>65.5M<br>65.5M<br>Norms + confidence head<br>27.5k<br>27.5k<br>27.5k<br>Total<br>295.7M<br>327.7M<br>327.7M
Table 1: Draft model size. Embedding and LM head are tied to the target, not carried by the draft.<br>Quality parity<br>Under greedy decoding, a draft token is only accepted if it matches the target model’s distribution. On rejection, the target model's own token takes its place. The emitted sequence is therefore identical to baseline greedy by construction, so benchmark accuracy (pass@1 or exact match) is...