Sparse Attention with Persistent State Machines – High‑Sparsity LLM Accelerator

yusuke_esaka1 pts0 comments

Persistent State Machine: A Formal Computational Paradigm for High-Sparsity LLM Attention Acceleration [Version 6.0] | Zenodo

Skip to main

You are using an outdated browser. Please upgrade your browser to improve your experience.

We experienced an issue with file uploads on Zenodo between 28 July 23:00 UTC and 29 July 08:15 UTC. The issue has now been resolved, and you can retry any uploads that did not go through. We apologize for the inconvenience.

Published July 29, 2026

| Version v6

Preprint

Open

Persistent State Machine: A Formal Computational Paradigm for High-Sparsity LLM Attention Acceleration [Version 6.0]

Authors/Creators

Esaka, Yusuke1

Show affiliations

1.

Cosmos Administrative Scrivener Office & Independent Researcher

Description

Persistent State Machine: A Formal Computational Paradigm for High-Sparsity LLM Attention Acceleration (Version 3.3)

ABSTRACT:

The von Neumann memory wall—the bandwidth and energy gap between computation and data storage—has become the dominant bottleneck of Large Language Model (LLM) inference. In autoregressive decoding, every generated token requires re-streaming the entire Key-Value (KV) cache across DRAM, consuming energy four to five orders of magnitude above the arithmetic cost.

This paper introduces the Persistent State Machine (PSM), a formal computational paradigm where computation is broadcast as instructions to stationary in-memory cells that evaluate state transitions locally. We define PSM as a mathematical 7-tuple, prove its representation equivalence and exponential reduction in explicit lookup-table implementation complexity over Deterministic Finite Automata (DFA), and establish its formal equivalence to Deterministic Linear Bounded Automata (DLBA) under linear bounded memory constraint N = O(n), characterizing the deterministic space complexity class DSPACE(O(n)).

We present the Active State-machine Memory Architecture (ASMA), a proposed silicon architecture implementing PSM for LLM KV-cache attention. Under the analytical model and operating assumptions described in Section 4 (INT4 precision, N=4,096 sequence length basis), ASMA is projected to reduce system bus traffic by up to 99.47% and net step energy by 99.0% (Horowitz 45 nm CMOS energy model basis) against GPU baselines.

Note: All quantitative results in this paper are derived from mathematical proofs, arithmetic calculations, analytical energy models (Horowitz 2014), and Python software simulations. No physical silicon fabrication, FPGA synthesis, or gate-level timing analysis has been performed at the time of this publication. Synthesizable Verilog-2001 RTL source code is provided as a reference design.

Japanese Patent Application No. 2026-177318 (Patent Pending).

Files

024_Zenodo_ASMA_Paper_v6_0_FullProofMaster.pdf

Files<br>(85.7 kB)

Name<br>Size

Download all

024_Zenodo_ASMA_Paper_v6_0_FullProofMaster.pdf

md5:9716415934c789f04a11eecbc393b8ef

85.7 kB

Preview

Download

782

Views

32

Downloads

Show more details

All versions<br>This version

Views

Total views

782

Downloads

Total downloads

32

Data volume

Total data volume

1.1 MB<br>0 Bytes

More info on how stats are collected....

Versions

External resources

Indexed in

OpenAIRE

Communities

Details

DOI

DOI Badge

DOI

10.5281/zenodo.21679919

Markdown

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21679919.svg)](https://doi.org/10.5281/zenodo.21679919)

reStructuredText

.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.21679919.svg<br>:target: https://doi.org/10.5281/zenodo.21679919

HTML

Image URL

https://zenodo.org/badge/DOI/10.5281/zenodo.21679919.svg

Target URL

https://doi.org/10.5281/zenodo.21679919

Resource type<br>Preprint

Publisher<br>Zenodo

Rights

License

Creative Commons Attribution 4.0 International

The Creative Commons Attribution license allows re-distribution and re-use of a licensed work on the condition that the creator is appropriately credited.

Read more

Citation

Export

Technical metadata

Created

July 29, 2026

Modified

July 29, 2026

Jump up

This site uses cookies. Find out more on how we use cookies

Accept all cookies<br>Accept only essential cookies

zenodo state https attention persistent machine

Related Articles