[2605.29901] Dissecting the Black Box: Circuit-Level Analysis of LLM Vulnerability Detection

-->

Computer Science > Cryptography and Security

arXiv:2605.29901 (cs)

[Submitted on 28 May 2026]

Title:Dissecting the Black Box: Circuit-Level Analysis of LLM Vulnerability Detection

Authors:Syafiq Al Atiiq, Chun Zhou, Christian Gehrmann<br>View a PDF of the paper titled Dissecting the Black Box: Circuit-Level Analysis of LLM Vulnerability Detection, by Syafiq Al Atiiq and 2 other authors

View PDF<br>HTML (experimental)

Abstract:Large language models (LLMs) can detect software vulnerabilities, but how do they actually identify vulnerable code? We address this question using mechanistic interpretability; analyzing the internal computations of a neural network to understand its reasoning this http URL Circuit Tracer on Gemma-2-2b, we trace the computational pathways activated when the model classifies 472 C/C++ code samples as vulnerable or safe. Our analysis reveals a surprising finding: the model primarily relies on safety detectors, attention heads that recognize safe coding patterns, rather than directly detecting vulnerability signatures. When these safety detectors fail to activate, the model classifies code as vulnerable. We identify the critical neural components: specific attention heads in early layers (L5, L7) that focus on safety patterns, and Multilayer Perceptron (MLP) neurons in Layer 7 that encode vulnerability-related features. Ablation experiments confirm their causal role; removing Layer 11 drops vulnerability detection accuracy from 100% to 6%, while ablating just 20 neurons in Layer 7 reduces it by 50%.Our findings show that LLM vulnerability detection uses sparse, interpretable circuits (only 16% of model capacity), enabling circuit-level explanations for security predictions and targeted improvements to detection systems.

Comments:<br>11 pages, 6 figures. Supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP)

Subjects:

Cryptography and Security (cs.CR); Machine Learning (cs.LG)

Cite as:<br>arXiv:2605.29901 [cs.CR]

(or<br>arXiv:2605.29901v1 [cs.CR] for this version)

https://doi.org/10.48550/arXiv.2605.29901

Focus to learn more

arXiv-issued DOI via DataCite

Submission history<br>From: Chun Zhou [view email]<br>[v1]<br>Thu, 28 May 2026 13:23:13 UTC (2,123 KB)

Full-text links:<br>Access Paper:

View a PDF of the paper titled Dissecting the Black Box: Circuit-Level Analysis of LLM Vulnerability Detection, by Syafiq Al Atiiq and 2 other authors<br>View PDF<br>HTML (experimental)<br>TeX Source

view license

Current browse context:

cs.CR

next >

new<br>recent<br>| 2026-05

Change to browse by:

cs<br>cs.LG

References & Citations

NASA ADS<br>Google Scholar

Semantic Scholar

export BibTeX citation<br>Loading...

BibTeX formatted citation

&times;

loading...

Data provided by:

Bookmark

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Which authors of this paper are endorsers? |<br>Disable MathJax (What is MathJax?)