Uncovering Neural Geometry in Vision Models With Block-Sparse Featurizers

Research

The Neural Geometry Series

Uncovering Neural Geometry in Vision Models With Block-Sparse Featurizers

We introduce Block-Sparse Featurizers (BSF), a family of methods to<br>decompose a model's activations into multidimensional subspaces rather<br>than single directions. Applied to vision models, we find that BSFs<br>find interpretable, multidimensional features which offer a more<br>parsimonious explanation of model internals; that those features enable<br>fine-grained steering; and that most concepts in the models are<br>multidimensional.

Authors

Thomas Fel*,1

Matthew Kowal*,1

Mozes Jacobs*,1,2

Dron Hazra*,1

Usha Bhalla*,1

Lee Sharkey1

Lucius Bushnaq1

Satchel Grant1

Tal Haklay1

Thomas Icard1,3

Can Rager1

Michael Pearce1

Daniel Wurgaft1,3

Aiden Swann1,3

Fenil Doshi1,2

Siddharth Boppana1

Curt Tigges1

Nick Cammarata1

Thomas Serre4

Vasudev Shyam1

Owen Lewis1

Thomas McGrath1

Jack Merullo†,1

Ekdeep Singh Lubana†,1

Atticus Geiger†,1

* Equal contribution

† Equal senior contribution

1 Goodfire

2 Harvard University

3 Stanford University

4 Brown University

Published

July 7, 2026

Full Paper

Read on arXiv →

Code

GitHub →

Blog post by Thomas Fel, Matthew Kowal, and Michael Byun

Picture a tree.

Now make it small, a sapling; then tall, an oak; then enormous, a redwood;<br>then tiny again, a bonsai. Turn its leaves summer green, then autumn red,<br>then strip them bare. Change its species: birch, pine, willow. Each one is<br>still a tree, but they differ along many axes of variation. The concept<br>"tree" is not a one-dimensional idea, but a multidimensional space of<br>variation.Even before the emergence of LLMs, many have argued we should think of concepts as regions in structured geometric spaces, e.g. Peter Gärdenfors in Conceptual Spaces: The Geometry of Thought.

Trees are just one example here; our world is full of intricate structure.<br>So are the internals of neural networks! But the most popular methods we use to understand what's<br>happening inside neural networks, like sparse autoencoders (SAEs) and<br>transcoders, treat each internal concept as a single straight line – an<br>assumption which is surprisingly powerful, but which nonetheless doesn't<br>effectively capture the naturally curved, multidimensional structures we<br>know exist in model representations.

In this post, we explain our new paper which introduces a family of<br>techniques – Block-Sparse Featurizers (BSF) – that we think provide a simple,<br>viable alternative. BSFs decompose model activations into subspaces, which<br>empirically contain multidimensional shapes.

We believe this is a significant step forward in understanding neural networks on<br>their own terms. Applied to vision models, we find that BSFs find<br>interpretable, multidimensional features which better explain model<br>activations; that those features enable fine-grained steering; and that<br>most concepts in the models are multidimensional.

Contents

Introduction: A toy model of manifolds in superposition

As part of interpretability's broader goal of understanding what goes on<br>inside neural networks, we want to know what a model is "thinking" at any<br>given moment. The primary way we do thisBut not the only way! E.g., see our work on decomposing weights. is by reading the activations<br>flowing through the model and mapping them to meaningful concepts. In their<br>raw form, though, those activations are just large vectors with no<br>convenient labels. So how do we disentangle them into meaningful patterns?

That question can be reframed as a different one: how are a model's<br>representations structured? If we know that, then we can choose the best<br>method to disentangle them.

If you've been following this series of posts, you can probably guess where<br>this is going. We've discussed the growing body of research, from Goodfire<br>as well as many others, showing that concepts are often represented inside<br>neural networks as manifolds – multidimensional, curved shapes. We've also<br>discussed the mismatch between these complex representations and the tools<br>we use to find them: the most popular methods, like SAEs and transcoders,<br>look only for straight lines.This straight-line assumption has gotten SAEs quite far! We want the simplest possible building blocks for what a network computes, and a straight line is about as simple as a shape can be. Straight lines are also easy to work with computationally. But we think it's a fundamentally limiting assumption, if we want to fully understand model activations.

To get an intuition for the consequences of this mismatch, consider the toy<br>model above. We constructed clean artificial activations by summing samples<br>from predefined manifolds – implicitly stacking the manifolds in<br>superposition. This forms a simplified model of how we think real neural<br>network activations are structured. Then we use two different featurizer<br>methods to try to recover the known underlying structure from the<br>activations.

On...