A Tree and a Server Walk Into a Core… | Charlie Holland's Blog

Skip to main content<br>A Tree and a Server Walk Into a Core&#x2026;<br>A Tree and a Server Walk Into a Core&#x2026;<br>July 10, 2026

© 2026 Charlie Holland

A Tree and a Server Walk Into a Core…

A Tree and a Server Walk Into a Core…

Table of Contents

1. TLDR   tldr

2. About   emacs carnival programming

3. The Combinatorial Woes of M×N   leverage architecture

4. Tree-sitter: a Fresh Parse Tree on Every Keystroke   treesitter parsing

4.1. Looking at Concrete Trees

4.1.1. Fibona-tree

4.1.2. Scaling Up: quicksort

4.1.3. Zooming Out: a Whole Module

4.2. What Emacs Does with the Tree   fontlock navigation

5. LSP: Tapping a Compiler's Brain   lsp architecture

5.1. Eglot: LSP, the Emacs way   eglot builtins

6. The Road Into Core   history upstream

7. Try It   handson

1. TLDR   tldr

Two technologies have endowed every editor with IDE superpowers this past decade: tree-sitter , a generic incremental parser that gives your editor a live syntax tree of your code, and the Language Server Protocol (LSP) , a generic client-server protocol that gives your editor code intelligence features like linting, type-checking, auto-completion, and code navigation. Each one replaces an M×N fan-out of per-language, per-editor hacks with a single, language- and editor-agnostic protocol. Emacs has merged support for both into its core, and shipped them in Emacs 29.1 (three years ago this month!). This post explains what each one does, visualizes a real-life tree-sitter parse tree, and recounts how both technologies made their way upstream into the glorious Emacs core.

An interactive visualization of the syntax tree tree-sitter builds from a small Python module — hover any node for details, click to zoom into a subtree. Two more of these live further down the post.

2. About   emacs carnival programming

This is my entry for July's Emacs Carnival, hosted by Andy over at Plain DrOps. The theme is Programming , i.e. Emacs as a programming environment. I've done these before (May was about deep Emacs patterns, June about Emacs teaching you Emacs), and this month I want to push back on my least favorite meme: that Emacs is arcane; rusty and dusty; old news; the "E ditor for M iddle A ged C omputer S cientists".

Emacs has been under active development by a growing number of contributors, and is surprisingly modern. As two language-agnostic protocols for IDE features arose (tree-sitter for syntax and LSP for semantics), Emacs developers quickly merged support for both into core. More interestingly, they made them usable in the editor by integrating them into the Emacs machinery that predates them by decades.

The first suggested topic in the carnival is "Language Specific Setups" and, given the utility of tree-sitter and LSP, I couldn't help but misinterpret this suggestion intentionally and create a post on my "Language Generic Setup".

3. The Combinatorial Woes of M×N   leverage architecture

Traditionally, in most IDEs "support for language X" meant one implementation per editor, per language.

Syntax highlighting was a mixed bag of regular expressions that approximated the language's grammar.

Navigation ("jump to the function I'm in") was basically regex guesswork. Completion and go-to-definition, if they were even available, came from language-specific plugins of wildly varying quality.

With M editors and N languages, the world had to write (and maintain, and debug) M×N of these plugins. The modern fix is the classic programmer's reflex of establishing a generic layer, or protocol, in the middle, and paying a much more affordable M+N tax instead (the framing the LSP community itself uses).

This reflex is older than any editor in the diagram below, by the way. It's the UNCOL argument from 1958, which suggested putting one universal intermediate language between M source languages and N machines, and M×N compilers become M+N. UNCOL itself never shipped, but its argument became the standard justification for compiler intermediate representations, and the argument is just as valid for editor tooling.

graph TB<br>subgraph before["Before: every editor reimplements every language (M × N)"]<br>direction LR<br>A1[Emacs] --- B1[Python]<br>A1 --- B2[Rust]<br>A1 --- B3[TypeScript]<br>A2[Vim] --- B1<br>A2 --- B2<br>A2 --- B3<br>A3[VS Code] --- B1<br>A3 --- B2<br>A3 --- B3<br>end<br>subgraph after["After: one generic layer in the middle (M + N)"]<br>direction LR<br>C1[Emacs] --> MID(("tree-sitter+ LSP"))<br>C2[Vim] --> MID<br>C3[VS Code] --> MID<br>MID --> D1[Python grammar & server]<br>MID --> D2[Rust grammar & server]<br>MID --> D3[TypeScript grammar & server]<br>end<br>before ~~~ after

Tree-sitter provides the answer to "what is this text, structurally?" — it can tell the editor that characters 4 through 7 are a function name. LSP provides the answer to "what does this text mean, in this project?" — it knows that the function is defined in another file, is called from twelve places, and is missing an argument, etc…. These two...