Three years of writing a browser engine, in 50 posts

jaytaph · CodeMusings

Fifty parts of Gosub Browser Progress

Part 50 went up this week, which felt like a good moment to look back at the whole series. Three years, from "I don't know Rust and I don't know how browsers work" to a modular engine with a multi-process proof-of-concept.

Here is the whole thing, one line per part. It reads a bit like a changelog of the project, and a bit like a changelog of me learning Rust.

It started here: The one where he is writing a browser... - the announcement, including the tiny issue that I didn't know Rust yet.

Tokenizer (pt1-pt9)

pt1 - First days with Rust, and how much less explicit it is than Go.

pt2 - Fighting Rust over how to structure the thing. Unclear who won.

pt3 - Named character references, finally done. They took far too long.

pt4 - Character entities complete, so whole strings tokenize correctly.

pt5 - More data states filled in, plus generated testcases to check them.

pt6 - A test harness that runs the tokenizer against the html5 testsuite.

pt7 - Over 5000 of 6000 token tests passing, with utf-8 as the sticking point.

pt8 - The tokenizer passes all html5lib-tests and lands in main.

pt9 - Tokenizer done, so on to tree construction.

Tree construction and the borrow checker (pt10-pt23)

pt10 - Starting to get the hang of Rust. The borrow checker yells less.

pt11 - Document tree generation runs straight into ownership problems.

pt12 - An arena holding all nodes, referenced by node_id.

pt13 - Still wrestling the borrow checker, with an LLM as a second pair of eyes.

pt14 - First attempts at the adoption agency algorithm.

pt15 - Something snapped and the borrow checker finally made sense.

pt16 - A blog post on why this project exists, which blew up on Hacker News.

pt17 - Still stuck on test 23 of the tree-construction tests.

pt18 - Contributions start coming in, and my long-lived branch gets out of sync.

pt19 - Test 23 defeated. The adoption agency works.

pt20 - Filling in more tests, and struggling with Rust while doing it.

pt21 - All 112 tests in test1.dat pass. Many files to go.

pt22 - Parsing live documents from the actual web.

pt23 - 99.89% of the tree-construction tests pass.

Network stack and CSS (pt24-pt30)

pt24 - Designing the whole flow, and realising how much is still unwritten.

pt25 - The DNS component is finished.

pt26 - A week of yak shaving on the way to a http-to-rendertree pipeline.

pt27 - A working css3 parser, at the cost of the previous one.

pt28 - Porting csstree from JavaScript to Rust, which goes better than expected.

pt29 - Real-life CSS parses, at the cost of ignoring the rest of the engine.

pt30 - Tying the html parser and the css3 parser together.

First pixels (pt31-pt43)

pt31 - "dive", a text-based user agent built with ratatui.

pt32 - Tree-walking to get rudimentary text on screen.

pt33 - Turning a CST into stylesheet structures usable for selectors.

pt34 - Slower going, with the CSS system still far from finished.

pt35 - Refactoring to break the cyclic dependency between html5 and css3.

pt36 - The CSS3 and HTML5 crates finally work together.

pt37 - Taking stock of the components, and thinking about event loops and async.

pt38 - The v2 engine proof-of-concept merges into main.

pt39 - We've got a browser: a standalone useragent that loads tabs and renders them.

pt40 - Typography work, after the planning went sideways as usual.

pt41 - Font manager setbacks: the font libraries are not interchangeable.

pt42 - The render pipeline split into separate phases, tiling and painting included.

pt43 - The rendering pipeline PoC turns html and css into actual pixels.

Putting it together (pt44-pt50)

pt44 - Back at it after a quiet stretch, now with AI in the toolbox.

pt45 - The big three merged: network stack, rendering pipeline and engine api.

pt46 - Modularity pays off, with several browsers built on the same engine.

pt47 - A new gosub.io, with a nautical theme.

pt48 - The engine back in a useful state, with ModuleConfig and RenderConfig.

pt49 - A refurbished Mac M1 to build and test on real hardware.

pt50 - Cleanup, and a multi-process proof-of-concept.

On to the next fifty.

#rust

#gosub

Nobody amused yet.

jaytaph

Codemuser extraordinaire

Loves building crazy and insane stuff. Happiest when left alone. All I wanted was a Pepsi, just a Pepsi.

Joined Mar 24, 2023 Following Followers Posts 70 Upvotes

RSS

Three years of writing a browser engine, in 50 posts

Related Articles

(no title)

Scientists reverse brain aging, with a nasal spray

NHS apologises and admits Palantir have access to identifiable patient data

AI has torched the market for junior programmers

Al Vigier: Canada's AI strategy shouldn't include secret Palantir bills