isopolis — behind the scenes

0 · why<br>1a · rendering the city<br>1b · ground truth, nano banana<br>1c · the training set<br>2a · a LoRA TL;DR<br>2b · Modal<br>2c · inference<br>2d · the overlap plan<br>3a · manual checking<br>3b · regenerations<br>3c · the water stamp<br>3d · the tile pyramid<br>4a · the app<br>5 · what's next<br>7 · reading & credits

0Why [Why not!?]

Isopolis is by far my most ambitious side-project to date. Throughout this report, you'll see how I would have either not attempted this project, or abandoned it at least 10 times without coding agents. If it feel like I am gushing over them, it is because I am. I'm stoked for the golden era of side-projects!

Isopolis is a continuous, zoomable, clickable<br>isometric pixel-art map of SF. All ~121 km² of it which works out to roughly a<br>22-gigapixel image. A few months ago, I saw<br>isometric.nyc<br>and was immediately sold on making it for SF. Also, I wanted to recreate the opening credits scene of Silicon Valley show, which was as hilarious as it was accurate. I started with a simple plan to just replicate the NY pipeline, for SF. Poor, naive me.

“We choose to do these things not because they are easy, but because we<br>thought they were going to be easy.”

The basic idea is simple: we can train a tiny LORA on a few ground truth pairs of 3D map render and ghiblified image, then run the model over the whole city. Each part of the pipeline, was supposed to be easy.

Steps:

Get the 3D google tiles of SF

Generate a few ground truth pairs of 3D render and ghiblified image via some SOTA image model

Train a LORA on those pairs

Run the model over the whole city (with some inference strategy to ensure continuous style)

Make maptiles out of the generated images

Build a snazzy app around it

... PROFIT???

alcatraz, before & after · drag the divider

<>

input · 3D tiles render

output · pixel art

Alcatraz aka The Rock. Left of the divider is Google's photorealistic mesh<br>rendered by my "camera"; right is the final map after multiple rounds of inference, color and water correction.

civic center

palace of fine arts

southeast hills

1Data

1aRendering the city

The source is Google Photorealistic 3D Tiles . Isometric.nyc explored and rejected the use of 3d building data. It is pretty insane that US gov has free LIDAR data for every city in the US available to the public. I spent All map math happens in a single fixed local ENU tangent<br>plane . The camera is orthographic, azimuth 22.5°, elevation<br>30°. I chose it by eye from comparison renders after a few trials. This was all vibes based, no real science here. The intended effect was a classic Sim City view of the city. The Ground point (e, n, h) → map pixel math works out roughly so:

u = (e·cos&thinsp;α − n·sin&thinsp;α)/s,<br>v = −[(e·sin&thinsp;α + n·cos&thinsp;α)·sin&thinsp;φ + h·cos&thinsp;φ]/s

top view · azimuth 22.5°

view up · 22.5°

both street grids keep a diamond look

side view · elevation 30° → 2:1 dimetric

ground

30°<br>camera

1 m of ground

→ sin 30° = 0.5 px on screen

One camera for the whole city.

The capture process was basically Playwright driving the three.js page headless, a worker pool rendering blocks in parallel. The tiles' textures have baked lighting. Shading them again would<br>double-light the city. I learnt this the hard way. There are errors that build up with the single camera over the whole city, but they are negligible at our city scale. In the end, we ended up with 440 blocks of 1024² renders.

Roadblocks: Speed, errors.

This sort of scraping is actually against Google's TOS. So the process was painfully slow and error-prone. The workers often got rate-limited. But Claude Code prevailed! I had a loop set up with the error rate and the render time. It was able to go from ~3 mins to ~20 seconds per render.

a raw orthographic render (r_4_-6), straight out of the capture worker

1bGround truth, with nano banana

Now that we have the 3D tiles, we need to generate the ground truth pairs for training! Similar to isometric.nyc, I generated a few ghibli-style pixel-art images using Google's Nano Banana. SF terrain is very interesting. There are quite a few distinct features like skyscrapers in FiDi, the hills in the southeast, 2 iconic bridges, lots of coastline, piers, parks, suburban grids and lots of water. I generated a ton of images and curated from them. Getting consistent style was a challenge. There was a LOT of manual trial and error. But as usual, Claude Code added this feature to the dev app that allowed me to select the best images and approve them.

Roadblocks: Style consistency

Getting consistent style was a huge challenge. I spent a ton of time generating 8 reference images using the Pro model. Once I was satisfied with the style, I froze those 8 images and used them as references for all subsequent generations via a smaller model.

I created one training set with ~60 image pairs, and another with ~100. These were the basis for the two training runs we will describe in the next section. The total cost...