Sentinel-2 Paint: Recreate Any Image from Real Satellite Imagery – GeoSpatial ML

What colors does the Earth actually come in? We built Sentinel-2 Paint to find out: a script that scours the Sentinel-2 archive for patches of every color the planet shows, and a browser app for playing with the results. Upload a photo and the app rebuilds it as a mosaic of real satellite imagery, nearest color per cell; pick a color and it shows the chip of Earth whose average color is closest, then flies to the exact scene and lat/lon where Sentinel-2 observed it. The idea is in the same spirit as NASA’s fun Your Name in Landsat tool, which spells out your name using Earth features shaped like letters — we do the same trick with color instead of shape. The palette behind the live app comes from 368 scenes and contains 92,916 unique Earth colors so far — about 0.55% of the 16.7 million colors in the 8-bit RGB cube. All the code is at github.com/calebrob6/sentinel2-paint.

A picture of one of the Sentinel-2 satellites, recreated by Sentinel-2 Paint (i.e. rebuilt entirely from 32×32 patches of its own imagery!)

How it works

We made a harvesting pipeline that runs against the Microsoft Planetary Computer in five steps:

Sample scenes. We start with a curated list of color-rich places and seasons (salt-evaporation ponds, tulip fields, glaciers, coral lagoons, autumn foliage, geothermal pools, red mine tailings) then do endless random sampling over continental land for the long tail.

Download. Each scene’s 8-bit RGB visual asset as a Cloud Optimized GeoTIFF.

Grid. Non-overlapping 32×32 patches, ~117k per full 10980×10980 tile.

Reduce. Each patch becomes its mean (R, G, B); we keep one thumbnail and geolocation per unique color and drop off-swath and nodata patches.

Pack. Sprite-atlas JPEGs plus a compact JSON index (color, atlas coordinates, lon/lat, source scene) that a static web page can load.

The palette store is append-only, so a harvest can run for hours and resume at any time — a ~10 minute harvest.py run already gives a usable starter palette, and longer runs fill out the rare colors. The full ~93k-color palette is only ~285 MB before packing.1

Sixty-four of the most vivid patches in the palette, sorted by hue, each from a different Sentinel-2 scene: algal blooms in Lake Natron, sulfur springs in the Danakil Depression, lithium evaporation ponds in the Atacama, melt ponds on the Greenland ice sheet — and, supplying most of the saturated blues and purples on land, warehouse roofs in China.

The app

The webapp is a single static HTML file with a shared pigment picker driving two tabs. The picker only offers colors Sentinel-2 actually contains: regions of the gamut with no imagery are dimmed,2 and any pick snaps to the nearest real Earth patch with the snap distance shown — or, if you are lucky, “exact match — this color exists on Earth”. Browsing the picker is a quick way to see the Earth’s gamut, and how much of the RGB cube simply never shows up at 10 m resolution.

Canvas tab. Upload any image (or click a built-in example) and the app recreates it from the palette, nearest color per cell. A Detail slider sets the mosaic resolution (24 to 400 cells on the long side), and a Random slider swaps each cell, at random, for one of the patches within a bounded ΔRGB of the best match, which breaks up the repeated chips you would otherwise get in flat regions like skies (Re-roll reshuffles those random choices without re-uploading). Hovering a tile enlarges it and flashes its origin on the globe, clicking a tile flies there, and Save PNG re-renders every tile at its native 32×32 resolution (up to a ~16,000 px output).

Map tab. A Leaflet globe over Esri World Imagery with an ambient field of ~1,500 palette dots sampled across the planet. Clicking a dot grabs its color, and picking any color flies to the exact place it was observed and pins it.

The canvas tab: the NASA logo rebuilt entirely from real Sentinel-2 patches, nearest color per cell. Hovering a tile traces it back to its source scene and coordinates.

The map tab: a Leaflet globe showing where each palette color was observed. Picking a color flies to the exact place Sentinel-2 saw it.

Everything runs client side: nearest-color lookup is fast enough to run per-cell in the browser,3 the mosaic builds in chunks with a progress bar, and the sprite atlases stream in the background while color matching works immediately. There is no backend — the whole thing is sprite atlases and one JSON file on a static web server, in the same spirit as TerraBit and DeltaBit.

Links

Live demo: calebrob.com/sentinel2-paint: rebuild a photo out of Sentinel-2 patches.

Code: github.com/calebrob6/sentinel2-paint: harvesting pipeline, palette store, and the webapp (MIT).

Data: Sentinel-2 L2A on the Microsoft Planetary Computer: the imagery archive behind the palette.

Footnotes

Each unique color costs exactly 3,072 bytes on disk (a raw 32×32×3 thumbnail in patches.bin) plus one JSON...