Handsum: An LQIP Image File Format | Nigel Tao

Nigel Tao

Handsum: An LQIP Image File Format

Low Quality Image Placeholders (LQIPs) are very small (both in terms of file<br>size and pixel dimensions) images that load very quickly, providing immediate<br>visual feedback while full-resolution images load slowly in the background.<br>They are small-in-file-size enough that it’s feasible to base64-encode and<br>inline them into a web page, minimizing network round-trips and improving page<br>load times.

Existing LQIP techniques include the Blurhash and<br>Thumbhash custom image file formats, as<br>well as just using low-quality JPEG or WebP and very small pixel dimensions.<br>This blog post introduces Handsum, another LQIP custom image file format.

Handsum files are also a fixed size-in-bytes: 48 bytes at the lowest quality<br>setting and 147 bytes at the highest. For a desktop, native or command-line<br>app, if you’re storing your thumbnail images in a database, this lets you use<br>fixed-size columns, which can be faster and simpler to iterate<br>over.

Handsum is based on the Discrete Cosine Transform (DCT), also used by the<br>Blurhash<br>algorithm, the<br>Thumbhash algorithm and JPEG<br>itself. Understanding how the relatively simple Handsum format works should<br>help you if you ever wanted to understand how the relatively complicated JPEG<br>format works (the JPEG<br>specification is a 186-page PDF<br>file; Handsum is simpler).

Web Page Example

Here’s an annotated screenshot of a Handsum web page<br>demo. It uses a Handsum decoder written in<br>C,<br>compiled to Wasm.

Comparison

Here’s Handsum compared to PNG, Thumbhash (which is pretty similar to<br>Blurhash), WebP, ETC2 and JPEG. You might want to open this image in its own<br>tab, zoom in and pan around.

There’s a lot going on here. Let’s break it down.

Each row concerns a famous image, as seen on Wikipedia:<br>earthrise,<br>la-grande-jatte,<br>lincoln,<br>mona-lisa,<br>parliament,<br>pearl-earring,<br>starry-night,<br>tsunami,<br>van-eyck,<br>water-lillies.

The first column, “Original”, is the image itself, scaled to fit inside a 32×32<br>pixel bounding box (while preserving the aspect ratio). For example, the<br>500×745 pixel Mona Lisa image is resized to 21×32.

Every other column is that original image after round tripping: encoding in<br>some lossy fashion, then decoding.

The numbers (like 1368 or 615) above each cell is the encoded size-in-bytes.<br>More bytes are clearly correlated with more quality. The question is how small<br>(in terms of size-in-bytes) can you get while keeping acceptable quality.

The second column, “PNG 16”, simply scales the image down to fit inside a 16×16<br>pixel bounding box, encoding it losslessly as a PNG. (Yeah, we could possibly<br>get a slightly smaller size-in-bytes if we pngcrushed it, or used WebP<br>lossless, but it’s probably not that big a difference). Decoding produces a<br>within-16×16 image (e.g. a 11×16 Mona Lisa) which we then upsample (with a<br>bi-linear filter) back to 32 pixels wide or high.

The third column uses the Thumbhash LQIP<br>codec. This produces extremely small encodings, weighing only 24–27 bytes.

The next four columns use Handsum, at each of its four quality settings. The<br>“16” in “Hsum 16 q=1” means that, like “PNG 16”, the codec per se produces<br>something that fits inside a 16×16 bounding box, so we upsample to 32 to get an<br>image that’s better compared with the other cells.

The four columns after that use WebP lossy, based on VP8, at quality 0, 25<br>and 75. The “WebP 16” columns encode (and thus decode) within-16×16 (and,<br>again, we upsample). The “WebP 32” column encodes within-32×32 image (and no<br>upsampling is needed on decode).

The next two columns use ETC2 (wrapped in a PKM / PACKMAN container), the<br>Ericsson Texture Compression format, mandatory in OpenGL ES<br>3.0<br>(Appendix C.1 of the linked spec) and similar to other texture formats like BCn<br>/ DXTn / S3TC. ETC2 is designed to be decoded on GPUs (with high<br>parallelization), but you can decode on CPUs too, like any other image format.<br>ETC2 is an interesting and under-appreciated image format, but diving into that<br>is out of scope of this blog post. Again, “16” means encoding the<br>lower-resolution source image and upsampling after decode, “32” means encoding<br>the higher-resolution source image.

The last two columns are JPEG, at quality 75 (which is roughly comparable to,<br>but not exactly the same as, WebP’s “quality = 75”). Again, “16” vs “32” means<br>lower-res-and-upsample vs higher-res.

At Handsum’s q=1 (lowest quality), it’s roughly comparable to a bigger (higher<br>file size), better (better visual quality) Thumbhash. Bigger is in relative<br>terms. The absolute difference is barely 24 bytes. By “better visual quality” I<br>mean that, if you told me to guess the famous image based on the thumbnail,<br>then for Mona Lisa or Girl with a Pearl Earring, I’d say that that’s<br>plausible for Handsum (q=1) but implausible for Thumbhash. Still better than<br>the modern potato art that WebP 16 q=0 produces, though.

At Handsum’s q=4 (highest quality), it’s roughly comparable to WebP...