A field guide to rendering: from triangles to Gaussians Skip to content<br>Every rendering algorithm ever written answers the same question: given a description of a 3D scene, what color is each pixel on the screen? The scene might be a pile of triangles, a cloud of scanned points, or a neural network trained on photographs, but the output is always the same flat grid of colored dots. What separates one renderer from the next is not the question but the strategy used to answer it, and almost every strategy falls into one of two camps.<br>The first camp works object-order . It loops over the geometry and, for each piece, figures out which pixels that piece covers, then writes color into them. You can think of it as scattering: the renderer throws the scene at the screen and lets each primitive land where it may. Rasterization, the technique behind every real-time game, is the canonical example, and so is drawing a point cloud.<br>The second camp works image-order . It loops over the pixels and, for each one, asks what part of the scene is visible along the line of sight through that pixel. This is gathering: every pixel reaches back into the world and collects the light that arrives at it. Ray tracing, path tracing, ray marching, and NeRF all gather.<br>Neither order is more correct than the other; they trade different costs. Object-order does a bounded amount of work per primitive and maps beautifully onto parallel hardware, but it struggles the moment an effect needs global knowledge of the scene, like a shadow or a reflection. Image-order can answer those global questions directly, because a pixel that has found the first surface can simply cast another ray to ask about the light reaching it, but the cost of finding what each ray hits grows with the complexity of the scene.<br>This guide walks the major families in that frame. Parts 1 and 2 cover the classical renderers, the ones that take an authored scene, a model somebody built, and turn it into an image. Part 3 covers the recent and stranger turn: renderers that take a captured scene, reconstructed from ordinary photographs, and where the line between reconstruction and rendering all but disappears. Each section has a stepped, interactive demo so you can watch the pixels fill in.<br>Part 1: Scattering geometry to the screen<br>Rasterization<br>Rasterization is the workhorse. If you are reading this on a screen that updated smoothly as you scrolled, something rasterized it. The idea is old and direct: take a geometric primitive, usually a triangle, and determine which pixels it covers.<br>The lineage runs through the earliest days of computer graphics. In 1965 Jack Bresenham published an integer-only algorithm for stepping a plotter pen along the pixels closest to an ideal line[01], establishing the incremental, integer-arithmetic spirit that graphics hardware still favors, even though modern GPUs settle triangle coverage a different way, with an edge function per side rather than Bresenham-style edge walking. The other foundational piece arrived in 1974, when Edwin Catmull’s PhD thesis introduced both the z-buffer, a per-pixel depth store that resolves which surface is nearest without sorting the geometry first, and texture mapping, wrapping an image onto a surface[02]. Those two ideas, incremental rasterization and a depth buffer, are still the spine of every GPU.<br>The modern pipeline is a fixed sequence of stages. Vertices enter in model space and are transformed into world space, then into the camera’s view space, then projected onto the 2D image plane; the projection is what makes distant things smaller. Each triangle is then set up for rasterization, where an edge function for each of its three sides tests, for a candidate pixel, which side of the edge the pixel lies on. A pixel inside all three edges is covered. For every covered pixel, or fragment, the renderer runs a depth test against the z-buffer and, if the fragment is nearer than what is already there, runs a shader to compute its color and writes both color and depth.<br>The reason this dominates real-time rendering is the shape of its cost. The work per triangle is bounded and predictable, and every fragment is independent of every other, so the whole process fans out across the thousands of cores on a GPU with almost no coordination. Rasterization does not naturally know about the rest of the scene, which is why shadows, reflections, and indirect light are all bolted on as separate passes and tricks, but for putting a lot of triangles on screen sixty or more times a second, nothing else comes close.<br>Step through the pipeline here, from vertices to shaded fragments:<br>Loading

Voxels<br>Not all object-order geometry is triangles. A voxel representation stores the scene as a 3D grid of cells, the volumetric cousin of the 2D pixel. Instead of describing a surface with its boundary, you fill space with little cubes and mark each one as solid or empty, or give it a density and a color. Minecraft is the famous example, but the...