Power Foam – unified 3DGS representation for both rasterization and ray-tracing

Power Foam

Abstract

Method

Quantitative

Qualitative

Fisheye

Ray tracing

BibTeX

Unifying Real-Time Differentiable Ray Tracing and Rasterization

Shrisudhan Govindarajan*1,4,

Daniel Rebain*2,

Dor Verbin3,

Kwang Moo Yi2,

Anish Prabhu4,

Andrea Tagliasacchi1,5

1Simon Fraser University 2University of British Columbia 3Google DeepMind 4Google 5University of Toronto

*Equal contribution

arXiv

Paper

Code

Your browser does not support the video tag.

We unify real-time differentiable ray tracing and rasterization in a single scene representation.

Abstract

We introduce a differentiable 3D representation that unifies the ray tracing capabilities of foam-based ray tracing with the efficiency of modern rasterization pipelines. While prior foam representations enable constant-time ray traversal through an explicit volumetric partition of space, their potentially unbounded cells hinder efficient tile-based rasterization. We address this limitation by generalizing Voronoi foams to bounded power diagrams with controllable cell extents, enabling spatially bounded primitives without requiring expensive Delaunay triangulations during training. We further introduce an oriented surface formulation that explicitly models interfaces between interior and exterior regions, and decouple geometry from appearance by embedding differentiable texture directly on these surfaces. Together, these contributions yield a representation that preserves state-of-the-art ray tracing efficiency while achieving rasterization performance competitive with current generation 3DGS, providing a practical path toward unified real-time differentiable rendering.

Method

Our recipe is to employ Voronoi at every scale: a bounded power diagram gives us 3D geometry made of cells with controllable extent, a 2D Voronoi on each cell provides texture and displacement, and a spherical Voronoi on each texture site captures directional radiance.

Bounded power diagram

Voronoi texture

Spherical Voronoi

Our goal is to construct a representation that can be both rasterized and ray traced, but while foam structures are natively amenable to ray tracing, efficient rasterization requires bounded primitives that an unbounded foam lacks. Without such bounds, testing a cell's intersection with image tiles demands an unwieldy projected convex hull in screen space and often spans large regions where the cell is fully occluded. The simplest remedy is to restrict each Voronoi foam cell to its intersection with a rasterization-friendly bounding primitive such as a sphere — a structure already provided by computational geometry as the weighted α-complex, or more specifically its dual, which we refer to as the bounded power diagram. As illustrated below, the Voronoi diagram (left) builds cell faces from planes equidistant to the cell sites and the power diagram (center) builds them from per-cell radii; using those radii as bounding spheres (right) then ensures that all cell boundaries have gradients with respect to every cell parameter.

Voronoi diagram

Power diagram

Bounded power diagram

Interactive Drag the red sites to move them, or drag any circle edge to resize it.

Reset

Ray traversal additionally requires an adjacency graph between neighbouring cells: Radiant Foam obtained this from the Delaunay triangulation of its sites (left), and an unbounded power diagram would analogously require a regular triangulation (center). The bounded power diagram, in contrast, needs only its α-complex (right, blue), which drops edges between non-overlapping spheres and is therefore cheaper to build. We can simplify construction even further by replacing it with the &Ccaron;ech complex — the graph of all pairwise-overlapping spheres — which is a strict superset of the α-complex (right, blue + green). This approximation costs only a small amount of rendering speed while leaving the final output exactly correct.

Delaunay triangulation

Regular triangulation

α-complex (blue) & &Ccaron;ech complex (blue + green)

Interactive Drag the red sites to move them, or drag any circle edge to resize it.

Reset

Finally, for our decoupled geometry and appearance framework, the dipole face acts as a proxy for macro-scale geometry, while detail sites \( s_i \) are optimized to capture high-frequency geometric and appearance details without increasing the primitive count. As illustrated below by zooming into a leaf-tip cell of the Garden scene, displacement values \( d_i \) associated with each detail site push the surface up or down locally along the axis of the dipole, and our soft Voronoi formulation distributes both these displacements and the directional radiance \( c_i \) of each detail site across the dipole plane (shown top-down, with displacement, and side, left to right).

overlays (used in place of foreignObject for Safari compatibility) sit at exactly the right coordinates. -->

that scales by HEATMAP_SCALE around the marker centre (287+30.5,...

Power Foam – unified 3DGS representation for both rasterization and ray-tracing

Related Articles

(no title)

Scientists reverse brain aging, with a nasal spray

NHS apologises and admits Palantir have access to identifiable patient data

AI has torched the market for junior programmers

Al Vigier: Canada's AI strategy shouldn't include secret Palantir bills