DiffGI: Differentiable Geometry Images for High-Fidelity Thin-Shell 3D Generation

👕" />

Your browser does not support the video tag.

DiffGI generates high-fidelity, thin-shell 3D geometry (such as garments) end-to-end from a<br>single image or pattern — in seconds, on a CPU.

Main Contributions

01

Thin-Shell & Non-Manifold 3D

Leverages multi-chart geometry images to effectively learn thin-shell, non-manifold surfaces —<br>geometry that watertight volumetric models cannot represent.

02

High Fidelity, End-to-End

Continuous TSDF + differentiable marching squares backpropagate 3D surface losses end-to-end,<br>yielding subpixel-accurate boundaries.

03

Fast & Lightweight

Diffusion over a compact 32×32 latent generates 3D in ~1.2 s on a consumer GPU — down to<br>CPU-only devices.

DiffGI Pipeline

An input 3D mesh is mapped to a 2D TSDF geometry image and compressed by the DiffGI-VAE into a<br>compact 32×32 latent. The decoder reconstructs the TSDF map, and a Differentiable Marching Squares (DMS)<br>module extracts the 3D surface from it. Because DMS is differentiable, pixel-space losses on the TSDF and<br>position maps — together with a geometry-aware normal rendering loss — propagate end-to-end, from the<br>rendered 3D surface all the way back to the 2D latent. A transformer-based latent diffusion model is then<br>trained on this latent space for conditional generation.

The Effect of Our Representation

DiffGI (ours)

Conventional GI

The same garment recovered from our DiffGI — a continuous, differentiable TSDF geometry image<br>(left) — versus a conventional geometry image built on a binary occupancy map (right). DiffGI<br>keeps thin shells and boundaries sharp and intact, while the occupancy-based representation leaves them<br>torn and aliased.

GT chart boundaries in UV space, rasterized at 256×256 : binary occupancy snaps to the pixel<br>grid, while our TSDF follows the dashed GT boundary with subpixel accuracy .

Results

Single-View Image-to-3D

From a single front-view image to a complete 3D garment. Against TRELLIS, TRELLIS.2, and GarmageNet,<br>DiffGI recovers cleaner boundaries with far more compact meshes (≈23K vertices on average).

Additional image-to-3D results, shown with the corresponding 2D DiffGI tensors.

Image-to-3D on GarmageSet — best geometric accuracy with an order of magnitude fewer vertices.

MethodAvg. VerticesCD↓MD (F1)↑dH↓BCD↓

TRELLIS109K3.440.2815.38N/A<br>TRELLIS.2380K11.010.2769.7012.44<br>GarmageNet526K4.310.2023.765.64<br>Ours (DiffGI)23K1.350.488.422.91

Reconstruction Fidelity (DiffGI-VAE)

VAE reconstructions on GarmageSet and ABO. DiffGI-VAE produces sharper boundaries and better thin-shell<br>preservation than Omages and GarmageNet.

Reconstruction & encoding fidelity — best scores with the most compact representation.

MethodRep. Size<br>ABO CD↓ABO EMD↓ABO JSD↓ABO NC↑<br>Garmage CD↓Garmage EMD↓Garmage JSD↓Garmage NC↑

Omages64×64×40.890.250.920.891.310.171.790.95<br>GarmageNetN×72————2.190.2132.610.94<br>Ours32×32×40.830.230.890.830.460.161.240.96

Label-Conditioned Generation (ABO)

DiffGI generates thin frames and open-boundary furniture with fewer staircase artifacts than Omages.

Latent Space Interpolation

Smooth interpolations across furniture categories reveal a well-structured, continuous DiffGI latent manifold.

Peak memory and inference latency — DiffGI scales from servers down to CPU-only edge devices.

MethodHardwarePeak VRAM (GB)↓Time (s)↓

TRELLIS-imageRTX A6000 Ada16.284.52<br>OmagesRTX A6000 Ada2.4952.0<br>Ours-ImageRTX A6000 Ada3.220.80<br>Ours-ImageRTX 4070 (12GB)3.221.21<br>Ours-ImageMacBook M4 (CPU)—8.52

Acknowledgements

We thank Hyun Kang, Seungoh Han, Sihun Cha, and Dong-sig Kang for valuable discussions and feedback<br>throughout this work. We are also grateful to CLO Virtual Fashion for providing the research<br>environment and resources that made this work possible.

BibTeX

@inproceedings{shim2026diffgi,<br>title = {DiffGI: Differentiable Geometry Images for High-Fidelity Thin-Shell 3D Generation},<br>author = {Shim, Eungjune and Lee, Hansol and Ju, Eunjung},<br>booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},<br>year = {2026}