NVIDIA Axolotl3D Reconstructs Hidden Object Geometry
NVIDIA introduced Axolotl3D, a model that reconstructs complete 3D shapes from partial images and point clouds, offering a breakthrough for robotics simulations and 3D asset editing.
NVIDIA's Spatial Intelligence Lab has unveiled Axolotl3D at the ECCV 2026 conference. This unified 3D shape completion model is designed to reconstruct the hidden, unobserved geometry of objects using incomplete visual data. While traditional image-to-3D generators often struggle with occluded surfaces and inconsistent camera angles, Axolotl3D solves this by simultaneously processing images, visibility masks, camera poses, and partial point clouds to predict missing areas while keeping visible surfaces aligned.
The model builds upon the Hunyuan3D-DiT diffusion transformer architecture, paired with a ShapeVAE decoder. To process diverse inputs, Axolotl3D utilizes DINOv2 to extract semantic and visual features from images, while employing VecSetX to encode the geometric data from point clouds. Feature-pyramid layers then fuse these inputs into multimodal condition tokens. During training on large 3D mesh datasets, the system learns to handle varying numbers of views and missing data patterns, allowing a single checkpoint to perform multiple reconstruction and editing tasks.
In evaluations, Axolotl3D achieved state-of-the-art results on the Toys4K and OmniObject3D datasets under both clean and synthetically occluded conditions. Practitioners can leverage the model for sparse-view image-to-3D generation by pairing it with camera and point predictions from Pi3X. It also enables geometry-guided editing, where an edited image guides changes while point constraints anchor the untouched regions, and helps prepare simulation assets by completing partially captured objects for robotics pipelines.
For AI practitioners, Axolotl3D offers a blueprint for multimodal 3D conditioning: encoding evidence sources separately, aligning them in a single coordinate frame, and training a single generator across diverse observation regimes. However, the model is limited to discrete, object-level reconstructions rather than entire rooms. It also requires upstream camera pose estimators. While NVIDIA demonstrated the model using Kaolin's web user interface, the company has not yet released a public checkpoint or source code.
This is our own summary of reporting by AlphaSignal
