Visual Geometry Router
Projects VGGT-Ω features into requested cameras with confidence-aware, two-layer visibility routing.
-Diff
Visual Geometry Meets Diffusion
for Sparse-View Novel View Synthesis
1 XPeng Motors2 The Chinese University of Hong Kong3 Tsinghua University
Effective 21K checkpoint · Full resolution · 80 continuous frames
Full-Resolution Long-Sequence Demo
From six input images, VGGT-Diff generates a continuous 80-frame novel-view sequence at full resolution. The model was initialized from a 20K half-resolution checkpoint, then received only 1K steps of full-resolution continuation training.
1080p autoplay · 24 fps · 40 seconds · sound on when browser permits
Training status.
This effective 21K checkpoint is still training. Fine details and cross-view consistency are expected to continue improving. Fully trained full-resolution checkpoints will be released on the VGGT-Diff GitHub project page.
Six Full-Resolution Input Views
832 × 480 each
Abstract
We present VGGT-Diff, a geometry-routed multi-view diffusion model for sparse-view novel view synthesis. Existing NVS methods face a fundamental trade-off: reconstruction-based approaches preserve observed geometry but struggle to synthesize unseen regions, while diffusion-based methods provide strong generative priors yet rely on implicit source-to-query correspondence. VGGT-Diff bridges these regimes by routing visual geometry latents from VGGT-Ω into a pretrained video diffusion model.
A confidence-aware Visual Geometry Router transforms 3D points and appearance-bearing tokens into query-aligned conditions while preserving front and back surface evidence. Joint target-view denoising is regularized by Point-Track Residual Consistency, which aligns predicted-clean residuals along reliable 3D tracks. Uncertainty-aware training and inference further improve robustness to imperfect geometry.
Method
Joint Multi-View Diffusion
Processes source and query slots together so requested views exchange information during denoising.
Point-Track Consistency
Couples remaining denoising errors along reliable 3D tracks without suppressing view-dependent appearance.
+ Visual Geometry Conditioning
The router begins from a sharp depth-selected anchor, preserves secondary-surface evidence near uncertain occlusions, and learns only a residual correction from layered features and routing statistics.
Results
18.104DL3DV PSNR↑ best
0.222DL3DV LPIPS↓ best
16.296Mip-NeRF 360 PSNRzero-shot
−26.7%Chamfer-L1vs. FrameCrafter
Compared with prior baselines, VGGT-Diff synthesizes novel views that remain more faithful to the real scene, better preserving its original geometry, object identity, and visual content. The predicted images also align more accurately with the requested camera viewpoints and their corresponding ground-truth views, producing more consistent spatial layouts and fewer viewpoint or content mismatches.
Ground Truth
Loading scene
+ Cross-View Geometry
To test whether multiple generated views agree on a single 3D structure, we jointly synthesize eight target views from the same six input views and reconstruct them with two frozen geometry models: VGGT-Ω, our primary probe, and Pi3, an independent reconstructor never used to train or condition VGGT-Diff. Each point cloud is compared with the reference reconstructed from the corresponding ground-truth views under identical cameras and settings. Agreement under both probes shows that the gains reflect stronger cross-view geometry rather than evaluator bias.
| Method | Chamfer-L1 ↓ | F-score@1% ↑ | F-score@2% ↑ |
|---|---|---|---|
| LVSM | 0.2387 | 0.1116 | 0.2218 |
| SEVA | 0.1944 | 0.2050 | 0.3244 |
| FrameCrafter | 0.1000 | 0.3086 | 0.4793 |
| VGGT-Diff | 0.0733 | 0.4231 | 0.6041 |
| Method | Chamfer-L1 ↓ | F-score@1% ↑ | F-score@2% ↑ |
|---|---|---|---|
| LVSM | 0.3610 | 0.0673 | 0.1525 |
| SEVA | 0.2514 | 0.1640 | 0.2751 |
| FrameCrafter | 0.1404 | 0.2140 | 0.3880 |
| VGGT-Diff | 0.1174 | 0.3089 | 0.5021 |
Evaluated across 52 DL3DV scenes. Metrics are trajectory-normalized and should be compared within each reconstructor. Lower Chamfer-L1 and higher F-scores indicate stronger cross-view geometric consistency.
Video & Trajectory Demos
Twenty full-resolution 80-frame continuous-view comparisons generated from six input views. Every sequence preserves its original GT-left / VGGT-Diff Prediction-right layout and embedded label bar.
Lightweight previews play automatically · Select a scene for the original resolutionCitation
@article{chen2026vggtdiff,
title={VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis},
author={Chen, Kangjie and Li, Xiangyu and Zhang, Dongbin and Zheng, Chaoda and Chen, Shijia and Deng, Jinhao and Lin, Hongbin and Choo, Sin Wai and Wang, Minqi and Yang, Minghao and Zhong, Dake and Song, Guorui and Zhang, Yu and Liu, Xianming and Wang, Boyang},
journal={arXiv preprint arXiv:2609.33253},
year={2026}
}