Visual Geometry Router
Projects VGGT-Ω features into requested cameras with confidence-aware, two-layer visibility routing.
-Diff
Visual Geometry Meets Diffusion
for Sparse-View Novel View Synthesis
1 XPeng Motors2 The Chinese University of Hong Kong3 Tsinghua University
Featured films · Full-resolution novel views
From six input images, VGGT-Diff synthesizes geometrically consistent novel views across continuous camera trajectories.
Updated pose-difficulty results with continuous-view footage from the earlier trajectory checkpoint.
Model weights.
Download the released 100K half-resolution NVS, full-resolution NVS, or full-resolution 80-frame trajectory checkpoint. See the Model Zoo for download and inference instructions.
Six Full-Resolution Input Views
832 × 480 each
Abstract
We present VGGT-Diff, a geometry-routed multi-view diffusion model for sparse-view novel view synthesis. Existing NVS methods face a fundamental trade-off: reconstruction-based approaches preserve observed geometry but struggle to synthesize unseen regions, while diffusion-based methods provide strong generative priors yet rely on implicit source-to-query correspondence. VGGT-Diff bridges these regimes by routing visual geometry latents from VGGT-Ω into a pretrained video diffusion model.
A confidence-aware Visual Geometry Router transforms 3D points and appearance-bearing tokens into query-aligned conditions while preserving front and back surface evidence. Joint target-view denoising is regularized by Point-Track Residual Consistency, which aligns predicted-clean residuals along reliable 3D tracks. Uncertainty-aware training and inference further improve robustness to imperfect geometry.
Method
Joint Multi-View Diffusion
Processes source and query slots together so requested views exchange information during denoising.
Point-Track Consistency
Couples remaining denoising errors along reliable 3D tracks without suppressing view-dependent appearance.
+ Visual Geometry Conditioning
The router begins from a sharp depth-selected anchor, preserves secondary-surface evidence near uncertain occlusions, and learns only a residual correction from layered features and routing statistics.
Results
19.283DL3DV PSNR↑ latest checkpoint
0.198DL3DV LPIPS-Alex↓ latest checkpoint
17.217Mip-NeRF 360 PSNRzero-shot
+1.179 dBDL3DV PSNR gainvs. earlier VGGT-Diff
Latest quantitative checkpoint: 100K effective steps at 192 × 336 on 9,510 DL3DV training scenes, followed by 6K full-resolution adaptation steps at 480 × 832. The formal results below use six input views and independent target-view inference.
Compared with prior baselines, VGGT-Diff synthesizes novel views that remain more faithful to the real scene, better preserving its original geometry, object identity, and visual content. The predicted images also align more accurately with the requested camera viewpoints and their corresponding ground-truth views, producing more consistent spatial layouts and fewer viewpoint or content mismatches.
Ground Truth
Loading scene
+ Cross-View Geometry
To test whether multiple generated views agree on a single 3D structure, we jointly synthesize eight target views from the same six input views and reconstruct them with two frozen geometry models: VGGT-Ω, our primary probe, and Pi3, an independent reconstructor never used to train or condition VGGT-Diff. Each point cloud is compared with the reference reconstructed from the corresponding ground-truth views under identical cameras and settings. Agreement under both probes shows that the gains reflect stronger cross-view geometry rather than evaluator bias.
| Method | Chamfer-L1 ↓ | F-score@1% ↑ | F-score@2% ↑ |
|---|---|---|---|
| LVSM | 0.2387 | 0.1116 | 0.2218 |
| SEVA | 0.1944 | 0.2050 | 0.3244 |
| FrameCrafter | 0.1000 | 0.3086 | 0.4793 |
| VGGT-Diff | 0.0733 | 0.4231 | 0.6041 |
| Method | Chamfer-L1 ↓ | F-score@1% ↑ | F-score@2% ↑ |
|---|---|---|---|
| LVSM | 0.3610 | 0.0673 | 0.1525 |
| SEVA | 0.2514 | 0.1640 | 0.2751 |
| FrameCrafter | 0.1404 | 0.2140 | 0.3880 |
| VGGT-Diff | 0.1174 | 0.3089 | 0.5021 |
These cross-view geometry results are from the earlier VGGT-Diff evaluation, not the new 100K + 6K checkpoint. Evaluated across 52 DL3DV scenes. Metrics are trajectory-normalized and should be compared within each reconstructor. Lower Chamfer-L1 and higher F-scores indicate stronger cross-view geometric consistency.
Video & Trajectory Demos
Explore twenty 80-frame trajectories generated from six input views. Each plays forward and back at 24 FPS. Select a scene to watch its prediction; matching ground truth remains available for comparison.
VGGT-Diff prediction · 80-frame pathevalnative_1de58be51569
Full-resolution 832 × 480 · 24 FPS · forward and reverse
1 / 20
Explore trajectoriesSelect a scene to play it above
Citation
@article{chen2026vggtdiff,
title={VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis},
author={Chen, Kangjie and Li, Xiangyu and Zhang, Dongbin and Zheng, Chaoda and Chen, Shijia and Deng, Jinhao and Lin, Hongbin and Choo, Sin Wai and Wang, Minqi and Yang, Minghao and Zhong, Dake and Song, Guorui and Zhang, Yu and Liu, Xianming and Wang, Boyang},
journal={arXiv preprint arXiv:2609.33253},
year={2026}
}