Pith. sign in

REVIEW 3 major objections 5 minor 50 references

OmniX predicts dense 3D point trajectories for every pixel from any-view, any-time video with large camera motion by separating motion into a compact set of dynamic tokens.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 08:51 UTC pith:4RSPODS5

load-bearing objection Solid feed-forward 4D trajectory work with a real architecture and a big synthetic engine; the headline SOTA is still mostly measured on the authors’ own UE5 split, so treat the transfer claim carefully. the 3 major comments →

arxiv 2607.10840 v1 pith:4RSPODS5 submitted 2026-07-12 cs.CV

OmniX: Any-view and Any-time 4D Reconstruction via Feed-forward Trajectory Fields

Yanqin Jiang , Tengfei Wang , Zhengwei Wang , Chenjie Cao , Junta Wu , Wenhan Luo , Weiming Hu , Jin Gao
show 1 more author
Chunchao Guo
This is my paper
classification cs.CV
keywords 4D reconstruction3D point trackingtrajectory fieldssparse spatiotemporal attentionfeed-forward reconstructionsynthetic 4D datasetdynamic scene understanding
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Feed-forward 4D reconstruction has been stuck between two incomplete options: per-frame static point clouds that ignore motion, or trajectory methods that only work under small camera motion. OmniX claims a single forward pass can recover dense 3D trajectories for every pixel even when viewpoints swing widely or inputs are temporally discontinuous. The key move is to treat 3D motion as sparse and low-rank, so a small set of dynamic tokens can parameterize trajectory fields for the whole scene while static geometry is predicted separately. To train this, the authors built an automatic Unreal Engine 5 data engine that yields 80K scenes and 1.28M multi-view videos with full depth, pose, and trajectory labels. The resulting model sets new numbers on dense trajectory prediction and 3D point tracking and stays competitive on depth and camera pose, showing that large-motion 4D reconstruction is now feasible without iterative tracking loops.

Core claim

OmniX establishes that dense 3D point trajectories for every pixel can be predicted in one feed-forward pass from videos with large camera motion by explicitly disentangling dynamic foreground motion from static geometry and representing that motion with a compact set of dynamic tokens whose trajectory fields are upsampled to every pixel.

What carries the argument

Sparse Spatiotemporal Attention (SSA) plus the Deformable Trajectory Sampling Head (DTSH): SSA selects the top-ρ% dynamic tokens, expands them temporally, and lets them cross-attend to all image tokens to produce low-rank trajectory transformation bases; DTSH then deformably samples and weights those bases so every pixel obtains a trajectory while keeping token selection differentiable.

Load-bearing premise

The central claim rests on 3D motion being sparse and low-rank enough that a small fraction of dynamic tokens can accurately parameterize trajectories for every pixel, and on synthetic multi-view data transferring to real large-motion scenes.

What would settle it

A controlled test set of real multi-view videos with large camera motion (greater than roughly 90–180 degrees) and dense ground-truth 3D trajectories in which OmniX’s foreground and all-point APD3D/EPE fall behind or match iterative trackers and prior dense trajectory methods after identical scale alignment.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Any-view, any-time video collections—including temporally disjoint clips and hybrid image-video sets—can be turned into complete dynamic point-cloud sequences without iterative point tracking.
  • Dense 3D trajectories become a practical intermediate for downstream tasks such as embodied simulation, AR/VR content, and autonomous driving perception under large viewpoint change.
  • The same sparse-token motion representation can be reused for video depth and camera pose, keeping those tasks competitive while adding trajectory output.
  • A public UE5-style data engine supplying 80K annotated dynamic scenes lowers the barrier to training future large-motion 4D models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the low-rank motion assumption holds more broadly, similar sparse dynamic tokens could compress other dense correspondence problems (optical flow, scene flow) without full self-attention among all queries.
  • The domain gap between the synthetic engine and real large-motion capture remains the most direct place to pressure-test whether the reported gains are architectural or data-driven.
  • Once dense trajectories are cheap, long-horizon 4D scene models can treat trajectory fields as a first-class persistent state rather than re-inferring motion at every step.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. OmniX is a feed-forward 4D reconstruction model that predicts dense 3D point trajectories for every pixel from any-view, any-time inputs, including monocular videos, disjoint video pairs, and hybrid image-video sets under large camera motion. Building on a multi-view transformer backbone (DepthAnything3-style), it predicts per-frame depth, rays, and cameras via a DPT head, then lifts points to trajectories with a Sparse Spatiotemporal Attention (SSA) module that selects top-ρ% dynamic tokens, expands them temporally, and cross-attends to all image tokens to produce sparse trajectory transformation bases (Eqs. 2–8). A Deformable Trajectory Sampling Head (DTSH) upsamples these bases to per-pixel transformations with a dynamic-score residual (Eqs. 9–11). Training uses a new UE5 data engine (80K scenes, 1.28M multi-view videos with depth, pose, and dense trajectories) plus public datasets. The paper reports large gains on its own dense-trajectory benchmark (Table 1), SOTA on TAPVid-3D 16-frame clips (Table 2), competitive KITTI depth and Sintel/TUM pose (Tables 3–4), efficiency benefits from sparsity (Table 5), and ablations of SSA components, ρ, and data scale (Tables 6–8, Fig. 4).

Significance. If the results hold under fairer controls, the work is a solid contribution to feed-forward 4D reconstruction: it targets a real limitation of prior dense trajectory methods (small camera motion / monocular bias), proposes a concrete sparse low-rank motion parameterization (SSA + DTSH) that is more efficient than iterative trackers, and ships a large multi-view 4D synthetic engine that the community currently lacks. Strengths include a clear architecture, multi-task supervision (Eqs. 12–18), efficiency measurements, and ablations that isolate cross-attention and data scale. The any-view/any-time formulation and competitive secondary tasks (depth, pose) increase practical value for embodied AI and AR/VR pipelines. The main significance is therefore empirical and systems-level rather than a new theoretical guarantee.

major comments (3)
  1. [Table 1, §4.1–4.2] Table 1 is the primary support for the dense any-view/any-time claim under large camera motion, but evaluation is only on the authors’ own UE5 validation split (600 videos / 40 scenes from the same engine used for training). Baselines (TraceAnything, VDPM) were trained on different and smaller corpora and are not re-trained on the UE5 data, so the large APD3D/EPE margins cannot be cleanly attributed to SSA/DTSH versus data scale and domain match. A matched-data re-training of at least one strong baseline, or an independent dense large-motion real benchmark with full trajectory GT, is needed to substantiate the central transfer claim.
  2. [§3.1–3.2, Eqs. 3–11; Tables 6–8] The load-bearing modeling assumption—that 3D motion is sparse/low-rank enough that top-ρ% dynamic tokens (ρ=20) plus deformable sampling of trajectory bases (Eqs. 3–11) parameterize accurate per-pixel trajectories for all pixels across all views and times—is only stress-tested on the custom UE set (Tables 6–8, Fig. 4). Failure modes under dense multi-object motion, heavy occlusion, or non-rigid deformation that violate sparsity are not quantified on real data beyond sparse TAPVid-3D and qualitative DAVIS/Waymo figures. Reporting dense trajectory metrics on a real multi-view sequence (even with partial GT or pseudo-GT) would strengthen the claim.
  3. [§4.1 Data Preparation; Table 2] Training mixes the large UE5 corpus with eight public datasets, but the paper does not report an ablation that isolates the contribution of UE5 versus public data on the real TAPVid-3D / KITTI / Sintel metrics, nor does it quantify domain gap (e.g., synthetic-only vs full mix on Table 2). Without this, it remains unclear how much of the real-world sparse-tracking gains come from the architecture versus the new synthetic scale.
minor comments (5)
  1. [§2.1, §3.2, §4.1] Several typos and formatting issues: “pinoreer” (§2.1), “multi-veiw” (§4.1), inconsistent spacing in compound words (“addressthis”, “Inthiswork”), and broken math rendering in the SSA attention description (e.g., “˜↥×d”).
  2. [Table 1 caption; §4.2] Table 1 caption notes unequal training data; this caveat should also appear in the main text discussion of SOTA claims so readers do not over-read the margins.
  3. [§3.3, §4.1] Hyperparameters β, α, λ1, λ2, Huber ε, and exact κ schedule are only partially specified; a full list would aid reproducibility.
  4. [Fig. 2, Fig. 3] Fig. 2 and Fig. 3 are informative but dense; adding a short note on how trajectories are colored/selected (especially background points) would improve readability.
  5. [§2.2] Related work mentions concurrent 4RC and Track4World; a short explicit comparison of sparse attention vs their designs would help position the contribution.

Circularity Check

0 steps flagged

No significant circularity: standard supervised feed-forward regression of trajectories/depth/poses to external geometric labels; architecture and sparse-token ansatz are modeling choices, not tautologies.

full rationale

The paper's derivation chain is a conventional deep-learning pipeline: a transformer backbone plus DPT head recovers per-frame 3D geometry (depth, rays, optional poses); a Sparse Spatiotemporal Attention module selects top-ρ% dynamic tokens, expands them temporally, and cross-attends to image tokens to predict a compact set of trajectory-transformation bases (Eqs. 4–8); a Deformable Trajectory Sampling Head then bilinearly samples and weights those bases to obtain per-pixel transformations (Eqs. 9–11) that are applied to the 3D points. All quantities are supervised by confidence-aware ℓ1 / Huber / BCE losses against ground-truth depth, rays, trajectories, poses and (optional) dynamic masks that come from the authors' UE5 engine plus eight public datasets (Eqs. 12–18). Nothing is defined in terms of the quantity it is claimed to predict; the sparse/low-rank motion assumption and the weighted-basis representation (Eq. 3) are explicit modeling choices, not self-definitional identities. Evaluation of the primary dense-trajectory metrics occurs on a held-out split of the same synthetic engine, and ablations are likewise run only on that data, but this is ordinary train/val practice rather than a circular reduction of a claimed first-principles result. Independent public benchmarks (TAPVid-3D, KITTI, Sintel, TUM-dynamics) and qualitative real-world figures supply external checks. No uniqueness theorem, load-bearing self-citation chain, or fitted parameter renamed as prediction appears. Consequently the reported SOTA numbers are empirical outcomes of supervised training, not forced by construction.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 4 invented entities

The central claim rests on standard transformer/DPT geometry backbones, the modeling assumption that 3D motion is sparse/low-rank enough for top-ρ% dynamic tokens, several hand-chosen training hyperparameters, and author-invented modules (SSA, DTSH, trajectory-field bases) whose value is shown only by the paper’s own ablations and benchmarks. No formal axioms; free parameters are ordinary ML knobs.

free parameters (5)
  • top-ρ% dynamic token fraction = 20
    Controls how many tokens are treated as dynamic; ablated at 10/20/30 and fixed at 20 for efficiency (Table 7). Directly affects capacity and the sparse-field assumption.
  • dynamic mask loss weight κ = 0.01
    Set to 0.01 for token- and pixel-level BCE regularization when GT masks exist (Sec. 4.1).
  • number of SSA blocks M = 8
    Architecture depth of the trajectory module; fixed at 8 without exhaustive search reported.
  • loss balance weights β, α, λ1, λ2 and Huber ε
    Task and confidence-loss hyperparameters in Eq. 12–16; not fully enumerated with sensitivity analysis.
  • training resolution, sequence length, LR, steps = 280x504, T=16, lr=2e-4, 64k steps
    280×504, 16 frames, AdamW 2e-4, ~64k steps on 64 GPUs; standard but claim-dependent training choices.
axioms (4)
  • domain assumption 3D motion in natural dynamic scenes is sparse and low-rank enough that a small set of dynamic tokens can parameterize trajectory fields for all pixels (Sec. 3.1, abstract).
    Load-bearing modeling premise for SSA efficiency and DTSH upsampling; supported by ablations but not proven generally.
  • domain assumption Per-frame depth/ray geometry from a DepthAnything3-style transformer plus rigid SE(3)-like trajectory transforms (Eq. 1–2) suffice to represent non-rigid 4D scenes when bases are blended per pixel.
    Standard in recent feed-forward 4D work; non-rigid motion is approximated by weighted combination of bases (Eq. 3).
  • domain assumption Synthetic UE5 composites of static environments and dynamic objects, plus listed public datasets, are adequate supervision for real-world large-motion tracking.
    Primary dense-trajectory benchmark is the authors’ own validation split; transfer is assumed from mixed training.
  • standard math Standard multi-head attention, DPT heads, confidence-weighted L1, and BCE losses are valid training objectives for joint geometry and trajectory regression.
    Ordinary deep-learning practice; not novel mathematical claims.
invented entities (4)
  • Sparse Spatiotemporal Attention (SSA) with dynamic tokens no independent evidence
    purpose: Select top-ρ% tokens, expand them temporally, and cross-attend to all image tokens to predict trajectory transformation bases without full dense spatiotemporal attention.
    Core architectural invention; independent evidence is only the paper’s ablations and benchmarks, not an external physical law.
  • Deformable Trajectory Sampling Head (DTSH) no independent evidence
    purpose: Upsample sparse trajectory fields to per-pixel transforms via multi-scale deformable sampling of bases and dynamic scores (Eq. 9–11).
    Enables dense output while keeping token selection differentiable; validated only inside this paper.
  • Trajectory field as weighted combination of sparse transformation bases no independent evidence
    purpose: Compact motion representation τ = Σ wk τk (Eq. 3) inspired by Shape-of-Motion but used feed-forward for all pixels.
    Modeling construct; not independently measured outside the method.
  • UE5-based automatic 4D data engine and 80K-scene dataset no independent evidence
    purpose: Supply multi-view videos with depth, pose, and dense 3D trajectory annotations under large camera motion.
    Major contribution; independent evidence would require public release and third-party use, not stated in the manuscript.

pith-pipeline@v1.1.0-grok45 · 19414 in / 3998 out tokens · 51863 ms · 2026-07-14T08:51:42.850061+00:00 · methodology

0 comments
read the original abstract

Previous feed-forward 4D reconstruction methods either predict per-frame static point clouds, ignoring foreground motion, or estimate point cloud trajectories while being limited to small camera motions. This restricts their ability to aggregate observations over time and reconstruct complete dynamic scenes under large viewpoint changes. To address this limitation, we propose OmniX, a feed-forward 4D reconstruction framework that predicts dense 3D point trajectories for every pixel from videos with large camera motion. OmniX decouples dynamic motion modeling from static geometry prediction and represents motion using a compact set of dynamic tokens. By leveraging the sparse and low-rank structure of 3D motion, these tokens generate trajectory fields for all pixels across all images while efficiently preserving global interactions. To facilitate training, we further build an automatic UE5-based 4D data engine and introduce a large-scale dataset containing 80K scenes and 1.28M multi-view videos with full geometric annotations. OmniX achieves state-of-the-art performance on dense 3D point trajectory prediction and 3D point tracking, while also demonstrating competitive results on video depth estimation and camera pose estimation.

Figures

Figures reproduced from arXiv: 2607.10840 by Chenjie Cao, Chunchao Guo, Jin Gao, Junta Wu, Tengfei Wang, Weiming Hu, Wenhan Luo, Yanqin Jiang, Zhengwei Wang.

Figure 1
Figure 1. Figure 1: Framework of OmniX. Built upon DepthAnything3 [16], OmniX predicts depth, ray maps, and camera parameters for the input images, from which 3D points are recovered. A trajectory module then predicts transformations for these 3D points. For efficiency, we propose a Sparse Spatiotemporal Attention (SSA) mechanism, which selects dynamic tokens, expands them across timestamps as queries, and lets them cross-att… view at source ↗
Figure 2
Figure 2. Figure 2: Visualization of dense 3D point trajectory prediction under challenging camera motion. The first two scenes are from our custom dataset, and the last two row features a scene from the DexYCB [2] and Waymo [34] datasets. Each scene is shown as a 3×5 grid, where the top row presents input images at different timesteps. The first column of the second and third rows shows input images from different viewpoints… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison of 3D point tracking on the DAVIS dataset [29]. Efficiency. Tab. 5 compares the inference efficiency of different methods when evaluated on 16-frame clips. TraceAnything [18] achieves the highest efficiency, attributed largely to its simple architecture, which only adds a tracking head to predict trajectory curves for pixels. However, its accuracy remains much lower than that of othe… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison for the ablation study of SSA components [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 9 linked inside Pith

  1. [1]

    In: Proceedings of the European Conference on Com- puter Vision

    Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Proceedings of the European Conference on Com- puter Vision. pp. 611–625 (2012)

    2012

  2. [2]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Chao,Y.W.,Yang,W.,Xiang, Y.,Molchanov,P.,Handa,A., Tremblay, J.,Narang, Y.S., Van Wyk, K., Iqbal, U., Birchfield, S., et al.: DexYCB: A benchmark for capturing hand grasping of objects. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9044–9053 (2021)

    2021

  3. [3]

    arXiv preprint arXiv:2507.16443 (2025)

    Deng, K., Ti, Z., Xu, J., Yang, J., Xie, J.: VGGT-Long: Chunk it, loop it, align it–pushing VGGT’s limits on kilometer-scale long RGB sequences. arXiv preprint arXiv:2507.16443 (2025)

    Pith/arXiv arXiv 2025

  4. [4]

    Epic Games: Unreal Engine 5 (2022),https://www.unrealengine.com/en-US/ unreal-engine-5, accessed: 2026-06-24

    2022

  5. [5]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Feng, H., Zhang, J., Wang, Q., Ye, Y., Yu, P., Black, M.J., Darrell, T., Kanazawa, A.: St4RTrack: Simultaneous 4D reconstruction and tracking in the world. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 8503–8513 (2025)

    2025

  6. [6]

    International Journal of Robotics Research32(11), 1231–1237 (2013)

    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: The KITTI dataset. International Journal of Robotics Research32(11), 1231–1237 (2013)

    2013

  7. [7]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Hu, Y., Cheng, C., Yu, S., Guo, X., Wang, H.: VGGT4D: Mining motion cues in visual geometry transformers for 4D scene reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 414–424 (2026)

    2026

  8. [8]

    arXiv preprint arXiv:2604.14268 (2026)

    HY-World, T., Cao, C., Zuo, X., Wang, Z., Zhang, Y., Wu, J., Liu, Z., Gong, Y., Liu, Y., Yuan, B., et al.: HY-World 2.0: A multi-modal world model for recon- structing, generating, and simulating 3D worlds. arXiv preprint arXiv:2604.14268 (2026)

    Pith/arXiv arXiv 2026

  9. [9]

    Jin,L.,Tucker,R.,Li,Z.,Fouhey,D.,Snavely,N.,Holynski,A.:Stereo4D:Learning howthingsmovein3Dfrominternetstereovideos.arXivpreprintarXiv:2412.09621 (2024)

    Pith/arXiv arXiv 2024

  10. [10]

    Joo, H., Simon, T., Li, X., Liu, H., Tan, L., Gui, L., Banerjee, S., Godisart, T.S., Nabbe, B., Matthews, I., et al.: Panoptic Studio: A massively multiview system for socialinteraction.IEEETransactionsonPatternAnalysisandMachineIntelligence 16(2017)

    2017

  11. [11]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Karaev, N., Rocco, I., Graham, B., Neverova, N., Vedaldi, A., Rupprecht, C.: Dy- namicStereo: Consistent dynamic depth from stereo videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13229– 13239 (2023)

    2023

  12. [12]

    In: Proceedings of the European Conference on Computer Vision

    Karaev, N., Rocco, I., Graham, B., Neverova, N., Vedaldi, A., Rupprecht, C.: Co- Tracker: It is better to track together. In: Proceedings of the European Conference on Computer Vision. pp. 18–35. Springer (2024)

    2024

  13. [13]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Karhade, J., Keetha, N., Zhang, Y., Gupta, T., Sharma, A., Scherer, S., Ramanan, D.: Any4D: Unified feed-forward metric 4D reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14578– 14589 (2026)

    2026

  14. [14]

    arXiv preprint arXiv:2509.13414 (2025) Omnix: 4D Reconstruction via Trajectory Fields 17

    Keetha, N., Müller, N., Schönberger, J., Porzi, L., Zhang, Y., Fischer, T., Knapitsch, A., Zauss, D., Weber, E., Antunes, N., et al.: MapAnything: Universal feed-forward metric 3D reconstruction. arXiv preprint arXiv:2509.13414 (2025) Omnix: 4D Reconstruction via Trajectory Fields 17

    Pith/arXiv arXiv 2025

  15. [15]

    Advances in Neural Information Processing Systems37, 82149–82165 (2024)

    Koppula, S., Rocco, I., Yang, Y., Heyward, J., Carreira, J., Zisserman, A., Brostow, G., Doersch, C.: TAPVid-3D: A benchmark for tracking any point in 3D. Advances in Neural Information Processing Systems37, 82149–82165 (2024)

    2024

  16. [16]

    arXiv preprint arXiv:2511.10647 (2025)

    Lin, H., Chen, S., Liew, J., Chen, D.Y., Li, Z., Shi, G., Feng, J., Kang, B.: Depth Anything 3: Recovering the visual space from any views. arXiv preprint arXiv:2511.10647 (2025)

    Pith/arXiv arXiv 2025

  17. [17]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Ling, L., Sheng, Y., Tu, Z., Zhao, W., Xin, C., Wan, K., Yu, L., Guo, Q., Yu, Z., Lu, Y., et al.: DL3DV-10K: A large-scale scene dataset for deep learning-based 3D vision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 22160–22169 (2024)

    2024

  18. [18]

    arXiv preprint arXiv:2510.13802 (2025)

    Liu, X., Xiao, Y., Chen, D.Y., Feng, J., Tai, Y.W., Tang, C.K., Kang, B.: Trace Anything: Representing any video in 4D via trajectory fields. arXiv preprint arXiv:2510.13802 (2025)

    arXiv 2025

  19. [19]

    arXiv preprint arXiv:2510.10726 (2025)

    Liu, Y., Min, Z., Wang, Z., Wu, J., Wang, T., Yuan, Y., Luo, Y., Guo, C.: WorldMirror: Universal 3D world reconstruction with any-prior prompting. arXiv preprint arXiv:2510.10726 (2025)

    arXiv 2025

  20. [20]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Liu, Y., Liu, Y., Jiang, C., Lyu, K., Wan, W., Shen, H., Liang, B., Fu, Z., Wang, H., Yi, L.: HOI4D: A 4D egocentric dataset for category-level human-object in- teraction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21013–21022 (2022)

    2022

  21. [21]

    arXiv preprint arXiv:1711.05101 (2017)

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

    Pith/arXiv arXiv 2017

  22. [22]

    arXiv preprint arXiv:2603.02573 (2026)

    Lu, J., Xu, J., Hu, W., Zhu, R., Zhao, C., Yeung, S.K., Shan, Y., Liu, Y.: Track4World: Feedforward world-centric dense 3D tracking of all pixels. arXiv preprint arXiv:2603.02573 (2026)

    arXiv 2026

  23. [23]

    arXiv preprint arXiv:2602.10094 (2026)

    Luo, Y., Zhou, S., Lan, Y., Pan, X., Loy, C.C.: 4RC: 4D reconstruction via condi- tional querying anytime and anywhere. arXiv preprint arXiv:2602.10094 (2026)

    Pith/arXiv arXiv 2026

  24. [24]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Mehl, L., Schmalfuss, J., Jahedi, A., Nalivayko, Y., Bruhn, A.: Spring: A high- resolutionhigh-detaildatasetandbenchmarkforsceneflow,opticalflowandstereo. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4981–4991 (2023)

    2023

  25. [25]

    In: Proceedings of the International Conference on Learning Representations (2025)

    Ngo, T., Zhuang, P., Kalogerakis, E., Gan, C., Tulyakov, S., Lee, H.Y., Wang, C.: DELTA: Dense efficient long-range 3D tracking for any video. In: Proceedings of the International Conference on Learning Representations (2025)

    2025

  26. [26]

    arXiv preprint arXiv:2508.01170 (2025)

    Ngo, T.D., Mirzaei, A., Qian, G., Liang, H., Gan, C., Kalogerakis, E., Wonka, P., Wang, C.: DELTAv2: Accelerating dense 3D tracking. arXiv preprint arXiv:2508.01170 (2025)

    arXiv 2025

  27. [27]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Pan, X., Charron, N., Yang, Y., Peters, S., Whelan, T., Kong, C., Parkhi, O., Newcombe, R., Ren, Y.C.: Aria Digital Twin: A new benchmark dataset for ego- centric 3D machine perception. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 20133–20143 (2023)

    2023

  28. [28]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Peebles, W., Xie, S.: Scalable diffusion models with transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 4195–4205 (2023)

    2023

  29. [29]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Perazzi, F., Pont-Tuset, J., McWilliams, B., Van Gool, L., Gross, M., Sorkine- Hornung, A.: A benchmark dataset and evaluation methodology for video object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 724–732 (2016)

    2016

  30. [30]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Ranftl, R., Bochkovskiy, A., Koltun, V.: Vision transformers for dense prediction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 12179–12188 (2021) 18 Y. Jiang et al

    2021

  31. [31]

    In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems

    Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 573–580 (2012)

    2012

  32. [32]

    arXiv preprint arXiv:2601.09499 (2026)

    Sucar, E., Insafutdinov, E., Lai, Z., Vedaldi, A.: V-DPM: 4D video reconstruction with dynamic point maps. arXiv preprint arXiv:2601.09499 (2026)

    arXiv 2026

  33. [33]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Sucar, E., Lai, Z., Insafutdinov, E., Vedaldi, A.: Dynamic Point Maps: A versatile representation for dynamic 3D reconstruction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 7295–7305 (2025)

    2025

  34. [34]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B., et al.: Scalability in perception for autonomous driving: Waymo open dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2446–2454 (2020)

    2020

  35. [35]

    In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition

    Wang, J., Chen, M., Karaev, N., Vedaldi, A., Rupprecht, C., Novotny, D.: VGGT: Visual geometry grounded transformer. In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition. pp. 5294–5306 (2025)

    2025

  36. [36]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Wang, Q., Chang, Y.Y., Cai, R., Li, Z., Hariharan, B., Holynski, A., Snavely, N.: Tracking everything everywhere all at once. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 19795–19806 (2023)

    2023

  37. [37]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Wang, Q., Ye, V., Gao, H., Zeng, W., Austin, J., Li, Z., Kanazawa, A.: Shape of Motion: 4D reconstruction from a single video. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 9660–9672 (2025)

    2025

  38. [38]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wang, Q., Zhang, Y., Holynski, A., Efros, A.A., Kanazawa, A.: Continuous 3D per- ception model with persistent state. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10510–10522 (2025)

    2025

  39. [39]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wang, S., Leroy, V., Cabon, Y., Chidlovskii, B., Revaud, J.: DUSt3R: Geometric 3D vision made easy. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20697–20709 (2024)

    2024

  40. [40]

    arXiv preprint arXiv:2507.13347 (2025)

    Wang, Y., Zhou, J., Zhu, H., Chang, W., Zhou, Y., Li, Z., Chen, J., Pang, J., Shen, C., He, T.:π 3: Permutation-equivariant visual geometry learning. arXiv preprint arXiv:2507.13347 (2025)

    Pith/arXiv arXiv 2025

  41. [41]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Xiao, Y., Wang, J., Xue, N., Karaev, N., Makarov, Y., Kang, B., Zhu, X., Bao, H., Shen, Y., Zhou, X.: SpatialTrackerv2: Advancing 3D point tracking with explicit camera motion. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 6726–6737 (2025)

    2025

  42. [42]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Xiao, Y., Wang, Q., Zhang, S., Xue, N., Peng, S., Shen, Y., Zhou, X.: Spatial- Tracker: Tracking any 2D pixels in 3D space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20406–20417 (2024)

    2024

  43. [43]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yang, J., Sax, A., Liang, K.J., Henaff, M., Tang, H., Cao, A., Chai, J., Meier, F., Feiszli, M.: Fast3R: Towards 3D reconstruction of 1000+ images in one for- ward pass. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21924–21935 (2025)

    2025

  44. [44]

    Advances in Neural Information Processing Systems38, 135284–135303 (2026)

    Zhang, B., Ke, L., Harley, A., Fragkiadaki, K.: TAPIP3D: Tracking any point in persistent 3D geometry. Advances in Neural Information Processing Systems38, 135284–135303 (2026)

    2026

  45. [45]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Zhang, C., Le Moing, G., Koppula, S., Rocco, I., Momeni, L., Xie, J., Sun, S., Sukthankar, R., Barral, J.K., Hadsell, R., et al.: Efficiently reconstructing dynamic scenes one D4RT at a time. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7382–7392 (2026)

    2026

  46. [46]

    In:ProceedingsoftheInternationalConferenceonLearningRepresentations(2025) Omnix: 4D Reconstruction via Trajectory Fields 19

    Zhang,J.,Herrmann,C.,Hur,J.,Jampani,V.,Cole,F.,Sun,D.,Yang,M.H.,etal.: MonST3R: A simple approach for estimating geometry in the presence of motion. In:ProceedingsoftheInternationalConferenceonLearningRepresentations(2025) Omnix: 4D Reconstruction via Trajectory Fields 19

    2025

  47. [47]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Zheng, Y., Harley, A.W., Shen, B., Wetzstein, G., Guibas, L.J.: PointOdyssey: A large-scale synthetic dataset for long-term point tracking. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 19855–19865 (2023)

    2023

  48. [48]

    Zhou, K., Wang, Y., Chen, G., Chang, X., Beaudouin, G., Zhan, F., Liang, P.P., Wang, M.: PAGE-4D: Disentangled pose and geometry estimation for 4D percep- tion (2026)

    2026

  49. [49]

    arXiv preprint arXiv:2509.12201 (2025)

    Zhou, Y., Wang, Y., Zhou, J., Chang, W., Guo, H., Li, Z., Ma, K., Li, X., Wang, Y., Zhu, H., et al.: OmniWorld: A multi-domain and multi-modal dataset for 4D world modeling. arXiv preprint arXiv:2509.12201 (2025)

    arXiv 2025

  50. [50]

    arXiv preprint arXiv:2507.11539 (2025)

    Zhuo, D., Zheng, W., Guo, J., Wu, Y., Zhou, J., Lu, J.: Streaming 4D visual geometry transformer. arXiv preprint arXiv:2507.11539 (2025)

    Pith/arXiv arXiv 2025