Pith. sign in

REVIEW 4 major objections 5 minor 72 references

Object Concepts Emerge from Motion

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Motion boundaries can serve as a label-free supervisory signal for learning instance-level visual representations.

desk verdict A genuinely simple and reproducible motion-pretraining recipe with consistent but modest gains, whose headline claims outrun the evidence — worth reviewing seriously, but needs controlled baselines and validation of the pseudo-labels. read the letter →

arxiv 2505.21635 v1 pith:7UAV2HIL submitted 2025-05-27 cs.CV

Haoqian Liang , Xiaohui Wang , Zhichao Li , Ya Yang , Naiyan Wang This is my paper
classification cs.CV
keywords unsupervisedrepresentationlearningmotionboundaryobject-centricrepresentationsopticalflowclusteringcontrastivevisualinstancemonoculardepthestimation3Dobjectdetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that object concepts can be learned without any labels by watching motion. The authors generate pseudo instance masks from optical flow clustering on roughly 2,700 hours of unlabeled driving video, then use these masks in contrastive learning to train single-image encoders. They report that this motion-based pretraining outperforms supervised ImageNet-22K and self-supervised baselines on monocular depth estimation, 3D object detection, and occupancy prediction. If true, motion would provide a scalable, calibration-free route to instance-level visual representations, complementing the category-level features of existing foundation models.

What carries the argument

The central machinery is a BFS pixel-clustering algorithm that converts optical flow into pseudo instance masks: for each pixel passing a forward-backward consistency check, neighboring pixels whose flow vectors differ by less than $\theta_f = 1.5$ are grouped, and clusters smaller than $\theta_s = 100$ pixels are discarded. These masks feed a contrastive loss that pulls features from the same mask together, pushes features from different masks apart with margin $m = 1.0$, and ignores background-background pairs so incomplete pseudo-labels do not corrupt training. The loss turns raw motion discontinuities into a training signal for a single-image encoder built on a backbone plus a feature pyramid network.

What would settle it

Take a held-out subset of the pretraining videos, run the BFS clustering, and compare every pseudo-mask against human instance-segmentation annotations: if boundary recall is low, or if clusters systematically split along texture or parallax gradients rather than object boundaries, the central motion-boundary premise is not doing the claimed work.

Watch

Extended reading notes

Core claim

The paper claims that motion boundaries provide an unsupervised supervisory signal for learning features that capture the visual instance: a distinct, coherent object entity rather than merely a semantic category. Under rigid scene motion, the spatial gradient of the optical flow field is proportional to the gradient of depth, so flow discontinuities serve as proxies for depth discontinuities and hence object boundaries. The authors implement this by clustering optical flow with a simple breadth-first search, obtaining pseudo instance masks without human labels or camera calibration, and then training a contrastive objective on those masks. Across monocular depth, 3D object detection, and 3D occupancy prediction, the resulting features outperform supervised and self-supervised pretraining baselines, generalize to unseen indoor scenes, and are complementary to DINOv2 features when fused.

Load-bearing premise

The whole method depends on the assumption that where optical flow changes sharply, an object boundary is present; if that link fails, the pseudo-masks teach the model about parallax, texture edges, or flow noise instead of objects.

Editorial extensions

If this is right

  • A single-image encoder pretrained only on motion-derived masks can replace supervised ImageNet-22K pretraining on monocular depth, 3D detection, and occupancy prediction, across both ResNet and Swin backbones.
  • Features learned from motion are complementary to category-level features from DINOv2; concatenating them improves depth estimation beyond either representation alone.
  • Because the pipeline needs no camera calibration, it can be applied to any large, unstructured video collection without annotation.
  • Although trained on outdoor driving video, the features segment objects in unseen indoor, egocentric, and robot-manipulation scenes, suggesting transfer of object structure rather than memorized appearances.
  • Instance-level features benefit both low-level tasks like depth and high-level tasks like 3D detection, indicating that object composition is a useful shared abstraction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the core premise would be to evaluate the pseudo-masks against ground-truth instance boundaries on a held-out video subset; if boundary recall is low or clusters systematically follow parallax or texture gradients, the motion-boundary assumption is not doing the claimed work.
  • If optical flow quality improves through synthetic-data pretraining, the same motion-mask pipeline should scale to egocentric and unconstrained web video, where the paper reports current flow limits the approach.
  • The reported complementarity with DINOv2 suggests a joint pretraining objective combining a motion-mask contrastive term with a self-distillation term could produce a single backbone with both instance-level and category-level abstractions.
  • A controlled pretraining experiment on static-camera video with only independently moving objects would isolate whether moving-object boundaries, rather than depth parallax from ego-motion, drive the learned object concepts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a self-supervised pretraining method for visual encoders that uses motion information from raw video. It generates pseudo instance masks by applying BFS clustering to optical flow with thresholds theta_f = 1.5 and theta_s = 100, on about 2,700 hours of driving video from OpenDV-YouTube and nuPlan. The resulting masks are used to train Swin and ResNet backbones with a contrastive loss: pixels in the same motion cluster are pulled together, pixels in different clusters are pushed apart, and background-background pairs are ignored. The method is evaluated on monocular depth estimation on KITTI, 3D object detection on nuScenes, and 3D occupancy prediction on nuScenes, with comparisons against ImageNet-22K supervised pretraining, SimMIM, DINOv2, and other baselines. The central claim is that motion boundaries provide an unsupervised supervisory signal that yields object-instance features, and that models trained this way outperform prior supervised and self-supervised baselines across model sizes.

Significance. If the central claim were fully substantiated, this would be a valuable contribution: a scalable, calibration-free, label-free alternative to existing visual foundation models that captures instance-level structure, with evidence across low-level (depth) and high-level (detection, occupancy) tasks. The paper's pipeline is simple and reproducible, uses a large amount of unlabeled video, and the experiments demonstrate measurable gains in several configurations, including consistent improvements over ImageNet-22K pretraining for most backbones. The complementarity with DINO features shown in Table 5 is also a useful empirical finding. However, the current significance is limited by two gaps: the motion-to-object-boundary link is asserted rather than validated quantitatively, and the paper's universal superiority claim is contradicted by some of its own results. Because the core value depends on whether the pseudo-labels actually correspond to object instances, the missing validation is load-bearing.

major comments (4)
  1. [§3.1 and §3.2] Equation (3) is a chain-rule identity showing that flow gradients follow depth gradients under rigid motion, but it does not establish that depth discontinuities are object boundaries. Depth changes also occur at surface orientation changes, occlusion edges, and non-object scene structures such as walls or ground-plane boundaries. The paper's central mechanism therefore rests on an unproven domain assumption. This would be acceptable if the pseudo-labels were quantitatively validated against ground-truth instance masks, but no precision, recall, boundary IoU, or similar measure is reported. The qualitative examples in Fig. 3 and the acknowledgment of under-segmentation in Sec. 3.2 do not fill this gap. I request a quantitative evaluation of pseudo-label quality on a labeled subset (e.g., KITTI or nuScenes annotations), and an ablation comparing motion-based clustering against a depth-discontinuity baseline to isolate whether the gains come from object instances or from generic depth edges.
  2. [Abstract, Tables 1–6] The claim that models 'outperform previous supervised and self-supervised baselines' is not supported by the full set of results. In Table 1, SimMIM (Swin-L, window 12) achieves SILog 6.542 while Ours (Swin-L, window 7) achieves 6.558, so the baseline wins on the primary depth metric. In Table 6, DINOv2 (ViT-L) achieves RayIoU 39.0 while Ours (Swin-L) achieves 38.7. In Table 4, DINOv2 (ViT-L) achieves mAP 42.05 versus Ours (Swin-L) 41.79. These are not isolated footnotes; they directly bear on the abstract's claim that the method offers 'a compelling alternative to existing vision foundation models.' The text should be revised to say that the method is competitive and often better in specific settings, and the cases where baselines are stronger should be analyzed rather than omitted.
  3. [§4.1, Tables 1 and 6] The baseline comparison is not controlled for model capacity and context. Section 4.1 states: 'All Swin models use a window size of 7, while the B and L variants of SimMIM and Semantic-SAM used for comparison adopt a larger window size of 12.' A larger window gives the baseline more spatial context, which is usually beneficial. Therefore, the fact that Ours is worse than SimMIM on Table 1 may partly reflect this design choice rather than the pretraining method alone. Conversely, some claimed wins over SimMIM could be due to other differences in training setup. To support the 'outperforms' claim, the authors should either report SimMIM and Semantic-SAM with window size 7, or clearly separate the effect of the pretraining objective from the effect of window size, for example by training Ours with window 12 as well.
  4. [§3.2, §4.2, Eq. (4)] The training signal could reinforce flow clustering artifacts rather than object instances. The BFS clustering uses fixed thresholds θf=1.5 and θs=100, which are not validated against any ground truth. The loss in Eq. (4) pulls together all pixels in the same cluster, including pixels that may belong to a parallax region, a texture edge, or a flow estimation error. Furthermore, the rule that 'the label with the largest pixel count is background' is an assumption: if a large object fills the frame, its pixels would be ignored as background. The qualitative evidence in Sec. 4.2 that the model distinguishes objects 'not annotated in the pseudo-labels' is presented as generalization, but it could equally indicate that the pseudo-labels themselves are noisy and the model learns some other grouping. This alternative explanation should be addressed explicitly, for example by measuring whether the learned features separate instances better than the pseudo-labels do, or by ablating the clustering thresholds and reporting pseudo-label quality versus downstream performance.
minor comments (5)
  1. [§3.1, Eq. (3)] The notation in Eq. (3) is imprecise: ∇F is a 2x2 Jacobian (or a vector of two component gradients), while dφ/dD is not defined as a matrix. The chain rule should be written with an explicit Jacobian so that the dimensional relationship is clear.
  2. [§1 and §2.1] The statement 'we are the first to leverage motion as an unsupervised supervisory signal' is too strong, since the paper itself cites earlier work on motion- and flow-guided object discovery, such as [49] and [59]. The novelty claim should be qualified to reflect the specific combination of simple BFS clustering, contrastive learning, and large-scale driving video.
  3. [§3.2] The sentence 'We retained all samples with at least two pseudo-label(i.e. at least one foreground cluster)' contains a typo and a potentially confusing parenthetical; the writing should be cleaned up.
  4. [§3.2 and §4.1] The paper calls the framework 'fully label-free,' but it relies on VideoFlow, an optical flow model that is trained with supervision on labeled synthetic data. The label-free claim should be scoped to the downstream training data or the pseudo-label generation process, not extended to the entire pipeline.
  5. [§4.3] In Table 1, the claim that 'our model consistently outperforms both supervised ImageNet-22K pretraining and models pretrained on the Semantic-SAM' is technically true for the rows shown, but the table also includes SimMIM, which outperforms Ours in the Swin-L row. The sentence should be rephrased to acknowledge the SimMIM comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the motion-derived pseudo-labels are used only as pretraining inputs, and all central quantitative claims are validated on external downstream benchmarks independent of those labels.

full rationale

The pretraining pipeline is self-contained: motion-based pseudo-labels from BFS clustering of optical flow (Sec. 3.2) supervise a contrastive loss (Eq. 4), and the resulting encoder is transferred to three downstream tasks with external benchmarks (KITTI depth, nuScenes detection and occupancy). Those benchmarks provide independent grounding; no downstream label enters the pretraining, and no fitted parameter is relabeled as a prediction. The geometric link between flow gradients and depth gradients (Eq. 3) is a chain-rule identity under a rigid-scene assumption; it does not define object boundaries, so it is an unverified domain assumption rather than a circular reduction. The paper explicitly acknowledges that pseudo-labels under-segment due to weak motion cues or flow errors (Sec. 3.2) and that performance is greatly limited by optical flow quality (Sec. 5.2). The fixed clustering thresholds are not validated against ground truth, but they are inputs to the pretraining, not predictions. The only overlapping-author citation ([4]) is used merely to contrast calibration requirements and is not load-bearing. Qualitative feature visualizations are illustrative, not the basis of the quantitative claims. No circularity by construction is present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a small set of hand-set parameters and two domain assumptions: that motion boundaries mark object boundaries and that reliable optical flow can be obtained at scale. There are no invented physical entities. The strongest extra premises are the rigid-scene pinhole model and the background-label heuristic.

free parameters (3)
  • flow clustering threshold θf = 1.5
    Maximum L2 difference between neighboring flow vectors to be considered the same cluster (Sec 3.2). Chosen by hand; no sensitivity analysis.
  • min cluster size θs = 100
    Minimum number of pixels for a cluster to be kept as an instance (Sec 3.2). Chosen by hand.
  • loss margin m = 1.0
    Margin for pulling apart features of different pseudo-labels in the contrastive loss (Eq 4).
assumptions (4)
  • domain assumption Motion boundaries align with object boundaries; flow discontinuities occur at depth discontinuities, which mark object boundaries.
    Sec 3.1 states this as the key geometric insight, derived from Eq (3), but the chain rule only shows flow gradients scale with depth gradients; the link to object boundaries is assumed, not proven.
  • domain assumption The scene is rigid except for independently moving objects, and a pinhole camera model applies for the optical flow relationship in Eq (1)-(3).
    Used to derive flow as a function of depth and ego-motion; real driving scenes contain non-rigid motion, occlusions, and rolling shutter.
  • domain assumption Optical flow from VideoFlow and the forward-backward consistency check are reliable enough for pixel clustering.
    The entire pseudo-label pipeline depends on flow quality; the paper notes in Sec 5.2 that performance is limited by optical flow.
  • ad hoc to paper The label with the largest pixel count is background, and ignoring background-background losses is safe.
    Sec 3.3 defines y=0 as the largest cluster; this avoids collapsing background but is a domain-specific heuristic for driving videos.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Object Concepts Emerge from Motion." pith.science (2026). https://pith.science/paper/7UAV2HIL

@misc{pith2026250521635,
  author       = {Pith},
  title        = {Pith review of: Object Concepts Emerge from Motion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7UAV2HIL}},
  note         = {Machine review of arXiv:2505.21635}
}
read the original abstract

Object concepts play a foundational role in human visual cognition, enabling perception, memory, and interaction in the physical world. Inspired by findings in developmental neuroscience - where infants are shown to acquire object understanding through observation of motion - we propose a biologically inspired framework for learning object-centric visual representations in an unsupervised manner. Our key insight is that motion boundary serves as a strong signal for object-level grouping, which can be used to derive pseudo instance supervision from raw videos. Concretely, we generate motion-based instance masks using off-the-shelf optical flow and clustering algorithms, and use them to train visual encoders via contrastive learning. Our framework is fully label-free and does not rely on camera calibration, making it scalable to large-scale unstructured video data. We evaluate our approach on three downstream tasks spanning both low-level (monocular depth estimation) and high-level (3D object detection and occupancy prediction) vision. Our models outperform previous supervised and self-supervised baselines and demonstrate strong generalization to unseen scenes. These results suggest that motion-induced object representations offer a compelling alternative to existing vision foundation models, capturing a crucial but overlooked level of abstraction: the visual instance. The corresponding code will be released upon paper acceptance.

Figures

Figures reproduced from arXiv: 2505.21635 by the authors.

Figure 1
Figure 1. Comparisons with feature maps learned by our method and different visual foundation [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Pipeline of the proposed method. developmental insights, our approach leverages the observation that coherent motion often indicates objecthood—an idea supported by infant perception studies and geometric reasoning in dynamic scenes. In this section, we introduce our method, which consists of three key components: (1) a geometric analysis revealing how motion boundaries correlate with object boundaries, (2) a data p… view at source ↗
Figure 3
Figure 3. Examples of the pseudo-label generation results and the output features. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Similarity visualization for a set of reference points. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Examples of feature maps in out of domain scenes. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Examples of the pseudo-label generation results and the output features. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

72 extracted references · 51 canonical work pages

  1. [1]

    Self-supervised learning from images with a joint-embedding predictive architecture

    Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. InCVPR, 2023

    2023

  2. [2]

    BEiT: BETR pre-training of image transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEiT: BETR pre-training of image transformers. In ICLR, 2022

    work page 2022

  3. [3]

    Revisiting feature prediction for learning visual representations from video

    Adrien Bardes, Quentin Garrido, Jean Ponce, Xinlei Chen, Michael Rabbat, Yann LeCun, Mahmoud Assran, and Nicolas Ballas. Revisiting feature prediction for learning visual representations from video. TMLR, 2024

    work page 2024

  4. [4]

    Unsupervised scale-consistent depth and ego-motion learning from monocular video

    Jiawang Bian, Zhichao Li, Naiyan Wang, Huangying Zhan, Chunhua Shen, Ming-Ming Cheng, and Ian Reid. Unsupervised scale-consistent depth and ego-motion learning from monocular video. InNeurIPS, 2019

    work page 2019

  5. [5]

    MONet: Unsupervised scene decomposition and representation.arXiv preprint arXiv:1901.11390, 2019

    Christopher P Burgess, Loic Matthey, Nicholas Watters, Rishabh Kabra, Irina Higgins, Matt Botvinick, and Alexander Lerchner. MONet: Unsupervised scene decomposition and representation.arXiv preprint arXiv:1901.11390, 2019

    arXiv 1901

  6. [6]

    nuscenes: A multimodal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. InCVPR, 2020. 10

    work page 2020

  7. [7]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. InICCV, 2021

    2021

  8. [8]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. InICML, 2020

    2020

Show all 72 references
  1. [9]

    Improved baselines with momentum contrastive learning.arXiv preprint arXiv:2003.04297, 2020

    Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning.arXiv preprint arXiv:2003.04297, 2020

    2003 arXiv

  2. [10]

    An empirical study of training self-supervised vision transformers

    Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. InICCV, 2021

    2021

  3. [11]

    OpenMMLab’s pre-training toolbox and benchmark

    MMPreTrain Contributors. OpenMMLab’s pre-training toolbox and benchmark. https://github.com/ open-mmlab/mmpretrain, 2023

    2023

  4. [12]

    Vision transformers need registers

    Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. InICLR, 2024

    2024

  5. [13]

    ImageNet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. InCVPR, 2009

    2009

  6. [14]

    With a little help from my friends: Nearest-neighbor contrastive learning of visual representations

    Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, and Andrew Zisserman. With a little help from my friends: Nearest-neighbor contrastive learning of visual representations. InICCV, 2021

    2021

  7. [15]

    Depth map prediction from a single image using a multi-scale deep network

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network. InNeurIPS, 2014

    2014

  8. [16]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. InCVPR, 2012

    2012

  9. [17]

    Ego4D: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4D: Around the world in 3,000 hours of egocentric video. InCVPR, 2022

    2022

  10. [18]

    Multi-object representation learning with iterative variational inference

    Klaus Greff, Raphaël Lopez Kaufman, Rishabh Kabra, Nick Watters, Christopher Burgess, Daniel Zoran, Loic Matthey, Matthew Botvinick, and Alexander Lerchner. Multi-object representation learning with iterative variational inference. InICML, 2019

    2019

  11. [19]

    Bootstrap your own latent-a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. InNeurIPS, 2020

    2020

  12. [20]

    Tan et al

    K. Tan et al. H. Caesar, J. Kabzan. NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles. InCVPR ADP3 workshop, 2021

    2021

  13. [21]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InCVPR, 2016

    2016

  14. [22]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. InCVPR, 2020

    2020

  15. [23]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. InCVPR, 2022

    2022

  16. [24]

    Development of perceptual completion originates in early infancy

    Scott P Johnson and Susan A Johnson. Development of perceptual completion originates in early infancy. Psychological Science, 14(6):553–559, 2003

    2003

  17. [25]

    Discriminative clustering for image co-segmentation

    Armand Joulin, Francis Bach, and Jean Ponce. Discriminative clustering for image co-segmentation. In CVPR, 2010

    2010

  18. [26]

    Multi-class cosegmentation

    Armand Joulin, Francis Bach, and Jean Ponce. Multi-class cosegmentation. InCVPR, 2012

    2012

  19. [27]

    Perception of object unity in young infants.Infant Behavior and Development, 11(2):161–180, 1983

    Philip J Kellman and Elizabeth S Spelke. Perception of object unity in young infants.Infant Behavior and Development, 11(2):161–180, 1983

    1983

  20. [28]

    Panoptic feature pyramid networks

    Alexander Kirillov, Ross Girshick, Kaiming He, and Piotr Dollár. Panoptic feature pyramid networks. In CVPR, 2019

    2019

  21. [29]

    A new neural framework for visuospatial processing.Nature Reviews Neuroscience, 12(4):217–230, 2011

    Dwight J Kravitz, Kadharbatcha S Saleem, Chris I Baker, and Mortimer Mishkin. A new neural framework for visuospatial processing.Nature Reviews Neuroscience, 12(4):217–230, 2011. 11

    2011

  22. [30]

    Segment and recognize anything at any granularity

    Feng Li, Hao Zhang, Peize Sun, Xueyan Zou, Shilong Liu, Chunyuan Li, Jianwei Yang, Lei Zhang, and Jianfeng Gao. Segment and recognize anything at any granularity. InECCV, 2024

    2024

  23. [31]

    BEVFormer: learning bird’s-eye-view representation from LiDAR-camera via spatiotemporal transformers

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Qiao Yu, and Jifeng Dai. BEVFormer: learning bird’s-eye-view representation from LiDAR-camera via spatiotemporal transformers. TPAMI, 47(3):2020–2036, 2024

    2020

  24. [32]

    V A-DepthNet: A variational approach to single image depth prediction

    Ce Liu, Suryansh Kumar, Shuhang Gu, Radu Timofte, and Luc Van Gool. V A-DepthNet: A variational approach to single image depth prediction. InICLR, 2023

    2023

  25. [33]

    Fully sparse 3D occupancy prediction

    Haisong Liu, Yang Chen, Haiguang Wang, Zetong Yang, Tianyu Li, Jia Zeng, Li Chen, Hongyang Li, and Limin Wang. Fully sparse 3D occupancy prediction. InECCV, 2024

    2024

  26. [34]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. InICCV, 2021

    2021

  27. [35]

    Object-centric learning with slot attention

    Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention. InNeurIPS, 2020

    2020

  28. [36]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InICLR, 2019

    2019

  29. [37]

    Unsupervised discovery of object-centric neural fields

    Rundong Luo, Hong-Xing Yu, and Jiajun Wu. Unsupervised discovery of object-centric neural fields. arXiv preprint arXiv:2402.07376, 2024

    2024 arXiv

  30. [38]

    David Marr.Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. 1982

    1982

  31. [39]

    Object exploration and object knowledge in young infants: A view from developmental psychology.Cognition, Brain, and Consciousness, 2001

    Amy Needham. Object exploration and object knowledge in young infants: A view from developmental psychology.Cognition, Brain, and Consciousness, 2001

    2001

  32. [40]

    DINOv2: Learning robust visual features without supervision.TMLR, 2023

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. DINOv2: Learning robust visual features without supervision.TMLR, 2023

    2023

  33. [41]

    Open X-embodiment: Robotic learning datasets and RT-X models

    Abby O’Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open X-embodiment: Robotic learning datasets and RT-X models. InICRA, 2024

    2024

  34. [42]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. InNeurIPS, 2019

    2019

  35. [43]

    BEiT v2: Masked image modeling with vector-quantized visual tokenizers.arXiv preprint arXiv:2208.06366, 2022

    Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Ye, and Furu Wei. BEiT v2: Masked image modeling with vector-quantized visual tokenizers.arXiv preprint arXiv:2208.06366, 2022

    2022 arXiv

  36. [44]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InICML, 2021

    2021

  37. [45]

    NDDepth: Normal-distance assisted monocular depth estimation

    Shuwei Shao, Zhongcai Pei, Weihai Chen, Xingming Wu, and Zhengguo Li. NDDepth: Normal-distance assisted monocular depth estimation. InICCV, 2023

    2023

  38. [46]

    IEBins: Iterative elastic bins for monocular depth estimation

    Shuwei Shao, Zhongcai Pei, Xingming Wu, Zhong Liu, Weihai Chen, and Zhengguo Li. IEBins: Iterative elastic bins for monocular depth estimation. InNeurIPS, 2023

    2023

  39. [47]

    VideoFlow: Exploiting temporal cues for multi-frame optical flow estimation

    Xiaoyu Shi, Zhaoyang Huang, Weikang Bian, Dasong Li, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, Jifeng Dai, and Hongsheng Li. VideoFlow: Exploiting temporal cues for multi-frame optical flow estimation. InICCV, 2023

    2023

  40. [48]

    Localizing objects with self-supervised transformers and no labels

    Oriane Siméoni, Gilles Puy, Huy V V o, Simon Roburin, Spyros Gidaris, Andrei Bursuc, Patrick Pérez, Renaud Marlet, and Jean Ponce. Localizing objects with self-supervised transformers and no labels. In BMVC, 2021

    2021

  41. [49]

    Locate: self-supervised object discovery via flow-guided graph-cut and bootstrapped self-training

    Silky Singh, Shripad Deshmukh, Mausoom Sarkar, and Balaji Krishnamurthy. Locate: self-supervised object discovery via flow-guided graph-cut and bootstrapped self-training. InBMVC, 2023

    2023

  42. [50]

    Selective search for object recognition.IJCV, 104:154–171, 2013

    Jasper RR Uijlings, Koen EA Van De Sande, Theo Gevers, and Arnold WM Smeulders. Selective search for object recognition.IJCV, 104:154–171, 2013. 12

    2013

  43. [51]

    Two cortical visual systems.Analysis of Visual Behavior, 1982

    Leslie G Ungerleider and Mortimer Mishkin. Two cortical visual systems.Analysis of Visual Behavior, 1982

    1982

  44. [52]

    Object cosegmentation

    Sara Vicente, Carsten Rother, and Vladimir Kolmogorov. Object cosegmentation. InCVPR, 2011

    2011

  45. [53]

    Unsupervised image matching and object discovery as optimization

    Huy V V o, Francis Bach, Minsu Cho, Kai Han, Yann LeCun, Patrick Pérez, and Jean Ponce. Unsupervised image matching and object discovery as optimization. InCVPR, 2019

    2019

  46. [54]

    Large-scale unsupervised object discovery

    Van Huy V o, Elena Sizikova, Cordelia Schmid, Patrick Pérez, and Jean Ponce. Large-scale unsupervised object discovery. InNeurIPS, 2021

    2021

  47. [55]

    DCDepth: Progressive monocular depth estimation in discrete cosine domain

    Kun Wang, Zhiqiang Yan, Junkai Fan, Wanlu Zhu, Xiang Li, Jun Li, and Jian Yang. DCDepth: Progressive monocular depth estimation in discrete cosine domain. InNeurIPS, 2024

    2024

  48. [56]

    Image as a foreign language: BEiT pretraining for vision and vision-language tasks

    Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, et al. Image as a foreign language: BEiT pretraining for vision and vision-language tasks. InCVPR, 2023

    2023

  49. [57]

    Self- supervised transformers for unsupervised object discovery using normalized cut

    Yangtao Wang, Xi Shen, Shell Xu Hu, Yuan Yuan, James L Crowley, and Dominique Vaufreydaz. Self- supervised transformers for unsupervised object discovery using normalized cut. InCVPR, 2022

    2022

  50. [58]

    TokenCut: Segmenting objects in images and videos with self-supervised transformer and normalized cut.TPAMI, 45(12):15790–15801, 2023

    Yangtao Wang, Xi Shen, Yuan Yuan, Yuming Du, Maomao Li, Shell Xu Hu, James L Crowley, and Dominique Vaufreydaz. TokenCut: Segmenting objects in images and videos with self-supervised transformer and normalized cut.TPAMI, 45(12):15790–15801, 2023

    2023

  51. [59]

    4D unsupervised object discovery

    Yuqi Wang, Yuntao Chen, and Zhao-Xiang Zhang. 4D unsupervised object discovery. InNeurIPS, 2022

    2022

  52. [60]

    Unsupervised object discovery and co-localization by deep descriptor transformation.PR, 88:113–126, 2019

    Xiu-Shen Wei, Chen-Lin Zhang, Jianxin Wu, Chunhua Shen, and Zhi-Hua Zhou. Unsupervised object discovery and co-localization by deep descriptor transformation.PR, 88:113–126, 2019

    2019

  53. [61]

    CroCo: Self-supervised pre-training for 3D vision tasks by cross-view completion

    Philippe Weinzaepfel, Vincent Leroy, Thomas Lucas, Romain Brégier, Yohann Cabon, Vaibhav Arora, Leonid Antsfeld, Boris Chidlovskii, Gabriela Csurka, and Jérôme Revaud. CroCo: Self-supervised pre-training for 3D vision tasks by cross-view completion. InNeurIPS, 2022

    2022

  54. [62]

    SimMIM: A simple framework for masked image modeling

    Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. SimMIM: A simple framework for masked image modeling. InCVPR, 2022

    2022

  55. [63]

    Revealing the dark secrets of masked image modeling

    Zhenda Xie, Zigang Geng, Jingcheng Hu, Zheng Zhang, Han Hu, and Yue Cao. Revealing the dark secrets of masked image modeling. InCVPR, 2023

    2023

  56. [64]

    BEVFormer v2: Adapting modern image backbones to bird’s-eye- view recognition via perspective supervision

    Chenyu Yang, Yuntao Chen, Hao Tian, Chenxin Tao, Xizhou Zhu, Zhaoxiang Zhang, Gao Huang, Hongyang Li, Yu Qiao, Lewei Lu, et al. BEVFormer v2: Adapting modern image backbones to bird’s-eye- view recognition via perspective supervision. InCVPR, 2023

    2023

  57. [65]

    Generalized predictive model for autonomous driving

    Jiazhi Yang, Shenyuan Gao, Yihang Qiu, Li Chen, Tianyu Li, Bo Dai, Kashyap Chitta, Penghao Wu, Jia Zeng, Ping Luo, Jun Zhang, Andreas Geiger, Yu Qiao, and Hongyang Li. Generalized predictive model for autonomous driving. InCVPR, 2024

    2024

  58. [66]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. InCVPR, 2024

    2024

  59. [67]

    Depth anything v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. InNeurIPS, 2024

    2024

  60. [68]

    Decoupled contrastive learning

    Chun-Hsiao Yeh, Cheng-Yao Hong, Yen-Chi Hsu, Tyng-Luh Liu, Yubei Chen, and Yann LeCun. Decoupled contrastive learning. InECCV, 2022

    2022

  61. [69]

    Neural window fully-connected CRFs for monocular depth estimation

    Weihao Yuan, Xiaodong Gu, Zuozhuo Dai, Siyu Zhu, and Ping Tan. Neural window fully-connected CRFs for monocular depth estimation. InCVPR, 2022

    2022

  62. [70]

    HEAP: unsupervised object discovery and localization with contrastive grouping

    Xin Zhang, Jinheng Xie, Yuan Yuan, Michael Bi Mi, and Robby T Tan. HEAP: unsupervised object discovery and localization with contrastive grouping. InAAAI, 2024

    2024

  63. [71]

    Unsupervised learning of depth and ego-motion from video

    Tinghui Zhou, Matthew Brown, Noah Snavely, and David G Lowe. Unsupervised learning of depth and ego-motion from video. InCVPR, 2017

    2017

  64. [72]

    Edge boxes: Locating object proposals from edges

    C Lawrence Zitnick and Piotr Dollár. Edge boxes: Locating object proposals from edges. InECCV, 2014. 13 A Pseudo-codes for Pixel Cluster For all optical flow data generated by VideoFlow, we perform a simple Breadth-First Search(BFS) to segment moving objects. Alg. 1 provides a...

    2014

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.