Pith. sign in

REVIEW 4 major objections 4 minor 50 references

Tell Me What to Track: Infusing Robust Language Guidance for Enhanced Referring Multi-Object Tracking

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TellTrack claims that rebalancing detection-query training and injecting language directly into decoder queries raises referring multi-object tracking accuracy, reporting 55.63 HOTA on Refer-KITTI and 37.67 on Refer-KITTI-V2.

desk verdict Solid RMOT engineering integration with SOTA numbers, but the threshold sweeps don't match the final reported numbers, so the margin is not yet verifiable. read the letter →

arxiv 2412.12561 v3 pith:IBSUOLKJ submitted 2024-12-17 cs.CV cs.AI

Wenjun Huang , Yang Ni , Hanning Chen , Yirui He , Ian Bryant , Yezi Liu , Mohsen Imani This is my paper · ORCID
classification cs.CVcs.AI
keywords referringmulti-objecttrackinglanguage-guidedtransformerdecoderquerymatchingcross-modalfusionautonomousdrivingHOTAend-to-end
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Referring multi-object tracking (RMOT) asks a tracker to follow only the objects described by a sentence such as "a person riding a bike" across a video. The paper argues that transformer-based RMOT models have two fixable weaknesses: detection queries for newborn objects are trained far less often than track queries for already-tracked objects, and language guidance is fused only into image features, never reaching the decoder queries directly. TellTrack responds with three moves: a collaborative query matching strategy that lets detection queries also match existing targets in intermediate decoder layers, a referring-infused query adaptation that adds the sentence embedding to query content before and inside the decoder, and a cross-modal encoder that runs deformable multi-scale self-attention before textual cross-attention. The method reports HOTA of 55.63% on Refer-KITTI and 37.67% on Refer-KITTI-V2, surpassing the previous best by 3.22 and 1.49 percentage points, with inference speed essentially unchanged. The intended contribution is a reusable recipe for stronger end-to-end language-guided tracking.

What carries the argument

The load-bearing objects are the two query types in a DETR-style decoder: detection queries, which are supposed to fire when a target first appears, and track queries, which carry identities across frames. The paper's central identity is that a target activates its detection query exactly once but its track query many times, so detection queries are undertrained; CQM rebalances this by extending the auxiliary bipartite matching in intermediate decoder layers to include existing targets, while keeping strict one-to-one matching in the final layer. The other machinery is the sentence embedding added element-wise to the content part of queries (pre-decoder adaptation) or prepended as an extra query that participates in self-attention only (in-decoder adaptation), plus a cross-modal encoder that reorders the deformable-attention and text-cross-attention steps. Together these carry the argument that explicit linguistic guidance has to reach the query level, not just the feature map, and that training signal has to reach detection queries more often.

What would settle it

Run TellTrack and the strongest baseline on Refer-KITTI-V2 with all thresholds fixed by validation-set selection and report test HOTA; if the gap falls below the claimed 1.49 points, the headline result depends on threshold choice. Separately, replace CQM by simply training detection queries for twice as many gradient updates; if HOTA matches, the matching rule itself is not the active ingredient.

Watch

Extended reading notes

Core claim

The paper's central claim is that the imbalanced activation of detection versus track queries is a measurable bottleneck in referring multi-object tracking, and that this bottleneck can be relieved during training without changing inference. The mechanism is collaborative query matching (CQM): in auxiliary decoder layers, detection queries are matched not only with newborn targets but also with existing targets, so they receive gradients in more frames and learn to localize objects the track queries already know. Around this, the paper adds a referring-infused query adaptation that injects the sentence embedding into the content part of queries before and inside the decoder, and a cross-modal encoder that runs deformable multi-scale self-attention before textual cross-attention so that text interacts with structured rather than raw visual features. The reported result is a HOTA of 55.63% on Refer-KITTI and 37.67% on Refer-KITTI-V2, with each ablated component contributing positively and inference speed essentially unchanged.

Load-bearing premise

The reported margin assumes the inference thresholds $\beta_{\rm ref}=0.3$ and $\beta_{\rm obj}=0.7$ were chosen on a validation split rather than by tuning on the test set, because the paper's threshold sweeps do not state which split they used.

Editorial extensions

If this is right

  • If CQM's activation-rebalancing explanation is right, any end-to-end MOT model with one-to-one bipartite matching could improve newborn detection by allowing intermediate-layer matches to existing targets, independent of language.
  • Putting sentence embeddings directly in decoder queries means RMOT no longer depends solely on early feature fusion; the decoder can be treated as a language-conditioned set predictor.
  • The reported gains come with negligible inference cost (15.08 versus 15.13 FPS), so the improvements are compatible with real-time autonomous-driving settings if thresholds are calibrated.
  • Because CQM only changes auxiliary training losses, the same pretrained backbone and initialization can be reused, making the recipe easy to graft onto other transformer trackers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper leaves implicit is that sentence-level embedding may be too coarse for expressions that require distinguishing two objects of the same category by attributes or spatial relations; injecting word-level or phrase-level embeddings into queries would be a direct test of whether finer linguistic control helps.
  • Because threshold sweeps are reported only for the full model, the components' individual contributions at other operating points are unknown; evaluating each component across the full beta_ref--beta_obj grid would clarify whether CQM improves detection or merely shifts the precision-recall tradeoff.
  • CQM's logic generalizes beyond referring tasks to any set-prediction tracker where newborn targets are rare; a cheap test is to apply the same auxiliary matching rule to MOTR-style trackers on a standard MOT benchmark and check whether detection recall rises.
  • The paper compares against published numbers; an apple-to-apple re-run of the strongest baselines with the same backbone, initialization, and training schedule would separate architectural gains from training-length and hyperparameter effects.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes TellTrack, an end-to-end transformer-based referring multi-object tracking method with three main components: Collaborative Query Matching (CQM), which lets detection queries match existing targets in auxiliary decoder layers to compensate for the imbalance between newborn and existing targets; Referring-Infused Query Adaptation (RIQA), which injects a sentence embedding into decoder queries either before or inside the decoder; and a Cross-Modal Encoder (CME), which applies deformable attention to the image feature pyramid before cross-attending to word embeddings. Experiments on Refer-KITTI and Refer-KITTI-V2 report HOTA of 55.63 and 37.67, respectively, exceeding the previous best results by 3.22 and 1.49 HOTA points. Ablations in Sec. 4.4 claim that each proposed component contributes positively, and threshold sweeps are reported for the inference thresholds beta_ref and beta_obj.

Significance. If the reported results are reproducible, the work is a useful engineering contribution to RMOT. The CQM idea of increasing detection-query activation during auxiliary training is simple and plausible, and the RIQA and CME components address a real limitation in existing indirect language fusion. The paper has several strengths: it trains end-to-end, requires no post-processing such as NMS, reports a comparable inference speed of 15.08 FPS versus 15.13 FPS for TempRMOT, and provides qualitative examples showing language-conditioned behavior. At the same time, the claimed margins rest on a threshold-selection protocol that is not documented, and the ablation tables contain internal inconsistencies. These issues are fixable, but until they are resolved, the headline improvement cannot be independently verified.

major comments (4)
  1. [Sec. 4.1, Sec. 4.4, Tables 4-5] The paper never states whether the beta_ref and beta_obj sweeps in Tables 4 and 5 were performed on a validation split or on the same test partition used for Table 1. The final inference thresholds beta_ref=0.3 and beta_obj=0.7 are selected from these sweeps, and Table 4 shows HOTA varying from 29.99 to 37.22 across beta_ref, so test-set threshold selection could account for a substantial part of the claimed 1.49-point margin on Refer-KITTI-V2. Please specify the split used for the sweeps; if thresholds were tuned on the test set, the comparison should be re-run with thresholds fixed independently of the test partition.
  2. [Table 1 vs. Tables 4-5] The final configuration is internally inconsistent across tables. Table 1 reports HOTA=37.67, DetA=24.09, AssA=58.92 for TellTrack on Refer-KITTI-V2, whereas both Table 4 (beta_ref=0.3) and Table 5 (beta_obj=0.7) report HOTA=37.22, DetA=23.38, AssA=59.25 for the configuration stated as optimal in Sec. 4.4. The text does not explain this discrepancy. If Tables 4-5 were computed on a validation subset and Table 1 on the test subset, that split must be stated explicitly so the results can be checked.
  3. [Table 2 vs. Table 3, Sec. 4.4] The ablation tables do not agree on the effect of RIQA. Table 2 lists the RIQA-only condition as HOTA=35.66, but Table 3 reports 35.37 for in-decoder RIQA, 35.45 for pre-decoder with detection queries only, 35.88 for pre-decoder with track queries only, and 36.23 for pre-decoder with both; no row in Table 3 equals 35.66. In addition, the text attributes a +3.32 AssA and +8.02 AssPr improvement to RIQA, but those numbers match Table 3's pre-decoder-both row relative to the base row, not the RIQA row in Table 2, whose differences are +2.84 and +0.48. Please state which RIQA variant is used in Table 2 and reconcile the two tables.
  4. [Sec. 4.3, Table 1] All reported results, including the headline margins of +3.22 and +1.49 HOTA, come from a single run with no error bars or significance testing. Given the sensitivity of HOTA to threshold choices documented in Table 4, and the fact that the margin over HFF-Tracker on Refer-KITTI-V2 is 1.49 points, the paper should report results over multiple seeds or at least state the evaluation protocol used to produce Table 1 so that the robustness of the claimed improvement can be assessed.
minor comments (4)
  1. [Abstract, Sec. 4.3, Sec. 6] The abstract reports a +3.42% improvement while the introduction, Sec. 4.3, and conclusion report +3.22%; please reconcile the headline number and specify which dataset and configuration each number refers to.
  2. [Sec. 3.1, Sec. 3.2.2, Fig. 1] There are several typographical errors, including 'activited' in Sec. 3.1, 'catenating' around Eq. (12), and inconsistent spacing in 'TellTrack' in Fig. 1; a copyedit would improve readability.
  3. [Eq. (17)] The indicator in Eq. (17) is written as 'I refers to I{cDet!=empty}'; please define the indicator function formally, as the current notation is ambiguous.
  4. [Sec. 4.1] Please consider adding a reproducibility statement, including whether code and trained models will be released, since the paper does not currently state this.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claims are empirical benchmark measurements with ablations, not derivations from fitted inputs or self-citation.

full rationale

TellTrack's central claims are empirical. The reported HOTA improvements (55.63% on Refer-KITTI, 37.67% on Refer-KITTI-V2) are measured against external benchmarks, not derived from the method's own assumptions or fitted parameters. Each proposed component (CQM, RIQA, CME) is an architectural modification whose contribution is assessed through ablations in Table 2; none of the components is defined in terms of the final metric, and no 'prediction' is obtained by renaming a fitted quantity. The paper's only noticeable self-citation ([11], TaskCLIP) appears in a related-work discussion of two-stage referring methods and is not load-bearing for any core result. The threshold-sweep concern regarding beta_ref and beta_obj is a potential experimental-validity issue about unspecified selection splits, but it is not circularity: the thresholds are inference hyperparameters, not derived quantities, and the paper does not claim to predict HOTA from them. The inconsistency between Table 1 and Tables 4/5 is a reproducibility/checkability concern, not a circular one. No equation or construction in the paper reduces the claimed result to its own inputs, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or neural entities, but it introduces several fitted thresholds and relies on domain assumptions about query activation and matching. The main free parameters are the inference thresholds, which are tuned via sweeps and directly affect the headline HOTA numbers.

free parameters (3)
  • referring threshold βref = 0.3
    Chosen via sweep in Table 4 to maximize HOTA; no validation/test split stated.
  • object threshold βobj = 0.7
    Chosen via sweep in Table 5 to maximize HOTA; no validation/test split stated.
  • RIQA variant = Pre-decoder infusion into both detection and track queries (inferred)
    The paper evaluates multiple RIQA designs in Table 3 but does not explicitly state which one is used in the final TellTrack model, and the Table 2 RIQA-only score does not match the best Table 3 variant.
assumptions (3)
  • domain assumption The imbalance between newborn and existing target activations is a major cause of RMOT underperformance.
    Stated in Sec. 1 without dataset statistics on the number of newborn vs. existing target assignments.
  • domain assumption DETR-style one-to-one matching with auxiliary losses is an appropriate training paradigm for RMOT.
    The CQM modification extends the standard matching without questioning the joint training objective.
  • domain assumption The sentence embedding added to query content parts provides useful guidance without disrupting existing track queries.
    RIQA is motivated by SAM but RMOT has newborn target detection; no analysis of interference is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tell Me What to Track: Infusing Robust Language Guidance for Enhanced Referring Multi-Object Tracking." pith.science (2026). https://pith.science/paper/IBSUOLKJ

@misc{pith2026241212561,
  author       = {Pith},
  title        = {Pith review of: Tell Me What to Track: Infusing Robust Language Guidance for Enhanced Referring Multi-Object Tracking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IBSUOLKJ}},
  note         = {Machine review of arXiv:2412.12561}
}
read the original abstract

Referring multi-object tracking (RMOT) is an emerging cross-modal task that aims to localize an arbitrary number of targets based on a language expression and continuously track them in a video. This intricate task involves reasoning on multi-modal data and precise target localization with temporal association. However, prior studies overlook the imbalanced data distribution between newborn targets and existing targets due to the nature of the task. In addition, they only indirectly fuse multi-modal features, struggling to deliver clear guidance on newborn target detection. To solve the above issues, we conduct a collaborative matching strategy to alleviate the impact of the imbalance, boosting the ability to detect newborn targets while maintaining tracking performance. In the encoder, we integrate and enhance the cross-modal and multi-scale fusion, overcoming the bottlenecks in previous work, where limited multi-modal information is shared and interacted between feature maps. In the decoder, we also develop a referring-infused adaptation that provides explicit referring guidance through the query tokens. The experiments showcase the superior performance of our model (+3.42%) compared to prior works, demonstrating the effectiveness of our designs.

Figures

Figures reproduced from arXiv: 2412.12561 by the authors.

Figure 1
Figure 1. Tracking pipelines for SOT, MOT, and our TellTrack for [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of TellTrack. The transformer-based framework with a memory bank accepts a video frame, a language expression, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of conventional query matching and CQM. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Overview of Cross-modal Encoder (CME). The im [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 4
Figure 4. Figure 4: Overview of Referring-Infused Query Adaptation: (a) [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison. Red bounding boxes are the [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: The bottom panels show all visible objects detected by [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 33 canonical work pages

  1. [1]

    Character-level language modeling with deeper self-attention

    Rami Al-Rfou, Dokook Choe, Noah Constant, Mandy Guo, and Llion Jones. Character-level language modeling with deeper self-attention. In Proceedings of the AAAI conference on artificial intelligence, pages 3159–3166, 2019. 6

    work page 2019

  2. [2]

    Tracking without bells and whistles

    Philipp Bergmann, Tim Meinhardt, and Laura Leal-Taixe. Tracking without bells and whistles. In Proceedings of the IEEE/CVF international conference on computer vision, pages 941–951, 2019. 8

    work page 2019

  3. [3]

    Fully-convolutional siamese networks for object tracking

    Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, and Philip HS Torr. Fully-convolutional siamese networks for object tracking. In Computer Vision–ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8- 10 and 15-16, 2016, Proceedings, Part II 14, pages 850–865. Springer, 2016. 1

    work page 2016

  4. [4]

    Visual object tracking using adaptive corre- lation filters

    David S Bolme, J Ross Beveridge, Bruce A Draper, and Yui Man Lui. Visual object tracking using adaptive corre- lation filters. In 2010 IEEE computer society conference on computer vision and pattern recognition, pages 2544–2550. IEEE, 2010. 1

    work page 2010

  5. [5]

    End-to-end referring video object segmentation with multi- modal transformers

    Adam Botach, Evgenii Zheltonozhskii, and Chaim Baskin. End-to-end referring video object segmentation with multi- modal transformers. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 4985–4995, 2022. 8

    work page 2022

  6. [6]

    Learning a neu- ral solver for multiple object tracking

    Guillem Bras ´o and Laura Leal-Taix ´e. Learning a neu- ral solver for multiple object tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6247–6257, 2020. 8

    work page 2020

  7. [7]

    Memot: Multi-object tracking with memory

    Jiarui Cai, Mingze Xu, Wei Li, Yuanjun Xiong, Wei Xia, Zhuowen Tu, and Stefano Soatto. Memot: Multi-object tracking with memory. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8090–8100, 2022. 8

    work page 2022

  8. [8]

    A computational approach to edge detection

    John Canny. A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelli- gence, (6):679–698, 1986. 6

    work page 1986

Show all 50 references
  1. [9]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In European confer- ence on computer vision, pages 213–229. Springer, 2020. 5, 6, 8

    2020

  2. [10]

    Refergpt: Towards zero-shot referring multi-object tracking

    Tzoulio Chamiti, Leandro Di Bella, Adrian Munteanu, and Nikos Deligiannis. Refergpt: Towards zero-shot referring multi-object tracking. In Proceedings of the Computer Vi- sion and Pattern Recognition Conference, pages 3849–3858,

  3. [11]

    Taskclip: Extend large vision-language model for task oriented object detec- tion

    Hanning Chen, Wenjun Huang, Yang Ni, Sanggeon Yun, Fei Wen, Hugo Latapie, and Mohsen Imani. Taskclip: Extend large vision-language model for task oriented object detec- tion. arXiv preprint arXiv:2403.08108, 2024. 8

    2024 arXiv

  4. [12]

    Motchallenge: A benchmark for single-camera multiple target tracking

    Patrick Dendorfer, Aljosa Osep, Anton Milan, Konrad Schindler, Daniel Cremers, Ian Reid, Stefan Roth, and Laura Leal-Taix´e. Motchallenge: A benchmark for single-camera multiple target tracking. International Journal of Computer Vision, 129:845–881, 2021. 8

    2021

  5. [13]

    Talk2car: Taking control of your self-driving car

    Thierry Deruyttere, Simon Vandenhende, Dusan Grujicic, Luc Van Gool, and Marie-Francine Moens. Talk2car: Taking control of your self-driving car. arXiv preprint arXiv:1909.10838, 2019. 1

    1909 arXiv

  6. [14]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint a...

    2010 arXiv

  7. [15]

    ikun: Speak to trackers without retraining

    Yunhao Du, Cheng Lei, Zhicheng Zhao, and Fei Su. ikun: Speak to trackers without retraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19135–19144, 2024. 7, 8

    2024

  8. [16]

    Lasot: A high-quality benchmark for large-scale single ob- ject tracking

    Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, and Haibin Ling. Lasot: A high-quality benchmark for large-scale single ob- ject tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, page...

  9. [17]

    Vi- sual prompt tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Vi- sual prompt tuning. In European conference on computer vision, pages 709–727. Springer, 2022. 5

    2022

  10. [18]

    Video object segmentation with language referring expressions

    Anna Khoreva, Anna Rohrbach, and Bernt Schiele. Video object segmentation with language referring expressions. In Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Re- vised Selected Papers, Part IV 14, pages 123–141. Springer,

    2018

  11. [19]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 2, 4

    2023

  12. [20]

    Cognitive disentanglement for referring multi-object tracking

    Shaofeng Liang, Runwei Guan, Wangwang Lian, Daizong Liu, Xiaolou Sun, Dongming Wu, Yutao Yue, Weiping Ding, and Hui Xiong. Cognitive disentanglement for referring multi-object tracking. Information Fusion , page 103349,

  13. [21]

    Glus: Global-local reasoning unified into a single large lan- guage model for video segmentation

    Lang Lin, Xueyang Yu, Ziqi Pang, and Yu-Xiong Wang. Glus: Global-local reasoning unified into a single large lan- guage model for video segmentation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 8658–8667, 2025. 8

    2025

  14. [22]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In 9 Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedi...

    2014

  15. [23]

    Focal loss for dense object detection

    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ar. Focal loss for dense object detection. In Pro- ceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017. 6

    2017

  16. [24]

    Learning to assemble neural module tree networks for visual grounding

    Daqing Liu, Hanwang Zhang, Feng Wu, and Zheng-Jun Zha. Learning to assemble neural module tree networks for visual grounding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4673–4682, 2019. 8

    2019

  17. [25]

    Dab-detr: Dynamic anchor boxes are better queries for detr

    Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, and Lei Zhang. Dab-detr: Dynamic anchor boxes are better queries for detr. arXiv preprint arXiv:2201.12329, 2022. 4

    2022 arXiv

  18. [26]

    Hota: A higher order metric for evaluating multi-object tracking

    Jonathon Luiten, Aljosa Osep, Patrick Dendorfer, Philip Torr, Andreas Geiger, Laura Leal-Taix´e, and Bastian Leibe. Hota: A higher order metric for evaluating multi-object tracking. International journal of computer vision, 129:548– 578, 2021. 6

    2021

  19. [27]

    Exploring simple 3d multi-object tracking for autonomous driving

    Chenxu Luo, Xiaodong Yang, and Alan Yuille. Exploring simple 3d multi-object tracking for autonomous driving. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10488–10497, 2021. 1

    2021

  20. [28]

    Generation and comprehension of unambiguous object descriptions

    Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan L Yuille, and Kevin Murphy. Generation and comprehension of unambiguous object descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 11–20, 2016. 8

    2016

  21. [29]

    Trackformer: Multi-object track- ing with transformers

    Tim Meinhardt, Alexander Kirillov, Laura Leal-Taixe, and Christoph Feichtenhofer. Trackformer: Multi-object track- ing with transformers. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 8844–8854, 2022. 8

    2022

  22. [30]

    Conditional detr for fast training convergence

    Depu Meng, Xiaokang Chen, Zejia Fan, Gang Zeng, Houqiang Li, Yuhui Yuan, Lei Sun, and Jingdong Wang. Conditional detr for fast training convergence. In Proceed- ings of the IEEE/CVF international conference on computer vision, pages 3651–3660, 2021. 4

    2021

  23. [31]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  24. [32]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024. 2

    2024 arXiv

  25. [33]

    Generalized in- tersection over union: A metric and a loss for bounding box regression

    Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Generalized in- tersection over union: A metric and a loss for bounding box regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 658–666,

  26. [34]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 8

    2017

  27. [35]

    Anchor detr: Query design for transformer-based detector

    Yingming Wang, Xiangyu Zhang, Tong Yang, and Jian Sun. Anchor detr: Query design for transformer-based detector. In Proceedings of the AAAI conference on artificial intelli- gence, pages 2567–2575, 2022. 4

    2022

  28. [36]

    Hyperseg: Towards univer- sal visual segmentation with large language model

    Cong Wei, Yujie Zhong, Haoxian Tan, Yong Liu, Zheng Zhao, Jie Hu, and Yujiu Yang. Hyperseg: Towards univer- sal visual segmentation with large language model. arXiv preprint arXiv:2411.17606, 2024. 8

    2024 arXiv

  29. [37]

    Referring multi- object tracking

    Dongming Wu, Wencheng Han, Tiancai Wang, Xingping Dong, Xiangyu Zhang, and Jianbing Shen. Referring multi- object tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 14633– 14642, 2023. 1, 2, 4, 6, 7, 8

    2023

  30. [38]

    Language as queries for referring video object seg- mentation

    Jiannan Wu, Yi Jiang, Peize Sun, Zehuan Yuan, and Ping Luo. Language as queries for referring video object seg- mentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4974– 4984, 2022. 8

    2022

  31. [39]

    Visa: Reasoning video object segmentation via large language models

    Cilin Yan, Haochen Wang, Shilin Yan, Xiaolong Jiang, Yao Hu, Guoliang Kang, Weidi Xie, and Efstratios Gavves. Visa: Reasoning video object segmentation via large language models. In European Conference on Computer Vision, pages 98–115. Springer, 2024. 8

    2024

  32. [40]

    Bridging the gap between end-to-end and non-end-to- end multi-object tracking

    Feng Yan, Weixin Luo, Yujie Zhong, Yiyang Gan, and Lin Ma. Bridging the gap between end-to-end and non-end-to- end multi-object tracking. arXiv preprint arXiv:2305.12724,

    arXiv

  33. [41]

    Generalizing multiple ob- ject tracking to unseen domains by introducing natural lan- guage representation

    En Yu, Songtao Liu, Zhuoling Li, Jinrong Yang, Zeming Li, Shoudong Han, and Wenbing Tao. Generalizing multiple ob- ject tracking to unseen domains by introducing natural lan- guage representation. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, pages 3304–3...

    2023

  34. [42]

    Motrv3: Release-fetch super- vision for end-to-end multi-object tracking

    En Yu, Tiancai Wang, Zhuoling Li, Yuang Zhang, Xiangyu Zhang, and Wenbing Tao. Motrv3: Release-fetch super- vision for end-to-end multi-object tracking. arXiv preprint arXiv:2305.14298, 2023. 8

    2023 arXiv

  35. [43]

    Modeling context in referring expres- sions

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expres- sions. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 69–85. Springer, 2016. 1

    2016

  36. [44]

    Motr: End-to-end multiple- object tracking with transformer

    Fangao Zeng, Bin Dong, Yuang Zhang, Tiancai Wang, Xi- angyu Zhang, and Yichen Wei. Motr: End-to-end multiple- object tracking with transformer. In European Conference on Computer Vision, pages 659–675. Springer, 2022. 2, 8

    2022

  37. [45]

    Fairmot: On the fairness of detection and re-identification in multiple object tracking

    Yifu Zhang, Chunyu Wang, Xinggang Wang, Wenjun Zeng, and Wenyu Liu. Fairmot: On the fairness of detection and re-identification in multiple object tracking. International journal of computer vision, 129:3069–3087, 2021. 7

    2021

  38. [46]

    Bytetrack: Multi-object tracking by associating every detection box

    Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Fucheng Weng, Zehuan Yuan, Ping Luo, Wenyu Liu, and Xinggang Wang. Bytetrack: Multi-object tracking by associating every detection box. In European conference on computer vision, pages 1–21. Springer, 2022. 7

    2022

  39. [47]

    Motrv2: Bootstrapping end-to-end multi-object tracking by pre- trained object detectors

    Yuang Zhang, Tiancai Wang, and Xiangyu Zhang. Motrv2: Bootstrapping end-to-end multi-object tracking by pre- trained object detectors. In Proceedings of the IEEE/CVF 10 Conference on Computer Vision and Pattern Recognition , pages 22056–22065, 2023. 8

    2023

  40. [48]

    Bootstrapping referring multi-object tracking

    Yani Zhang, Dongming Wu, Wencheng Han, and Xingping Dong. Bootstrapping referring multi-object tracking. arXiv preprint arXiv:2406.05039, 2024. 2, 4, 6, 7, 8

    2024

  41. [49]

    Hff-tracker: A hierarchical fine-grained fusion tracker for referring multi- object tracking

    Zeyong Zhao, Yanchao Hao, Minghao Zhang, Qingbin Liu, Bo Li, Dianbo Sui, Shizhu He, and Xi Chen. Hff-tracker: A hierarchical fine-grained fusion tracker for referring multi- object tracking. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 10528–10536, 2025. 7

    2025

  42. [50]

    Deformable detr: Deformable trans- formers for end-to-end object detection

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable trans- formers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020. 2, 4, 5, 6 11

    2010 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.