{"id":"e9fafd20-c1b5-4d35-b8fb-205123a6db75","arxiv_id":"2505.23704","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"CLDTracker improves language-guided visual tracking by building a bag of diverse textual descriptions from CLIP and GPT-4V and updating them across frames, achieving top normalized precision on five benchmarks and best AO on GOT-10k.","lead":"CLDTracker is a visual object tracker that enriches the description of the tracked target with multiple text cues, including class names, low-level attributes, detailed GPT-4V captions, and semantic context, and it updates these cues over time during tracking. On six standard benchmarks it reports gains over prior language-aware trackers, such as CiteTracker, with the largest improvements on LaSOT and TNL2K.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GOT-10k results may violate the official training protocol, undermining the headline SOTA claim on that benchmark.","rationale":"The reader flagged the GOT-10k protocol ambiguity in passing but identified the static Bt and dictionary coverage as the weakest assumption. I see the GOT-10k training protocol issue as more concrete and more damaging: it directly affects a headline result (best AO and SR0.75 on GOT-10k) and is checkable from the released code. The static Bt limitation is acknowledged by the authors and is a soft spot, but the paper provides ablations (Tables 4, 5, 6, 12) showing TTFUM and prompt robustness, and the failure case is explicitly shown; it does not invalidate the central claim. The GOT-10k issue, if it lands, removes one of the six benchmark claims and distorts comparisons with protocol-compliant trackers. I therefore keep the reader's CONDITIONAL verdict (no change) because the concern is addressable by disclosing or re-running training, and the remaining benchmark results may still support the method's value. My disagreement is with the relative weighting: the GOT-10k protocol is more immediately decisive than the static Bt for assessing the paper's headline quantitative claims.","tokens_in":41560,"tokens_out":5407,"duration_ms":50927,"concrete_test":"Inspect the released training configuration (e.g., data list files) to determine whether the GOT-10k evaluation model was trained solely on the GOT-10k training split or on a mixture including TrackingNet, LaSOT, and COCO2017. If the mixture was used, retrain CLDTracker from scratch on GOT-10k only and re-evaluate on the GOT-10k test set; if the AO drops below CiteTracker's 74.7%, the SOTA claim on GOT-10k is invalid.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central SOTA claim includes the highest AO (77.5) and SR0.75 (75.6) on GOT-10k. Section 4.1 states: 'We trained CLDTracker following standard VOT practice, using the training splits of GOT-10k, TrackingNet, and LaSOT. In addition, we also utilized COCO2017.' GOT-10k's official protocol explicitly prohibits using external datasets for training when reporting results on its test set. Section 4.2 acknowledges this requirement but says only that the model was evaluated 'under similar conditions'—a phrase that does not resolve the contradiction. If the GOT-10k evaluation used the mixed-data model, then its results are not comparable to protocol-compliant trackers such as CiteTracker and HIPTrack, and the claim of 'best AO and SR0.75 on GOT-10k' is unsupported. This is the most load-bearing concern because it invalidates a headline quantitative result rather than a peripheral discussion point. The paper must disclose the precise training data used for the GOT-10k model and, if external data was used, retrain under the protocol or withdraw the GOT-10k claims.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"CLDTracker proposes a vision-language tracker that extends the OSTrack/CiteTracker architecture with a bag of textual descriptions (Bt) built offline from the first frame via predefined class/attribute dictionaries, CLIP matching, GPT-4V captions, and semantic/contextual enrichment, plus a Temporal Text Feature Update Mechanism (TTFUM) that averages search-region text features over a temporal window. The paper reports results on LaSOT, LaSOText, TrackingNet, TNL2K, OTB99-Lang, GOT-10k, and VastTrack, claiming state-of-the-art or top-tier performance, and provides extensive ablations of each component, robustness tests to noisy text, and an efficiency analysis.","tokens_in":41735,"tokens_out":4838,"duration_ms":45807,"significance":"The idea of using a diverse, multi-perspective text bag rather than a single static phrase is timely and the experimental infrastructure is substantial: 38 competitors, multiple benchmarks, component-wise ablations, robustness checks, and a useful unseen-class generalization test on VastTrack (Table 15). The efficiency improvement over CiteTracker (35.31 vs. 15.35 FPS with lower MACs) is also a concrete practical strength. However, the 'SOTA' claim is not uniform: the method is second on several benchmarks and metrics. More importantly, the GOT-10k training-protocol ambiguity, the absence of error bars, and the fact that code and Bt are not yet available prevent the results from being fully verified at this stage. If the protocol issue is resolved and the artifacts are released, the work would be a solid contribution to VLM-based tracking.","major_comments":[{"comment":"Section 4.1 states that CLDTracker was trained on GOT-10k, TrackingNet, LaSOT, and COCO2017, while Section 4.2 acknowledges that GOT-10k's official protocol prohibits external training data and only says the model was evaluated 'under similar conditions.' If the reported GOT-10k AO=77.5 and SR0.75=75.6 were produced by the mixed-data model, they are not comparable to protocol-compliant trackers such as CiteTracker (AO 74.7) or HIPTrack (AO 77.4), and the claim of 'highest scores in both AO and SR0.75' is unsupported. The authors must disclose the exact training data used for the GOT-10k model and, if external data were used, retrain under the official protocol or withdraw the GOT-10k claims.","section":"Sec. 4.1 and Sec. 4.2, Table 1"},{"comment":"The margins separating CLDTracker from the runner-up are small on several decisive metrics (e.g., GOT-10k AO 77.5 vs. HIPTrack 77.4; LaSOText S 53.1 vs. VLT_OST-384 53.5; OTB99-Lang S 77.8 vs. VLT_OST-384 78.2). Single-run evaluations with no variance estimates or significance tests cannot support 'consistently top-tier' or 'best' claims. Please report mean and standard deviation over at least three seeds, or provide official leaderboard significance analysis if available.","section":"Table 1"},{"comment":"The abstract and introduction claim 'temporally-adaptive vision-language representations,' but the Bt is constructed once from the first frame and TTFUM only averages features from the same static bag over a temporal window (Eq. 8). Section 4.9 explicitly acknowledges that 'the use of a static Bt, generated only from the first frame, limits adaptability to significant appearance changes over time.' This limitation weakens the temporal-modeling contribution as stated; either the method should regenerate or augment descriptions online, or the claims should be explicitly scoped to 'temporally refined text features from a static bag.'","section":"Sec. 3.2.1 and Sec. 4.9"},{"comment":"The Bt is filtered using CLIP similarity to the first-frame visual feature FI with threshold tau=0.8, and the same CLIP text encoder and FI-derived features are used during inference to select prompts via Eq. (6). This creates a potential selection bias: the Bt is optimized to match CLIP's own embedding rather than to be semantically faithful. A control experiment using human-written captions or an independent captioning model not filtered by CLIP would help show that the gains come from comprehensive language grounding rather than from exploiting CLIP's self-consistency.","section":"Sec. 3.2.1(e) and Eq. (6)"}],"minor_comments":[{"comment":"There is a typo in the introduction: 'Thereofore' should be 'Therefore.'","section":"Sec. 2.1"},{"comment":"The symbol tau is used both as the synonym-retrieval threshold in Eq. (2) and as the filtering threshold in Sec. 3.2.1(e); please rename one to avoid confusion.","section":"Sec. 3.2.1(d) and Sec. 3.2.1(e)"},{"comment":"Lines 6-7 write the class and attribute selection as a joint argmax over (j,k), but the objective adds two separate similarity terms; clarify that the class and attribute are selected independently.","section":"Algorithm 1"},{"comment":"The abstract says six benchmarks, but Section 4.2 also lists VastTrack as an evaluation benchmark; please reconcile the count or describe VastTrack as an additional generalization test.","section":"Sec. 4.2"},{"comment":"The abstract and the Research Data Availability statement say code and models are publicly available, while Section 4.9 says they 'will be made available'; please make the GitHub repository and Bt accessible at the time of publication or state a concrete release date.","section":"Abstract and Sec. 4.9"},{"comment":"The column header 'Baseline VL Model (WS)' is visually ambiguous because the same header spans all WS blocks; please add explicit WS labels under each block or restructure the table so the w/ and w/o Prompt Adapter configurations are immediately clear.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The GOT-10k training-protocol issue is the main barrier: if the authors cannot clarify whether external data were used for the reported GOT-10k numbers, the headline SOTA claim on that benchmark is not verifiable. The paper otherwise contains a solid set of ablations, but the static-Bt limitation and the absence of error bars should be addressed before acceptance. I would also verify the GitHub link during the revision process, since the abstract promises availability that the text later qualifies."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is good: build a bag of diverse textual descriptions (class, attributes, GPT-4V, semantic/contextual) and update them over time via TTFUM. This is a clear step beyond CiteTracker's fixed attribute set and JointNLT's single text. The ablations are the best part—they systematically test each component, window size, prompt adapter, and robustness to noisy text. The attribute-wise analysis and efficiency breakdown are also useful. This is a serious piece of work in the language-guided tracking niche. The main problem is the GOT-10k results. Section 4.1 says the model was trained on GOT-10k, TrackingNet, LaSOT, and COCO2017. Section 4.2 acknowledges GOT-10k's official protocol prohibits external training data, then says they evaluated 'under similar conditions'—which does not resolve the contradiction. If the GOT-10k numbers come from the mixed-data model, they are not comparable to CiteTracker or HIPTrack, and the headline claim of best AO on GOT-10k is unsupported. This needs a straight answer: retrain under the protocol or explicitly state the comparison is not protocol-compliant. This is the most load-bearing issue. A second issue: the abstract and data availability statement promise code and the Bt, but the GitHub link appears to have nothing. For a method that depends on hand-merged dictionaries and GPT-4V outputs, that is a real reproducibility gap. Smaller issues: no error bars, and 'SOTA' overstates the ranking (second on three benchmarks). The paper itself admits this in Section 4.5, but the abstract does not. The hard selection of one description via argmax (Eq. 6-7) is never ablated; I would like to see a weighted-average comparison. The static-Bt limitation is honestly acknowledged, so that is fine. The mild self-referentiality of using CLIP to generate and filter text is worth mentioning but not a red flag. Who is this for: people working on vision-language tracking, and anyone looking for a careful ablation template. The paper should go to peer review, but it needs a major revision: fix the GOT-10k protocol issue, make code and Bt available, and tone down the SOTA claim. The core contribution is worth publishing.","headline":"Solid contribution with strong ablations, but the GOT-10k protocol contradiction and missing code/data keep me from trusting the headline SOTA claim.","tokens_in":42406,"tokens_out":4710,"would_cite":true,"duration_ms":43029,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CLDTracker claims that a comprehensive bag of textual descriptions, updated over time, lets vision-language trackers outdo 38 prior methods on six benchmarks.","keywords":["visual object tracking","vision-language models","comprehensive language description","bag of textual descriptions","temporal text feature update","prompt adapter","GPT-4V","CLIP"],"falsifier":"Take a set of sequences where a target changes appearance irreversibly after initialization (for example, a player changes jersey, or a vehicle is repainted) and a same-category distractor appears later. If CLDTracker's success on that subset is no better than CiteTracker's, or if injecting a random first-frame description into the bag produces a collapse comparable to the L0 zero-text ablation, then the claim that comprehensiveness plus TTFUM repairs drift is not supported.","tokens_in":41301,"feed_emoji":"🎯","tokens_out":7252,"duration_ms":65817,"temperature":0.7,"pith_summary":"CLDTracker claims that visual object tracking improves when the target is given a comprehensive, multi-perspective text description instead of the single phrase most benchmarks provide. From the first frame, the method builds a bag of textual descriptions covering class name, attribute phrase, GPT-4V-generated caption, and semantic/context enrichment, then aligns these texts with visual features and adaptively updates their weights over time. Across six benchmarks, the paper reports the best success score on LaSOT (74.0 AUC), the best normalized precision on all five language-annotated benchmarks, and the best AO (77.5) and SR0.75 (75.6) on GOT-10k among the 38 compared methods. The central claim is that a static, single-sentence description is a bottleneck in vision-language tracking, and that comprehensiveness plus temporal updating removes that bottleneck.","feed_headline":"One tracker tops six benchmarks by writing richer target descriptions","feed_subtitle":"A first-frame caption bag plus temporal text updates lifts LaSOT AUC to 74.0 and GOT-10k AO to 77.5.","key_machinery":"The carrying object is the bag of textual descriptions $\\mathcal{B}_t$, built offline from the first frame: the best CLIP-matched class and attribute phrase, a GPT-4V caption, and a semantic/context enrichment containing synonyms, perturbed forms, task-specific phrases, and a high-level concept. Two learned modules carry the argument: a Prompt Adapter (CoCoOp-style conditioned context vectors) selects the most relevant text per frame, and the Temporal Text Feature Update Mechanism (TTFUM) computes attention weights from the absolute difference between the exemplar text and the average of past search-region texts, producing $T_{\\mathrm{att}}$. The final correlation $\\mathrm{Corr}(F_{\\mathrm{frame}}, \\tilde{T}_e) = (1 + T_{\\mathrm{att}}(\\tilde{T}_e)) \\ast F_{\\mathrm{frame}}$ uses the text feature as a convolution kernel over the visual feature map, which is what converts language into a spatially localized target prediction.","core_discovery":"On its own terms, the paper's discovery is that many-to-one text–image alignment beats one-to-one alignment in tracking. CLDTracker generates a bag of texts from the first frame, selects the most relevant one per frame with a learned prompt adapter, and continuously reweights past textual features through a temporal update mechanism; the chosen text embedding is then used as a convolution kernel against the visual feature map. The paper reports consistent gains over CiteTracker, its vision-language baseline, and argues that these gains show richer language inputs supply disambiguating cues where short benchmark phrases fail. It also reports strong generalization on VastTrack, including on classes absent from the constructed dictionaries, positioning the method as capable of tracking novel categories without retraining.","pith_inferences":["A testable extension follows from the paper's own failure case: if the bag of descriptions were regenerated periodically instead of only from the first frame, long-occlusion sequences should improve, which is exactly what the limitation section predicts.","The TTFUM attention formula treats textual grounding as a dynamical state, suggesting the same averaging-plus-difference scheme could be applied to streaming open-vocabulary detection or re-identification where object appearance drifts.","The dictionary coverage constraint of 940 classes and 23,899 attributes could be relaxed by replacing fixed vocabularies with on-the-fly retrieval, which might extend the reported gains to truly arbitrary objects.","If the paper's mechanism is the real driver, then holding the visual branch fixed and varying only the richness of text inputs should produce monotone accuracy gains, which is an ablation not reported in the current paper."],"forward_implications":["If correct, language-guided tracking no longer needs hand-written per-frame sentences: one offline-generated bag plus a temporal update matches or beats manual annotations on TNL2K and OTB99-Lang.","The normalized-precision gains imply the method is more robust to annotation style, not just to target appearance, so it could absorb noisy or paraphrased descriptions in deployed systems.","The computational path of pre-encoded text features reused across frames implies comprehensive text can be added at 35.31 FPS with 54.25G MACs, roughly a third of the cost of the CiteTracker baseline.","Generalization to 95 unseen VastTrack classes suggests the pipeline can track novel categories without retraining, as long as the prompt adapter can condition on visual features alone."],"supporting_citations":[{"why":"Supplies the CiteTracker baseline and the visual–text correlation fusion scheme that CLDTracker extends, and its attribute-only descriptions are the comparison point throughout.","marker":"Li et al. (2023b)"},{"why":"Provides the OSTrack visual branch, ViT backbone, and prediction head that CLDTracker reuses for target state estimation.","marker":"Ye et al. (2022)"},{"why":"CLIP encoders perform class and attribute dictionary matching and encode the bag of textual descriptions.","marker":"Radford et al. (2021)"},{"why":"GPT-4V generates the detailed captions, task-specific phrases, and high-level concepts added to the bag.","marker":"Yang et al. (2023)"},{"why":"Source of the hand-merged attribute dictionary and a major vision-language tracking comparator in the evaluation.","marker":"Guo et al. (2024)"},{"why":"CoCoOp supplies the instance-conditioned prompt learning used by the Prompt Adapter to select descriptions per frame.","marker":"Zhou et al. (2022a)"},{"why":"LaSOT provides training data and the primary long-term benchmark, and the paper's headline success score is measured on it.","marker":"Fan et al. (2019)"},{"why":"GOT-10k supplies the zero-overlap training and evaluation protocol and the AO/SR metrics where CLDTracker reports its best result.","marker":"Huang et al. (2021)"},{"why":"VastTrack supplies the unseen-class test set used to claim generalization beyond the predefined dictionaries.","marker":"Peng et al. (2024)"}],"fun_headline_variants":["Richer text descriptions boost visual tracking on six benchmarks","Text bag plus temporal updates powers SOTA tracking","CLDTracker: language-driven tracking with richer descriptions","Six benchmark wins via rich language descriptions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a bag of texts generated once from the first frame, together with TTFUM's averaging of past frame features, remains a faithful description of the target for the whole sequence; if the first-frame descriptions are wrong or go stale during long occlusion, the correlation kernel in Eq. (9) is guided by misleading language.","fun_headline_variants_meta":{"raw":{"variants":["Richer text descriptions boost visual tracking on six benchmarks","Text bag plus temporal updates powers SOTA tracking","CLDTracker: language-driven tracking with richer descriptions","Six benchmark wins via rich language descriptions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000163,"raw_usage":{"total_tokens":1263,"prompt_tokens":984,"completion_tokens":279,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":220}},"tokens_in":600,"tokens_out":279,"duration_ms":3273,"temperature":1.0,"reasoning_tokens":220,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:39:09.708162+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of sequences where a target changes appearance irreversibly after initialization (for example, a player changes jersey, or a vehicle is repainted) and a same-category distractor appears later. If CLDTracker's success on that subset is no better than CiteTracker's, or if injecting a random first-frame description into the bag produces a collapse comparable to the L0 zero-text ablation, then the claim that comprehensiveness plus TTFUM repairs drift is not supported.","supporting_citations":[{"cited_title":", author Kim, J","cited_arxiv_id":null,"evidence_quote":"CLIP encoders perform class and attribute dictionary matching and encode the bag of textual descriptions."},{"cited_title":", author Zhang, Z","cited_arxiv_id":null,"evidence_quote":"Source of the hand-merged attribute dictionary and a major vision-language tracking comparator in the evaluation."},{"cited_title":", author Lin, L","cited_arxiv_id":null,"evidence_quote":"LaSOT provides training data and the primary long-term benchmark, and the paper's headline success score is measured on it."},{"cited_title":", author Gao, J","cited_arxiv_id":null,"evidence_quote":"VastTrack supplies the unseen-class test set used to claim generalization beyond the predefined dictionaries."}],"review_version":1}