{"id":"bb3e4d35-5ab9-4982-adb1-3fdf4c124013","arxiv_id":"2412.17807","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The authors propose the CRMOT task, the CRTrack benchmark of 13 scenes and 221 language descriptions, and the CRTracker method, which combines CrossMOT-style tracking with APTM text-image matching.","lead":"This paper introduces a new computer-vision task, Cross-view Referring Multi-Object Tracking, where a system must track all people matching a natural-language description across multiple synchronized camera views while keeping each person's identity consistent between views.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim rests on a custom prediction module with six manually fixed thresholds and two fusion weights, tuned without a validation split; a sensitivity check could show the margin is not robust.","rationale":"The reader's weakest_assumption already identified the absence of a validation split and the risk of threshold overfitting. My stress-test confirms this is the most load-bearing concern: the prediction module's thresholds and fusion weights are numerous, arbitrary-looking, and appear directly in the final configuration without any sensitivity study. This directly threatens the central claim because the comparison against baselines could be biased if the constants were chosen on the test set. The metric truncation issue (Eq. 4) is real but would make the reported margin conservative, not inflated, so it is not the primary threat. The baseline weakness is important but secondary: even with stronger baselines, the method could still be competitive, whereas if the thresholds are overfit, the reported performance may not generalize. The proposed concrete test would settle the concern by showing whether validation-based tuning preserves the margin. Since the paper can address this with an additional experiment, the CONDITIONAL verdict remains appropriate; no change is needed.","tokens_in":13785,"tokens_out":5788,"duration_ms":55159,"concrete_test":"Hold out one training scene (e.g., 'Floor') as a validation set. Train the model on the remaining six training scenes, then grid-search Tas in {0.3, 0.5, 0.7}, Tss in {0.6, 0.75, 0.9}, Ths in {10, 30, 50}, and beta in {0.05, 0.1, 0.2} on the validation scene. Fix the best validation configuration and evaluate on the official in-domain test set (Circle, Gate2, Side). Report CVRIDF1 and CVRMA. If the validation-tuned configuration yields CVRIDF1 more than 5 points below the published 54.88, the headline margin is not robust to hyperparameter selection and the SOTA claim is weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim that CRTracker surpasses the best adapted baseline by 31.45% CVRIDF1 and 25.83% CVRMA depends on a hand-crafted prediction module (Algorithm 1) with thresholds Tas=0.5, Tss=0.75, Ths=30, s1=3, s2=3, s3=1, plus fusion weights alpha=0.01 (Eq. 5) and beta=0.1 (Eq. 9). These constants are stated in Section 4 without any validation split or sensitivity analysis. If they were selected by inspecting the in-domain test set, the reported margin could be inflated: a small change such as Ths from 30 to 20 or beta from 0.1 to 0.3 might substantially reduce CVRIDF1, particularly on the hard 'Side' scene (37.97). The ablation only tests the prediction module on/off, not the hyperparameters, so the robustness of the headline numbers is unknown. This is the weakest link in the state-of-the-art claim because it directly affects the validity of the comparison: without validation-based tuning, we cannot rule out that the configuration was overfit to the test set.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a new task, Cross-view Referring Multi-Object Tracking (CRMOT), which extends single-view referring multi-object tracking to synchronized multi-view sequences, and introduces CRTrack, a benchmark built from DIVOTrack and CAMPUS with 13 scenes and 221 language descriptions. The authors also present CRTracker, an end-to-end method that combines CrossMOT and APTM with a hand-crafted prediction module that converts frame-to-frame association results and fusion scores into final trajectories. The paper reports that CRTracker outperforms two single-view RMOT baselines, TransRMOT and TempRMOT, adapted with MvMHAT, by large margins on in-domain and cross-domain test sets.","tokens_in":14135,"tokens_out":3652,"duration_ms":32087,"significance":"If the claims are supported, the paper makes a useful contribution by identifying a new task formulation, releasing a benchmark with 82K frames and 221 descriptions, and providing a working baseline and open code. The motivation that multi-view observation can resolve appearance ambiguity in referring tasks is plausible, and the dataset annotation pipeline using invariant attributes plus GPT-4o is a practical method. The reported gains are large, but the evaluation protocol needs strengthening regarding hyperparameter validation, baseline selection, and metric construction before the state-of-the-art claim is fully convincing.","major_comments":[{"comment":"The central SOTA claim rests on hyperparameters that are not validated. In Section 4 (Settings) and Algorithm 1, the fusion weights α=0.01, β=0.1 and thresholds Tas=0.5, Tss=0.75, Ths=30, s1=3, s2=3, s3=1 are fixed without a validation split or sensitivity analysis. Because the reported margins (31.45% CVRIDF1 and 25.83% CVRMA in-domain; 8.74% and 1.92% cross-domain) are the principal claim, this is load-bearing. Please add a validation split, report sensitivities over plausible ranges, and state how the constants were chosen.","section":"Experiments / Settings"},{"comment":"The comparison in Table 2 is limited to two single-view RMOT methods (TransRMOT and TempRMOT) each adapted with MvMHAT. The abstract and conclusion claim \"state-of-the-art\", but no cross-view MOT baseline (e.g., CrossMOT itself) or two-stage RMOT method (e.g., iKUN or LaMOT) is evaluated. If one of these were competitive, the claimed superiority could fail. Please add at least one strong cross-view baseline and, where feasible, a two-stage RMOT baseline; report results over multiple runs with error bars to establish statistical significance.","section":"Experiments / Quantitative Results (Table 2)"},{"comment":"Equation (4) truncates each CVMA at zero before averaging, with max(CVMA, 0). This discards negative values that indicate a large number of false positives, so CVRMA can overstate performance and is not strictly comparable across methods with different false-positive behavior. Please report the untruncated average as a secondary metric, and state how many language descriptions are affected by the truncation in the reported numbers.","section":"Evaluation Metrics (Eq. 4)"},{"comment":"The benchmark contribution is a core part of the paper, but the annotation section provides no measure of agreement or ambiguity. The authors state that descriptions were \"carefully\" manually checked, yet no inter-annotator agreement, description-level statistics, or examples of ambiguous descriptions are given. Please add a small human-evaluation study or at least per-description statistics to support the validity of the 221 language descriptions as ground truth.","section":"Benchmark / Dataset Annotation"}],"minor_comments":[{"comment":"The column header \"A verage Number of frames\" contains a spacing typo and should be \"Average Number of Frames per View\" for clarity.","section":"Table 1"},{"comment":"The text lists several hyperparameters in running prose; a compact notation table or list would improve reproducibility, especially since Algorithm 1 references many threshold symbols.","section":"Section 4"},{"comment":"The pipeline diagram is dense; labeling the components and adding a legend for the frozen, merging, and extraction symbols would aid readability.","section":"Figure 4"},{"comment":"The \"Questions and Replies\" section reads like reviewer responses embedded in the paper; integrating this content into a limitations or discussion section would make the paper more self-contained.","section":"Supplementary Material / Questions and Replies"}],"recommendation":"major_revision","confidential_remarks":"The paper introduces a new task and benchmark that could be of interest to the journal, but the evaluation protocol needs strengthening before acceptance. The main concern is the lack of a validation split for the manually chosen thresholds and fusion weights; a sensitivity analysis would address this. The author Q&A in the supplementary suggests the paper may have already undergone one round of review; no issue with that, but the response should be integrated formally."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The real contribution here is the task and the benchmark, not the method. CRMOT is a sensible extension of RMOT: use overlapping multi-view footage to resolve attributes that are invisible in any single view, then track every person matching a language description while keeping IDs consistent across views. The CRTrack benchmark is built from DIVOTrack and CAMPUS, with 13 scenes, 221 descriptions, and an annotation protocol that focuses on invariant attributes and uses GPT-4o plus manual checking. That is a solid, useful piece of infrastructure for the multi-camera tracking and vision-language communities. The proposed CVRIDF1 and CVRMA metrics are reasonable, even if the max(CVMA,0) truncation is a bit crude.\n\nThe method, CRTracker, is an honest engineering combination of CrossMOT for tracking, APTM for text-image matching, and a hand-designed prediction module. That is fine for a first baseline on a new benchmark, and the ablation shows the prediction module helps. However, the headline SOTA numbers — 31.45% and 25.83% over the best adapted baseline in-domain — are not well supported. The prediction module has at least six manually fixed thresholds and two fusion weights, all set without a reported validation split and without sensitivity analysis. The stress-test concern is on target: if those constants were chosen by inspecting the test set, the margin could be inflated. The baselines are also weak: only two single-view RMOT methods adapted with MvMHAT, no direct cross-view tracker like CrossMOT, and no error bars anywhere. I am not saying the result is false; I am saying the evidence presented does not nail it down.\n\nI do want to credit the authors for putting a Questions and Replies section in the supplementary. It directly addresses the \"why cross-view\" motivation and the single-reference-multiple-people issue, and the supplementary openly admits the method suffers ID switching on the 6330-frame ParkingLot clip. That is the sort of candor that makes me trust the benchmark more than the SOTA claim.\n\nBottom line: this paper deserves peer review and likely publication after major revision. The benchmark and task definition are valuable; the evaluation needs a validation split, sensitivity analysis for the thresholds, error bars, and at least one stronger cross-view baseline. I would bring it to the reading group to discuss benchmark design and threshold-tuning pitfalls. I would cite it for the benchmark, not for the method's performance.\n\nRecommendation: engage seriously; require the evaluation fixes before it is accepted.","headline":"A new cross-view referring MOT task and benchmark that are worth having, wrapped around a method whose SOTA claim rests on manually tuned thresholds without a validation split.","tokens_in":14528,"tokens_out":1732,"would_cite":true,"duration_ms":20171,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes that referring multi-object tracking should operate on synchronized cross-view videos instead of a single view, and reports that its CRTracker method outperforms single-view baselines by 31.45% in CVRIDF1 on the new…","keywords":["cross-view multi-object tracking","referring multi-object tracking","language-guided tracking","multi-view video understanding","benchmark construction","attribute-based annotation","prediction module","identity consistency"],"falsifier":"Re-run the comparison with a single-view referring tracker that uses a more powerful vision-language backbone (e.g., grounded-language models) plus the same MvMHAT association, or re-tune the prediction module thresholds on a held-out validation split instead of the test set; if CVRIDF1 margin drops below the claimed 31.45% or the ranking changes, the central claim would be weakened.","tokens_in":13578,"feed_emoji":"🎥","tokens_out":5584,"duration_ms":44605,"temperature":0.7,"pith_summary":"The paper proposes that referring multi-object tracking should be done across synchronized overlapping camera views rather than within a single view. In a single view, the attributes named in a language description—coat color, carried items, shoes—are often occluded or out of frame, so a tracker cannot reliably tell which person the text refers to. The authors introduce the Cross-view Referring Multi-Object Tracking (CRMOT) task, build the CRTrack benchmark with 13 scenes and 221 language descriptions, and present CRTracker, an end-to-end method that combines a cross-view multi-object tracker with a vision-language matcher and a prediction module. On CRTrack, CRTracker outperforms the best single-view based baseline by 31.45% CVRIDF1 in-domain and 8.74% cross-domain. If correct, this establishes cross-view information as a necessary ingredient for robust language-conditioned tracking.","feed_headline":"Language-guided tracking gains 31% with cross-view","feed_subtitle":"A new CRMOT benchmark and CRTracker match people across cameras, fixing hidden attributes in referring tracking.","key_machinery":"The load-bearing mechanism is the fusion of cross-view tracking with language–attribute matching, capped by a prediction module. CRTracker reuses CrossMOT's detection and Re-ID heads (single-view, cross-view, full), merges APTM image encodings with the full Re-ID feature, and computes both text scores and attribute scores through APTM's cross encoder; the two scores are fused with a weight and fed, together with frame-to-frame association tracks, into a prediction module. That module treats tracks as detections and fusion scores as confidences, then uses three thresholds (average fusion score, single-view fusion score, hit score) to keep or drop each track. The same pipeline works for cross-domain evaluation without adaptation.","core_discovery":"The central claim is that cross-view observations solve the failure mode where a person's appearance in one camera does not show all attributes described in the language query. The authors argue that by associating the same person across overlapping views, the network can assemble a complete appearance, judge whether the description matches, and keep one identity across views. As evidence, they construct CRTrack from DIVOTrack and CAMPUS, annotate objects with invariant attributes (headwear, coat, trousers, shoes, held items, transportation), generate descriptions with GPT-4o, and define CVRIDF1 and CVRMA metrics that average per-language cross-view IDF1 and matching accuracy. On the in-domain test set, CRTracker reaches 54.88% CVRIDF1, far above the 23.30% of TransRMOT and 22.18% of TempRMOT; cross-domain, it still leads with 12.52% vs 3.78%. The proposed prediction module, which converts frame-to-frame association results and fused text–attribute scores into output trajectories, accounts for a 7.34% CVRIDF1 gain in the ablation.","pith_inferences":["Because the prediction module's thresholds are fixed constants (Tas=0.5, Tss=0.75, Ths=30), a natural extension is to learn them from a validation split; if the thresholds are overfit to the test scenes, the real cross-domain margin may be smaller than reported.","The same cross-view aggregation idea could be applied to other vision-language grounding tasks such as referring expression segmentation or visual question answering under occlusion, not just tracking.","The paper evaluates only two single-view baselines; the claim of 'state of the art' would be stronger with baselines from the cross-view MOT literature (e.g., ReST or other multi-camera trackers) adapted to the referring task, which the authors did not include."],"forward_implications":["CRMOT becomes a new evaluation task: any future method that wants to score on CRTrack must handle synchronized multi-view input and per-view identity consistency, not just single-view referring.","The observed margins imply that single-view referring trackers, when evaluated on occluded scenes, are penalized by invisible attributes; adding cross-view association to them (as done with MvMHAT) is not enough—the architecture needs to integrate cross-view appearance encoding.","The prediction module's threshold-based filtering suggests that trajectory confidence can be derived from per-view fusion scores, which could be transplanted to other multi-view tracking or retrieval pipelines.","The benchmark's long sequences (up to thousands of frames per description) stress temporal consistency, so future work can focus on long-range identity retention."],"supporting_citations":[{"why":"Supplies the DIVOTrack dataset and the CrossMOT baseline method on which CRTracker's detection and Re-ID heads are built.","marker":"Hao et al. 2024"},{"why":"Supplies the CAMPUS dataset used for the cross-domain test scenes in CRTrack.","marker":"Xu et al. 2016"},{"why":"Supplies the MvMHAT cross-view association algorithm used both to adapt single-view baselines and in CRTracker's inference.","marker":"Gan et al. 2021"},{"why":"Supplies the APTM framework for attribute prompt and text matching that CRTracker uses to compute text and attribute scores.","marker":"Yang et al. 2023"},{"why":"TransRMOT, the end-to-end single-view referring tracker used as the primary baseline in both in-domain and cross-domain comparisons.","marker":"Wu et al. 2023"},{"why":"TempRMOT, another end-to-end referring tracker baseline whose performance is compared against CRTracker.","marker":"Zhang et al. 2024"}],"fun_headline_variants":["Cross-view tracking reads hidden looks from other cameras","Cross-view boosts referring tracking by 31 points","One camera misses details; second view fills them in","New benchmark and model for cross-view referring MOT"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported edge assumes that adding MvMHAT to the two single-view trackers makes them fair, strong baselines, and that the fixed thresholds in the prediction module were not chosen by looking at the test set; if a stronger baseline or a validation-based threshold selection were used, the margin could shrink.","fun_headline_variants_meta":{"raw":{"variants":["Cross-view tracking reads hidden looks from other cameras","Cross-view boosts referring tracking by 31 points","One camera misses details; second view fills them in","New benchmark and model for cross-view referring MOT"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000291,"raw_usage":{"total_tokens":1729,"prompt_tokens":1004,"completion_tokens":725,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":665}},"tokens_in":620,"tokens_out":725,"duration_ms":7476,"temperature":1.0,"reasoning_tokens":665,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:08:11.845654+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the comparison with a single-view referring tracker that uses a more powerful vision-language backbone (e.g., grounded-language models) plus the same MvMHAT association, or re-tune the prediction module thresholds on a held-out validation split instead of the test set; if CVRIDF1 margin drops below the claimed 31.45% or the ranking changes, the central claim would be weakened.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the DIVOTrack dataset and the CrossMOT baseline method on which CRTracker's detection and Re-ID heads are built."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MvMHAT cross-view association algorithm used both to adapt single-view baselines and in CRTracker's inference."}],"review_version":1}