{"id":"67ccd9f7-98aa-4d93-bcce-c8c14a167ac1","arxiv_id":"2608.10932","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CamDistill matches direct geometric feature injection for temporally grounded, compositional camera-motion recognition while removing the 3D teacher at inference, and CamChoreo provides the first real-video benchmark for this task.","lead":"This paper introduces a new benchmark where camera movements in real videos are labeled as time intervals with multiple simultaneous motions, plus a training method that learns geometry-aware camera tokens without running a heavy 3D model at test time. The method matches direct geometric feature injection while adding almost no inference cost, advancing camera-motion understanding from clip-level tags to fine-grained temporal recognition.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No inter-annotator agreement is reported for CamChoreo, and unresolved clips are discarded; systematic annotator bias on hard pairs (Dolly/Zoom, Pan/Truck) could distort every ranking, including the 67.5-vs-67.5 match that anchors the central claim.","rationale":"The reader's weakest_assumption identifies exactly the load-bearing gap I find. The central empirical claim—CamDistill matches CamInject—is only as strong as the benchmark ground truth used to measure both methods. An unmeasured inter-annotator agreement leaves open the possibility that hard perceptual distinctions are systematically biased; because both methods are trained and evaluated on the same labels, the exact match could be an artifact of that shared bias rather than evidence that distilled camera tokens preserve geometric knowledge. I considered two alternative concerns: (1) GCTE capacity confounding the distillation signal, and (2) absence of variance estimates. The capacity concern is partially addressed by the lambda_cam sweep in Figure 4 and the VGGT-vs-VGGT-Omega teacher ablation in Table 5, though a fully explicit lambda=0 control with numbers would strengthen it. The variance concern is real but secondary; a 0.5 F1 shift would weaken but not destroy the 'matches' claim. The annotation-reliability concern is therefore the most load-bearing, and the proposed re-annotation test would settle it. Since this is exactly the basis for the reader's CONDITIONAL verdict, no verdict adjustment is needed.","tokens_in":24971,"tokens_out":7555,"duration_ms":77819,"concrete_test":"Re-annotate a random subset of at least 300 CamChoreo clips with two independent annotator panels using the same protocol but no consensus step; compute per-label Krippendorff's alpha and boundary agreement at 0.1s tolerance, plus the confusion matrix for Dolly/Zoom and Pan/Truck. If alpha for these hard pairs is below 0.6 or pairwise agreement is below 70%, re-evaluate CamDistill vs CamInject on the high-agreement subset; if the 67.5-vs-67.5 gap moves by more than 1 F1 point, the central match claim is not robust to label reliability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's load-bearing claim—CamDistill matches CamInject without the 3D teacher at inference—is an empirical comparison on CamChoreo. The ground truth for that comparison is produced by five annotators who mark motion-consistent intervals and direction-aware labels, then cross-review and discard unresolved clips (Section 3.2; Appendix D). No inter-annotator agreement statistic (e.g., Cohen's kappa or Krippendorff's alpha) is reported for either segment boundaries or label sets. The taxonomy explicitly requires distinguishing Dolly from Zoom by parallax and Pan from Truck by translation, cues that the paper itself shows are hard for models (Section 5.3, Appendix M). If annotators systematically disagree on these pairs, the consensus label may reflect the majority annotator's perceptual bias rather than ground truth. Because both CamDistill and CamInject are trained and evaluated on the same labels, a shared bias would preserve their apparent match while invalidating the benchmark's claim to measure geometric camera-motion understanding. The absence of any variance estimate across training runs compounds this: the exact 67.5-vs-67.5 equality in Table 2 could also be within run-to-run noise. The concern is not that the labels are wrong; it is that the paper provides no evidence they are right enough to support the central comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formulates camera-motion understanding as temporally grounded compositional recognition, where a video must be segmented into motion-consistent intervals and each interval labeled with the complete set of active direction-aware movements. It introduces CAMCHOREO, a benchmark of 4,229 real single-shot clips from YouTube with 8,591 segments and 14,258 motion instances over a 20-label taxonomy, annotated by five experts. It also proposes CAMDISTILL, which attaches a lightweight Geometry-aware Camera Token Extractor (GCTE) to a frozen vision encoder, trains it to regress VGGT-Omega camera tokens via a cosine-distance distillation loss, and removes the teacher at inference. The central empirical claim is that CAMDISTILL matches CAMINJECT, direct injection of teacher features, on CAMCHOREO (67.5 vs 67.5 micro F1 on the 4B backbone; 67.8 vs 68.3 on 8B) while adding only about 0.1 s per clip and 1.8 GB memory over the base model. The paper also reports external transfer to CameraBench and CMVQA and extensive ablations.","tokens_in":25254,"tokens_out":8475,"duration_ms":71377,"significance":"If the central claim holds, the paper is a solid contribution: it defines a more realistic camera-motion task, releases a new benchmark, and shows that geometric knowledge from a 3D foundation model can be compressed into lightweight per-frame tokens and discarded at inference. The strengths are the controlled comparison (same backbone, same training data, only the teacher's presence at inference varied), the lambda_cam=0 partial control in Figure 4, the external transfer evaluation, and the careful latency/memory measurements. The main weakness is that benchmark reliability is not quantified and all experiments are single runs, leaving the headline 'matches' claim under-supported.","major_comments":[{"comment":"The benchmark's ground truth is load-bearing for every reported model ranking, including the central CamDistill-equals-CamInject comparison, yet no inter-annotator agreement statistic is provided. The protocol describes five annotators, cross-review, and discarding unresolved clips, but report no Cohen's kappa or Krippendorff's alpha for either segment boundaries or the 20-label sets, nor a reliability analysis for the hard distinctions (Dolly versus Zoom, Pan versus Truck) that the taxonomy explicitly requires. Additionally, Appendix C says annotators 'independently replace the pseudo labels,' which suggests they saw model-generated suggestions; this could introduce anchoring bias. Please report agreement statistics on a per-family or per-pair basis, clarify whether pseudo-labels were visible during annotation, and discuss the selection effect of discarding unresolved clips.","section":"Section 3.2 / Appendix C-D"},{"comment":"All results are single runs, and the claim that CamDistill 'matches' CamInject is based on differences of 0.0 to 0.5 micro F1 points (67.5 vs 67.5 for 4B; 67.8 vs 68.3 for 8B) and 0.4 to 0.5 SegDet@0.5 points. Without multiple seeds, confidence intervals, or a paired significance test, these differences are within typical run-to-run noise, so the equality could be coincidental. Please provide variance estimates or significance tests for the key CamDistill-versus-CamInject comparisons on the 4B and 8B backbones.","section":"Section 5.2, Table 2"},{"comment":"The unqualified abstract statement that CamDistill 'matches the accuracy of direct feature injection' overstates the 8B results: on the 8B backbone CamDistill trails CamInject by 0.5 micro F1 and 1.3 macro F1 (Table 2). The body text acknowledges this, but the abstract and conclusion should either qualify the claim (e.g., 'matches on the 4B backbone with a small gap at 8B') or provide statistical evidence that the 8B gap is within noise.","section":"Abstract / Section 5.2"}],"minor_comments":[{"comment":"The Cam-Motion-7B row states that the model yields valid outputs for only 19 of 4,229 clips; please clarify whether the reported scores are computed on those 19 clips alone and specify which subset, as the current wording is ambiguous.","section":"Section 5.2, Table 2"},{"comment":"The x-axis includes a tick at lambda_cam = 0 but this value is not labeled; please label it explicitly and report the lambda=0 result in the text or a table, since it is the key control for isolating the distillation objective.","section":"Figure 4"},{"comment":"The notation c_i^{(m)}, x_i^{(ell_m)}, and the block index m should be defined in the text before their first use in the equations to avoid ambiguity.","section":"Section 4.2, Eqs. (2)-(3)"},{"comment":"Line 7, 'Save the post-frame-wise cross-attention tokens in the last block as the frame-level branch,' is ambiguous because the loop notation suggests it executes every iteration; reword to state that the save occurs after the final block.","section":"Appendix K, Algorithm 1"},{"comment":"The manuscript inconsistently uses both CAMCHOREO and CamChoreo (and similarly CAMDISTILL/CamDistill); please unify the capitalization style throughout.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The main reason for major revision is the missing reliability evidence for the benchmark and the absence of variance estimates for the central comparison; both are fixable. I also recommend that the authors temper the headline 'matches' claim in the abstract, as the 8B gap is not trivially negligible without statistical support."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The benchmark is the real contribution. CamChoreo is the first real-video camera-motion benchmark I know of that combines variable-length temporal segments with direction-aware multi-label annotations, and the annotation statistics show that temporal and compositional structure is common, not rare. The task formulation is sensible and the gap it fills is real. The method comparison is also well designed: same backbone, same training data, same prompt, only the presence of the 3D teacher at inference differs. The distillation-weight sweep with lambda=0 as a control, the external transfer results on CameraBench and CMVQA, and the per-class error decomposition all give the central claim a fair amount of support.\n\nThe soft spot is the one the stress-test note flags: there is no inter-annotator agreement statistic for CamChoreo. Five annotators label every clip, cross-review disagreements, and discard unresolved clips, but the paper never reports kappa or anything similar. Given that the taxonomy requires distinguishing Dolly from Zoom by parallax and Pan from Truck by translation, and the paper's own error analysis shows those are exactly the confusions the model makes, systematic annotator disagreement on those pairs would propagate directly into the benchmark labels. Both methods are trained and evaluated on the same labels, so the CamDistill-vs-CamInject comparison would survive shared label bias as an internal result, but the stronger claim that CamChoreo measures geometric camera-motion understanding depends entirely on annotation reliability. That needs to be quantified. Also, all numbers are single runs with no variance, so the exact 67.5-vs-67.5 match in Table 2 could be within run-to-run noise. That is a lesser concern because the gap is tiny across many metrics, not just one, but it should still be reported.\n\nThe other limitations are minor: no artifacts are shipped yet, and the training set is license-restricted, though the benchmark annotations are promised. The geometry-only baseline is not tuned and its poor SegDet score is expected; I would not over-read it.\n\nWho this is for: people working on video MLLMs, camera understanding, spatial intelligence, and geometry distillation. They will get a solid task definition, a useful benchmark, and a clearly explained distillation mechanism. The paper deserves a serious referee; my verdict would be conditional accept with a request for inter-annotator agreement, multi-run variance, and artifact release. This is exactly the kind of work that should go through peer review rather than be desk-rejected.","headline":"A genuinely useful benchmark and a well-controlled distillation comparison, but the missing inter-annotator agreement and single-run results leave the anchor claim softer than the tables suggest.","tokens_in":25788,"tokens_out":1710,"would_cite":true,"duration_ms":18007,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Camera-motion model keeps 3D teacher's accuracy after dropping it","keywords":["camera motion","temporal grounding","compositional recognition","multimodal large language model","knowledge distillation","3D geometry","video understanding","benchmark"],"falsifier":"Re-annotate a random subset of CamChoreo (say 300 clips) with a fresh expert panel following the same protocol and measure inter-annotator agreement on boundaries and label sets, especially for Dolly/Zoom, Pan/Truck, and compound labels. If per-label agreement falls below a high threshold, the benchmark's difficulty numbers and the distillation parity claim lose their footing. A second check: train a linear probe on CamDistill's student tokens to predict the teacher's camera pose; if the probe achieves low accuracy, the tokens may not carry the geometry the paper attributes to them.","tokens_in":74,"feed_emoji":"🎥","tokens_out":5327,"duration_ms":164408,"temperature":0.7,"pith_summary":"The paper argues that camera-motion understanding should be treated as a temporally grounded, compositional recognition problem: a model must split a shot into motion-consistent intervals and, for each interval, list every simultaneous movement (pan, dolly, tilt, zoom, and so on). To support this, it introduces CamChoreo, a benchmark of 4,229 real single-shot clips with expert-annotated segments, 20 direction-aware labels, and compound motion in 44.2% of segments. It then shows that current multimodal large language models score poorly because their vision encoders are semantic, not geometric. The central claim is that distilling a frozen 3D geometry teacher's per-frame camera tokens into lightweight student tokens (CamDistill) matches the accuracy of directly injecting the teacher's features (CamInject) while removing the 3D model at inference. A sympathetic reader should care because, if true, expensive geometric supervision can be compressed into a cheap per-frame stream that is free at test time.","feed_headline":"Dropping the 3D teacher costs nothing for camera-motion AI","feed_subtitle":"CamDistill matches direct 3D feature injection on the new CamChoreo benchmark, adding only 0.1 s per clip.","key_machinery":"The load-bearing mechanism is the Geometry-aware Camera Token Extractor (GCTE), a lightweight branch attached to the frozen vision encoder. GCTE predicts one camera token per frame using alternating frame-wise cross-attention (each camera token reads only its own frame's intermediate visual features) and global camera self-attention (tokens exchange temporal context). A cosine-distance distillation loss aligns the per-frame student tokens with the teacher's cached 2048-dimensional camera tokens. At inference the teacher is removed, leaving a compact geometry-aware stream inserted before each frame's visual tokens.","core_discovery":"The paper's central discovery is that the benefit of an expensive 3D geometry model for camera-motion recognition can be preserved without running it at test time. Using VGGT-Ω's per-frame camera token as a distillation target, CamDistill trains a Geometry-aware Camera Token Extractor (GCTE) on frozen intermediate vision features, then discards the teacher. On CamChoreo, CamDistill-4B reaches 67.5 frame-level micro F1, exactly matching CamInject-4B's 67.5, with SegDet@0.5 within 0.4; the 8B version trails direct injection by 0.5 micro F1. Inference latency stays at 10.2 seconds per clip versus 10.1 seconds for the base model and 16.0 seconds for CamInject. The distilled tokens also transfer to external clip-level benchmarks (CameraBench and CMVQA), improving on both the base model and a CameraBench-tuned specialist.","pith_inferences":["The paper does not report inter-annotator agreement; an independent re-annotation study could reveal how much of the 20-point gap over closed-source models is label noise versus model capability.","If the student tokens truly capture pose-associated geometry, they should be decodable into camera pose by a linear probe; the paper does not test this, but such a probe would directly test whether distillation transfers geometry rather than only task-specific cues.","Because the teacher is removed at inference, the same approach could be used with an even larger or multi-view teacher at training time only, pushing accuracy without affecting inference cost.","The benchmark restricts to single-shot clips; extending to edited multi-shot video would test whether temporally grounded compositional labels survive cut boundaries and narrative structure."],"forward_implications":["If the parity holds, video multimodal LLMs can be trained once with a 3D teacher and shipped without it, retaining most of the geometric benefit at near-base inference cost.","Temporally grounded, compositional camera labels become a practical target for large-scale supervised training, since the benchmark's annotations support precise segment-level evaluation.","Rare geometry-dependent classes (Zoom, Roll, Focus Shift) gain more from distillation than frequent ones, so geometry supervision should be a standard component for camera-aware video models.","The same token-distillation scheme could generalize to other geometry-heavy video tasks, such as object-motion or depth-based reasoning, wherever a teacher produces per-frame tokens.","External transfer results imply the distilled representation carries reusable camera cues, not just benchmark-format memorization."],"supporting_citations":[{"why":"Supplies the feed-forward 3D teacher whose camera tokens are the distillation target.","marker":"(Wang et al., 2025b)"},{"why":"Supplies the stronger teacher (VGGT-Ω) that improves distilled accuracy by 1.7 micro F1.","marker":"(Wang et al., 2026a)"},{"why":"Provides the knowledge-distillation objective the paper adapts as a cosine loss on tokens.","marker":"(Hinton et al., 2015)"},{"why":"Provides the vision transformer layer structure whose intermediate features GCTE taps.","marker":"(Dosovitskiy et al., 2020)"},{"why":"Provides the Qwen3-VL backbones used for all SFT, CamInject, and CamDistill runs.","marker":"(Bai et al., 2025b)"},{"why":"Provides the CameraBench external clip-level benchmark used to test transfer.","marker":"(Lin et al., 2026)"},{"why":"Provides the CMVQA synthetic benchmark and the geometry-guided MLLM baseline.","marker":"(Feng et al., 2026)"},{"why":"Provides TransNetV2, used to split source videos into single-shot clips for CamChoreo.","marker":"(Soucek & Lokoc, 2024)"}],"fun_headline_variants":["Camera-motion AI drops 3D model, keeps accuracy","Distilled camera tokens match 3D injection on video motion","CamDistill: 3D-free camera motion at same accuracy","No 3D teacher needed: camera motion AI matches injection","Lightweight tokens replace 3D geometry for camera motion"],"cache_read_input_tokens":27904,"weakest_assumption_plain":"The load-bearing premise is that the five annotators' temporal segments and 20-label sets are reliable ground truth; no inter-annotator agreement statistic is reported, so if hard distinctions like Dolly versus Zoom or Pan versus Truck are inconsistent across annotators, the benchmark rankings and the CamDistill–CamInject parity would rest on noisy labels.","fun_headline_variants_meta":{"raw":{"variants":["Camera-motion AI drops 3D model, keeps accuracy","Distilled camera tokens match 3D injection on video motion","CamDistill: 3D-free camera motion at same accuracy","No 3D teacher needed: camera motion AI matches injection","Lightweight tokens replace 3D geometry for camera motion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000192,"raw_usage":{"total_tokens":1406,"prompt_tokens":1063,"completion_tokens":343,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":679,"completion_tokens_details":{"reasoning_tokens":257}},"tokens_in":679,"tokens_out":343,"duration_ms":3652,"temperature":1.0,"reasoning_tokens":257,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:00:06.263257+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate a random subset of CamChoreo (say 300 clips) with a fresh expert panel following the same protocol and measure inter-annotator agreement on boundaries and label sets, especially for Dolly/Zoom, Pan/Truck, and compound labels. If per-label agreement falls below a high threshold, the benchmark's difficulty numbers and the distillation parity claim lose their footing. A second check: train a linear probe on CamDistill's student tokens to predict the teacher's camera pose; if the probe achieves low accuracy, the tokens may not carry the geometry the paper attributes to them.","supporting_citations":[],"review_version":1}