{"id":"b43f3736-d07b-47a0-9e3a-f4dba1839e1f","arxiv_id":"2411.08753","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"LangView uses per-view caption accuracy against view-agnostic narrations as pseudo-labels to train a view selector that outperforms heuristics and prior baselines on Ego-Exo4D and LEMMA.","lead":"A model picks the best camera angle from multi-view instructional videos, trained with only the video's text narration as weak supervision instead of expensive human view labels. A generalist might care because automatic view selection could save hours of manual video editing in how-to, sports, and instructional content.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Core proxy claim rests on a small, extreme-pair human study while automatic metrics are coupled to the training signal; the central claim lacks strong independent support.","rationale":"The reader correctly identifies the pseudo-labeler's dependence on finetuned captioners as a weak point, but the deeper load-bearing concern is that the only direct evidence for the central proxy claim—that captioning accuracy indicates human informativeness—comes from a small human study with modest margins, while the automatic metrics are coupled to the training signal. The paper's own failure cases show that humans sometimes prefer views that differ from the pseudo-labeler's choice in nuanced ways, suggesting the proxy is not universally aligned. A larger, more granular human study would directly test whether CIDEr ranking matches human preference across all views, not just the extremes. The conditional verdict is appropriate, but the conditions should include this validation, not just releasing code and error bars. My concern does not reject the paper; it identifies the specific evidence that would make the central claim robust.","tokens_in":27509,"tokens_out":3706,"duration_ms":34882,"concrete_test":"Conduct a human preference study on 150 randomly sampled clips per dataset (Ego-Exo4D, LEMMA), with at least 20 participants per clip, asking them to compare all pairwise combinations of the pseudo-labeler's ranked views (not just best vs. worst). Aggregate pairwise choices into a per-view human quality score, then compute the Spearman correlation between these scores and the pseudo-labeler's CIDEr scores for the same views. If the correlation is weak (rho < 0.3) or non-monotonic, the core proxy assumption fails and the conditional acceptance should be reconsidered.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central hypothesis is that CIDEr agreement between a view's caption and the view-agnostic narration predicts human informativeness. This CIDEr score is used to generate pseudo-labels, and the main automatic evaluation metrics (CIDEr, METEOR, V-IoU, N-IoU, NC-IoU) are all caption-narration agreement measures, so the quantitative gains may partly reflect optimizing the same objective used for evaluation. The only independent validation is the human study, but it is small (70 pairs per dataset, 10 participants) and shows modest margins: for pseudo-label quality, best vs. worst wins are 53.3% and 46.7% on Ego-Exo4D and LEMMA, with losses of 28.9% and 38.9%; for view selection, the win rate over Hand-object on LEMMA is only 43.3% vs. 41.1%. The study tests only extreme pairs (best vs. worst) and best model vs. one baseline, not the full ranking, so it cannot establish that CIDEr is a reliable fine-grained proxy. If the proxy is weak or non-monotonic among mid-ranked views, the pseudo-labels and the learned selector are trained on a flawed signal, undermining the claim that LangView selects the most informative view for humans.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LangView, a weakly supervised approach to best-view selection in multi-view instructional videos. Given a multi-view clip and a view-agnostic narration, a pseudo-labeler uses off-the-shelf video captioners (Video-Llama with two LLM decoders and VideoChat2), finetuned on the target datasets, to caption each view; the views are ranked by CIDEr similarity between the predicted caption and the ground-truth narration, and a rank aggregator produces a best-view pseudo-label set. A view selector, built on an EgoVLPv2/TimeSformer encoder with a view classification head and an auxiliary relative-camera-pose classification head, is trained with a min-over-pseudo-labels cross-entropy loss plus a weighted pose loss. At inference, only the multi-view video is used, without narrations or camera poses. Experiments on Ego-Exo4D and LEMMA report automatic captioning metrics (CIDEr, METEOR, V/N/NC-IoU) and a pairwise human study, claiming consistent improvement over baselines.","tokens_in":27782,"tokens_out":4713,"duration_ms":40128,"significance":"If the central proxy holds, the paper makes a useful contribution: it replaces expensive best-view labels with widely available view-agnostic narrations and shows a concrete way to mine those narrations for view-selection supervision. The inclusion of a human evaluation is a genuine strength, as it provides a signal that is not directly aligned with the pseudo-labeling objective. The relative-pose auxiliary task is a sensible mechanism for increasing view sensitivity, and the supplementary material is thorough, including a 3-fold evaluation, pseudo-labeling cost analysis, and additional ablations. The main weakness is that the independent evidence for the core hypothesis is currently thin: the human study is small and tests only extreme pairs, while the automatic metrics share the same captioner families, ground-truth narrations, and metric family as the training signal.","major_comments":[{"comment":"The automatic evaluation is partially aligned with the training signal by construction. The pseudo-labeler ranks views using CIDEr between per-view captions and the ground-truth narration (Sec. 3.2), and the automatic metrics (CIDEr, METEOR, V-IoU, N-IoU, NC-IoU) are computed by captioning the selected view with the same captioner families (Video-Llama, VideoChat2) and comparing to the same ground-truth narrations. A selector that optimizes the pseudo-labeling objective will therefore tend to score higher on these metrics even if it does not improve human informativeness. The authors should provide an automatic evaluation that is more independent of the training signal, for example by using a captioner that was not used in pseudo-labeling, or by reporting metrics that do not involve caption-narration agreement at all.","section":"Sec. 3.2 and Sec. 4.1 (evaluation metrics)"},{"comment":"The human evaluation results are more modest than the text claims. For pseudo-label quality, the best view wins only 53.3% and 46.7% on Ego-Exo4D and LEMMA, with loss rates of 28.9% and 38.9%. For view prediction on LEMMA, the win rate over Hand-object is 43.3% versus a 41.1% loss rate, a difference of only 2.2 percentage points. These numbers do not support the unqualified claim that 'our selected views are preferred significantly more than the two top baselines.' The paper should report confidence intervals or per-participant analysis, and should temper the claim for the LEMMA Hand-object comparison.","section":"Table 2 (human evaluation)"},{"comment":"The ablation without captioner finetuning yields near-collapse (CIDEr 0.4, METEOR 12.2, V-IoU 1.4), showing that the pseudo-labeling pipeline is entirely dependent on finetuning the captioners on target-dataset narrations. This dependency is load-bearing for the claimed 'weak supervision from language': the narrations are used not only as a ranking signal but also to finetune the captioners themselves. The paper should explicitly acknowledge this and discuss the extent to which the method's success relies on having a large set of in-domain clip-narration pairs for captioner finetuning.","section":"Supp. Table 4 and Sec. 6.2"},{"comment":"Gains over the strongest baselines are small on several metrics: on Ego-Exo4D, CIDEr is 13.5 vs. 12.9 for Body-area and METEOR is 48.4 vs. 48.2; on LEMMA, CIDEr is 42.7 vs. 42.1 and METEOR is 74.4 vs. 73.8. No error bars, standard deviations, or number of runs are reported, so it is unclear whether these differences are within noise. The authors should report variance (e.g., bootstrapped confidence intervals or multiple training runs) for the main table, or at least for the comparison against the strongest baseline.","section":"Table 1 (main results)"}],"minor_comments":[{"comment":"There are several typos: 'c.f.' should be 'cf.' (Sec. 3.4 and elsewhere), 'we against observe' appears in Supp. Sec. 6.2, and 'and and' appears in Sec. 4.2 in the qualitative examples paragraph.","section":"Throughout"},{"comment":"The caption states 'Significance, p ≤ 0.05' but does not name the statistical test or how ties were handled in the significance computation; please specify the test and the tie-handling procedure.","section":"Table 2 caption"},{"comment":"The phrase 'state-of-the-art baselines' overstates the baseline set, which includes simple heuristics (Ego-only, Random, Random-exo). Suggest rephrasing to 'a range of baselines, including heuristics and prior methods.'","section":"Abstract and Sec. 4.2"},{"comment":"The captioner training uses 'a maximum of 1.6 million iterations' for both datasets; please clarify whether this is the same budget for LEMMA and Ego-Exo4D given their very different sizes, and whether early stopping was applied.","section":"Supp. Sec. 10.2.1"}],"recommendation":"major_revision","confidential_remarks":"The work is from a strong group and the core idea is appealing, but the evidence for the central hypothesis is not yet conclusive. The automatic metrics are partly coupled to the training objective, and the human study, while a good step, is too small and too restricted (extreme pairs only) to establish that CIDEr-based pseudo-labels are a reliable fine-grained proxy. I would like to see an independent automatic evaluation and a more detailed human study before recommending acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"LangView is worth a look: the core idea is to score each view by how well its caption agrees with a view-agnostic narration, then use that score as a pseudo-label for training a view selector. That combination is new as far as I know, and the paper is well engineered. The relative-pose auxiliary task is a sensible regularizer, and the ablations are honest, including the supplementary table showing that without finetuning the captioners the pseudo-labels collapse (CIDEr 0.4). The human study, while small, does support the central hypothesis: in pairwise comparisons, the pseudo-labeler's best views beat its worst views by 53% to 29% on Ego-Exo4D and 47% to 39% on LEMMA, and the learned selector beats body-area on both datasets. That is real evidence.\n\nSoft spots. The automatic metrics are partly circular by construction: CIDEr, METEOR, and the IoU metrics all compare captions from the chosen view to the same ground-truth narrations that generated the pseudo-labels. So the headline gains (e.g., CIDEr 13.5 vs 12.9 on Ego-Exo4D) partly measure how well the model optimizes its own training objective. The independent validation is the human study, but it is limited: 70 pairs, 10 participants, and only extreme pairs (best vs. worst) or one-vs-one against a baseline. On LEMMA, the win rate over Hand-object is 43.3% vs. 41.1%—a two-point margin they call significant without reporting confidence intervals. The stress-test concern about mid-ranked views is less severe than it sounds because the pseudo-labels are binary (they only need to identify the best view, not a full ranking), but a finer-grained human eval would strengthen the claim. No code or data is released, which matters for a method whose behavior depends heavily on finetuned captioners.\n\nThe citation pattern is fine; related work is covered. This is not a take-down. The idea is solid, the experiments are thorough, and the limitations are acknowledged in the paper itself.\n\nBottom line: this paper deserves peer review, not a desk reject. I would accept with revisions: release code and data splits, report variance or confidence intervals for both human and automatic metrics, and re-evaluate with a captioner not used during training. If those are addressed, I'd cite it.","headline":"A clean weakly-supervised view selection idea with genuine novelty, but the independent human validation is thinner than the claims; deserves a serious review with requests for code, variance, and an unseen captioner.","tokens_in":28294,"tokens_out":2805,"would_cite":true,"duration_ms":27551,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The view that best predicts a view-agnostic narration of an activity is the most informative view, and a selector trained on this language proxy needs no best-view labels.","keywords":["view selection","multi-view video","weak supervision","video captioning","pseudo-labeling","instructional videos","relative camera pose","egocentric video"],"falsifier":"Take a held-out set of multi-view clips with narrations, compute the pseudo-labeler's per-view CIDEr scores, and collect human pairwise preferences among all views, not just the extremes. If the top-scored view loses to a lower-scored view at or above chance, or if the CIDEr ranking agrees with human preference no better than the Hand-object or Body-area heuristics, the core proxy is not doing the work. A cheaper quantitative check is to compare the pseudo-labeler's top view against a human-labeled best view on a dataset with such labels; chance-level agreement would refute the claim.","tokens_in":27314,"feed_emoji":"🎥","tokens_out":7543,"duration_ms":62631,"temperature":0.7,"pith_summary":"This paper tackles a practical problem: of the several cameras recording the same instructional activity, which view should a viewer watch at each moment. Its central claim is that the camera view whose per-view caption most closely matches a view-agnostic narration of the activity is the view a human would find most informative. To turn that claim into a method, LangView first finetunes captioners on the target datasets, ranks every view by how well its predicted caption scores against the narration with CIDEr, and aggregates ranks across captioners into best-view pseudo-labels. A view selector is then trained on those pseudo-labels with an auxiliary relative-camera-pose prediction task, so that at inference it needs only the multi-view video. The paper reports that this weakly supervised model outperforms heuristic baselines and prior view-selection methods on Ego-Exo4D and LEMMA, both in automatic metrics and in pairwise human preference.","feed_headline":"Caption accuracy picks the best camera view without labels","feed_subtitle":"A weakly supervised model ranks views by how well they retell the activity, beating camera heuristics on two datasets.","key_machinery":"The load-bearing object is the best-view pseudo-labeler. For each training clip it produces one captioned version per view using three off-the-shelf video captioners, scores every caption against the view-agnostic narration using the CIDEr metric, ranks the views per captioner, and aggregates the top-ranked views into a (possibly multi-element) pseudo-label set. The second piece is the view selector: a shared TimeSformer visual encoder feeding a view classification head and a relative camera pose prediction head; pose prediction is posed as classification over discretized relative-pose angle bins for every view pair, and acts as a regularizer that prevents the encoder from collapsing all views into narration-predictive but viewpoint-insensitive representations. The training loss is the min over pseudo-labels of the cross-entropy of the predicted view, plus a weighted pose classification loss.","core_discovery":"The discovery the paper aims to establish is that language can isolate the informative viewpoint in a multi-view instructional video: the more accurately an individual view predicts a view-agnostic text summary of the activity, the more informative that view is. The paper operationalizes this by using an ensemble of video captioners to caption each view independently, scoring each caption against the ground-truth narration with CIDEr, and taking the consensus top-ranked views as pseudo-labels. It then shows that a view classifier can be trained on these pseudo-labels alone, provided it is regularized by an auxiliary relative camera pose predictor that keeps the visual features sensitive to viewpoint. The result is a selector that, at test time, consumes only the multi-view video and returns a best view per clip, with reported gains over heuristics, caption-length scoring, and prior cinematographic baselines on both evaluation datasets.","pith_inferences":["Beyond the paper: a natural next test is replacing CIDEr with a learned or human-calibrated caption-to-narration similarity, since captioning metrics reward lexical overlap and might rank views differently when captioners are verbose or paraphrastic.","Beyond the paper: the same 'predictive accuracy against view-agnostic text' recipe should carry over to other multi-view settings with descriptions, such as broadcast sports, surveillance with operator logs, or user-generated event coverage.","Beyond the paper: because the pseudo-labeler is an ensemble of captioners, the whole pipeline can inherit future captioner improvements without architectural change, so the selector should improve as captioners get better at naming visible entities and interactions.","Beyond the paper: the min-over-pseudo-labels loss implicitly assumes the selector should pick the easiest-to-predict among the plausible best views; a ranking loss over the captioners' numeric view scores might make better use of the pseudo-labeler's output."],"forward_implications":["Best-view labels can be replaced by view-agnostic narrations wherever such narrations already exist, removing the most expensive annotation bottleneck in view selection.","The auxiliary relative-pose task measurably improves selection: in the paper's ablations, dropping it hurts performance on most metrics, while the rank aggregator mainly protects noun and noun-chunk overlap.","The approach transfers to both a multi-exo setup (Ego-Exo4D) and a single-exo setup (LEMMA), so it is not tied to a particular camera topology.","At inference the system is text-free and pose-free: language and camera geometry are used only to manufacture training signal.","Pseudo-label quality depends on finetuned captioners; with frozen off-the-shelf captioners the paper reports near-zero CIDEr pseudo-labels, so captioner finetuning on the target domain is a required step."],"supporting_citations":[{"why":"Supplies one of the three captioners whose per-view predicted narrations are scored against the view-agnostic ground truth.","marker":"[122]"},{"why":"Supplies the other captioner used in the pseudo-labeler, adding independent per-view caption predictions to the consensus.","marker":"[64]"},{"why":"Provides the Ego-Exo4D dataset with five time-synced views and view-agnostic narrations, the main evaluation and training ground.","marker":"[37]"},{"why":"Provides the LEMMA dataset with ego and single-exo views and narrations, testing the method on a different camera topology.","marker":"[56]"},{"why":"Defines the CIDEr metric used inside the ranker to compare per-view captions with narrations and to evaluate selected views.","marker":"[108]"},{"why":"Supplies the METEOR metric used to evaluate how well the selected view's caption agrees with the narration.","marker":"[7]"},{"why":"Provides the EgoVLPv2 visual encoder, pretrained on Ego-Exo4D, that serves as the backbone of the view selector.","marker":"[86]"}],"fun_headline_variants":["Caption accuracy picks best camera view without labels","Weak supervision lets captions choose informative views","Use language to find the best viewpoint in videos","View selection via captioning accuracy, no labels","How captions reveal the optimal camera angle"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire method assumes that the CIDEr score between a view's predicted caption and the view-agnostic narration reliably tracks how informative that view would be for a human viewer; if captioners describe views in ways that do not reflect what is visible, the pseudo-labels and the selection model trained on them are wrong.","fun_headline_variants_meta":{"raw":{"variants":["Caption accuracy picks best camera view without labels","Weak supervision lets captions choose informative views","Use language to find the best viewpoint in videos","View selection via captioning accuracy, no labels","How captions reveal the optimal camera angle"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000165,"raw_usage":{"total_tokens":1245,"prompt_tokens":934,"completion_tokens":311,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":243}},"tokens_in":550,"tokens_out":311,"duration_ms":3929,"temperature":1.0,"reasoning_tokens":243,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T21:23:09.209605+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of multi-view clips with narrations, compute the pseudo-labeler's per-view CIDEr scores, and collect human pairwise preferences among all views, not just the extremes. If the top-scored view loses to a lower-scored view at or above chance, or if the CIDEr ranking agrees with human preference no better than the Hand-object or Body-area heuristics, the core proxy is not doing the work. A cheaper quantitative check is to compare the pseudo-labeler's top view against a human-labeled best view on a dataset with such labels; chance-level agreement would refute the claim.","supporting_citations":[{"cited_title":"Lawrence Zitnick, and Devi Parikh","cited_arxiv_id":null,"evidence_quote":"Defines the CIDEr metric used inside the ranker to compare per-view captions with narrations and to evaluate selected views."},{"cited_title":"Egovlpv2: Egocentric video-language pre-training with fusion in the backbone","cited_arxiv_id":null,"evidence_quote":"Provides the EgoVLPv2 visual encoder, pretrained on Ego-Exo4D, that serves as the backbone of the view selector."}],"review_version":1}