{"id":"ad4147e3-4814-48d8-ab2e-6c4d921784c2","arxiv_id":"2508.05857","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A multi-view model with head aggregation, uncertainty-based gaze selection, and epipolar scene attention outperforms single-view gaze target estimation and enables cross-view prediction.","lead":"Researchers show that gaze target estimation, the task of finding where a person is looking, becomes more accurate and can even work across cameras when multiple synchronized views are used. The paper introduces the first multi-view gaze dataset and a model that exploits the extra view, including the ability to predict a gaze target in a camera that does not show the person.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-view GTE's absolute-depth pipeline is never stress-tested: RANSAC scale/shift is fit on background, then applied to the person; no depth-error sensitivity analysis supports the Table 3 claim.","rationale":"Stress-test pass. The paper's central multi-view claim (Table 1, ablations) is supported by leave-one-scene-out experiments and sensible ablations; the modules (HIA/UGS/ESA) each contribute, and the same-learning-rate results in S12 suggest limited overfitting, though per-scene beta remains. Camera-parameter sensitivity is tested in S10. The baseline reproduction in S13 is good practice. The weak point is the cross-view extension. The reader identified the absolute-depth assumption; I agree and sharpen it: the RANSAC fit is performed on static background while the eye lies on the person, so the scale/shift transfer is not automatically valid. No experiment tests this, and the paper's own limitation (S14) acknowledges reliance on 3D reconstruction without quantifying failure modes. This does not invalidate the multi-view claim, but it makes the cross-view claim conditional on a robustness that should be demonstrated. Therefore the verdict remains CONDITIONAL, with the condition being a depth-error sensitivity analysis.","tokens_in":21666,"tokens_out":7868,"duration_ms":77539,"concrete_test":"Perturbation test on the MVGT validation splits: for each cross-view sample, re-run the S4 pipeline but corrupt the estimated scale/shift (a,b) by multiplicative noise in {±5%, ±10%, ±20%}, and separately add a person-specific depth bias (e.g., +10 cm to the person's depth before unprojection) to simulate background/person inconsistency. Recompute cross-view Dist/AP and compare to Table 3. Threshold: if Dist increases by >0.02 for ±10% noise, the pipeline is too sensitive. Additionally, on the ordinary multi-view splits where the head is visible in both views, triangulate the 3D eye location from the two views and compare with the cross-view pipeline's recovered eye location; report median error. If median eye error is small (<5 cm) and perturbed runs stay within noise of Table 3, the concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Sec 4.6 and S4 describe the cross-view pipeline: Dust3R reconstructs the static scene from six calibration images, then for each new input image a RANSAC fit (Eq. 9) estimates scale/shift a,b between Metric3D depth and the reconstructed absolute depth, masking out the person and calibration cube. The 3D eye location is then computed by applying a,b to the person's monocular depth. This is a load-bearing step for the cross-view result (Table 3, Dist 0.188 vs 0.248): if the person's depth does not share the same scale/shift as the masked static background, the eye is mislocated in 3D and, after transformation to the primary view, the FoV heatmap is systematically shifted. The paper provides no validation of the estimated absolute eye location (e.g., against triangulation when the head is visible in two views) and no sensitivity analysis to depth errors; S10 jitters camera parameters, not depth. S14 acknowledges the reliance on 3D reconstruction but does not quantify when it fails. Because cross-view GTE is a headline capability, this untested assumption is the weakest support for the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the first multi-view gaze target estimation (GTE) method. It takes a pair of calibrated camera views, extracts head embeddings through a cross-attention Head Information Aggregation (HIA) module, selects the more reliable predicted 3D gaze vector via an Uncertainty-based Gaze Selection (UGS) module, and shares scene context through Epipolar-based Scene Attention (ESA). The method is evaluated on the new MVGT dataset, containing 13,686 synchronized images from 6 cameras across 4 scenes, with gaze targets annotated via a laser-pointer protocol. The main experiments use leave-one-scene-out cross-validation and compare against single-view GTE baselines on primary-view images. A cross-view extension predicts the gaze target in a view that does not contain the person, using the person image from another view and an absolute-depth pipeline based on Dust3R and Metric3D alignment. The paper reports consistent improvements when the reference view shows the person's head, and large gains in the cross-view setting.","tokens_in":21947,"tokens_out":9342,"duration_ms":103356,"significance":"If the results hold, this is a meaningful new capability and dataset: it is the first multi-view GTE method and dataset, with a novel cross-view setting that single-view methods cannot address. The method is modular, and the ablations show that each proposed module contributes to the overall improvement. The evaluation has several strengths: held-out leave-one-scene-out evaluation, a controlled comparison in which the same primary-view images are averaged over their reference pairs, a re-implementation of the Tafasca baseline with reproduced GazeFollow numbers, and supplementary analyses of uncertainty calibration and attention behavior. However, the two headline claims—(i) significant gains over single-view baselines and (ii) the cross-view capability—currently rest on a 4-scene evaluation without per-scene variance reporting and on an unvalidated absolute-depth alignment step. These issues are load-bearing for the central claims and need to be addressed before the results are fully convincing.","major_comments":[{"comment":"The cross-view result is the paper's headline capability, but the depth-alignment step that makes it possible is never validated. Eq. (9) fits a RANSAC scale/shift (a,b) between Metric3D depth and the Dust3R reconstruction on static background pixels, with the person and calibration cube masked out; the same (a,b) is then applied to the person's monocular depth to obtain the metric 3D eye location. Nothing demonstrates that the person region shares the same scale/shift as the static background. If it does not, the transformed eye position in the primary view is systematically wrong, and the FoV heatmap is misplaced. This is not a side detail: Table 3 reports Dist 0.188 vs. 0.248 for the strongest adapted baseline, so the claimed cross-view margin depends directly on this step. The sensitivity analysis in S10 perturbs camera parameters, not depth; S14 acknowledges reliance on 3D reconstru","section":"Sec. 4.6 / S4, Eq. (9) / Table 3"},{"comment":"The statistical support for the claim that the method 'significantly outperforms' single-view baselines is incomplete. The evaluation uses only four scenes (leave-one-scene-out), but no per-scene breakdown, standard error, confidence interval, or significance test is reported. The headline differences in Table 1 are small—e.g., 0.129 vs. 0.141 Dist. in the head-visible/target-visible cell—and could plausibly be driven by a single scene. Moreover, S1 shows per-scene tuning of the learning rate and the in/out loss weight (β), and the cross-view task uses learning rates of 1e-7 or 1e-8 depending on the scene. S12 shows that fixing the learning rate at one value degrades the reported numbers (0.134 vs. 0.129 for the general multi-view task; 0.199 vs. 0.188 for cross-view), so part of the reported advantage is training-protocol-dependent. Please report all four fold results, means ± standard","section":"Sec. 5.2 / Table 1 / S1 / S12"},{"comment":"The ground-truth gaze vectors used to train the gaze estimator are computed from 'pseudo' monocular depth maps, and the paper argues that the additive shift can be ignored because Metric3D has 'low depth distortion and shift.' This is an important assumption: if a nonzero depth shift remains, the 3D vector from eye to target is not simply a scaled version of the true vector, because the shift is multiplied by the pixel-coordinate term in Eq. (5). The paper does not quantify the residual shift of Metric3D on the MVGT scenes or the effect of this residual on the FoV heatmap. Since this pseudo-depth assumption is shared with some single-view baselines (e.g., Tafasca), it may not change the comparison, but it should be validated or at least discussed quantitatively for the final heatmap computation.","section":"Sec. 4.3 / Eq. (2)"}],"minor_comments":[{"comment":"The sentence beginning 'when we use a depth estimation model that has low depth distortion and shift...' is unclear: 'V(u,v) will only be the same' should be rephrased. The mathematical reasoning about scale and shift cancellation needs to be stated precisely.","section":"Sec. 4.3"},{"comment":"The ESA epipolar sampling is underspecified: how are the 48 points along the epipolar line chosen (uniform in pixel coordinates? bilinear interpolation?), and are the sampled features taken from a single feature-map scale? This detail is necessary for reproducibility.","section":"Sec. 4.4 / Fig. 6"},{"comment":"S11 cites '[75]' (a hedge-fund portfolio construction paper) as a transformer-architecture reference alongside [13] and [64]. This citation appears unrelated to transformer architectures and should be removed or replaced.","section":"S11 / References"},{"comment":"The explanation 'This ensures the same total number of testing samples' is slightly misleading. Averaging over reference pairs reweights primary views that appear in different numbers of pairs; the weighting scheme should be stated explicitly.","section":"Sec. 5.2"},{"comment":"The statement that the first model is 'applicable without any assumptions' overstates its scope: the model still requires calibrated camera parameters and head bounding boxes in both views. Suggest softening this sentence.","section":"S14"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid first step toward multi-view GTE, and the cross-view setting is genuinely novel. The main weaknesses are empirical: the cross-view absolute-depth step is not validated, and the 4-scene evaluation lacks variance reporting and uses per-scene hyperparameter tuning. These are fixable with additional experiments and analysis. I would also ask the editor to verify the S11 citation [75], which appears accidental."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First multi-view GTE paper, and it's a legitimate step forward: new task, new dataset with calibrated cameras, and a clean leave-one-scene-out setup that avoids the usual circularity trap. The main result — that a second view with a visible face improves target error (Dist 0.129 vs 0.141) — holds up under the ablation they ran, and the HIA/UGS/ESA modules each earn their keep. They also ship code and data, which is more than most papers in this area.\n\nThe weak point is the cross-view extension in Sec 4.6/5.4. The idea is clever: use Dust3R to reconstruct the static scene once, then fit scale/shift between Metric3D depth and the reconstruction on background pixels, and apply that transform to the person's depth to locate the eye in 3D. But the paper never validates that fitted transform on the person. There is no sensitivity analysis to depth error (S10 jitters camera parameters, not depth), and no check of the estimated eye location against triangulation when the head is visible in two views. Since Table 3's cross-view result (0.188 vs 0.248) is a headline capability, this is a real gap. The paper acknowledges the reliance in S14 but does not quantify when it breaks.\n\nOther concerns are minor. Four scenes is small, and there are no error bars, but leave-one-scene-out at least tests generalization. The per-scene learning rates look like tuning, but S12 shows the same-LR version is only slightly worse (0.134 vs 0.129 for the main metric). The comparison with single-view baselines is fairly controlled.\n\nVerdict: the central multi-view claim is solid and the dataset is a useful contribution. The cross-view claim is promising but should be treated as conditional until they demonstrate robustness to depth misalignment. This deserves peer review — a serious referee should ask for the depth sensitivity analysis and per-scene results, but not reject the paper for them.","headline":"First multi-view gaze-target paper with a new dataset and a solid main result; the cross-view claim leans on an unvalidated depth-alignment step that needs a sensitivity analysis.","tokens_in":22454,"tokens_out":2746,"would_cite":true,"duration_ms":26102,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two calibrated views improve gaze target estimation, and the same pipeline can localize targets in a view that never sees the person.","keywords":["multi-view gaze target estimation","gaze following","head information aggregation","uncertainty-based gaze selection","epipolar scene attention","cross-view gaze estimation","MVGT dataset","calibrated multi-camera setup"],"falsifier":"Run the cross-view pipeline in a scene where the 3D reconstruction comes from an earlier state (e.g., furniture moved) while keeping the same cameras, and measure normalized distance on gaze targets. If Dist. stays near 0.188, the depth-alignment assumption is not load-bearing; if it rises toward the 0.248 adapted-baseline level, the assumption is essential. A second check: plot angular error of predicted gaze vectors against predicted $\\sigma$; UGS's selection only works while that correlation holds.","tokens_in":21583,"feed_emoji":"👀","tokens_out":11971,"duration_ms":99887,"temperature":0.7,"pith_summary":"The paper aims to establish that gaze target estimation — predicting where in a scene a person is looking — should not be locked to a single camera. Using a pair of synchronized, calibrated views, with one view treated as primary and the other as reference, the proposed model consistently beats single-view methods whenever the second view supplies useful face or scene information, and can even locate the target in a view that never shows the person. The authors introduce the first multi-view GTE dataset, MVGT, collected with a laser-pointer annotation protocol that yields precise targets without image artifacts. On MVGT, with the reference view showing the head, normalized target distance drops to 0.129 versus 0.141 for the best single-view baseline; in the cross-view setting, distance is 0.188 versus 0.248 for the strongest adapted baseline. If correct, the work moves GTE from a single-camera restriction to the multi-camera setups already common in public spaces.","feed_headline":"Two cameras beat one at gaze target estimation","feed_subtitle":"A second face view cuts error to 0.129 and finds targets in frames the person never appears in.","key_machinery":"The load-bearing object is the field-of-view (FoV) heatmap, a gaze-target prior computed from a predicted 3D gaze vector $g$ and monocular depth: each pixel's vector to the eye, $V_{(u,v)}$, is scored by $M_f(u,v)=\\max(0, V_{(u,v)}\\cdot g/(\\|V_{(u,v)}\\|\\|g\\|))$, with exponential decay below 0.9. Three modules improve that prior. HIA cross-attends head tokens while concatenating the relative rotation $R_{21}=R_1R_2^{-1}$ into the keys/values. UGS trains the gaze estimator with the aleatoric-uncertainty loss $\\mathcal{L}_{\\mathrm{gaze}}=\\frac{1}{2\\sigma^2}(1-\\hat{g}\\cdot g/(\\|\\hat{g}\\|\\|g\\|))+\\frac{1}{2}\\log\\sigma^2$, then replaces the high-$\\sigma$ view's gaze vector with the low-$\\sigma$ vie","core_discovery":"The central claim is that two calibrated views, fused at three specific points in the pipeline, suffice to improve gaze target estimation beyond any single view. The Head Information Aggregation (HIA) module cross-attends head tokens from the other view, injecting the relative rotation between cameras so the head embedding becomes geometry-aware; the Uncertainty-based Gaze Selection (UGS) module predicts an aleatoric uncertainty $\\sigma$ for each view's 3D gaze vector and replaces the less reliable prediction with the better one transformed through camera extrinsics; and the Epipolar-based Scene Attention (ESA) module lets scene tokens attend only to tokens sampled along the corresponding ep","pith_inferences":["Beyond the paper: UGS's uncertainty-based replacement is a generic mechanism; the same calibrated-pair recipe (predict a per-view confidence, rotate the better prediction into the other view) could be dropped into other multi-view fusion tasks, such as multi-view action or attention prediction, wherever one view is occluded.","Beyond the paper: Because the cross-view path needs only a one-time scene reconstruction, a practical deployment could reconstruct an empty room once and then run continuously; the untested sensitivity is whether moving furniture or people changes the scale/shift alignment enough to break the FoV prior.","Beyond the paper: The ablation ties the gains to explicit camera parameters; a testable extension is to learn relative camera pose from a few views and see whether the multi-view gain survives, which would lift the calibration requirement the authors name as a limitation in Section S14."],"forward_implications":["In multi-camera environments such as lecture halls or stores, gaze following can be made more accurate than the best single-view model whenever at least one camera captures the subject's face clearly; the paper reports 0.129 versus 0.141 Dist. in that setting.","Gaze targets outside the field of view of the camera that sees the person become estimable: the model predicts the target in a second view using only the person's appearance, a capability single-view GTE lacks.","More cameras help without retraining: selecting the lowest-uncertainty pair gives Dist. 0.130 with two views, 0.121 with four, and 0.118 with six (Table S2).","The benefit scales with face visibility: error reduction over the single-view version is 23.7% for forward-facing, 23.2% for sideways, and 9.3% for backward heads.","The MVGT dataset and its laser-pointer annotation protocol give the community a calibrated, multi-view GTE benchmark with precise target labels, enabling the head/target visibility breakdown used in evaluation."],"supporting_citations":[{"why":"Pretraining corpus that defines the single-view GTE task and supplies the model initialization before MVGT fine-tuning.","marker":"[51]"},{"why":"Source of the FoV-heatmap gaze-prior technique the pipeline extends; reimplemented as a single-view baseline.","marker":"[56]"},{"why":"Depth-aware single-view GTE baseline that the multi-view method is compared against in the head-visible setting.","marker":"[43]"},{"why":"Video GTE baseline and related task definition used in the single-view comparison table.","marker":"[10]"},{"why":"Multi-view stereo reconstruction model that produces the absolute-depth scene used to transfer the eye location across views in cross-view GTE.","marker":"[66]"},{"why":"Monocular metric depth model whose per-image depth maps are aligned to the reconstruction to build pseudo point clouds and FoV heatmaps.","marker":"[72]"},{"why":"Aleatoric uncertainty formulation that UGS adopts; the paper relies on higher predicted uncertainty accompanying larger gaze-vector error.","marker":"[36]"},{"why":"Uncertainty-aware gaze loss that trains the sigma prediction used by UGS for view selection.","marker":"[11]"},{"why":"Cross-view baseline adapted from video gaze following; supplies the strongest adapted comparison in the cross-view experiment.","marker":"[52]"},{"why":"AprilTag-based extrinsic calibration that yields the real-world camera parameters consumed by HIA, UGS, and the cross-view module.","marker":"[47]"}],"fun_headline_variants":["Multi-view gaze estimation beats single-camera limits","Two cameras sharpen gaze target accuracy","Cross-view fusion tackles occlusion and out-of-view targets","Gaze target from another camera alone: multi-view method","Head-aware multi-view gaze selection improves accuracy"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise, which the paper itself flags in Section S14, is that cross-view GTE requires calibrated cameras and a prior 3D reconstruction, and that the RANSAC scale-and-shift fit between a per-image monocular depth map and that reconstruction yields near-absolute depth for every new frame; if the alignment is wrong, the FoV heatmap is misplaced and the cross-view prediction fails.","fun_headline_variants_meta":{"raw":{"variants":["Multi-view gaze estimation beats single-camera limits","Two cameras sharpen gaze target accuracy","Cross-view fusion tackles occlusion and out-of-view targets","Gaze target from another camera alone: multi-view method","Head-aware multi-view gaze selection improves accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00073,"raw_usage":{"total_tokens":3092,"prompt_tokens":716,"completion_tokens":2376,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":460,"completion_tokens_details":{"reasoning_tokens":2320}},"tokens_in":460,"tokens_out":2376,"duration_ms":19555,"temperature":1.0,"reasoning_tokens":2320,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T23:06:13.419510+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the cross-view pipeline in a scene where the 3D reconstruction comes from an earlier state (e.g., furniture moved) while keeping the same cameras, and measure normalized distance on gaze targets. If Dist. stays near 0.188, the depth-alignment assumption is not load-bearing; if it rises toward the 0.248 adapted-baseline level, the assumption is essential. A second check: plot angular error of predicted gaze vectors against predicted $\\sigma$; UGS's selection only works while that correlation holds.","supporting_citations":[{"cited_title":"Where are they looking? In Advances in Neu- ral Information Processing Systems","cited_arxiv_id":null,"evidence_quote":"Pretraining corpus that defines the single-view GTE task and supplies the model initialization before MVGT fine-tuning."},{"cited_title":"Child- play: A new benchmark for understanding children’s gaze behaviour","cited_arxiv_id":null,"evidence_quote":"Source of the FoV-heatmap gaze-prior technique the pipeline extends; reimplemented as a single-view baseline."},{"cited_title":"Patch- level gaze distribution prediction for gaze following","cited_arxiv_id":null,"evidence_quote":"Depth-aware single-view GTE baseline that the multi-view method is compared against in the head-visible setting."},{"cited_title":"Detecting attended visual targets in video","cited_arxiv_id":null,"evidence_quote":"Video GTE baseline and related task definition used in the single-view comparison table."},{"cited_title":"Dust3r: Geometric 3d vi- sion made easy","cited_arxiv_id":null,"evidence_quote":"Multi-view stereo reconstruction model that produces the absolute-depth scene used to transfer the eye location across views in cross-view GTE."},{"cited_title":"Metric3d: Towards zero-shot metric 3d prediction from a single image","cited_arxiv_id":null,"evidence_quote":"Monocular metric depth model whose per-image depth maps are aligned to the reconstruction to build pseudo point clouds and FoV heatmaps."},{"cited_title":"What uncertainties do we need in bayesian deep learning for computer vision? Advances in Neural Information Processing Systems, 30, 2017","cited_arxiv_id":null,"evidence_quote":"Aleatoric uncertainty formulation that UGS adopts; the paper relies on higher predicted uncertainty accompanying larger gaze-vector error."},{"cited_title":"Gaze estimation for as- sisted living environments","cited_arxiv_id":null,"evidence_quote":"Uncertainty-aware gaze loss that trains the sigma prediction used by UGS for view selection."},{"cited_title":"Following gaze in video","cited_arxiv_id":null,"evidence_quote":"Cross-view baseline adapted from video gaze following; supplies the strongest adapted comparison in the cross-view experiment."},{"cited_title":"Apriltag: A robust and flexible visual fiducial system","cited_arxiv_id":null,"evidence_quote":"AprilTag-based extrinsic calibration that yields the real-world camera parameters consumed by HIA, UGS, and the cross-view module."}],"review_version":1}