{"id":"1cdebe67-3953-49d1-bb93-9d7f2147891e","arxiv_id":"2504.12535","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A real-time navigation system uses the internal feature maps of a video classifier to locate and annotate the inferior vena cava in cardiac ultrasound without any spatial training labels.","lead":"This paper describes an AI system that guides someone holding an ultrasound probe to find the inferior vena cava in the heart by drawing a green circle on the live image. The appeal is that a novice, using a cheaper handheld ultrasound machine, could get a view that normally requires a trained sonographer.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Localization rates are computed only on videos where the decision model already found the IVC, so the 97%/99.67% numbers exclude all false negatives and overstate end-to-end navigation.","rationale":"The reader's weakest assumption is exactly the conditional filtering, and I agree that it is the most load-bearing concern. The paper's strongest quantified claims are the 97%, 99.67%, and 100% localization rates, and the deployment logic in Algorithm 1 ties localization to a positive decision. Without the sensitivity denominator, the system-level claim that the model guides a user to the IVC is not supported by the localization numbers. The RAP transfer result is less affected by this filter but rests on only 30 patients and has no baseline comparison. I would not reject the paper: the method is plausible, the clinical 62% result is an independent whole-task data point, and the failure mode is an evaluation gap rather than an internal contradiction. The verdict should remain CONDITIONAL pending an unfiltered end-to-end evaluation.","tokens_in":5876,"tokens_out":4612,"duration_ms":49219,"concrete_test":"Run Algorithm 1 plus the decision model on an unfiltered random sample of the 4,000-video hospital dataset with manual IVC-presence labels (or on all 91 Butterfly scans, using every video, not only the 916 positive ones). Report three numbers: decision-model sensitivity on true IVC videos, conditional localization accuracy as in the paper, and end-to-end localization accuracy = (true IVC videos correctly localized) / (all true IVC videos). If the end-to-end number is materially below the reported 97% / 99.67%, the headline claim should be restated as conditional and the navigation claim weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central evidence for the headline localization claims is conditional in a way that matches the deployment logic but not the evaluation claim. Section 3.2 states the hospital test is 'a separate test set of 693 videos where the decision model detected an IVC' and the Butterfly test is '916 videos from 91 scans ... where the decision model detected an IVC.' Algorithm 1 only localizes when p_i > 0: for a negative decision, it 'stop[s] and do[es] not localize.' So the reported 97% and 99.67% accuracy are P(localize correctly | decision says IVC present). The real navigation task is P(find IVC) = P(decision positive | IVC present) × P(localize correctly | positive decision). The decision model's sensitivity on these datasets is not reported, so the end-to-end capability is unknown. If sensitivity is 85%, a 97% conditional rate becomes roughly 82% end-to-end; if sensitivity is lower, the gap is larger. The 100% RAP transfer result is not subject to this exact filter because all 30 test videos are sniff studies, but it is a single 30-patient sample with no baseline. The same manual-review protocol is used throughout, and no center-of-image or random-patch control is reported, so it is unclear how much of the localization accuracy is due to the algorithm rather than to the anatomical prior that the IVC is usually near the image center in positive clips.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes a real-time IVC localization system built on top of a binary video classifier. The authors train an X3D model to detect IVC presence in transthoracic echocardiogram videos, then apply a localization algorithm that extracts the feature map after the third ResNet block, normalizes it over the channel dimension, interpolates it to input resolution, keeps the n highest-valued pixels after removing black pixels and pixels farther than 40 pixels from the mean, and draws a 20-pixel green circle at the mean position. Localization is gated on a positive decision from the classifier. The authors report 97% localization accuracy on 693 hospital videos where the decision model detected an IVC, 99.67% on 916 Butterfly iQ videos with positive decisions, and 100% on 30 sniff videos from a SlowFast RAP classifier without retraining. They frame the contribution as object-detection-like localization without spatial supervision, and they mention a clinical trial of the decision-only navigation system at 62% accuracy.","tokens_in":6195,"tokens_out":7824,"duration_ms":77180,"significance":"If the results held as stated, the paper would demonstrate a practical and inexpensive route to spatially localizing an anatomical structure from a video-level classifier, with possible value for novice ultrasound operators and low-cost devices. The computational efficiency of the feature-map localization is a genuine strength, as is the attempt to test on an out-of-distribution device (Butterfly iQ) and on a different architecture and task (SlowFast RAP). The SHAP analysis is a useful sanity check that the decision model attends to the IVC. The claims are falsifiable and clinically relevant, but the manuscript does not provide code or data, and the evaluation has a conditional-filtering problem, lacks baselines, and rests on a small transfer set. The significance is therefore currently prospective rather than fully established.","major_comments":[{"comment":"The headline localization rates are conditional on the decision model's positive output. Algorithm 1 explicitly does not localize when the predicted probability is not positive, and the 693-video hospital set and 916-video Butterfly set are both described as videos 'where the decision model detected an IVC.' Thus the 97% and 99.67% numbers estimate P(correct localization | positive decision), not the probability that the navigation system finds the IVC in a real scan. The latter is P(true positive decision) × P(correct localization | positive decision), and the decision model's sensitivity is never reported. If sensitivity were 85%, a 97% conditional rate would become roughly 82% end-to-end; lower sensitivity increases the gap. Because the test sets are defined by model predictions rather than by ground-truth labels, it is also unclear whether false positives (videos with no IVC) are included. Please report the full confusion matrix on complete, ground-truth-labeled test sets and make end-to-end success the primary navigation metric.","section":"Section 3.2 and Algorithm 1"},{"comment":"No comparison to any baseline localization method is provided. The evaluation only asks a manual reviewer whether the 20-pixel circle contains the IVC, and positive clips are typically acquired with the IVC near the center of the image. To establish that Algorithm 1 contributes spatial information, the authors should compare against a center-of-image baseline, a random-patch control, a standard attribution map such as CAM or Grad-CAM thresholded the same way, and ideally a supervised object detector. Reporting distance between the predicted point and the IVC center, along with precision/recall for containment, would be more informative than a single binary accuracy.","section":"Sections 3.2-3.3"},{"comment":"The localization algorithm has several free parameters - the feature layer (3rd ResNet block), the number of retained pixels n, the 40-pixel outlier threshold, and the 20-pixel annotation radius - and Section 2.4 says these were 'adjusted' on a set of 90 hospital-grade IVC scans. The paper does not report how the tuning set was used, whether it is disjoint from the 693-video test set, or how sensitive the results are to each parameter. Without this information, the hospital result cannot be considered a fully out-of-sample estimate, and the phrase 'zero-shot on a RAP classification model' is stronger than what is shown because the algorithm's hyperparameters and layer choice were carried over from the IVC model rather than being intrinsic to the method. Please report the tuning protocol, dataset overlap, and a sensitivity analysis.","section":"Section 2.4 and Algorithm 1"},{"comment":"The transfer result rests on 30 patients from a single hospital machine, with 10 patients per RAP group. A 100% success rate over 30 independent trials has a Clopper-Pearson 95% lower bound of roughly 88%, so the result should be reported with a confidence interval. In addition, all 30 videos are sniff scans, so the IVC may be consistently framed; the authors should state the inclusion criteria and, to support the zero-shot generalization claim, test on at least one additional device or viewing protocol.","section":"Section 3.3"}],"minor_comments":[{"comment":"Algorithm 1 uses inconsistent notation: the input is (1, t, h, w) but the interpolation target is (1, tx, hx, wx), and it is not stated whether the 40-pixel and 20-pixel quantities are in original pixels or feature-map pixels.","section":"Section 2.3"},{"comment":"Figure 2's caption and Section 2.2 refer to a 'level bar' and to 'blackout' combinations without defining them; please add a color scale and explain how the SHAP values are normalized.","section":"Section 2.2"},{"comment":"In Section 3.1, '62% ± 9.98 accuracy' should specify whether this is a 95% confidence interval and over how many patients or trials; also clarify how the comparison to commercial systems [5,6] was performed.","section":"Section 3.1"},{"comment":"References [6] and [8] are the same paper and should be merged.","section":"References"},{"comment":"The text in Section 3.3 states the system 'has not been tested for guidance of cardiac views other than the IVC,' which is an appropriate caveat; the title and abstract should avoid implying a general-purpose visual navigation method beyond this single structure.","section":"Section 3.3"},{"comment":"The deployed system is said to be 'available on the Butterfly iQ app,' but the manuscript provides no version, link, or validation of the deployed implementation; clarify whether the evaluated model is exactly the deployed one.","section":"Section 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be an early arXiv version with several redactions (institution, IRB number, clinical-trial details) and no data availability or code release. For a paper claiming clinical deployment and a prospective clinical trial, the redactions and the conditional evaluation in Section 3.2 are significant barriers to independent verification. If the paper is submitted to a peer-reviewed venue, the editors may want to require a trial registry entry or protocol reference, as well as a clear statement of the relationship between the evaluated system and the commercially deployed app."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a real, deployed system: a video classifier (X3D) decides if the IVC is present, and a lightweight post-hoc saliency map on the third ResNet block draws a circle on the IVC in real time. Second, the reported 97% and 99.67% localization accuracies only count videos where the classifier already said 'IVC present.' If the classifier misses the IVC, the system stays silent, and the paper never tells you how often that happens on the test sets. The 62% clinical-trial number for the decision-only system suggests the gap could be large.\n\nWhat's genuinely good: the zero-shot transfer to the RAP classifier (SlowFast) and to the Butterfly iQ device is a meaningful robustness check. A 100% localization rate on 30 sniff studies is not a proof of anything, but it shows the feature-map signal transfers across architecture and task. The algorithm is simple enough to run in real time, and the paper is clearly written about the deployment logic.\n\nThe soft spots are the conditional evaluation, the missing baselines, and the novelty claim. No comparison to CAM or Grad-CAM, despite the method being essentially a channel-normalized, thresholded feature map. The claim to be the first to use learned feature maps for real-time guidance is not supported by the cited literature. The localization thresholds were tuned on 90 videos, and the manual-review protocol is described only briefly. None of these kill the paper, but they mean the headline numbers overstate what a novice will experience.\n\nWho is this for? A reader working on ultrasound guidance or interpretability for video classifiers will find the system description and transfer results useful. A methods researcher will find little that is new beyond a specific application.\n\nRecommendation: send it to peer review, but expect the reviewers to demand end-to-end accuracy numbers, a sensitivity analysis, and an honest positioning against CAM-style methods. If the authors can provide those, this could be a solid application paper.","headline":"A real-time IVC guidance system with a clever zero-shot transfer, but the headline localization numbers are conditional on the detector already finding the IVC, and the novelty claim ignores CAM-style methods.","tokens_in":6699,"tokens_out":1996,"would_cite":false,"duration_ms":20703,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A video classifier trained only on presence labels can point to the inferior vena cava in real time.","keywords":["AI guidance","deep learning","cardiac ultrasound","inferior vena cava","zero-shot localization","feature-map localization","video classification","real-time navigation"],"falsifier":"Run the full decision-plus-localization pipeline on an unselected set of ultrasound sweeps known to contain the IVC, including sweeps the decision model classifies as negative, and count how often the green circle actually lands on the IVC; if the missed detections are numerous, the conditional 97% and 99.67% figures would overstate real-world performance.","tokens_in":5717,"feed_emoji":"🫀","tokens_out":8892,"duration_ms":84604,"temperature":0.7,"pith_summary":"The paper claims that a video classifier trained only to decide whether the inferior vena cava is present can also act as a real-time visual guide: internal feature maps from a late network layer, normalized and interpolated back to the input image, mark the vessel's location without any spatial supervision. This turns a binary label—IVC present or not—into an on-screen cursor that tells an inexperienced operator where to look, and it works zero-shot on a different ultrasound device and on a model trained for a different, more abstract task. The consequence is that ultrasound guidance no longer needs annotated bounding boxes or explicit spatial losses, which lowers the barrier to AI-assisted cardiac imaging in clinics and on portable scanners. The authors report that the guidance system is undergoing clinical trials and is available on a handheld-device app.","feed_headline":"A label-only classifier points to the heart's IVC in real time","feed_subtitle":"The network that detects the vessel can also draw the on-screen guide, even on cheap handheld scanners.","key_machinery":"The central object is Algorithm 1, a lightweight localization step appended to a trained video classifier. It uses the feature map after the third ResNet block of the X3D network, the last layer that preserves both spatial and temporal information, normalizes the channel dimension, interpolates to original pixel space, selects the highest-valued pixels, filters out static black pixels and outliers farther than 40 pixels from the mean, and plots the mean point as a 20-pixel-radius circle. This feature map carries the same signal the paper verifies offline with an explanation method: the input pieces around the IVC are the ones that most change the decision when blacked out. Because the localization step is a few tensor operations rather than a costly per-pixel attribution run, it can execute in real time during scanning.","core_discovery":"The central discovery is that the feature maps of a decision model encode spatial location: the region with the largest normalized activation values after the third ResNet block of an X3D video classifier coincides with the IVC, and averaging those high-activation pixels produces a stable landmark. The localization algorithm runs only when the decision model's binary output is positive, then normalizes the feature map's channels, interpolates the tensor back to input resolution with a 3D spline, retains the highest-valued pixels, removes black pixels and outliers more than 40 pixels from the mean, and draws a 20-pixel-radius green circle at the mean location. On 693 hospital videos for which the decision model detected an IVC, it located the vessel in 97% of cases; zero-shot on 916 videos from a handheld low-cost device it succeeded in 99.67%; and zero-shot on a different model trained to estimate right atrial pressure from sniff videos, with no IVC-detection objective, it located the IVC in all 30 test cases.","pith_inferences":["The paper does not test this, but if the localization signal is a general property of classifiers with localized targets, the same recipe could convert classifiers for other one-location findings—such as gallbladder, fetal heartbeat, or lung consolidation—into real-time finders without detection training.","A discriminating experiment would separate two readings: the algorithm may be locating the anatomical object, or it may simply be highlighting whatever image region the classifier found most decision-relevant. Running Algorithm 1 on a classifier trained for a non-anatomical localized target would tell the difference.","Because localization is gated on a positive decision, the pipeline is a confirm-and-point system rather than a search system; the paper's own 62% clinical-trial accuracy for decision-only navigation suggests the harder part is finding the IVC, not drawing the circle once it is found."],"forward_implications":["IVC guidance can be delivered as an on-screen landmark by a model trained only on video-level presence labels; no bounding-box annotations or spatial loss functions are required.","The same decision-plus-localization design should transfer to other anatomical targets, since any binary video classifier whose decision depends on a spatially localized structure can be turned into a visual guide.","Because the localization step does not alter the decision model, the guidance system can be added to an already validated classifier without retraining.","Zero-shot results on a different device and a different architecture indicate that the localization signal is not tied to a single model or image quality, so portability to lower-cost scanners is plausible."],"supporting_citations":[{"why":"Provides the X3D video-recognition architecture whose third-ResNet-block feature map supplies the input to Algorithm 1.","marker":"[9]"},{"why":"Provides the SlowFast architecture used to show that the localization algorithm transfers zero-shot to a different model family.","marker":"[10]"},{"why":"Supplies the blackout-based explanation method used offline to verify that the input pieces with the largest effect on the decision lie around the IVC.","marker":"[11]"},{"why":"Provides the right-atrial-pressure classification model, the different-architecture, different-objective test on which Algorithm 1 reaches 100% localization.","marker":"[4]"},{"why":"Serves as the commercial instruction-based guidance system whose accuracy is the baseline for the clinical-trial comparison.","marker":"[5]"},{"why":"Serves as the second commercial guidance-system baseline for the reported clinical-trial accuracy.","marker":"[6]"}],"fun_headline_variants":["Zero-shot IVC localization from a binary classifier","Feature map from label model pinpoints IVC on cheap ultrasound","One model detects and locates IVC, even on handheld devices","AI's internal map draws guide for IVC on low-cost scanners","Decision classifier's features reveal IVC position in real time"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported localization accuracies are measured only on videos for which the decision model already said an IVC was present, so the numbers describe how well the system points when it detects, not how often it finds the IVC in the first place.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot IVC localization from a binary classifier","Feature map from label model pinpoints IVC on cheap ultrasound","One model detects and locates IVC, even on handheld devices","AI's internal map draws guide for IVC on low-cost scanners","Decision classifier's features reveal IVC position in real time"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000716,"raw_usage":{"total_tokens":3229,"prompt_tokens":969,"completion_tokens":2260,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":585,"completion_tokens_details":{"reasoning_tokens":2176}},"tokens_in":585,"tokens_out":2260,"duration_ms":15325,"temperature":1.0,"reasoning_tokens":2176,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:28:52.555248+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the full decision-plus-localization pipeline on an unselected set of ultrasound sweeps known to contain the IVC, including sweeps the decision model classifies as negative, and count how often the green circle actually lands on the IVC; if the missed detections are numerous, the conditional 97% and 99.67% figures would overstate real-world performance.","supporting_citations":[{"cited_title":"(2020, June)","cited_arxiv_id":null,"evidence_quote":"Provides the X3D video-recognition architecture whose third-ResNet-block feature map supplies the input to Algorithm 1."},{"cited_title":"(2019, October)","cited_arxiv_id":null,"evidence_quote":"Provides the SlowFast architecture used to show that the localization algorithm transfers zero-shot to a different model family."},{"cited_title":"M., & Lee, S.-I","cited_arxiv_id":null,"evidence_quote":"Supplies the blackout-based explanation method used offline to verify that the input pieces with the largest effect on the decision lie around the IVC."},{"cited_title":"I.,Moreno, M.,Ignatowski, D.,","cited_arxiv_id":null,"evidence_quote":"Serves as the commercial instruction-based guidance system whose accuracy is the baseline for the clinical-trial comparison."}],"review_version":1}