{"id":"1729af74-db02-4feb-8a1f-c83fc1e48e8a","arxiv_id":"2505.07511","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"MAIS adds a lightweight memory-attention bank to SAM-Med3D that stores past clicks and masks, improving interactive 3D segmentation refinement over memory-free fine-tuning on several medical datasets.","lead":"This paper introduces MAIS, a memory module for interactive medical image segmentation that remembers past user clicks and previous segmentations. The authors show it improves fine-tuned SAM-Med3D refinement on several CT/MRI benchmarks, especially when training data is scarce.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that MAIS consistently beats Ft-SAM3D is contradicted by the paper's own Table 1 on ACDC, where MAIS is worse at most interaction counts, and no significance testing is reported.","rationale":"I read the paper's contribution as a memory-attention mechanism on top of SAM-Med3D, and the evidence needed for that contribution is a consistent refinement gain over the no-memory fine-tuned variant under the same click protocol. That evidence is not established because Table 1 contains a direct counterexample on ACDC. The reader's simulation concern is real but secondary: the click simulator affects external validity, whereas the ACDC result is internal to the reported evaluation and therefore more load-bearing. I would keep the conditional verdict: the method is plausible and has a clear ablation, but the headline empirical claim must be revised or supported with significance testing before acceptance as stated.","tokens_in":9277,"tokens_out":6144,"duration_ms":55663,"concrete_test":"Reconstruct the per-case predictions behind Table 1 and compute paired differences (MAIS minus Ft-SAM3D) at each click count on ACDC, with bootstrapped 95% confidence intervals or a signed-rank test across at least 3 seeds. If the ACDC differences at 1-50 clicks remain negative or include zero, revise Section 3.2 to claim dataset-specific gains rather than consistent superiority; also clarify the asterisk/underlining semantics in Table 1, which are ambiguous as printed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3.2 the paper states that MAIS 'consistently outperform[s] Ft-SAM3D fine-tuning, particularly as the number of interactions increases' and that in the one-shot scenario MAIS 'significantly outperforms' Ft-SAM3D except on ACDC. Table 1 does not support this. On ACDC, MAIS is below Ft-SAM3D at the 1-click, 10-click, 20-click, and 50-click rows in the one-shot setting (61.38 vs 62.65, 70.46 vs 72.61, 71.85 vs 73.49, 74.09 vs 74.66), and it is also lower at the first click in the 10%, 50%, and 70% data columns. It overtakes Ft-SAM3D only at 150 clicks in the one-shot setting. AMOS-CT one-shot also shows MAIS below Ft-SAM3D at the first click (77.14 vs 77.58). With no error bars, confidence intervals, or repeated-seed statistics anywhere in the paper, the central claim of consistency rests on uncontrolled point estimates. If the mechanism is meant to help across sessions, the dataset where it does not help is not a minor exception; it is evidence against the claim as stated. The memory design may still be useful on some datasets, but the paper's strongest claim needs to be narrowed or statistically supported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MAIS, a memory-attention mechanism for interactive 3D medical image segmentation built on top of SAM-Med3D. A FIFO memory bank stores sparse click embeddings and dense previous-mask embeddings, and a memory attention block conditions the image embedding on these stored memories before the mask decoder produces the next segmentation. The authors fine-tune only the prompt encoder, mask decoder, and memory module on HaN-SEG, ACDC, AMOS-CT, and AMOS-MR, and compare MAIS against fine-tuned SAM-Med3D (Ft-SAM3D), zero-shot SAM3D, and oracle nn-UNet under one-shot and partial-data settings. They claim that MAIS consistently outperforms Ft-SAM3D, especially with more interactions and in low-data regimes, and that it approaches oracle performance even with 10% training data. The paper also reports ablation studies on memory bank size, sparse versus dense memory, and computational costs.","tokens_in":9613,"tokens_out":3825,"duration_ms":35305,"significance":"If the central claim holds, the paper would make a useful contribution: it introduces a lightweight, modular way to add temporal context to interactive segmentation, with clear potential to reduce redundant corrections in clinical annotation workflows. The strength of the paper is its focus on a practical setting (fine-tuning a foundation model on small target datasets) and its explicit accounting of parameter counts, GPU memory, and training and inference time. The reported gains on HaN-SEG and AMOS are encouraging. However, the paper's own Table 1 does not support the 'consistently outperform' claim, and the absence of error bars or significance tests makes the performance differences difficult to trust. The core idea is plausible and worth publishing, but the evidence as presented needs revision.","major_comments":[{"comment":"The statement that MAIS 'consistently outperform[s] Ft-SAM3D fine-tuning' is not supported by the data in Table 1. On ACDC in the one-shot setting, MAIS is below Ft-SAM3D at the 1-, 10-, 20-, and 50-click rows (61.38 vs. 62.65, 70.46 vs. 72.61, 71.85 vs. 73.49, and 74.09 vs. 74.66), and it is also lower at the first click in the 10%, 50%, and 70% data columns. MAIS exceeds Ft-SAM3D at 150 clicks in the one-shot case, but that value (76.54) is marked as underperforming the zero-shot SAM3D baseline (77.75). The 'consistently' claim therefore needs to be narrowed, or supported by a dataset-by-dataset analysis.","section":"Section 3.2, Table 1"},{"comment":"No error bars, confidence intervals, or repeated-seed statistics are reported anywhere in the paper. Given that the text uses the word 'significantly outperforms' to describe the comparison with Ft-SAM3D, the authors should either add statistical testing (e.g., paired tests across test subjects or multiple seeds) or remove the significance claim and present the numbers as point estimates.","section":"Section 3.2 and all experimental tables"},{"comment":"The user-interaction simulation samples clicks from regions where previous predictions were incorrect, but the paper does not validate this against real human correction behavior. Since the conclusions are framed in terms of clinical workflow benefits, the transfer of the measured gains to actual annotation sessions is untested. This limitation should be stated explicitly, and the clinical claims should be tempered accordingly.","section":"Section 2.3 and Conclusions"}],"minor_comments":[{"comment":"The keyword 'Fundation Models' should be spelled 'Foundation Models'.","section":"Keywords"},{"comment":"There is a missing space in 'vanillaSAM-Med3D' and a missing space after the colon in 'Sparse Memory: this memory bank'; these should be corrected.","section":"Section 2.1.2"},{"comment":"The sentence beginning 'we simulate sparse visual prompts...' should start with a capital letter, and the simulation details (how many clicks per iteration, how incorrect regions are sampled, and how positive versus negative clicks are assigned) should be described precisely enough for reproduction.","section":"Section 2.3"},{"comment":"The table formatting is dense and contains missing entries (dashes). The caption should also explain the asterisk notation more clearly, since the current text says 'underperforming SAM3D' but the reader must infer the comparison row by row.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The paper presents a reasonable and practically motivated idea, but the strongest claim in the abstract and Section 3.2 goes beyond what Table 1 shows, particularly on ACDC. The lack of any uncertainty quantification is a more general problem. I believe the paper can be made suitable for publication by revising the claims and adding proper statistical support, so I recommend major revision rather than rejection. I also note that no code is released, which would help verify the reported comparisons."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: MAIS is a sensible adaptation of SAM2's memory attention to 3D medical interactive segmentation, and the paper has a real, if incremental, empirical contribution. It deserves refereeing, but the central claim needs to be narrowed and supported with statistics.\n\nWhat's new: the memory-attention module stores past clicks and mask embeddings in a FIFO bank and conditions SAM-Med3D's decoder on them, omitting SAM2's memory encoder. The module is light: 2.84M parameters versus 7.31M for SAM2's memory module, and inference GPU memory goes up only 16% in the heaviest configuration. The ablations on memory size and sparse vs. dense memory are the right experiments, and they support the sparse+dense choice. On HaN-SEG, AMOS-CT, and AMOS-MR, MAIS beats fine-tuned SAM3D in most settings, especially with few training images and more clicks, and it approaches oracle nnU-Net on HaN-SEG with 10% data. That is a genuine result.\n\nThe soft spot is the paper's own wording. It says MAIS 'consistently outperform[s] Ft-SAM3D' and that in the one-shot setting it 'significantly outperforms' except on ACDC. Table 1 contradicts that. On ACDC one-shot, MAIS is below Ft-SAM3D at 1, 10, 20, and 50 clicks (61.38 vs 62.65 at one click) and only wins at 150. AMOS-CT one-shot also has MAIS behind at the first click. There are no error bars, no repeated seeds, no significance tests anywhere in the paper, so 'consistently' and 'significant' are doing more work than the data can support. The ACDC result is not a minor exception; it is a direct counterexample to the claim as stated. The click simulation—sampling from regions where the previous prediction was wrong—is plausible but unvalidated against human behavior, so the clinical-annotation claims should be softer. No code is released, which makes it harder to check the point estimates. The citation pattern looks fine; it builds on the relevant SAM/SAM2/SAM-Med3D line.\n\nOverall: this is a clean, useful idea with a solid complexity analysis and a suggestive but not conclusive empirical story. I'd recommend sending it to peer review—the right referee will ask for corrected claims, significance testing or repeated runs, and code. The direction is worth engaging with.","headline":"A useful memory-attention twist on SAM-Med3D with real gains on three datasets, but the paper's 'consistently outperform' claim is contradicted by its own ACDC numbers and no significance testing backs it.","tokens_in":10096,"tokens_out":3062,"would_cite":true,"duration_ms":25902,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MAIS shows that remembering past clicks and masks lets interactive segmentation keep improving where memoryless models plateau.","keywords":["interactive segmentation","memory attention","SAM-Med3D","foundation models","medical image segmentation","3D segmentation","low-data fine-tuning","user clicks"],"falsifier":"Run the same four-dataset protocol with clicks generated by a boundary-oriented or clinician-in-the-loop strategy instead of random samples from error regions; if MAIS's advantage over Ft-SAM3D disappears or narrows sharply, the claim that memory drives refinement in real workflows is falsified. A second test: replace the stored past interactions with random but equally sized memory stacks; if Dice gains persist, the improvement comes from extra capacity rather than from remembering what the user corrected.","tokens_in":9096,"feed_emoji":"🧠","tokens_out":6818,"duration_ms":55360,"temperature":0.7,"pith_summary":"The paper sets out to show that the plateau in interactive medical segmentation comes from missing memory: models such as SAM-Med3D treat every user click as a fresh event, discarding the history of corrections. To fix this, the authors propose a memory-attention mechanism (MAIS) that stores click embeddings and previous mask embeddings in a FIFO bank and conditions the current image embedding on that bank before mask decoding. Across CT and MRI datasets covering head-and-neck, heart, and abdomen, they report that MAIS consistently outperforms fine-tuned SAM-Med3D, with the gap widening as the number of clicks grows, and that it approaches or exceeds a nn-UNet oracle even when trained on 10% of the data. If this is right, interactive annotation with foundation models becomes a genuinely incremental process: each correction compounds earlier ones instead of being wasted.","feed_headline":"Memory bank lets segmentation improve with every click","feed_subtitle":"A memory-attention module on SAM-Med3D beats click-by-click fine-tuning and nears oracle accuracy at 10% data.","key_machinery":"The load-bearing object is the memory-attention block paired with a FIFO memory bank of the latest $N=60$ interaction embeddings, split into sparse click embeddings and dense mask embeddings. The block performs self-attention over the two memory stacks with a convolutional transformer, adds the self-attended dense output to the current image embedding, and then runs cross-attention between the sparse and dense memories; only after this conditioning does the mask decoder see the embedding. The design omits the separate memory encoder used in SAM2, which the paper reports keeps computational overhead low while preserving the refinement gains. This mechanism is what lets each new click draw on all earlier corrections.","core_discovery":"The central claim is that temporal context, not extra parameters or more data, is what makes repeated user corrections pay off. MAIS keeps the SAM-Med3D image encoder, prompt encoders, and mask decoder, and inserts a memory bank holding the latest $N=60$ sparse (click) and dense (mask) embeddings. A memory-attention block self-attends over the memory stacks, adds the resulting dense output to the image embedding, then cross-attends between the sparse and dense memories before the decoder produces the mask. In the paper's experiments this configuration beats the same architecture with the memory module removed on all four datasets, continues to improve through 50 to 150 clicks where the memoryless baseline flattens around 10, and reaches Dice scores comparable to an nn-UNet oracle trained on the full target data even when MAIS is fine-tuned on a tenth of it.","pith_inferences":["A natural extension the paper does not run is to apply the same memory bank to 2D SAM and to SAM2-style video object segmentation, since the mechanism is defined on embeddings and is backbone-agnostic; the paper only demonstrates it on SAM-Med3D.","The FIFO bank implies a session-boundary test: if memory persists across separate annotation sessions, clinicians could resume work where they left off; the paper evaluates only within a single simulated session.","Because gains appear largest in low-data regimes, a testable corollary is that memory attention acts partly as a regularizer or as a curriculum over the model's own mistakes; this could be checked by ablating memory during training versus inference only.","The simulated-click assumption is the main gap between the reported numbers and clinical use; a user study with real radiologists' clicks would be the direct validity test."],"forward_implications":["Interactive refinement no longer plateaus: MAIS keeps gaining up to 50–150 clicks, while the memoryless baseline saturates near 10 clicks.","Task-specific accuracy can be reached with dramatically less labeled data: with 10% of training data MAIS lands near or above oracle nn-UNet on several datasets.","Combining sparse and dense memory is essential; click-only memory underperforms and mask-only memory is weaker than the fused bank.","The memory module is cheap enough to retrofit: 2.84M parameters and roughly 6 seconds extra inference per image in the heaviest configuration.","The benefit transfers across CT and MRI and across head-neck, abdomen, and heart tasks, suggesting the mechanism is not dataset-specific."],"supporting_citations":[{"why":"Supplies the SAM-Med3D ViT backbone, pretrained weights, and prompt/decoder design that MAIS builds on and fine-tunes.","marker":"Wang et al., 2024b"},{"why":"Provides the memory-attention idea and the SAM2 architecture MAIS is inspired by and compares parameter efficiency against.","marker":"Ravi et al., 2024"},{"why":"Defines the SAM prompt-driven ViT segmentation paradigm that SAM-Med3D extends and MAIS inherits.","marker":"Kirillov et al., 2023"},{"why":"Serves as the oracle nn-UNet baseline trained from scratch that MAIS approaches in low-data settings.","marker":"Isensee et al., 2021"},{"why":"HaN-Seg dataset drives the memory-bank size and sparse-versus-dense ablation studies.","marker":"Podobnik et al., 2024"},{"why":"ACDC cardiac MRI dataset is one of the four external validation benchmarks.","marker":"Bernard and Lalande, 2018"},{"why":"AMOS CT and MR split provides the abdominal validation benchmarks.","marker":"Ji et al., 2022"},{"why":"SAM3D motivates the slice-wise-to-3D adaptation context that MAIS refines with memory.","marker":"Bui et al., 2024"}],"fun_headline_variants":["Memory bank turns repetitive clicks into sharper masks","Clicks that remember: temporal memory boosts segmentation","Temporal memory beats click-by-click fine-tuning","With memory, 10% data nears oracle accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that simulated clicks sampled from regions where the previous prediction was wrong behave like real human corrections; if real users click less often, on boundaries, or in more informative locations, the measured refinement gains may shrink.","fun_headline_variants_meta":{"raw":{"variants":["Memory bank turns repetitive clicks into sharper masks","Clicks that remember: temporal memory boosts segmentation","Temporal memory beats click-by-click fine-tuning","With memory, 10% data nears oracle accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1375,"prompt_tokens":807,"completion_tokens":568,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":423,"completion_tokens_details":{"reasoning_tokens":508}},"tokens_in":423,"tokens_out":568,"duration_ms":5407,"temperature":1.0,"reasoning_tokens":508,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:14:54.752899+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same four-dataset protocol with clicks generated by a boundary-oriented or clinician-in-the-loop strategy instead of random samples from error regions; if MAIS's advantage over Ft-SAM3D disappears or narrows sharply, the claim that memory drives refinement in real workflows is falsified. A second test: replace the stored past interactions with random but equally sized memory stacks; if Dice gains persist, the improvement comes from extra capacity rather than from remembering what the user corrected.","supporting_citations":[{"cited_title":"Han-seg: The head and neck organ-at-risk ct and mr segmentation challenge","cited_arxiv_id":null,"evidence_quote":"HaN-Seg dataset drives the memory-bank size and sparse-versus-dense ablation studies."}],"review_version":1}