The reviewed record of science sign in
Pith

arxiv: 2607.05310 · v1 · pith:VZDFKOP3 · submitted 2026-07-06 · cs.AI

Evaluating and Understanding Model Editing for Medical Vision Language Models

Guli Zhu , Chenwei Wu , Liyue Shen This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-07 18:02 UTCglm-5.2pith:VZDFKOP3record.jsonopen to challenge →

classification cs.AI
keywords model editingmedical vision-language modelsbenchmarklatent space geometryanisotropycone effectknowledge editingclinical AI
0
0 comments X

The pith

No medical VLM editor wins on all clinical criteria

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces M3Bench, a benchmark of 16,276 questions that evaluates model editing for medical vision-language models along ten clinically motivated axes: reliability, locality, and generality across image variation, text variation, modality shifts, and clinical composition, plus a novel temporal-consistency task. The authors evaluate four representative editors (LoRA, MEND, GRACE, BalancEdit) across six VLMs and find a fundamental trade-off: gradient-based editors (LoRA, MEND) reliably fix targeted errors and transfer to semantically similar cases but catastrophically corrupt unrelated knowledge, while memory-based editors (GRACE, BalancEdit) preserve locality but fail to generalize to compositional multi-finding cases and are highly sensitive to backbone-specific hyperparameter choices. The paper attributes this trade-off to the anisotropic geometry of medical VLM latent spaces, where concept embeddings cluster into a narrow cone. Gradient-based editors warp this cone globally, causing non-target concept drift; memory-based editors apply binary spatial gating that misses interleaved concepts. The cone effect is inherited from base VLMs and amplified by medical fine-tuning, which explains why editing is harder on medical models than on general ones and why hyperparameters do not transfer across backbones.

Core claim

The central mechanism is what the authors call the cone effect: medical VLM representations are profoundly anisotropic, concentrating into a narrow cone on the hypersphere (mean cosine similarity around 0.8, mean resultant length around 0.9). This geometric crowding means that gradient-based weight updates inevitably shift nearby non-target concepts, producing locality collapse, while memory-based binary activation spheres cannot separate interleaved single-finding and multi-finding representations, producing compositional-generalization failure. The cone tightness varies across backbones and worsens with medical fine-tuning, directly explaining the observed hyperparameter sensitivity of the

What carries the argument

M3Bench, a clinically grounded benchmark with 10 tasks (T0-T5) spanning reliability, image/text/modality/composition locality and generality, and temporal consistency; four editors taxonomized as gradient-based (LoRA, MEND) versus memory-based (GRACE, BalancEdit); latent-space geometric analysis via cosine-similarity distributions, mean resultant length, centroid-shift visualization, and activation-radius sweeps

If this is right

  • Any future medical VLM editor must address the cone effect directly, either by isotropizing representations before editing or by developing non-binary gating that can handle interleaved concepts.
  • Clinical deployment of model editing cannot rely on a single method; the choice between gradient-based and memory-based editors depends on whether the clinical use case prioritizes generality or locality.
  • Temporal consistency and compositional transfer remain unsolved for all tested editors, indicating that current editing techniques are not yet safe for longitudinal clinical reasoning or multi-finding cases.
  • Hyperparameters for memory-based editors must be recalibrated per backbone due to backbone-specific cone tightness, making out-of-the-box deployment impractical without per-model tuning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the cone effect is the root cause, then representation-level interventions such as whitening, contrastive re-alignment, or cone-axis decorrelation applied before editing could narrow the generality-locality gap without changing the editor itself.
  • The finding that medical fine-tuning amplifies anisotropy suggests the problem is not inherent to editing methods but is introduced during domain adaptation, raising the question of whether different medical fine-tuning objectives could preserve more isotropic geometry.
  • The BELoRA hybrid shows that combining memory-based routing with LoRA adapters improves over both parent methods but still fails on composition and temporality, implying that the binary-gating limitation is the binding constraint rather than the parameter-update mechanism.
  • The gradient incompatibility analysis (negative cosine similarity among same-layer edits under large radius) suggests that memory-based editors could benefit from gradient-aware key assignment that avoids grouping incompatible edits.

Load-bearing premise

The benchmark's clinical validity depends on an LLM-based pipeline correctly extracting standardized medical attributes (condition, anatomical site, modality, progression status) from raw QA pairs. If the LLM annotator systematically mislabels attributes, the controlled evaluation sets would not isolate the intended clinical variables, and all downstream performance comparisons would be confounded. The paper does not report human validation rates or inter-annotator agreement.

What would settle it

If a future editor that explicitly isotropizes the latent space before editing does not improve the locality-generality trade-off, the cone-effect explanation would be weakened as the primary causal mechanism.

Figures

Figures reproduced from arXiv: 2607.05310 by Chenwei Wu, Guli Zhu, Liyue Shen.

Figure 1
Figure 1. Figure 1: Post-deployment workflow and model editing performance. (a) Post [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Construction pipeline of our proposed Multimodal Medical Model [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Radar plot summarizing single and sequential editing performance across 10 tasks and 4 medical VLM backbones. General VLM results in Appendix. BalancEdit stands out as the most well-rounded, but composition, temporality, and overall generality remain challenging. BalancEdit (BE) achieves the strongest overall performance on most backbones (Tab. 1, Over￾all), reflecting consistently competitive reliability … view at source ↗
Figure 4
Figure 4. Figure 4: (a) Histograms of the cosine similarity between 1.5M pairs of embeddings across datasets. The average cosine similarity and mean resultant length are high and the minimum is above 0, indicating that the embedding space is a narrow cone. The same phenomenon exists for the pooled and the last hidden state. (b) Heatmap of cosine similarity for 5 different anatomic sites (500 image-text pairs each) in LLaVA￾Me… view at source ↗
Figure 5
Figure 5. Figure 5: Left: LoRA edits move all topic centroids of intended (Brain and Lung) and unintended targets (Abdomen and Heart), where BE restricts movement almost en￾tirely to the edited topics. Middle: BE strictly restricts it edit activation regions by setting a binary decision sphere. Right: LoRA fix targeted errors by broadly warping representations, and this global reshaping inevitably collides with neighboring cl… view at source ↗
Figure 6
Figure 6. Figure 6: Composition probe: single- vs. multi-finding geometry under editing. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Architecture sweep using BELoRA over varying model components and depths. Left: Task-wise results. Right: Harmonic mean of generality tasks, lo￾cality tasks, and all tasks. roughly 18M trainable parameters, whereas BalancEdit modifies the final MLP layer’s up_projection with 13.8M trainable parameters. This raises two natural questions: i) given a similar parameter budget, does LoRA’s stronger generality c… view at source ↗
Figure 8
Figure 8. Figure 8: Left: Different backbones have different optimal points (highest overall score marked by red circle), and the same hyperparameter may have opposite effects in differ￾ent backbones. Right: Extremely low or high radius result in degraded performance. 5.5 Backbone-Dependent Hyperparameter Sensitivity in Memory-Based Editors Memory-based editors such as BalancEdit store edit inputs and their correspond￾ing edi… view at source ↗
Figure 9
Figure 9. Figure 9: BE Failed Cases for T4-G and T5 on Huatuo34B [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Benchmark distribution (inner ring: modality; middle ring: body part; outer ring: lesion IDs). Center reports global counts (#images, #questions). T0: Reliability on Edited Errors. T0 is formed by sampling originally incor￾rect cases from the base dataset as edit instances. Each edit instance consists of a single edit request without additional probes, and tests whether the editor successfully corrects th… view at source ↗
read the original abstract

Model editing promises a fast, targeted way to correct post-deployment mistakes in medical vision-language models (VLMs) without costly retraining. However, existing multimodal model editing benchmarks focus on general-purpose tasks and do not reflect realistic clinical domain requirements and variability. To address this, we introduce M3Bench, a clinically grounded benchmark for multimodal model editing that evaluates whether an edit remains reliable, precise, and generalizable under the challenges of image and text variation, modality and protocol shifts, clinical knowledge composition, and temporal progression. M3Bench contains 16,276 questions spanning diverse anatomy, modalities, and specialties, and supports both single and sequential edits. By evaluating 4 representative editors across 6 medical and general VLMs, we find that no method excels across all criteria. Gradient-based editors achieve strong transfer but suffer from catastrophic locality violations, whereas memory-based methods preserve locality but lack compositional generality and exhibit high backbone-dependent hyperparameter sensitivity. We further attribute these failures to the latent space geometry of VLMs and how different editing methods shift its landscape. Overall, M3Bench establishes a rigorous clinical stress test for multimodal model editing and offers actionable guidance for safer post-deployment adaptation. The benchmark is publicly available at https://github.com/BioMed-AI-Lab-U-Michgan/M3Bench .

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 8 minor

Summary. This paper introduces M3Bench, a clinically grounded benchmark for evaluating model editing methods on medical vision-language models (VLMs). The benchmark defines 10 tasks spanning reliability, locality, generality (across image, text, modality, and composition axes), and temporal consistency, constructed from VQA-RAD, PMC-VQA, SLAKE, and PadChest-GR. The authors evaluate 4 editors (LoRA, MEND, GRACE, BalancEdit) across 6 VLMs (4 medical, 2 general) in both single and sequential editing settings, using free autoregressive generation rather than teacher-forced evaluation. The central findings are: (1) no single editing method dominates across all criteria; (2) gradient-based editors (LoRA, MEND) achieve strong reliability and generality but suffer catastrophic locality failures; (3) memory-based editors (GRACE, BalancEdit) preserve locality but lack compositional generality and exhibit backbone-dependent hyperparameter sensitivity. The authors attribute these tradeoffs to the anisotropic 'cone effect' in medical VLM latent spaces, supported by geometric analysis of embedding distributions, centroid shifts under editing, and a hybrid BELoRA ablation.

Significance. The paper makes a valuable contribution to the model editing literature by providing the first systematic, clinically grounded benchmark for multimodal model editing in the medical domain. The task design—particularly the clinically motivated axes of modality shift, compositional consistency, and temporal progression—goes meaningfully beyond existing general-domain editing benchmarks. The use of free autoregressive generation (avoiding teacher-forcing artifacts) is a methodological strength, as is the public release of the benchmark. The geometric analysis linking editing failures to representation anisotropy provides actionable mechanistic insight. The BELoRA hybrid ablation and the memory-collapse analysis (Appendix C.1) add depth. The finding that medical fine-tuning amplifies the cone effect (Table 14) is a notable observation with implications beyond editing.

major comments (3)
  1. §3.1, Stage 1 (Clinical Attribute Distillation): The entire benchmark construction pipeline depends on LLM-extracted structured attributes (condition, anatomical site, modality, progression status), yet no human validation, inter-annotator agreement, or accuracy estimate for this step is reported. This is load-bearing: if the LLM annotator systematically mislabels attributes (e.g., progression status for T5, semantic equivalence for T1G/T2G, multi-label findings for T4), then the controlled evaluation sets would not isolate the intended clinical variables, confounding all downstream performance comparisons. The paper should report at least a sampled human validation rate (e.g., 100–200 instances) for the distilled attributes, particularly for the more complex attributes like progression status and multi-label findings. Without this, the benchmark's clinical validity claims are not fully
  2. §5.1–5.3: The paper claims the cone effect 'drives' the observed editing tradeoffs (§5, 'This geometric bottleneck drives the observed behavioral differences'), but the evidence is primarily correlational. The authors show that representations are anisotropic (Fig. 4) and that different editors reshape the latent space differently (Fig. 5), but they do not directly demonstrate causality—for example, by comparing editing performance on representations with artificially reduced vs. preserved anisotropy, or by showing that cone tightness quantitatively predicts editing difficulty across backbones in a regression. The cross-backbone analysis in §5.5 and Table 14 provides suggestive evidence, but the causal framing should be softened or strengthened with a controlled test.
  3. §4, Hyperparameter Tuning: The paper states 'All method-specific hyperparameters are tuned to ensure optimal performance' but does not specify the tuning protocol—on which data split, using which metric, and with what search procedure. This is important for fair comparison because BalancEdit's α values differ by an order of magnitude across backbones (Appendix D.3: 0.2 for LLaVA-Med, 0.05 for Huatuo-7B), and the sensitivity analysis (§5.5, Fig. 8) shows performance varies dramatically with α. If hyperparameters were selected using the same evaluation data, this could bias the comparison. The paper should clarify the tuning protocol and ideally report results under a fixed (non-tuned) hyperparameter setting as a sensitivity check.
minor comments (8)
  1. §2.1: The model referred to as 'BioMed-Qwen' in the main text and 'BioMed-Qwen2-VL 2B' in Table 3 should use a consistent name. The reference [7] titles it differently again.
  2. §2.1: 'Qwen3.5-2B' is cited with reference [27] titled 'Qwen3.5-omni technical report'—the naming should be reconciled.
  3. Table 1: The 'Overall' column for LLaVA-Med shows MEND=0.07 and GRACE=0.07, which are identical to 2 decimal places. This is plausible but worth verifying given the harmonic mean computation.
  4. Fig. 3: The radar plots are small and some axis labels are difficult to read. Consider enlarging or providing a table equivalent for single-edit results in the main text (currently only in Appendix Tables 2–5).
  5. §3.2, Metrics: The temporality metric is defined as '1 - percentage an edit at earlier time introduces factual errors in later follow-up studies,' but the exact computation (e.g., what counts as a 'factual error' on the follow-up) could be specified more precisely, especially given the binary presence coding described in Appendix A.4.
  6. Appendix A.4, T5: The binary presence code maps stable/improving/worsening all to '11' (present in both). This collapses clinically distinct progression patterns; the paper should note this as a limitation or justify why this granularity is sufficient.
  7. §5.4: The BELoRA results are described as outperforming both BE and LoRA, but the specific numerical comparison is only in Tables 6–7 (harmonic means). A direct comparison to Table 1 numbers would help readers assess the magnitude of improvement.
  8. Reference [3] (Cai et al., 2025) appears to be about prompt politeness effects on LLMs and seems tangential to the context where it is cited (§3.2, T2G). Verify this is the intended citation.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the thorough and constructive assessment. The referee identifies three major concerns: (1) lack of human validation for LLM-extracted clinical attributes in the benchmark construction pipeline, (2) correlational rather than causal evidence for the cone effect driving editing tradeoffs, and (3) insufficient specification of the hyperparameter tuning protocol. We address each below and commit to revisions for all three points.

read point-by-point responses

  1. Referee: §3.1, Stage 1 (Clinical Attribute Distillation): No human validation, inter-annotator agreement, or accuracy estimate for LLM-extracted attributes is reported. This is load-bearing for benchmark validity.

    Authors: The referee is correct that this is a load-bearing step and that the current manuscript does not report validation statistics for it. We will address this in revision. Specifically, we will conduct a sampled human validation of the distilled attributes on 200 instances stratified across the four source datasets and the more complex attribute types (progression status, multi-label findings, semantic equivalence for T1G/T2G). We will report per-attribute accuracy and inter-annotator agreement (Cohen's kappa) between two human annotators, and we will include this as a new subsection in the Appendix. We agree that without this, the benchmark's clinical validity claims rest on an unverified annotation step. We will also report the LLM prompt template used for attribute distillation to support reproducibility. If the validation reveals systematic mislabeling on any attribute, we will report it transparently and discuss implications for the affected tasks. revision: yes

  2. Referee: §5.1–5.3: The paper claims the cone effect 'drives' editing tradeoffs, but evidence is correlational. No controlled test (e.g., artificially reduced vs. preserved anisotropy) or quantitative regression of cone tightness predicting editing difficulty is provided.

    Authors: We agree that the current evidence is primarily correlational and that the causal framing ('drives') is stronger than what we have demonstrated. We will make two changes. First, we will soften the causal language throughout §5, replacing 'drives' with 'is associated with' or 'explains' where appropriate, and adding an explicit statement that the geometric analysis provides mechanistic corroboration rather than a controlled causal test. Second, we will add a quantitative cross-backbone regression analysis: we will regress per-backbone editing performance (overall harmonic mean and locality/generality sub-scores) against cone tightness (mean resultant length R) across the six VLMs, and report the correlation coefficient and significance. This provides a stronger quantitative link than the current visual evidence alone. We note that a fully controlled causal test—e.g., artificially de-anisotropizing representations and re-evaluating editing—would require intervening on the representation geometry in a way that is non-trivial to do without confounding other model properties (e.g., via post-hoc projection, which changes the downstream computation). We will discuss this limitation explicitly and flag it as a direction for future work. revision: partial

  3. Referee: §4, Hyperparameter Tuning: The tuning protocol is not specified (data split, metric, search procedure). BalancEdit's α differs by an order of magnitude across backbones, and if hyperparameters were selected on evaluation data, comparisons could be biased.

    Authors: The referee raises a valid concern. We will clarify the tuning protocol in revision. To be transparent: hyperparameters were selected per-backbone by sweeping over the specified grid (e.g., α ∈ {0.01, 0.02, 0.05, 0.08, 0.1, 0.2, 0.5, 0.8} for BalancEdit) and selecting the value that maximized overall harmonic mean on the benchmark evaluation set. We acknowledge that this means hyperparameter selection used the same evaluation data, which could optimistically bias the reported numbers for methods with sensitive hyperparameters (particularly BalancEdit and GRACE). We will add this caveat explicitly. To address the concern about bias, we will additionally report results under a fixed (non-tuned) α setting across all backbones as a sensitivity check—specifically, we will report BalancEdit performance at a single shared α value (e.g., α = 0.1) for all backbones, showing how performance degrades when backbone-specific tuning is removed. This will make the backbone-dependent sensitivity visible in a controlled way. For LoRA and MEND, hyperparameters (rank, learning rate, epochs) were less sensitive and were set to standard values without per-backbone search; we will state this explicitly as well. revision: yes

Circularity Check

0 steps flagged

No significant circularity found; this is an empirical benchmark paper whose claims are grounded in external datasets, externally developed editing methods, and independent geometric measurements.

full rationale

This paper is a benchmark and evaluation study, not a derivation chain. The central claims (gradient-based editors trade generality for locality; memory-based editors show the reverse; cone effect explains these tradeoffs) are all supported by empirical experiments on externally developed datasets (VQA-RAD, PMC-VQA, SLAKE, PadChest-GR), externally developed editing methods (LoRA, MEND, GRACE, BalancEdit), and externally developed VLMs (LLaVA-Med, HuatuoGPT, BioMed-Qwen). The cone-effect analysis cites Liang et al. [19] (external authors) for the known phenomenon, then independently measures mean resultant length R and pairwise cosine similarities on 1.5M embedding pairs — this is an empirical observation, not a self-referential derivation. The BELoRA hybrid is a new construction evaluated empirically against its parent methods, not a renaming or definitional reduction. Hyperparameter tuning of α on the benchmark (Appendix D.3) is standard practice for an evaluation paper and does not constitute circularity in the sense of a 'prediction' being forced by a fit. The only minor concern is that hyperparameters are tuned on the same benchmark they are evaluated on, but this is a methodological limitation (potential overfitting to the benchmark), not a circular derivation. No step in the paper reduces to its inputs by construction.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The paper introduces no new entities (particles, forces, dimensions, etc.). It introduces a benchmark (M3Bench) and a hybrid editing variant (BELoRA), but these are methodological artifacts, not postulated physical or mathematical entities. The free parameters are all standard hyperparameters of existing editing methods, tuned per-backbone. The most load-bearing axiom is the unvalidated LLM annotation pipeline; the most ad-hoc assumption is the causal interpretation of the cone effect.

free parameters (6)
  • BalancEdit alpha (per backbone) = LLaVA-Med: 0.2, Qwen: 0.08, Huatuo-7B: 0.05, Huatuo-34B: 0.2
    Selected from {0.01, 0.02, 0.05, 0.08, 0.1, 0.2, 0.5, 0.8} based on benchmark performance (Appendix D.3). This is a per-backbone fitted hyperparameter, and the paper's own analysis (Sec. 5.5) shows optimal alpha does not transfer across backbones.
  • LoRA rank = 16
    Standard choice, not fitted to this benchmark specifically, but affects parameter budget comparisons with BalancEdit (Sec. 5.4 notes ~18M vs 13.8M trainable params).
  • LoRA learning rate = 5e-5
    Standard value, not benchmark-fitted.
  • MEND learning rate = 0.05
    Stated in Appendix D.2.
  • GRACE initial radius (eps-init) = 1.0
    Stated in Appendix D.1.
  • Number of sequential edits (main setting) = 200
    Chosen as a 'stronger stress test' (Appendix C.2); the ablation shows results are sensitive to this choice.
axioms (4)
  • domain assumption LLM-based clinical attribute distillation correctly extracts standardized medical facts (condition, anatomical site, modality, progression) from raw QA pairs and clinical notes.
    Invoked in Sec. 3.1 Stage 1. The entire benchmark construction depends on this pipeline producing accurate structured attributes. No human validation rate is reported.
  • domain assumption The selected 4 editing methods (LoRA, MEND, GRACE, BalancEdit) are representative of the editing method space.
    Invoked in Sec. 2.2 and Sec. 4. The paper taxonomizes into gradient-based and memory-based, but other categories (e.g., hypernetwork-based, prompt-based editing) are not evaluated.
  • standard math Free autoregressive generation is the appropriate evaluation mode for editing (vs. teacher forcing).
    Invoked in Sec. 4. Supported by citation [31] showing teacher-forced scores inflate editing performance. This is a reasonable methodological choice.
  • ad hoc to paper The cone effect (representation anisotropy) causally drives editing method tradeoffs.
    Invoked in Sec. 5.1-5.5. The paper provides correlational evidence (PCA visualizations, centroid shifts, cross-backbone alpha sensitivity) but does not prove causation — e.g., by showing that interventions reducing anisotropy improve editing outcomes.

pith-pipeline@v1.1.0-glm · 28548 in / 3553 out tokens · 220002 ms · 2026-07-07T18:02:33.632407+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 33 canonical work pages · 11 internal anchors

  1. [1]

    Nature medicine28(9), 1773–1784 (2022)

    Acosta, J.N., Falcone, G.J., Rajpurkar, P., Topol, E.J.: Multimodal biomedical ai. Nature medicine28(9), 1773–1784 (2022)

    work page 2022

  2. [2]

    Journal of medical Internet research26, e59505 (2024)

    AlSaad, R., Abd-Alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.A., Damseh, R., Sheikh, J.: Multimodal large language models in health care: applications, chal- lenges, and future outlook. Journal of medical Internet research26, e59505 (2024)

    work page 2024

  3. [3]

    arXiv preprint arXiv:2512.12812 (2025)

    Cai, H., Shen, B., Jin, L., Hu, L., Fan, X.: Does tone change the answer? evaluat- ing prompt politeness effects on modern llms: Gpt, gemini, llama. arXiv preprint arXiv:2512.12812 (2025)

    work page arXiv 2025

  4. [4]

    NEJM AI2(7), AIdbp2401120 (2025)

    de Castro, D.C., Bustos, A., Bannur, S., Hyland, S.L., Bouzid, K., Wetscherek, M.T., Sánchez-Valverde, M.D., Jaques-Pérez, L., Pérez-Rodríguez, L., Takeda, K., et al.: Padchest-gr: A bilingual chest x-ray dataset for grounded radiology report generation. NEJM AI2(7), AIdbp2401120 (2025)

    work page 2025

  5. [5]

    HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale

    Chen, J., Gui, C., Ouyang, R., Gao, A., Chen, S., Chen, G.H., Wang, X., Zhang, R., Cai, Z., Ji, K., Yu, G., Wan, X., Wang, B.: Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale (2024),https://arxiv. org/abs/2406.19280

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  6. [6]

    Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling

    Chen, X., Wu, Z., Liu, X., Pan, Z., Liu, W., Xie, Z., Yu, X., Ruan, C.: Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.17811 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  7. [7]

    Cheng, D., Huang, S., Zhu, Z., Zhang, X., Zhao, W.X., Luan, Z., Dai, B., Zhang, Z.: On domain-adaptive post-training for multimodal large language models (2025), https://arxiv.org/abs/2411.19930

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  8. [8]

    In: Pro- ceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

    Cheng, J., Zhao, X., Liu, S., Yu, X., Prakash, R., Codd, P.J., Katz, J.E., Lin, S.: Surgxbench: Explainable vision-language model benchmark for surgery. In: Pro- ceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 8188–8198 (2026)

    work page 2026

  9. [9]

    MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledge

    Du, Y., Jiang, K., Gao, Z., Shi, C., Zheng, Z., Qi, S., Li, Q.: Mmke-bench: A multimodal editing benchmark for diverse visual knowledge. arXiv preprint arXiv:2502.19870 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  10. [10]

    New England Journal of Medicine385(3), 283–286 (2021)

    Finlayson, S.G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I.S., Saria, S.: The clinician and dataset shift in artificial intelligence. New England Journal of Medicine385(3), 283–286 (2021)

    work page 2021

  11. [11]

    Guo, D., Hu, M., Guan, Z., Hartvigsen, T., Li, S.: Balancedit: Dynamically balancing the generality-locality trade-off in multi-modal model editing (2025), https://arxiv.org/abs/2505.01343

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  12. [12]

    Hartvigsen, T., Sankaranarayanan, S., Palangi, H., Kim, Y., Ghassemi, M.: Aging with grace: Lifelong model editing with discrete key-value adaptors (2023),https: //arxiv.org/abs/2211.11031

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  13. [13]

    Iclr1(2), 3 (2022) Evaluating and Understanding Model Editing for Medical VLMs 17

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. Iclr1(2), 3 (2022) Evaluating and Understanding Model Editing for Medical VLMs 17

    work page 2022

  14. [14]

    In: Proceedings of the 33rd ACM International Conference on Multimedia

    Hu,N.,Duan,X.,Zhang,J.,Kang,G.:Enhancingvisualrelianceintextgeneration: A bayesian perspective on mitigating hallucination in large vision-language models. In: Proceedings of the 33rd ACM International Conference on Multimedia. pp. 4778–4787 (2025)

    work page 2025

  15. [15]

    Jama330(23), 2255–2257 (2023)

    Khera, R., Simon, M.A., Ross, J.S.: Automation bias and assistive ai: risk of harm from ai-driven clinical decision support. Jama330(23), 2255–2257 (2023)

    work page 2023

  16. [16]

    In: Proceedings of the 2023 conference on empirical methods in natural language processing

    Koopman, B., Zuccon, G.: Dr chatgpt tell me what i want to hear: How different prompts impact health answer correctness. In: Proceedings of the 2023 conference on empirical methods in natural language processing. pp. 15012–15022 (2023)

    work page 2023

  17. [17]

    Lau, J.J., Gayen, S., Ben Abacha, A., Demner-Fushman, D.: A dataset of clinically generatedvisualquestionsandanswersaboutradiologyimages.Scientificdata5(1), 180251 (2018)

    work page 2018

  18. [18]

    Advances in Neural Information Processing Systems36, 28541–28564 (2023)

    Li,C.,Wong,C.,Zhang,S.,Usuyama,N.,Liu,H.,Yang,J.,Naumann,T.,Poon,H., Gao, J.: Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems36, 28541–28564 (2023)

    work page 2023

  19. [19]

    Advances in Neural Information Processing Systems35, 17612–17625 (2022)

    Liang,V.W.,Zhang,Y.,Kwon,Y.,Yeung,S.,Zou,J.Y.:Mindthegap:Understand- ing the modality gap in multi-modal contrastive representation learning. Advances in Neural Information Processing Systems35, 17612–17625 (2022)

    work page 2022

  20. [20]

    In: 2021 IEEE 18th international symposium on biomedical imaging (ISBI)

    Liu, B., Zhan, L.M., Xu, L., Ma, L., Yang, Y., Wu, X.M.: Slake: A semantically- labeled knowledge-enhanced dataset for medical visual question answering. In: 2021 IEEE 18th international symposium on biomedical imaging (ISBI). pp. 1650–1654. IEEE (2021)

    work page 2021

  21. [21]

    NPJ Digital Medicine6(1), 226 (2023)

    Liu, F., Zhu, T., Wu, X., Yang, B., You, C., Wang, C., Lu, L., Liu, Z., Zheng, Y., Sun, X., et al.: A medical multimodal large language model for future pandemics. NPJ Digital Medicine6(1), 226 (2023)

    work page 2023

  22. [22]

    The Lancet Digital Health2(10), e537–e548 (2020)

    Liu, X., Rivera, S.C., Moher, D., Calvert, M.J., Denniston, A.K., Ashrafian, H., Beam, A.L., Chan, A.W., Collins, G.S., Deeks, A.D.J., et al.: Reporting guidelines forclinicaltrialreportsforinterventionsinvolvingartificialintelligence:theconsort- ai extension. The Lancet Digital Health2(10), e537–e548 (2020)

    work page 2020

  23. [23]

    Advances in neural information processing systems35, 17359–17372 (2022)

    Meng, K., Bau, D., Andonian, A., Belinkov, Y.: Locating and editing factual associ- ations in gpt. Advances in neural information processing systems35, 17359–17372 (2022)

    work page 2022

  24. [24]

    Mitchell, E., Lin, C., Bosselut, A., Finn, C., Manning, C.D.: Fast model editing at scale (2022),https://arxiv.org/abs/2110.11309

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  25. [25]

    In: International conference on machine learning

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PmLR (2021)

    work page 2021

  26. [26]

    npj Digital Medicine8(1), 252 (2025)

    Rosenthal, J.T., Beecy, A., Sabuncu, M.R.: Rethinking clinical trials for medical ai with dynamic deployments of adaptive systems. npj Digital Medicine8(1), 252 (2025)

    work page 2025

  27. [27]

    Qwen3.5-Omni Technical Report

    Team, Q.: Qwen3. 5-omni technical report. arXiv preprint arXiv:2604.15804 (2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  28. [28]

    ACM Computing Surveys57(3), 1–37 (2024)

    Wang, S., Zhu, Y., Liu, H., Zheng, Z., Chen, C., Li, J.: Knowledge editing for large language models: A survey. ACM Computing Surveys57(3), 1–37 (2024)

    work page 2024

  29. [29]

    Xie, J., Cao, P., Chen, Y., Chen, Y., Liu, K., Zhao, J.: Memla: Enhancing multilingual knowledge editing with neuron-masked low-rank adaptation (2024), https://arxiv.org/abs/2406.11566

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  30. [30]

    MedMKEB: A Comprehensive Knowledge Editing Benchmark for Medical Multimodal Large Language Models

    Xu, D., Wang, J., Chai, Z., Cao, Y., Wang, H., Zhang, H., Huang, Y.: Medmkeb: A comprehensive knowledge editing benchmark for medical multimodal large lan- guage models (2025),https://arxiv.org/abs/2508.05083 18 G. Zhu and C. Wu et al

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  31. [31]

    In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Yang, W., Sun, F., Tan, J., Ma, X., Cao, Q., Yin, D., Shen, H., Cheng, X.: The mirage of model editing: Revisiting evaluation in the wild. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 15336–15354 (2025)

    work page 2025

  32. [32]

    Radiology: Artificial Intelligence4(3), e210064 (2022)

    Yu, A.C., Mohajer, B., Eng, J.: External validation of deep learning algorithms for radiologic diagnosis: a systematic review. Radiology: Artificial Intelligence4(3), e210064 (2022)

    work page 2022

  33. [33]

    PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering

    Zhang, X., Wu, C., Zhao, Z., Lin, W., Zhang, Y., Wang, Y., Xie, W.: Pmc-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint arXiv:2305.10415 (2023) A Benchmark Dataset Details A.1 Data sources. The benchmark integrates samples from public medical datasets includingVQA- RAD [17],PMC-VQA [33],PadChest-GR [4], andSLAKE [20]. ...

    work page internal anchor Pith review Pith/arXiv arXiv 2023