{"id":"ba07f0ef-ff92-45c4-8200-9336ab9ce037","arxiv_id":"2608.05478","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A vector-first framework, GenGA, generates editable graphical abstracts from paper text and user images, together with a Structural Independence Coefficient metric for editing simplicity.","lead":"GenGA turns graphical abstract generation into an editable vector graphics problem, producing SVG summaries from paper text and user-supplied images. It also introduces a metric, SIC, meant to predict how easily a figure can be edited, and reports that GenGA beats raster-generation baselines on editability and matches human-made abstracts on several quality axes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SIC's format-dependent edge definitions make the headline editability comparison between vector GenGA and raster human-authored GAs uninterpretable without a vectorized human control.","rationale":"The reader's weakest-assumption analysis correctly identifies that SIC is computed via different instantiations for vector and raster inputs, so cross-format SIC comparisons are not measuring the same quantity. This is the most load-bearing concern because the paper's headline contribution—that GenGA yields 'superior editing simplicity' and even surpasses human-authored GAs—relies on Table 2's SIC column. The concern is sharpened by noting the missing control: the authors vectorize two raster baselines but not the human-authored GA baseline, making the comparison asymmetric. The paper does provide a user study correlating SIC with human editing performance, which is genuine supporting evidence, but the study's design ('visually identical figures that differ in SIC due to representation and structural complexity') means the correlation may reflect only that raster editing is harder than vector editing, not that SIC is a valid proxy for editing cost across the formats being compared. A concrete test—vectorizing human GAs and recomputing SIC, plus a targeted user study—would either confirm or refute the central claim. Because the reader's verdict is already CONDITIONAL and this test directly addresses the decisive gap, I recommend keeping the verdict CONDITIONAL: the paper should be accepted only if the vectorized human-GA control supports the claimed advantage.","tokens_in":18697,"tokens_out":4769,"duration_ms":46347,"concrete_test":"Run the test-set author-created GAs (the same 2,053 papers) through the same vectorization pipeline used in Section 4.1 (Gemini-3.1-Pro-Preview with the Prompt 1 vectorizer, no self-correction) and compute SIC on the resulting vector representations. If the vectorized human GAs achieve mean SIC comparable to GenGA (e.g., > 0.8), then GenGA's claimed editability advantage over human GAs is a format artifact, not a structural improvement. Additionally, have users edit these vectorized human GAs in draw.io with the Section 4.4 protocol; if editing time and success rate match GenGA's, the SIC-based superiority claim fails.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central editability claim rests on Table 2, where GenGA's SIC (0.885–0.943) is compared with human-authored GAs' SIC (0.119). But Section 3.2 instantiates SIC differently per format: for vectors, edges have weight 1 only when elements share explicit grouping, transforms, or clipping, and 0 otherwise; for rasters, nodes are segmentation regions and edge weights are exp(-boundary strength), Eq. (3). These measure different constructs: one counts explicit structural coupling in a drawing-tool sense, the other approximates perceptual region boundaries. A human-authored GA, rasterized and segmented, will naturally have low SIC because region boundaries are weak within coherent figures; this does not mean a human GA is harder to edit when opened in a vector editor. The paper never vectorizes human-authored GAs and recomputes SIC, despite doing exactly this for PaperBanana and NanoBanana-Pro baselines. The user study in Section 4.4 also uses 'visually identical figures that differ in SIC due to representation and structural complexity,' which conflates format effects with layout quality; it does not establish that SIC predicts editing cost across formats for equivalent vector structures. Therefore the 'superior editing simplicity compared to conventional methods, and even surpasses human-authored GAs' claim is unsubstantiated: the headline numbers may be an artifact of comparing a vector-native metric against a raster-bound metric.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Editable GA Generation, a task in which graphical abstracts are produced as structured, editable vector graphics rather than raster images. It proposes GenGA, a VLM-based pipeline with reference retrieval, raster-to-SVG reference vectorization, asset-aware SVG generation using placeholder constraints for user-provided images, and a self-correction loop. It also defines the Structural Independence Coefficient (SIC), a graph-based metric intended to measure editing simplicity. The evaluation uses 2,053 test papers from SciGA-145k; GenGA is compared with author-created GAs, NanoBanana-Pro, PaperBanana, and AutoFigure, including vectorized versions of the raster baselines. The paper reports higher SIC, competitive or higher CLIP-S and contribution QA accuracy, and strong user preference for GenGA.","tokens_in":18950,"tokens_out":7038,"duration_ms":63831,"significance":"If the cross-format SIC claim were established, the paper would be a useful step: it reframes GA generation as an editable structure-generation problem, provides a concrete vector-generation framework with detailed prompts, and introduces a metric for editing simplicity. The paper's strengths include a pipeline that preserves user-provided data as immutable assets, an unusually detailed supplementary prompt section, and a user study with 15 experienced researchers in which editing time and success rate correlate strongly with SIC (Figure 3). The central quantitative claim, however, rests on comparing SIC values computed with different instantiations of the metric, so the current evidence does not yet support the statement that GenGA surpasses human-authored GAs in editing simplicity.","major_comments":[{"comment":"SIC is instantiated differently for vector and raster figures, so the headline comparison in Table 2 is not apples-to-apples. In the vector instantiation, w_jk=1 only when elements share explicit grouping, transforms, or clipping; in the raster instantiation, nodes are segmentation regions and w_jk=exp(-b_jk). A human-authored GA, when rasterized and segmented, will typically have many weakly bounded regions and hence low SIC, but this does not mean it is harder to edit once opened in a vector editor. The authors vectorized NanoBanana-Pro and PaperBanana outputs before computing SIC, but did not apply the same vectorization to author-created GAs. The claim that GenGA 'surpasses human-authored GAs' in editing simplicity therefore requires either a vectorized human control or a careful argument that the two instantiations measure the same construct. Without this, the 0.885-0.943 vs. 0.119 comparison in Table 2 could be an artifact of format rather than of editing quality.","section":"Section 3.2, Eqs. (2)-(4); Table 2"},{"comment":"The manual-editing user study does not isolate SIC from representation. Participants were given 'visually identical figures that differ in SIC due to representation and structural complexity' (Section 4.4), so the observed correlation between SIC and editing time/success may be driven by the vector-vs-raster dichotomy rather than by the SIC value itself. As a validation of SIC as an editing-cost proxy, the experiment should vary structural independence within a fixed representation (e.g., different grouping or edge structure in otherwise matched SVGs), or report the correlation separately for vector and raster figures. As written, Figure 3 supports the statement that vector figures with high SIC are easier to edit than raster figures with low SIC, which is a weaker and less interesting claim.","section":"Section 4.4, Figure 3"},{"comment":"The binary edge definition makes SIC highly sensitive to the authoring style of the SVG rather than to the perceptual independence of elements. For example, if the generator places two visually separate but spatially aligned elements in the same <g> for convenience, they receive w_jk=1, lowering SIC; if the same layout is emitted with separate groups, SIC increases. The paper does not quantify this sensitivity, and the generator prompt (Prompt 2) explicitly enforces nested grouping, so the reported SIC values partly measure compliance with the prompt. A robustness analysis or a comparison with an independent re-grouping of the same figures is needed before SIC can be claimed as a general measure of editing cost.","section":"Section 3.2, vector instantiation; Prompt 2"}],"minor_comments":[{"comment":"The conclusion contains a broken sentence: 'This work establishes GA generation a foundation for grounded in real research workflows'; please revise.","section":"Section 6"},{"comment":"Please report the number of data points and whether the 15 participants are treated as repeated measures; the reported p-values (<10^-7, <10^-8) assume independence, which is not justified for 15 participants contributing multiple editing tasks.","section":"Figure 3"},{"comment":"Several rows without the self-correction loop achieve higher SIC (e.g., 0.948-0.959) than the full model (0.943), yet the text says the full model gives the best overall performance; please clarify this trade-off and state explicitly that the full model does not maximize SIC.","section":"Table 4"},{"comment":"GenGA's Overlap Ratio (0.146/0.163) is comparable to the vectorized raster baselines (0.164/0.169) and much higher than AutoFigure's 0.000, but the text explains this as intentional layout design. Since Overlap Ratio is defined as a layout-quality metric, please provide evidence (e.g., a breakdown excluding parent-child overlap) that the overlaps are benign.","section":"Table 2, Overlap Ratio"}],"recommendation":"major_revision","confidential_remarks":"The core idea is appropriate for ACM MM and the framework appears functional. My main concern is the cross-format SIC comparison; this is fixable by adding a vectorized human-authored baseline and re-running the user study with representation held constant, or by softening the editability claims. I do not see grounds for rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing: this paper is worth reading and worth sending to reviewers, but the central number that makes it exciting—SIC 0.943 for GenGA versus 0.119 for human-authored GAs—is not measuring the same construct on both sides. The system itself is well described and the task definition is genuinely new. Turning GA generation into a structured, editable vector problem rather than a raster synthesis problem is a real reframing, and the pipeline is coherent: reference retrieval, vectorization with placeholders for non-structural content, asset-aware generation, and a self-correction loop. The authors also deserve credit for shipping their prompts, describing the asset curation pipeline, and running a user study with 15 experienced researchers. The correlation between SIC and manual editing time within the study is real evidence, not hand-waving. The limitation they state about proprietary models is honest and accurate.\n\nThe soft spot is exactly where the stress-test lands. SIC is instantiated differently for vector and raster inputs. For vectors, edges exist only when elements share explicit grouping, transforms, or clipping; ungrouped SVG elements are independent by definition. For rasters, edges come from segmentation boundary strength. The prompt used for generation forces every element into nested groups, so GenGA outputs will nearly always have high SIC by construction. Comparing those scores to a raster human GA's segmentation-based SIC is not comparing editability in a shared sense. The paper vectorizes the NanoBanana-Pro and PaperBanana baselines before computing SIC, but it never vectorizes human-authored GAs and recomputes their SIC. That is the missing control, and without it the \"superior editing simplicity compared to human-authored GAs\" claim is unsubstantiated. The user study also uses \"visually identical figures that differ in SIC due to representation and structural complexity,\" which conflates format effects with layout difficulty; it does not establish that SIC predicts editing cost across formats for equivalent vector structures.\n\nA second, smaller issue: the abstract's claim of surpassing human GAs in conciseness and semantic alignment is only partially supported by Table 2. GenGA with Gemini beats the human baseline on conciseness, but GPT-5.2 does not, and CLIP-S differences are small on held-out test papers. The contribution QA and VLM-as-a-Judge results are useful but they are model-mediated and not a substitute for human judgment.\n\nMy bottom line: the system and the task deserve serious engagement, and the SIC idea is worth refining rather than discarding. But the paper needs a vectorized human-authored control condition and a re-analysis of the cross-format comparison before the headline claim is credible. I would send it to peer review, with that as a required revision.","headline":"A well-built vector GA generation system with a plausible editability metric, but the headline SIC comparison across vector and raster formats is apples-to-oranges and needs a vectorized human control before the central claim can stand.","tokens_in":19499,"tokens_out":1723,"would_cite":false,"duration_ms":18217,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims graphical abstracts should be generated as editable vector graphics, and that a new score—SIC—can measure how easy a figure is to edit.","keywords":["Graphical Abstracts","Editable Generation","Vector Graphics","SVG","Structural Independence Coefficient","Vision-Language Models","Data-Grounded Generation","Scientific Figure Generation"],"falsifier":"Run a blind editing study in which participants receive two visually identical figures, one a GenGA SVG and one a raster abstract whose SIC is equal under the raster formula, and must perform the same eight edits in draw.io; if editing time and success rate do not differ, SIC is not capturing practical editing cost. A second decisive test would give two figures with identical SIC but different perceptual grouping and check whether edit time tracks SIC or grouping.","tokens_in":18478,"feed_emoji":"🧩","tokens_out":5571,"duration_ms":49401,"temperature":0.7,"pith_summary":"The paper argues that graphical abstracts for academic papers should be generated as editable vector graphics rather than finished raster images, because researchers revise figures throughout writing and peer review. It introduces GenGA, a framework that takes a paper and optional asset images, retrieves a structurally similar reference abstract, and produces an SVG whose elements—text blocks, shapes, arrows, and placeholder slots for real data—can be moved, rewritten, or replaced individually in standard drawing tools. To make editability measurable, the paper defines the Structural Independence Coefficient (SIC), a score in [0,1] that estimates how often a local edit will stay local instead of propagating to other elements. Experiments report that GenGA outputs achieve higher SIC than human-authored and raster-generated abstracts, that users prefer them in pairwise comparisons, and that higher SIC correlates with faster and more successful manual edits. If the claims hold, automatic figure generation becomes a structural design problem that can be evaluated and improved against a concrete editability target.","feed_headline":"Graphical abstracts, now editable: GenGA outputs SVG, not pixels","feed_subtitle":"A structural-independence score measures editability, and users prefer the vector results over human-made figures.","key_machinery":"The load-bearing innovation is the Structural Independence Coefficient (SIC), a number in [0,1] computed by modeling a figure as a weighted complete graph of visual elements, activating each edge with probability equal to its dependency weight, and taking one minus the expected fraction of other elements connected to a randomly chosen edited element. In the vector instantiation, edges exist only where the SVG has explicit grouping, shared transforms, or clipping, so ungrouped elements are independent by definition; in the raster instantiation, elements come from Felzenszwalb segmentation and edge weights decay exponentially with boundary strength. SIC is what lets the paper call editability a measurable quantity, and what the GenGA framework is designed to maximize by emitting hierarchical, non-overlapping, semantically tagged SVG groups.","core_discovery":"The central claim is that editable GA generation is a well-defined task—producing a vector representation of a graphical abstract with explicit hierarchy and geometry metadata—and that GenGA solves it by combining four stages: semantic retrieval of a reference GA, vectorization of that reference into layout guidance, asset-aware generation that treats user-supplied images as fixed placeholders rather than generative targets, and a self-correction loop that iteratively improves semantic alignment and layout. Alongside the framework, the paper proposes SIC, computed from a weighted dependency graph of figure elements, as a metric of editing simplicity. In the evaluation, GenGA with the Gemini backbone reaches SIC 0.943 while human-authored GAs sit at 0.119 and raster-generation baselines stay below 0.19; GenGA also beats human-authored GAs on CLIP-S semantic alignment and contribution-QA accuracy, and wins 76.7 percent of pairwise user preferences with a 66.7 percent publishable rate. The authors read these results as evidence that editability and grounding in real data can be achieved without sacrificing visual quality.","pith_inferences":["SIC is not limited to graphical abstracts: any figure format—posters, diagrams, dashboards—could be scored with the same dependency-graph recipe, giving a general 'edit costs' metric for structured graphics.","The paper validates SIC by correlation with human edit time, but a stronger test would use SIC as a reward signal during generation; nothing in the framework prevents adding that optimisation loop.","The headline editability comparison mixes representations: vector SIC counts only explicit SVG grouping, while raster SIC infers elements from segmentation, so the two scores are not obviously measuring the same quantity across formats.","The method's dependence on proprietary vision-language models, which the paper names as a limitation, means a useful next step is distilling the pipeline into smaller open models that researchers could run locally on confidential papers."],"forward_implications":["Graphical abstract generation is repositioned as a structural editing problem: the output is a set of named, grouped SVG elements that any common drawing tool can open and modify.","SIC gives a concrete, automatable target for editability, so future systems can optimise or filter generated figures by predicted editing cost.","Treating user-provided assets as fixed placeholder slots prevents the model from hallucinating or distorting real experimental data during generation.","The self-correction loop shows measurable gains in semantic alignment, readability, and conciseness across iterations, converging after about four passes.","Because vector SVGs can be rasterised for review and re-edited afterwards, the same figure can be automatically improved and still manually refined."],"supporting_citations":[{"why":"Supplies the SciGA-145k dataset, the retrieval model Long-CLIP-4-Inter-GA-Rec, and the test set of 2,053 papers used in evaluation.","marker":"[21]"},{"why":"PaperBanana, the raster-based generation baseline whose high visual quality but low SIC frames the editability comparison; the VLM-as-a-judge protocol is adopted from this work.","marker":"[48]"},{"why":"AutoFigure, the hybrid raster-vector baseline whose background remains a single raster, used to show that partial vectorization yields limited editability.","marker":"[49]"},{"why":"NanoBanana-Pro, a raster image-generation model used as a baseline and compared after post-hoc vectorization.","marker":"[12]"},{"why":"CLIPScore, the metric used to measure semantic alignment between the paper abstract and the generated graphical abstract.","marker":"[13]"},{"why":"Felzenszwalb segmentation, used to instantiate nodes and edge weights when computing SIC for raster graphics.","marker":"[9]"},{"why":"LayerD, the layer decomposition model used to extract asset regions from original GAs for the evaluation set.","marker":"[41]"},{"why":"GPT-5.2, one of the two backbone models used as generator and evaluator in the experiments.","marker":"[33]"},{"why":"Gemini-3.1-Pro-Preview, the other backbone model used for generation, vectorization, review, and evaluation.","marker":"[11]"},{"why":"Bradley–Terry model, used to convert pairwise comparison results into preference scores in the user study.","marker":"[5]"}],"fun_headline_variants":["GenGA turns graphical abstracts into editable vectors","GenGA's vector GAs are 8x more editable than human-made","Editable graphical abstracts: GenGA surpasses humans","GenGA: vector GAs with data grounding and editability","Graphical abstracts get editable: GenGA outputs vectors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The editability comparison rests on the assumption that SIC computed from explicit SVG grouping (where any ungrouped element is independent by definition) measures the same quantity as SIC computed from raster segmentation, so the headline vector-versus-raster numbers can be directly compared.","fun_headline_variants_meta":{"raw":{"variants":["GenGA turns graphical abstracts into editable vectors","GenGA's vector GAs are 8x more editable than human-made","Editable graphical abstracts: GenGA surpasses humans","GenGA: vector GAs with data grounding and editability","Graphical abstracts get editable: GenGA outputs vectors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000223,"raw_usage":{"total_tokens":1496,"prompt_tokens":1023,"completion_tokens":473,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":639,"completion_tokens_details":{"reasoning_tokens":392}},"tokens_in":639,"tokens_out":473,"duration_ms":4529,"temperature":1.0,"reasoning_tokens":392,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T12:20:41.795970+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a blind editing study in which participants receive two visually identical figures, one a GenGA SVG and one a raster abstract whose SIC is equal under the raster formula, and must perform the same eight edits in draw.io; if editing time and success rate do not differ, SIC is not capturing practical editing cost. A second decisive test would give two figures with identical SIC but different perceptual grouping and check whether edit time tracks SIC or grouping.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the SciGA-145k dataset, the retrieval model Long-CLIP-4-Inter-GA-Rec, and the test set of 2,053 papers used in evaluation."},{"cited_title":"Progressive Feature Self-reinforcement for Weakly Supervised Semantic Segmentation","cited_arxiv_id":"2312.08916","evidence_quote":"AutoFigure, the hybrid raster-vector baseline whose background remains a single raster, used to show that partial vectorization yields limited editability."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"NanoBanana-Pro, a raster image-generation model used as a baseline and compared after post-hoc vectorization."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Felzenszwalb segmentation, used to instantiate nodes and edge weights when computing SIC for raster graphics."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"LayerD, the layer decomposition model used to extract asset regions from original GAs for the evaluation set."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GPT-5.2, one of the two backbone models used as generator and evaluator in the experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gemini-3.1-Pro-Preview, the other backbone model used for generation, vectorization, review, and evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Bradley–Terry model, used to convert pairwise comparison results into preference scores in the user study."}],"review_version":1}