{"id":"ded87f1c-15cb-4607-a619-a14f9ad8fe30","arxiv_id":"2605.27332","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"EdgeFlow augments VLMs with a Canny edge map prior to raise node, edge, and path F1 scores by 11-17 points on a real-world industrial flowchart dataset while showing no gain on a public synthetic benchmark.","lead":"The paper proposes EdgeFlow, which feeds a Canny edge map alongside the original flowchart image into a vision-language model to improve conversion into Mermaid syntax for industrial requirements engineering. A smart generalist might read it to see whether a simple deterministic image filter can make off-the-shelf AI more reliable on topology-heavy diagrams without any training.","discovery_kind":"unclear","skeptic_critique":{"model":"grok-4.3","headline":"Canny edge map may not supply structural information inaccessible to VLM from raw pixels","rationale":"The reader's weakest_assumption precisely isolates the causal mechanism required for the headline improvements to be credited to EdgeFlow. No stronger internal inconsistency appears from the supplied abstract; the UNVERDICTED status already reflects the abstract-only limitation, so the concern does not shift the verdict category.","tokens_in":1745,"tokens_out":303,"duration_ms":30139,"concrete_test":"Prompt the baseline VLM (same model and prompt template) on raw images alone to explicitly output detected nodes, edges, and paths; compare its F1 scores against the EdgeFlow-augmented results on the same IndusReqFlow samples. If raw-image F1 is within 3 pp of augmented F1, the structural-prior assumption does not hold.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim attributes F1 gains (17.39 pp node, 16.94 pp edge, 11.06 pp path) on IndusReqFlow to the deterministic Canny augmentation acting as a structural prior. This requires that the edge map conveys topology the VLM cannot already recover from raw pixel values via its internal representations. The abstract-only evaluation and the reported lack of gains on the public synthetic benchmark leave open the possibility that any observed improvement stems from prompt formatting, dual-image input handling, or dataset-specific image properties rather than novel structural information.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces EdgeFlow, a training-free approach that augments off-the-shelf VLMs with a deterministically extracted Canny edge map as a structural prior to improve flowchart image to Mermaid conversion for industrial requirements engineering. On the real-world IndusReqFlow dataset it reports gains of 17.39 pp node-level F1, 16.94 pp edge-level F1 and 11.06 pp path-level F1 relative to baseline VLMs; cross-evaluation on a public synthetic benchmark shows no significant improvement. The work emphasizes practical applicability without annotated data or fine-tuning.","tokens_in":1867,"tokens_out":528,"duration_ms":20434,"significance":"If the reported gains are robustly attributable to the edge-map augmentation supplying topology information inaccessible from raw pixels, the method offers a lightweight, reproducible way to enhance VLM reliability on topology-critical RE tasks. The explicit reporting of negative cross-dataset results is a strength, as is the focus on an industrial rather than purely synthetic corpus. These elements could help set expectations for future VLM-based RE tooling.","major_comments":[{"comment":"The central claim that the Canny edge map supplies structural information the VLM cannot already recover from raw pixel values (abstract and methods) is load-bearing for attributing the F1 deltas to the augmentation rather than prompt formatting or dual-image handling. The manuscript should provide a concrete test—e.g., an ablation comparing VLM outputs on raw images versus edge maps with identical prompts, or analysis of error patterns on topology elements—to rule out alternative explanations.","section":"Methods / Evaluation"},{"comment":"Table or results section reporting the 17.39 / 16.94 / 11.06 pp gains on IndusReqFlow: without accompanying dataset statistics (image resolution distribution, flowchart complexity metrics), per-category error analysis, or statistical significance tests, it is difficult to assess whether the improvements generalize beyond the specific characteristics of the industrial images.","section":"Evaluation"}],"minor_comments":[{"comment":"Clarify in the prompt-engineering subsection exactly how the original image and Canny map are presented to the VLM (separate images, concatenated, or described in text) to allow replication.","section":"Methods"},{"comment":"The abstract states 'no significant improvement' on the synthetic benchmark; the results section should report the exact delta values and confidence intervals for transparency.","section":"Evaluation"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their thoughtful comments, which help improve the clarity and rigor of our work. We address each major comment point by point below.","responses":[{"response":"We agree that isolating the contribution of the edge-map augmentation is important. While the cross-dataset evaluation (no improvement on synthetic data) provides some evidence that the gains are tied to the characteristics of industrial flowcharts where pixel-level topology may be harder to parse, we will strengthen this in the revision by adding an ablation study. Specifically, we will compare VLM performance using identical prompts on (1) raw images and (2) raw images paired with Canny edge maps. We will also include an analysis of error patterns focusing on topology elements such as node connections and path continuity.","revision_made":"yes","referee_comment":"[Methods / Evaluation] The central claim that the Canny edge map supplies structural information the VLM cannot already recover from raw pixel values (abstract and methods) is load-bearing for attributing the F1 deltas to the augmentation rather than prompt formatting or dual-image handling. The manuscript should provide a concrete test—e.g., an ablation comparing VLM outputs on raw images versus edge maps with identical prompts, or analysis of error patterns on topology elements—to rule out alternative explanations."},{"response":"We acknowledge that additional dataset details and statistical analysis would enhance the evaluation section. In the revised manuscript, we will include: (i) dataset statistics such as the distribution of image resolutions and metrics for flowchart complexity (e.g., average number of nodes, edges, and paths per diagram); (ii) per-category error analysis breaking down performance by flowchart types or complexity levels; and (iii) statistical significance tests (e.g., McNemar's test for paired comparisons) on the reported F1 improvements to confirm they are not due to chance.","revision_made":"yes","referee_comment":"[Evaluation] Table or results section reporting the 17.39 / 16.94 / 11.06 pp gains on IndusReqFlow: without accompanying dataset statistics (image resolution distribution, flowchart complexity metrics), per-category error analysis, or statistical significance tests, it is difficult to assess whether the improvements generalize beyond the specific characteristics of the industrial images."}],"tokens_in":1444,"tokens_out":487,"duration_ms":29834,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"EdgeFlow reports clear F1 gains on a new industrial flowchart dataset from adding Canny edges to VLMs, but the same change adds nothing on synthetic data, so the dataset and numbers are the real contribution.\n\nThe paper creates IndusReqFlow from real requirements documents and shows that feeding both the original image and its Canny edge map to an off-the-shelf VLM lifts node-level F1 by 17.39 points, edge-level F1 by 16.94 points, and path F1 by 11.06 points over the baseline. The method requires no training data or fine-tuning, which keeps the setup simple and immediately usable. Those concrete deltas on actual industrial material are the usable result.\n\nThe authors also state that the same augmentation produces no significant improvement on a public synthetic benchmark. They treat this as evidence that future work needs more industrial-style test sets.\n\nThe soft spot is the missing link between the edge map and the observed gains. The claim rests on the edge map supplying topology the VLM cannot already extract from raw pixels. The industrial results are consistent with that, but the synthetic case leaves open the possibility that prompt formatting, dual-image handling, or dataset-specific image traits drive the difference. No ablations or error analysis are described to separate those factors.\n\nThis is for people working on diagram-to-model conversion inside industrial requirements engineering. A reader who needs numbers on real flowcharts and a ready-to-try augmentation will find the dataset and deltas relevant. The work is grounded enough in concrete results on domain data to deserve peer review, though any review would likely press for tighter controls on why the method behaves differently across datasets.","headline":"EdgeFlow reports clear F1 gains on a new industrial flowchart dataset from adding Canny edges to VLMs, but the same change adds nothing on synthetic data, so the dataset and numbers are the real contribution.","tokens_in":2347,"tokens_out":423,"would_cite":false,"duration_ms":29525,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Augmenting VLMs with Canny edge maps improves flowchart-to-Mermaid conversion F1 by 11 to 17 points on industrial data without training.","keywords":["flowchart processing","vision language models","requirements engineering","Canny edge detection","Mermaid diagrams","topology preservation","industrial datasets","training-free augmentation"],"falsifier":"Running the identical VLM on the IndusReqFlow images with and without the Canny edge map channel and finding no difference in node, edge, or path F1 scores would falsify the benefit of the augmentation.","tokens_in":2651,"feed_emoji":"📊","tokens_out":678,"duration_ms":21972,"temperature":0.7,"pith_summary":"The paper introduces EdgeFlow to help vision-language models convert flowcharts from industrial requirements documents into editable Mermaid diagrams. It adds a Canny edge map directly to the model's input image as an extra structural signal. This is done without any labeled training data or model fine-tuning. Evaluation on a dataset of real industrial flowcharts shows clear gains in detecting nodes, edges, and paths compared to standard VLMs. The same method brings no clear benefit on a synthetic public benchmark, which the authors note points to the value of testing on domain-specific industrial images.","feed_headline":"Edge map boosts VLM flowchart F1 by 17 points on industrial data","feed_subtitle":"Canny edge augmentation improves node and edge detection in real requirement diagrams without training or fine-tuning.","key_machinery":"Canny edge map augmentation supplied as an additional input channel to supply topology information the VLM may miss from raw pixels alone.","core_discovery":"EdgeFlow augments a VLM's original input with a deterministically extracted Canny edge map acting as a structural prior to improve flowchart-to-Mermaid conversion, without requiring annotated training data or domain-specific model fine-tuning. On the IndusReqFlow dataset of real-world requirements, this yields node-level F1 gains of 17.39 percentage points, edge-level F1 gains of 16.94 percentage points, and path F1 gains of 11.06 percentage points over off-the-shelf VLMs. Cross-dataset tests on a public synthetic benchmark show no significant improvement.","pith_inferences":["Edge maps may help VLMs on other structured diagram types such as state machines or sequence diagrams where connectivity matters.","If industrial images contain more noise or compression artifacts than synthetic ones, the edge map could be acting as a denoising step rather than purely a topology cue.","Future work could test whether other deterministic image filters produce similar gains or if Canny edges are uniquely effective."],"forward_implications":["Improved path-level accuracy directly supports model-based testing of requirements.","The method works on real industrial flowcharts without any domain-specific fine-tuning.","Cross-dataset results indicate that synthetic benchmarks alone are insufficient for evaluating VLM-based RE tools.","The approach remains applicable to any VLM that accepts image inputs without architectural changes."],"fun_headline_variants":["Canny edges raise VLM flowchart node F1 by 17 points on real data","EdgeFlow uses edge map to gain 17 points in VLM flowchart F1","Canny edge addition yields 17 point F1 gain on industrial flowcharts","Edge map prior increases flowchart VLM F1 by 17 points for RE"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The Canny edge map supplies structural information that the VLM cannot already extract from the raw pixel values.","fun_headline_variants_meta":{"raw":{"variants":["Canny edges raise VLM flowchart node F1 by 17 points on real data","EdgeFlow uses edge map to gain 17 points in VLM flowchart F1","Canny edge addition yields 17 point F1 gain on industrial flowcharts","Edge map prior increases flowchart VLM F1 by 17 points for RE"]},"model":"grok-4.3","cost_usd":0.006119,"raw_usage":{"total_tokens":2915,"prompt_tokens":720,"num_sources_used":0,"completion_tokens":84,"cost_in_usd_ticks":61187000,"prompt_tokens_details":{"text_tokens":720,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2111,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":720,"tokens_out":84,"duration_ms":21205,"temperature":1.0,"reasoning_tokens":2111,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T15:43:54.409580+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the identical VLM on the IndusReqFlow images with and without the Canny edge map channel and finding no difference in node, edge, or path F1 scores would falsify the benefit of the augmentation.","supporting_citations":[],"review_version":1}