{"id":"513d4f76-d132-462e-a643-d56d7032f173","arxiv_id":"2606.27665","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"LLMVue uses GPT-4o to extract and merge component hierarchies and route transitions into executable page transition graphs for Vue.js apps, reporting high precision/recall on ten GitHub projects and better coverage than random exploration.","lead":"The paper introduces LLMVue, a system that prompts GPT-4o to build page transition graphs directly from Vue.js source code for automated GUI testing. A smart generalist might read it to see whether current large language models can replace manual model construction in web application testing.","discovery_kind":"unclear","skeptic_critique":{"model":"grok-4.3","headline":"LLM reliability for PTG extraction from Vue.js source is the load-bearing assumption","rationale":"The reader's weakest_assumption directly identifies the single point on which the entire evaluation chain depends. Because the full text is now notionally available, the concrete_test above would resolve whether that assumption holds in the reported experiments. No other internal inconsistency is visible from the abstract-level claims.","tokens_in":1742,"tokens_out":293,"duration_ms":30047,"concrete_test":"For the two largest projects in the 10-project corpus, manually construct the reference PTG from the raw .vue files and router configuration, then recompute precision, recall, and GED against the LLM-generated PTG using the paper's exact definitions; if any metric deviates by more than 15% on either project, the reliability claim is undermined.","verdict_should_be":"UNCHANGED","load_bearing_attack":"All headline results (high precision/recall, low GED, improved coverage vs random) presuppose that GPT-4o produces a faithful PTG by correctly inferring component hierarchies and route transitions across SFCs and router configs. The abstract reports aggregate metrics but provides no description of ground-truth construction, inter-annotator agreement, or systematic checks for omissions/hallucinations. If the LLM output contains even moderate structural errors, the PTG becomes invalid and the coverage/efficiency gains cannot be attributed to model-based testing.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes LLMVue, a framework that uses an LLM (GPT-4o) to extract a Page Transition Graph (PTG) from Vue.js source code. The approach infers component hierarchies and route transitions from single-file components and router configurations, then merges them into a unified executable PTG for model-based GUI testing. On ten open-source projects the constructed PTGs show high precision/recall and low graph edit distance relative to human references; LLM-guided testing also yields higher coverage and exploration efficiency than a random baseline under identical time limits. The work positions itself as the first application of LLMs to source-level PTG extraction for Vue.js.","tokens_in":1839,"tokens_out":522,"duration_ms":27314,"significance":"If the empirical claims hold after the missing methodological details are supplied, the paper would constitute a modest but useful contribution: the first reported use of LLMs for automated model extraction in Vue.js GUI testing. The evaluation on ten real projects and the dual comparison (human PTG + random baseline) are positive features. The central risk is that the headline results rest entirely on the unverified fidelity of the LLM-generated PTGs; without explicit validation of that step the coverage/efficiency gains cannot be confidently attributed to model-based testing.","major_comments":[{"comment":"The headline claims of high precision, recall, and low GED presuppose that the LLM outputs are faithful PTGs. The manuscript supplies no description of how the human reference graphs were constructed, no inter-annotator agreement figures, and no systematic audit for hallucinations or omissions in the extracted hierarchies and transitions.","section":"Evaluation"},{"comment":"The graph-merging algorithm that combines inferences across multiple SFCs and router files is not specified, nor are the prompts, error-handling strategies, or post-processing steps used with GPT-4o. These components are load-bearing for the PTG quality that underpins all subsequent testing results.","section":"Approach / LLMVue framework"},{"comment":"The reported coverage and efficiency improvements versus the random baseline lack raw per-project data, statistical significance tests, and implementation details of the baseline itself (e.g., how actions were sampled and whether the same execution harness was used).","section":"Results"}],"minor_comments":[{"comment":"The abstract states results on \"ten open-source Vue.js projects\" but does not name the projects or the selection criteria; adding this information would improve reproducibility.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. We address each major comment below and will revise the manuscript to supply the missing methodological details.","responses":[{"response":"We agree the validation of human reference graphs requires more detail. The revised manuscript will add a subsection explaining that two authors independently built the reference PTGs from source code and documentation, with discrepancies resolved by consensus. We will report inter-annotator agreement (Cohen's kappa) and describe a manual audit comparing LLM outputs to references for hallucinations or omissions.","revision_made":"yes","referee_comment":"[Evaluation] The headline claims of high precision, recall, and low GED presuppose that the LLM outputs are faithful PTGs. The manuscript supplies no description of how the human reference graphs were constructed, no inter-annotator agreement figures, and no systematic audit for hallucinations or omissions in the extracted hierarchies and transitions."},{"response":"We will expand the LLMVue framework section with a precise description of the graph-merging algorithm (node/edge union with conflict resolution based on router priority). The revised paper will include the exact GPT-4o prompts, error-handling (retries on parse errors), and post-processing steps (label normalization) either in the main text or an appendix.","revision_made":"yes","referee_comment":"[Approach / LLMVue framework] The graph-merging algorithm that combines inferences across multiple SFCs and router files is not specified, nor are the prompts, error-handling strategies, or post-processing steps used with GPT-4o. These components are load-bearing for the PTG quality that underpins all subsequent testing results."},{"response":"We will add a table with raw per-project coverage and efficiency numbers. Statistical significance will be assessed using paired Wilcoxon tests with reported p-values. The baseline description will be expanded to confirm uniform action sampling from available elements and use of the identical execution harness.","revision_made":"yes","referee_comment":"[Results] The reported coverage and efficiency improvements versus the random baseline lack raw per-project data, statistical significance tests, and implementation details of the baseline itself (e.g., how actions were sampled and whether the same execution harness was used)."}],"tokens_in":1490,"tokens_out":484,"duration_ms":27461,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that this paper applies an LLM to build the page transition graph directly from Vue.js single-file components and router files, then feeds it into model-based GUI testing. They claim this is the first such effort for the framework and show the resulting graphs have high precision and recall plus low edit distance to references on ten open-source projects, with the guided tester reaching higher coverage and efficiency than a random baseline under equal time limits.\n\nWhat is new is the specific pipeline: LLM prompting to infer hierarchies and transitions, followed by merging across files into one executable PTG. Prior model-based work on other web frameworks did not target Vue.js source extraction this way, so the combination fills a narrow but real gap.\n\nThe evaluation on real GitHub projects is a plus; it moves beyond toy examples. The coverage gains over random are the practical result readers will notice.\n\nThe load-bearing issue is exactly the one flagged in the stress test. All headline numbers depend on the LLM producing faithful graphs. The abstract gives no description of how the human reference graphs were constructed, no inter-annotator numbers, and no systematic audit for omitted routes or invented transitions. Without that, it is impossible to know whether the reported improvements come from the model-based approach or from lucky LLM outputs. The paper also omits prompt details, error-handling steps, and raw data, which weakens reproducibility.\n\nThis work is aimed at researchers in GUI testing and LLM-assisted software engineering who already care about model-based methods for web apps. A reader looking for a ready-to-use technique will find the idea clear but will need the missing validation details before adopting it.\n\nIt deserves peer review. The core idea is practical and scoped to a popular framework, so referees can usefully press on the validation gaps rather than reject outright.","headline":"LLMVue extracts PTGs from Vue.js source with GPT-4o and reports better coverage than random, but the validation of those graphs against LLM errors is missing.","tokens_in":2342,"tokens_out":445,"would_cite":false,"duration_ms":29149,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"An LLM can extract component hierarchies and route transitions from Vue.js source to build a usable page transition graph for model-based GUI testing.","keywords":["LLM","model-based testing","GUI testing","Vue.js","page transition graph","web applications","automated testing","GPT-4o"],"falsifier":"Manual inspection of the inferred PTG against actual runtime navigation behavior on a new set of Vue.js applications, counting any missing or spurious transitions that cause test execution to deviate from the real application.","tokens_in":2633,"feed_emoji":"","tokens_out":686,"duration_ms":31562,"temperature":0.7,"pith_summary":"The paper introduces LLMVue, a framework that prompts a large language model to analyze Vue.js single-file components and router files, inferring hierarchies and transitions that are then merged into one executable page transition graph. This addresses the difficulty of locating and normalizing transition candidates spread across multiple files. On ten open-source projects the resulting graphs achieve high precision and recall with low edit distance to manually constructed references. When the graphs guide testing, both statement coverage and exploration efficiency rise compared with a random baseline under identical time budgets. The work presents this as the first application of LLMs to source-level PTG construction for Vue.js GUI testing.","feed_headline":"LLM extracts page transition graphs from Vue.js source","feed_subtitle":"Graphs built this way guide GUI tests that reach higher coverage than random walks on ten open-source projects.","key_machinery":"LLMVue framework that prompts an LLM (GPT-4o) to infer component hierarchies and route transitions from Vue.js source and merges them into a unified page transition graph (PTG).","core_discovery":"LLMVue uses an LLM to infer component hierarchies and route transitions from raw Vue.js source, then merges the extracted information into a single page transition graph; the graphs obtained from ten real projects exhibit high precision and recall together with low graph edit distance, and the same graphs, when used to direct test execution, produce measurably higher coverage and faster exploration than random walks within the same time limit.","pith_inferences":["The same LLM extraction pattern could be applied to other single-file-component frameworks that separate routing from component logic.","If the PTG is kept up to date by re-running the LLM after code changes, regression test suites could be regenerated automatically.","The approach assumes source code is the sole input; extending it to combine source with runtime traces would address cases where dynamic routes are generated at runtime."],"forward_implications":["The constructed PTG can serve directly as a model for automated GUI test generation in Vue.js applications.","LLM-guided exploration achieves higher coverage than random exploration when both are limited to the same execution time.","Graph construction metrics remain stable across ten distinct open-source Vue.js projects.","Merging per-component and router-derived transitions into one PTG removes the manual normalization step previously required for model-based testing."],"fun_headline_variants":["LLM infers PTG for Vue.js model-based testing","LLMVue unifies Vue.js SFCs into single PTG","LLM builds transition graph from Vue.js code","Page transition graphs from LLM on Vue.js apps","LLMVue merges Vue.js routes into executable PTG"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The large language model can correctly and completely recover component hierarchies and route transitions from Vue.js source code without systematic omissions or hallucinations that would produce an invalid PTG.","fun_headline_variants_meta":{"raw":{"variants":["LLM infers PTG for Vue.js model-based testing","LLMVue unifies Vue.js SFCs into single PTG","LLM builds transition graph from Vue.js code","Page transition graphs from LLM on Vue.js apps","LLMVue merges Vue.js routes into executable PTG"]},"model":"grok-4.3","cost_usd":0.006717,"raw_usage":{"total_tokens":3134,"prompt_tokens":679,"num_sources_used":0,"completion_tokens":70,"cost_in_usd_ticks":67174500,"prompt_tokens_details":{"text_tokens":679,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2385,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":679,"tokens_out":70,"duration_ms":30817,"temperature":1.0,"reasoning_tokens":2385,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T04:20:37.565650+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Manual inspection of the inferred PTG against actual runtime navigation behavior on a new set of Vue.js applications, counting any missing or spurious transitions that cause test execution to deviate from the real application.","supporting_citations":[],"review_version":1}