{"id":"84fe5670-7777-49ac-97e1-e480b08041cd","arxiv_id":"2608.08880","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"FRAME augments CLIP and BERT embeddings with a UI component graph, spectral propagation, and Vietoris-Rips triangle weighting, improving screen retrieval and clustering on three Android UI benchmarks.","lead":"This paper introduces FRAME, a method that turns app screenshots into searchable feature fingerprints by combining image and text understanding with the spatial layout of buttons, menus, and text fields. It reports better retrieval and clustering results than four baselines on three Android UI benchmarks, with larger and more consistent gains on retrieval.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table III reports HR@k values that decrease as k increases, which is impossible under the paper's own 'at least one correct in top-k' definition; the hit-rate evidence needs recomputation before the retrieval claim is fully supported.","rationale":"The paper's augmentation idea is plausible and the availability of a replication package is a genuine strength. The strongest evidence for the central claim is retrieval improvement, but Table III contains a metric that cannot be what the paper defines it to be. This is not a stylistic issue: HR@k values that decrease with k are impossible for 'at least one correct in top-k,' so either the metric or the code is mislabeled. The reader's verdict already flags this in the rationale, but the reader's weakest_assumption focuses on hyperparameter transfer across datasets. I see the HR@k inconsistency as more load-bearing because it touches the quantitative basis of the headline retrieval result, not just generalization. If the recomputation shows the table is precision@k, the paper still has MRR and clustering evidence, so the appropriate disposition remains conditional pending correction, and the reader's CONDITIONAL verdict should stand. If the table is a simple labeling typo and recomputation yields a monotonic hit rate, the retrieval claim is strengthened. The clustering section would also benefit from repeated-run error bars for K-means, but that is secondary to the metric check.","tokens_in":17477,"tokens_out":4251,"duration_ms":42112,"concrete_test":"Using the replication package's raw ranked lists, recompute for every method and dataset HR@k = (number of queries with at least one same-label corpus screen in the top k) / (number of queries) for k = 1, 5, 10. Check monotonicity and compare against Table III. If the reported numbers match precision@k instead, the HR@k evidence in Table III is invalid and the significance tests for those rows must be re-run; if they match the definition, report the corrected table and confirm that MRR and clustering conclusions are unchanged.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-C defines HR@k as 'the percentage of queries for which an approach retrieves at least one correct screen within the top-k screens.' With that definition, HR@1 <= HR@5 <= HR@10 for every method, because any query satisfied at rank 1 is also satisfied at ranks 5 and 10. Table III shows the opposite ordering in every row: for Aurora FRAME, HR@1 = 0.5625, HR@5 = 0.4652, HR@10 = 0.4166; the same decreasing pattern holds for Screen2Vec, BERT, BLIP, and CLIP on all three datasets. Mathematically impossible values indicate that the reported column is not the hit rate defined in the paper, most plausibly a precision-at-k or per-rank statistic. Consequently, the HR@k rows and their paired t-test p-values cannot be used as evidence for the retrieval claim, even though the MRR rows, if computed correctly, still support an improvement. The central claim therefore rests on an unverified metric pipeline for part of its headline evidence. A secondary reproducibility concern is that the propagation weight w in Eq. (1) is never reported, so the structural contribution cannot be independently re-instantiated from the text alone.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FRAME, a neuro-symbolic UI screen embedding technique that augments CLIP and BERT embeddings by (i) normalizing screenshots, (ii) building a UI component graph from detected bounding boxes, (iii) propagating embeddings through a spectral graph filter, (iv) weighting component relationships via Vietoris-Rips triangulation, and (v) applying PCA. FRAME is evaluated on screen retrieval (MRR, HR@k) and clustering (nine metrics) across three datasets (Aurora, Avgust, Enrico) against CLIP, BLIP, BERT, and Screen2Vec. The authors report consistent retrieval improvements, clustering improvements over CLIP, and an ablation study isolating each component.","tokens_in":17692,"tokens_out":5591,"duration_ms":46806,"significance":"If the results hold, FRAME provides a training-free, model-agnostic structural prior that could benefit UI testing, bug localization, and design search tools. The paper ships an archived replication package and source code, and the ablation design is systematic. However, the current manuscript contains internal metric inconsistencies and overclaimed significance that must be resolved before the empirical claims can be accepted.","major_comments":[{"comment":"In Section III-C, HR@k is defined as 'the percentage of queries for which an approach retrieves at least one correct screen within the top-k screens.' Under this definition, HR@1 ≤ HR@5 ≤ HR@10 for every method, because a query satisfied at rank 1 is also satisfied at ranks 5 and 10. Table III, however, shows the opposite ordering in every row (e.g., Aurora FRAME: HR@1=0.5625, HR@5=0.4652, HR@10=0.4166; the same decreasing pattern appears for all baselines and all datasets). These values are mathematically impossible for the defined metric; the columns are likely a different statistic (e.g., precision-at-k or recall-at-k at that exact rank). The authors must recompute the correct hit rates, re-run the paired t-tests on those correct values, and update the retrieval claims in Section IV-A accordingly.","section":"Section III-C, Table III"},{"comment":"Section IV-A states that 'most results across all three benchmarks reaching statistical significance,' and the introduction claims FRAME 'significantly outperforms baselines across three diverse datasets.' Yet Table III reports p-values on Enrico of 0.0176 (HR@1), 0.0954 (HR@5), 0.0687 (HR@10), and 0.0586 (MRR). Only the HR@1 result is below 0.05, so the Enrico results are not consistently significant. The significance claims and the abstract's blanket wording must be revised to reflect the actual statistical outcomes, or additional evidence must be provided.","section":"Section IV-A, Table III"},{"comment":"Equation (1) defines the propagation update as S' = (I_N + w D^{-1/2} A D^{-1/2}) S, where w is 'a constant to balance the information from the original node/component with structural information from the neighbor nodes/components.' The manuscript never reports the value of w used in the experiments or any tuning procedure for it. Because this equation is the core of the propagation phase, the missing value prevents an independent re-instantiation of FRAME from the text and should be added to the experimental configuration (e.g., in Section III or the replication package).","section":"Section II-C, Eq. (1)"},{"comment":"The clustering evaluation in Section III-D and Table IV compares FRAME only to CLIP, and the results are reported from a single run with no variance estimates or significance tests. Absolute clustering accuracy is very low even for FRAME (0.0217 on Aurora, 0.0543 on Enrico, 0.1669 on Avgust). Given the paper's claim of superior clustering, the authors should report multiple K-means runs with different seeds (or other forms of variance), add a significance analysis, and compare against at least one additional baseline (e.g., BLIP or Screen2Vec) so that the clustering claim is not tied to a single comparison.","section":"Section III-D, Table IV"},{"comment":"The 300-px Manhattan edge threshold in Section II-B2 and the Rips radius epsilon=0.5 in Section II-D are selected by visual inspection on a small held-out subset of RICO, yet they are applied unchanged to Avgust, Aurora, and Enrico. The manuscript should report a sensitivity analysis of the retrieval (and clustering) results with respect to these two parameters, at least over a small grid around the chosen values, to demonstrate that the reported gains are not artifacts of dataset-specific tuning.","section":"Section II-B2 and II-D"}],"minor_comments":[{"comment":"Heron's formula in Eq. (2) is typeset with a bare 'p' instead of a square-root symbol; the notation A, B, C for edge lengths also conflicts with the adjacency matrix A in Eq. (1). The formula should be restated cleanly.","section":"Section II-D, Eq. (2)"},{"comment":"In the Avgust HR@10 row, the entry '0.00150MRR' has the MRR label run into the p-value; this formatting error should be fixed.","section":"Table III"},{"comment":"The abstract and introduction state that FRAME achieves '7.6 percentage points higher accuracy in clustering'; Table IV shows per-dataset gains of 2.0, 4.7, and 16.0 percentage points (Aurora, Enrico, Avgust), so the 7.6 figure appears to be an average and should be labeled as such.","section":"Abstract and Section I"},{"comment":"Section III-F states that the qualitative study uses a single randomly selected test image from Avgust; this makes the qualitative comparison in Section IV-D anecdotal. The paper should either present several examples or explicitly describe this as an illustrative case, not evidence of general behavior.","section":"Section III-F and IV-D"},{"comment":"The acronym expansion uses 'ReinForced' but the standard spelling is 'Reinforced'; please correct the acronym expansion in the abstract and Section I.","section":"Throughout"},{"comment":"The sentence 'The NG-C (contrast only) variant is the least effective configuration, ranking last on all eight dataset–metric combinations' is correct for Table VI, but the subsequent discussion of G-NC vs. NG-NC on Avgust could be clarified by reporting the tied HR@1 values (0.8784) explicitly in the text.","section":"Section IV-C, Table VI"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has promising ideas and a thorough ablation framework, but the internal metric inconsistency and overclaimed significance require careful revision. The authors should verify the calculation of the HR@k values in the replication package before resubmission, and they should ensure that the Enrico significance claims match the reported p-values. The clustering evaluation needs more rigor, and the missing propagation weight should be documented. These issues are fixable, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know two things about this paper before reading further. First, the core idea is worth taking seriously: FRAME augments CLIP and BERT embeddings with a UI component graph, spectral propagation, and Vietoris–Rips triangle weighting, and it does so without any training. That is a genuinely new combination for feature-oriented screen similarity, and the component ablations support the intuition that each piece earns its place. Second, the retrieval table as printed contains a statistical impossibility: HR@k, defined as \"at least one correct in top-k\", cannot decrease as k grows, yet every row of Table III shows HR@1 > HR@5 > HR@10. That is a red flag that the HR columns are not the hit rate described in the text.\n\nWhat the paper does well: the method is clearly specified, the datasets are established, the baselines are reasonable (CLIP, BLIP, BERT, Screen2Vec), and the MRR improvements on Aurora and Avgust are backed by paired t-tests. The clustering comparison against CLIP shows large relative gains on FMI, ARI, NMI, even if absolute accuracy is low. The authors also promise a replication package, which is exactly what this line of work needs.\n\nThe soft spots, in proportion: the HR@k issue is not minor. It undermines part of the headline evidence. The MRR rows may still be correct, but the paper needs to recompute the hit-rate curves and either report the actual definition or remove them. The clustering experiment is single-run with no error bars; for a method with several tuned hyperparameters (300 px edge threshold, epsilon=0.5, PCA dim, contrast factor, and especially the propagation weight w in Eq. (1), which is never reported), that is a real reproducibility gap. The low absolute clustering accuracies also deserve a more measured presentation than \"7.6 percentage points higher\" without context.\n\nMy overall take: the central claim—FRAME improves feature-oriented retrieval and clustering over CLIP and other baselines—is plausible and largely supported by the MRR results. But the impossible HR numbers and missing hyperparameters mean the paper is not ready as-is. It deserves a serious referee, not a desk rejection, because the method is novel and the evaluation, once cleaned up, would be a solid contribution to UI engineering.\n\nWho will get value: anyone working on screen similarity, UI testing transfer, duplicate bug reports, or design search. Recommend: send to peer review with a request for major revision, focusing on the metric recomputation and full hyperparameter disclosure.\n\nLet me know if you want to discuss further.","headline":"FRAME is a genuinely novel training-free UI embedding approach with a solid MRR result, but the impossible HR@k table and undisclosed hyperparameters (w) need major revision before the retrieval claim is fully trustworthy.","tokens_in":18294,"tokens_out":2564,"would_cite":false,"duration_ms":25593,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FRAME builds a spatial graph of UI components and uses spectral propagation plus Rips-complex weighting to make screen embeddings capture the features a screen affords, improving retrieval and clustering over CLIP and other baselines…","keywords":["screen retrieval","screen clustering","UI embedding","neuro-symbolic","Vietoris-Rips complex","embedding propagation","vision-language models","mobile UI understanding"],"falsifier":"Run FRAME on a held-out corpus of screens at a different resolution or from a different platform (for example, iOS or web) without retuning the 300-pixel threshold or the Rips radius, and compare its MRR and clustering accuracy with CLIP; if the margin disappears while a per-dataset tuned version restores it, the claim that the fixed graph geometry generalizes is false.","tokens_in":17211,"feed_emoji":"📱","tokens_out":8530,"duration_ms":83102,"temperature":0.7,"pith_summary":"FRAME tries to establish that app screens can be embedded so that similarity reflects the features a screen affords, rather than only its pixels or text. The paper argues that augmenting off-the-shelf vision-language embeddings with a symbolic UI component graph—detecting components, connecting them spatially, propagating embeddings along the graph, and weighting them by triangle areas from a Vietoris-Rips complex—produces embeddings that rank and cluster screens by function better than the base models. This matters because UI testing, bug report management, and design tools depend on retrieving or grouping screens that share functionality despite different visual styles. Across the Aurora, Avgust, and Enrico benchmarks, the paper reports up to 13% higher MRR (mean reciprocal rank) in screen retrieval and up to 7.6 percentage points higher clustering accuracy over strong baselines, with no additional training of the underlying models.","feed_headline":"FRAME ranks app screens by shared features, beats CLIP by 13%","feed_subtitle":"A graph of UI components plus geometry groups screens by what they do, not how they look.","key_machinery":"The load-bearing mechanism is a UI component neighborhood graph combined with two operations: (1) embedding propagation through the first-order spectral filter $S' = (I_N + w D^{-1/2} A D^{-1/2})S$, which mixes each component's representation with its neighbors and approximates a localized graph filter; and (2) geometric weighting through the Vietoris-Rips complex, in which open balls of radius $\\epsilon = 0.5$ around propagated embeddings form 2-simplices and the area of each triangle (Heron's formula) is used as the weight of the three involved components. Together these operations encode the symbolic prior that spatially grouped UI components correspond to screen features, so the final screen embedding preserves feature structure that flat pixel and text embeddings miss.","core_discovery":"FRAME takes a screenshot, normalizes it to greyscale with doubled contrast, detects UI component bounding boxes, and builds a graph whose nodes carry a CLIP image embedding and a BERT text embedding. Edges connect components within 300 px Manhattan distance. It mixes each node's embedding with its neighbors through a first-order spectral filter, constructs a Vietoris-Rips complex on the propagated embeddings, and uses the area of each 2-simplex (triangle) as a weight so that clustered components dominate the screen representation. The weighted, propagated component embeddings are concatenated with a whole-screen CLIP embedding and projected to 116 dimensions. The paper's central claim is that this graph-and-geometry augmentation makes the resulting embedding feature-oriented: screens that support the same function become closer in embedding space even when their visual designs differ, improving retrieval ranking and clustering on three UI benchmarks.","pith_inferences":["Replacing CLIP with a newer vision-language model should change the size of FRAME's gains, since the method inherits component-level visual discrimination from the base model.","The 300-pixel edge threshold and $\\epsilon=0.5$ Rips radius are tuned on RICO screens; at other resolutions or on web and desktop layouts these parameters would likely need re-tuning, and a sweep would show whether per-dataset tuning increases the margin over CLIP further.","A direct test of whether the geometric weighting carries the feature signal would compare FRAME with a variant that replaces triangle-area weights with uniform averaging over graph neighbors, since the paper does not ablate the Rips radius or edge threshold themselves."],"forward_implications":["FRAME can be plugged into existing screen-retrieval-based UI tools, such as test transfer, duplicate bug report detection, and design search, without retraining the underlying vision-language model.","Because FRAME augments rather than replaces CLIP and BERT, the embedding quality should improve automatically as those base models improve.","The clustering results indicate that FRAME separates functional screen types more cleanly than CLIP, which suggests it can support app categorization and design-pattern discovery from unlabeled screens.","Only a screenshot is needed at inference time, so FRAME works in settings where Android view hierarchies or other runtime metadata are unavailable."],"supporting_citations":[{"why":"Supplies the CLIP image embeddings used for both whole-screen and per-component visual encoding.","marker":"[24]"},{"why":"Supplies the BERT text embeddings used for lexical information extracted from each UI component.","marker":"[28]"},{"why":"Supplies the first-order localized spectral filter approximation on which FRAME's embedding propagation is based.","marker":"[31]"},{"why":"Provides the GUDHI implementation of the Vietoris-Rips complex used for geometric weighting.","marker":"[23]"},{"why":"Supplies the area-based triangulated embedding (ATE) method that turns 2-simplex areas into embedding weights.","marker":"[33]"},{"why":"Supplies the UI component detection technique that produces the bounding boxes and graph nodes.","marker":"[26]"},{"why":"Supplies the Avgust dataset labels used to evaluate screen retrieval and clustering.","marker":"[7]"},{"why":"Supplies the Aurora dataset labels used to evaluate screen retrieval and clustering.","marker":"[8]"},{"why":"Supplies the Enrico dataset labels used to evaluate screen retrieval and clustering.","marker":"[20]"},{"why":"Supplies the RICO dataset used for tuning the graph edge threshold and as the source for Aurora.","marker":"[16]"}],"fun_headline_variants":["FRAME: graph-geometry UI embeddings boost app screen search 13%","UI screens matched by function: FRAME beats CLIP by 13% MRR","Neuro-symbolic UI embedding clusters app screens by features, up 7.6 pts","FRAME: graph geometry in UI embeddings beats CLIP 13% MRR, 7.6 pts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the single 300-pixel Manhattan edge threshold and the single Rips radius of 0.5, chosen by visual inspection on a small held-out subset of the RICO dataset, also hold for the Avgust, Aurora, and Enrico datasets so that the triangles they create correspond to real UI features.","fun_headline_variants_meta":{"raw":{"variants":["FRAME: graph-geometry UI embeddings boost app screen search 13%","UI screens matched by function: FRAME beats CLIP by 13% MRR","Neuro-symbolic UI embedding clusters app screens by features, up 7.6 pts","FRAME: graph geometry in UI embeddings beats CLIP 13% MRR, 7.6 pts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000644,"raw_usage":{"total_tokens":2942,"prompt_tokens":909,"completion_tokens":2033,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":1938}},"tokens_in":525,"tokens_out":2033,"duration_ms":19993,"temperature":1.0,"reasoning_tokens":1938,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:22:05.321278+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FRAME on a held-out corpus of screens at a different resolution or from a different platform (for example, iOS or web) without retuning the 300-pixel threshold or the Rips radius, and compare its MRR and clustering accuracy with CLIP; if the margin disappears while a per-dataset tuned version restores it, the claim that the fixed graph geometry generalizes is false.","supporting_citations":[{"cited_title":"Semi-supervised classification with graph convolutional networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the first-order localized spectral filter approximation on which FRAME's embedding propagation is based."},{"cited_title":"Rips complex,","cited_arxiv_id":null,"evidence_quote":"Provides the GUDHI implementation of the Vietoris-Rips complex used for geometric weighting."},{"cited_title":"Vietoris-rips complex: A new direction for cross-domain cold-start recommendation,","cited_arxiv_id":null,"evidence_quote":"Supplies the area-based triangulated embedding (ATE) method that turns 2-simplex areas into embedding weights."},{"cited_title":"UIED: UI element detection — detecting UI elements from UI screenshots or drawings,","cited_arxiv_id":null,"evidence_quote":"Supplies the UI component detection technique that produces the bounding boxes and graph nodes."},{"cited_title":"Aurora: Navigating UI tarpits via automated neural screen understanding,","cited_arxiv_id":null,"evidence_quote":"Supplies the Aurora dataset labels used to evaluate screen retrieval and clustering."},{"cited_title":"Enrico: A high-quality dataset for topic modeling of mobile UI designs,","cited_arxiv_id":null,"evidence_quote":"Supplies the Enrico dataset labels used to evaluate screen retrieval and clustering."}],"review_version":1}