{"id":"472946dc-5245-4090-ab65-330dc739e1ba","arxiv_id":"2607.11424","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.5,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"DiffLens quantifies neighbor-, path-, and structure-based local sampling differences and visualizes them with interactive lenses so users can diagnose and compare graph sampling strategies.","lead":"DiffLens is a visualization system that quantifies and displays local differences between original and sampled graphs in three categories (neighbor, path, structure) via multicolor heatmaps and interactive lenses. It helps sampling users and developers compare strategies and diagnose where sampling distorts important local structure.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Table 2 numerical errors and incomplete metric reporting undermine the statistical claim that DiffLens yields higher-quality selections of sampling-critical elements.","rationale":"The Reader correctly flags the hand-selected factors (Table 1, §4.2–5) as a soft spot and notes the Table 2 anomalies in passing. However, the load-bearing support for the strongest claim is the controlled user-study result that DiffLens produces both faster and higher-quality selections. The factor taxonomy could be incomplete yet the system still help users; conversely, if the metric-quality numbers are artifacts of the corrupted table, the statistical half of the claim fails even if the factors are perfect. Because the paper already acknowledges the small convenience sample and the need for minor revisions, the appropriate verdict remains CONDITIONAL, but the concrete revision that must be required is a corrected Table 2 (or release of the raw selections) rather than merely an expanded discussion of factor coverage. The case studies remain informative and the system design is sound; the concern is narrowly about the quantitative evidence that underpins the comparative claim.","tokens_in":21812,"tokens_out":584,"duration_ms":7198,"concrete_test":"Request the authors' raw per-participant node/path/structure IDs for all three tasks and all three sampling rates; recompute the nine classic metrics (DC, EC, PR, BC, CC1, NC, CC2, NoC, SC) from the original email graph and re-run the paired t-tests. If any corrected mean difference loses significance or reverses direction, the claim of superior metric quality collapses.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim rests on the user study (N=12) showing both faster task completion and higher classic-metric values for the nodes/paths/structures users selected with DiffLens versus baseline (Table 2, paired t-tests p<0.05 / p<0.001). Inspection of Table 2 reveals clear copy-paste or transcription errors: under 30% sampling, Task 2 Baseline BC/CC1/NC are listed as 0.0072/0.0498/0.0014 (identical to Task 1 Baseline PR/EC/DC values) and Task 3 Baseline likewise re-uses the same three numbers; DiffLens 30% Task 2/3 rows are similarly corrupted. Because the paper never supplies the raw per-participant selections or the exact aggregation procedure that produced those averages, it is impossible to verify that the reported metric gains (and therefore the p-values) are real rather than artifacts of the corrupted table. The factor-completeness assumption identified by the Reader is secondary; even if the eight factors are adequate, the quantitative evidence that users actually locate more important distortions is currently untrustworthy.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper addresses the lack of systematic local evaluation of differences introduced by graph sampling. It surveys and taxonomizes existing metrics into neighbor-based, path-based, and structure-based categories, extracts eight representative factors via correlation analysis (Table 1), defines quantitative local difference scores (Eqs. 1–10), and presents DiffLens: a system with multicolor heatmaps plus three specialized interactive lenses that encode those scores. Two expert case studies on real networks and a within-subjects user study (N=12) with a baseline comparison claim that DiffLens enables faster and higher-quality identification of sampling-critical nodes, paths, and structures (Table 2, paired t-tests).","tokens_in":22172,"tokens_out":1062,"duration_ms":19880,"significance":"If the claims hold, DiffLens supplies the first interactive, lens-based tool focused on local (rather than global) sampling distortions, which is practically useful for both sampling algorithm developers and analysts who must choose strategies for downstream tasks such as classification or routing. The taxonomy, factor selection, and three families of difference scores are reusable contributions beyond the particular visual interface. The case studies concretely illustrate how the lenses surface bridge-node loss and triangle destruction that global metrics obscure. These strengths are tempered by the small, homogeneous participant pool and by numerical problems in the quantitative evaluation that currently prevent full verification of the statistical gains.","major_comments":[{"comment":"Table 2 (30 % columns for Tasks 2 and 3) contains clear copy-paste errors: Baseline BC/CC1/NC and DiffLens values are identical to the Task-1 DC/EC/PR numbers, and the same three numbers reappear for Task 3. Because the paper reports no raw per-participant selections and no aggregation procedure, the claimed metric gains and the associated p-values (p<0.05 / p<0.001) cannot be verified. This is load-bearing for the central quantitative claim that DiffLens yields higher-quality selections of sampling-critical elements.","section":null},{"comment":"Section 4.2 and Table 1: the eight representative factors are justified solely by Pearson correlations with a hand-chosen subset of classic metrics; several correlations are low or negative (e.g., neighbor quality with VoteRank/PageRank, triangle count with clustering coefficient). No ablation or downstream-task experiment demonstrates that visualizing only these factors is sufficient for users to diagnose the sampling distortions that actually matter for real analyses (link prediction, community detection, routing). The completeness assumption therefore remains untested.","section":null},{"comment":"Section 7.2 user study: N=12 graduate students already familiar with visualization and graph sampling, within-subjects design, no power analysis, and no external (non-student) validation. While common in HCI, the combination of the small homogeneous sample with the corrupted Table 2 makes the statistical superiority claim fragile; at minimum the study must be re-run or the raw data and corrected table supplied so that the p-values can be recomputed.","section":null}],"minor_comments":[{"comment":"Section 5.2, Eq. (5)–(7): the Euclidean distance on the full shortest-path / betweenness / connectivity matrices is O(|V|²) even when the lens is applied to a local region; the text claims a local-region setting but does not describe any pruning or approximation that would make interactive use scalable.","section":null},{"comment":"Figure 2 caption and surrounding text: the six sampling strategies and four local regions are never named or shown; readers cannot reproduce the percentage-of-global-difference numbers.","section":null},{"comment":"Typographical issues: “EV ALUATION”, “V oteRank”, “k′c_max” vs. “k′c−max” inconsistency, missing spaces after periods in several places, and “submitted to COMPUTER GRAPHICSForum” footer on every page.","section":null},{"comment":"Section 6.1: the multicolor heatmap assigns a single dominant difference category per community by averaging node ranks; the aggregation rule and the precise community-detection algorithm used are not specified, making the visual encoding non-reproducible.","section":null},{"comment":"Weights α, β, γ appear in Eqs. (1), (4), (8) and are said to be user-tunable, yet the control panel (Fig. 4a) and the user-study protocol never describe how (or whether) participants actually adjusted them; default equal weights are used throughout the evaluation.","section":null}],"recommendation":"major_revision","confidential_remarks":"The Table 2 corruption is severe enough that I would normally request the raw data and a corrected table before any further review rounds; if the authors cannot produce them, the quantitative half of the evaluation should be withdrawn and the paper re-evaluated on the case studies alone. The system itself looks solid and novel for the venue."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The core of this paper is a practical system for looking at local (not global) distortions that graph sampling introduces. They taxonomize differences into neighbor-, path-, and structure-based, pick eight representative factors via correlation checks, define node-level scores (Eqs. 1–10), and wrap them in a multicolor heatmap plus three purpose-built interactive lenses. That combination is new; prior sampling evaluation stayed at global statistics or visual-perception scores.\n\nThe case studies are the strongest part. Two domain experts walk through real networks (VIS co-authorship, a 2.4k-node graph) and show how the lenses surface missing bridges, lost high-quality neighbors, and broken triangles, then use that to pick among RNS/TIES/DFS. The formulas themselves are straightforward and the correlation table (Table 1) is only used for factor selection, not circular validation. Free weights (α, β, γ) are exposed to the user, which is honest.\n\nThe soft spots are real but contained. Table 2 has clear transcription errors: the 30 % columns for Tasks 2 and 3 simply reuse Task-1 metric values for both baseline and DiffLens. Without raw selections or the aggregation code, the reported metric gains and the p-values that rest on them cannot be trusted. N=12 convenience sample of already-familiar students is also thin, and there is no code release. Factor completeness is an assumption, not a proof, but it is secondary to the table problem. Scalability notes in the discussion are candid.\n\nThis is for people who build or choose sampling algorithms and need to diagnose local damage before a downstream task. It is not a theoretical advance, but it is a usable diagnostic tool that a visualization or network-mining venue should send to referees. Fix the table, release the data or code, and it is ready. I would bring it to reading group as a systems example and would cite the taxonomy and lens designs if I were writing on sampling evaluation.","headline":"Solid local-sampling viz system with clean taxonomy and lenses; user-study stats are undermined by obvious Table 2 copy-paste errors but the design still fills a real gap.","tokens_in":22737,"tokens_out":508,"would_cite":true,"duration_ms":14015,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"DiffLens turns graph-sampling distortions into three interactive local lenses so users can see where and why a sample broke the original network.","keywords":["graph sampling","local differences","interactive lenses","visual analytics","neighbor-based metrics","path-based metrics","structure-based metrics","heatmap guidance"],"falsifier":"A controlled task in which users armed with DiffLens systematically miss (or rank lower) the local substructures whose removal is known, by independent ground-truth metrics or expert judgment, to most degrade a concrete downstream task such as link prediction or community detection.","tokens_in":22732,"feed_emoji":"🔍","tokens_out":598,"duration_ms":6046,"temperature":0.7,"pith_summary":"Graph sampling shrinks large networks so they can be analyzed or drawn, but it inevitably changes local neighborhoods, paths, and community structure. Global averages hide those changes, so users cannot tell which regions of a sample are faithful and which are misleading. DiffLens first groups dozens of existing evaluation metrics into three families—neighbor-based, path-based, and structure-based—then defines compact quantitative measures for each family that can be computed at every node. A multicolor heatmap over the whole graph shows which kind of distortion dominates each region; three purpose-built interactive lenses then let a user zoom into any region and read the precise local causes (lost high-degree neighbors, stretched shortest paths, broken triangles, etc.). Case studies and a twelve-person controlled study show that people locate the most damaging local changes faster and more accurately than with a baseline that only displays the sampled graph. The practical result is a tool that both sampling developers and analysts can use to compare strategies and choose the sample that best preserves the structures that matter for their task.","feed_headline":"Three lenses show where graph sampling breaks a network","feed_subtitle":"Local neighbor, path and structure measures let users compare sampling strategies by the damage they actually cause","key_machinery":"The three difference-based lenses—neighbor “fishbowl,” path outer-ring, and structure “radar”—that encode the quantified local measures and support click-to-expand interactions that reveal the exact nodes or edges responsible for each distortion.","core_discovery":"Three families of local quantitative difference measures (neighbor count/quality, path length/betweenness/connectivity, triangle/triplet counts and proportion), paired with three corresponding interactive lens designs, let users diagnose and compare sampling-induced distortions at the level of individual regions rather than global averages.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Three lenses expose local neighbor path and structure damage from sampling","DiffLens maps regional distortions after graph sampling","Local measures and lenses diagnose sampling differences by region","Visual lenses compare how sampling breaks neighborhoods paths and structures","Spot sampling-induced local graph changes with neighbor path structure lenses"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That the eight hand-chosen representative factors are complete enough that visualizing only those factors correctly surfaces the sampling distortions that matter for real downstream tasks.","fun_headline_variants_meta":{"raw":{"variants":["Three lenses expose local neighbor path and structure damage from sampling","DiffLens maps regional distortions after graph sampling","Local measures and lenses diagnose sampling differences by region","Visual lenses compare how sampling breaks neighborhoods paths and structures","Spot sampling-induced local graph changes with neighbor path structure lenses"]},"model":"grok-4.5","effort":"low","cost_usd":0.004738,"raw_usage":{"total_tokens":1307,"prompt_tokens":728,"num_sources_used":0,"completion_tokens":78,"cost_in_usd_ticks":47380000,"prompt_tokens_details":{"text_tokens":728,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":501,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":728,"tokens_out":78,"duration_ms":5094,"temperature":1.0,"reasoning_tokens":501,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T05:40:26.881179+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"A controlled task in which users armed with DiffLens systematically miss (or rank lower) the local substructures whose removal is known, by independent ground-truth metrics or expert judgment, to most degrade a concrete downstream task such as link prediction or community detection.","supporting_citations":[],"review_version":1}