{"id":"f10b60a0-05b5-4d69-842a-d83724b5a035","arxiv_id":"1908.03852","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"StructureFlow splits inpainting into structure reconstruction on edge-preserved smooth images and texture generation via appearance flow, reporting competitive results on Places2, CelebA, and Paris StreetView.","lead":"A two-stage neural network fills in missing image regions by first reconstructing a smoothed structure image and then copying textures from intact regions using appearance flow. The method reports improved inpainting quality on several public datasets, though its own tables show the advantage is not uniform for large holes.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline claim of superior performance is undermined by the paper's own Places2 FID and user-study numbers; RTV sigma is also tuned on the Paris test set.","rationale":"The reader's weakest assumption was the RTV smoothing parameter sigma and whether the structure/texture separation is valid. I agree that this is a real soft spot, but I see an even more immediate problem for the central claim: the paper's own quantitative evidence does not consistently support 'superior performance.' Table 1 shows a worse FID than EdgeConnect on the hardest Places2 mask range, and Table 2 shows a lower user-study fooling rate on Places2. The abstract oversells the result relative to the data. The RTV sigma tuning on the Paris test set compounds this by making the comparison favorable to StructureFlow through a test-set-selected hyperparameter. I do not think the core idea is invalid: the two-stage structure-then-texture design is plausible, the ablations support the contribution of structure reconstruction and appearance flow, and the source code is available. The concern is about the strength of the performance claim, not about the method's soundness. Therefore the conditional verdict is appropriate, and I would not change it: the authors should either provide a stricter evaluation (held-out sigma, confidence intervals, full metric reporting) or soften the claim to 'competitive.' This is why I mark agreement as partial: I share the reader's concern about sigma, but I locate the most load-bearing issue in the consistency of the evidence for the headline claim.","tokens_in":10934,"tokens_out":4895,"duration_ms":54256,"concrete_test":"Use the released StructureFlow code on the Places2 test set with the PConv irregular masks, selecting sigma on a held-out validation split only (e.g., 1k images from the training split) and then freezing it; compute PSNR/SSIM/FID with bootstrap 95% confidence intervals over 10k test images, and run the MTurk protocol on a 600-image sample. If the 40-60% FID and user-study results still favor EdgeConnect while PSNR/SSIM favor StructureFlow, the abstract should be revised from 'superior' to 'competitive'.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim ('superior performance') rests on Tables 1 and 2. On Places2 with 40-60% masks, StructureFlow's FID is 22.3803 versus EdgeConnect's 19.0003, and the user-study fooling rate is 23.56% versus 26.36%, both favoring EdgeConnect. The paper emphasizes PSNR/SSIM, where StructureFlow wins. A claim of superiority requires consistent gains across distortion and perceptual metrics; the current evidence is mixed and selectively reported. The choice of the RTV structure scale sigma approximately 3 (Section 4.1) is also selected from Table 4, which evaluates sigma values 0, 1, 3, 6, and 9 on the Paris test set, i.e., test-set tuning of the core representation. Since the two-stage design and appearance-flow conditioning both depend on this representation, the reported gains may not transfer to a fixed hyperparameter or to other datasets. The central claim is therefore conditional on metric choice and on a test-tuned hyperparameter.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes StructureFlow, a two-stage image inpainting network. The first stage is a structure reconstructor that predicts edge-preserved smoothed images (obtained with RTV) for the missing regions, and the second stage is a texture generator that uses appearance flow, with Gaussian sampling and a sampling correctness loss, to synthesize high-frequency details. Experiments on Places2, CelebA, and Paris StreetView compare the method with Contextual Attention, Partial Convolution, and EdgeConnect, and are supplemented by ablation studies of the structure stage, the smoothing representation, and the appearance flow. The paper's abstract claims superior performance, although its own quantitative tables show mixed results on Places2, and the RTV smoothing scale is selected empirically on the Paris test set.","tokens_in":11146,"tokens_out":4268,"duration_ms":42817,"significance":"If the claims were fully supported, the paper would make a useful contribution: replacing edge maps with edge-preserved smooth images as the structure prior is a reasonable and potentially more informative representation, and the appearance-flow modifications address a known optimization difficulty in using flow for inpainting. The ablations do support the value of the structure reconstructor, the smoothed structure labels, and the presence of appearance flow, and the authors release source code, which strengthens reproducibility. However, the core performance claim is not consistently supported by the paper's own metrics, the key hyperparameter is tuned on a test set, and two of the three stated contributions (Gaussian sampling and sampling correctness loss) lack quantitative evaluation. These issues need to be addressed before the paper can be recommended for publication.","major_comments":[{"comment":"The abstract's claim of 'superior performance' is contradicted by the paper's own quantitative results. In Table 1, for Places2 masks of 40–60%, StructureFlow's FID is 22.3803 versus EdgeConnect's 19.0003, and in Table 2, the Places2 user-study fooling rate is 23.56% for StructureFlow versus 26.36% for EdgeConnect. The conclusion also retreats to 'competitive results.' Because the central claim of the paper is the performance advantage, these internal contradictions must be resolved by either providing evidence across all metrics and mask ranges or by softening the claim.","section":"Abstract; §4.2, Tables 1–2"},{"comment":"The choice of the RTV smoothing scale σ≈3 is made by evaluating σ ∈ {0,1,3,6,9} on the Paris test set. Since σ defines the ground-truth structure representation Sgt on which both stages are trained, selecting it on the test set means the reported improvements are partly an artifact of test-set hyperparameter fitting. Please select σ on a validation split, or alternatively show that the same σ is optimal across multiple datasets and mask ratios, and report sensitivity to this choice.","section":"§4.1, §4.3, Table 4"},{"comment":"Gaussian sampling and the sampling correctness loss are two of the three contributions listed in the paper, but their quantitative effect is not reported. The flow ablation in §4.3 only shows qualitative flow-field visualizations in Figure 6; no PSNR/SSIM/FID is given for a model with Bilinear sampling or for a model without the sampling correctness loss. Table 3 ablates the presence of flow as a whole, not these components, so the claimed improvements from these modifications are not substantiated.","section":"§4.3, Figure 6"}],"minor_comments":[{"comment":"The sentence 'We employ edge-preserved smooth method RTV [31] to obtain the training labels of the structure reconstructor Gt' should refer to Gs, since Gt is the texture generator.","section":"§4.1"},{"comment":"The user-study numbers are reported without statistical significance testing; the Paris gap between StructureFlow and EdgeConnect is only 0.24 percentage points, so the statement that the model achieves 'better results' there is not clearly supported.","section":"§4.2, Table 2"},{"comment":"Table 1 does not report FID for PConv (shown as '- - -'), even though FID is emphasized as a perceptual metric; this incomplete comparison should be fixed.","section":"§4.2, Table 1"},{"comment":"The symbol M is first used as a binary mask matrix and then as a coordinate set; please disambiguate the notation.","section":"§3.2, Eq. (10)"},{"comment":"The color-coding map used to visualize appearance flow fields in Figure 6 is not explained in the caption; please add a description of what the colors and intensities encode.","section":"§4.3, Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is clearly written and the code release is a plus. The main issue is that the abstract overstates the results; the authors' own tables undermine 'superior performance.' The test-set tuning of σ is a serious but fixable concern, and the flow ablations need numbers. I recommend asking for a revised version that aligns claims with evidence and adds the missing quantitative comparisons."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a worthwhile inpainting paper. The core idea is clean: instead of using edges or raw images as structure priors, it trains a first stage to reconstruct RTV-smoothed images (edge-preserved low-frequency structure), then a second stage generates textures using appearance flow. The Gaussian-sampling modification and the sampling-correctness loss are sensible fixes to a known failure mode of appearance flow in inpainting. The ablations are informative and isolate the contribution of each component: the structure stage, the smoothing labels, and the flow block all give measurable gains. Code is released, which is a concrete plus. The soft spots are mostly about the strength of the claims rather than the validity of the method. The abstract says 'superior performance,' but Table 1 shows a worse FID than EdgeConnect on Places2 for 40-60% masks (22.38 vs 19.00), and Table 2 shows a lower fooling rate on the Places2 user study (23.56% vs 26.36%). The paper wins on PSNR/SSIM and on the more structured datasets (Celeba, Paris), so the correct summary is 'state-of-the-art on structured scenes, competitive on general scenes' - not blanket superiority. The bigger methodological worry is test-set tuning of the RTV smoothing scale. Section 4.3 picks sigma=3 based on Paris test results across sigma values. That is precisely the kind of hyperparameter selection that can inflate gains on one dataset and not transfer. It does not kill the paper, but it means the advantage should be treated as conditional until fixed sigma is validated on held-out data. The absence of error bars or significance tests makes the quantitative comparisons weaker than they look, and a few implementation details (exact network depths, feature layer for the correctness loss) are omitted. None of this undermines the central architecture. The paper deserves a serious referee; the main fixes are rhetorical and methodological, not architectural. I'd recommend engaging with it, and I'd ask the authors to reposition the claim as 'competitive with advantages on structured scenes' and to report results with sigma held fixed across datasets, ideally with variance estimates.","headline":"A solid two-stage inpainting architecture with a real structural idea, but the 'superior performance' claim is undercut by the paper's own Places2 FID and user-study numbers.","tokens_in":650,"tokens_out":871,"would_cite":true,"duration_ms":22466,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two-stage inpainting that completes smooth structure before sampling texture outperforms single-stage models on public benchmarks.","keywords":["image inpainting","structure reconstruction","edge-preserved smoothing","appearance flow","texture generation","generative adversarial network","Gaussian sampling","sampling correctness loss"],"falsifier":"Train the same two-stage architecture with structure labels produced by a plain Gaussian low-pass filter instead of RTV, and compare on a held-out dataset: if there is no performance drop, the edge-preserving properties of RTV are not the cause of the gain. Alternatively, vary the RTV scale $\\sigma$ on a held-out set; if performance is flat across $\\sigma$, the claim that $\\sigma \\approx 3$ is the right separation scale is a tuning artifact.","tokens_in":10703,"feed_emoji":"🖼️","tokens_out":7815,"duration_ms":76546,"temperature":0.7,"pith_summary":"Image inpainting often fails in one of two ways: it invents implausible structures or smears textures. The paper proposes to fix this by splitting the task into structure reconstruction and texture generation. It argues that edge-preserved smooth images are the right structural representation, and that an appearance-flow module with Gaussian sampling plus a sampling-correctness loss makes long-range texture transfer trainable. The paper reports that the resulting two-stage network beats existing one-stage methods on several public datasets in both automatic metrics and human preference.","feed_headline":"Structure first, texture second wins image inpainting","feed_subtitle":"Rebuilding edge-preserved smooth structure first lets appearance flow sample real textures from intact regions.","key_machinery":"The load-bearing object is the edge-preserved smooth image produced by relative total variation (RTV) smoothing. At scale $\\sigma \\approx 3$ it removes high-frequency texture while retaining sharp edges, giving the structure reconstructor a training target that captures global layout without texture noise. The second load-bearing component is the appearance-flow module inside the texture generator: for each missing pixel it predicts a location in the visible feature map to sample from, and the warped features are concatenated with convolutional features to synthesize detail. Gaussian sampling widens the sampling footprint, and the sampling-correctness loss keeps the flow from locking onto bad local minima during training.","core_discovery":"The paper claims that an inpainting network can produce both meaningful structure and vivid texture by splitting the task into two learned stages. The first stage reconstructs an edge-preserved smooth image, rather than an edge map or the raw image, as the global structure target; the second stage uses the finished structure to guide an appearance-flow module that samples features from visible regions to synthesize texture. To stop the appearance flow from collapsing onto bad matches, the paper replaces bilinear sampling with Gaussian sampling and adds a sampling-correctness loss computed from a pretrained deep feature network. On scene, face, and street-view datasets, the paper reports higher PSNR and SSIM values than contextual-attention, partial-convolution, and edge-guided baselines, with competitive FID scores and higher human preference on structured scenes.","pith_inferences":["Beyond the paper, this structure-first separation could be tested on video inpainting, where temporal structure makes hallucinating plausible layouts even harder.","Beyond the paper, Gaussian sampling in appearance flow may transfer to other flow-based generation tasks such as view synthesis or image translation, with the sampling-correctness loss as a general guard against bad matches.","Beyond the paper, if the structure prior is the real driver, replacing the fixed RTV preprocessing with a learnable per-image structure extractor could yield a fully end-to-end variant that adapts the smoothing scale automatically."],"forward_implications":["Inpainting models can be built modularly: a structure reconstructor followed by a texture generator, instead of one monolithic generator.","Edge-preserved smooth images are a viable structural intermediate for inpainting, carrying layout and color without the noise of full texture.","Appearance flow with Gaussian sampling can establish long-range correspondences between visible and missing regions, reducing blur and artifacts.","The method enables practical editing workflows such as masking out unwanted objects or editing the reconstructed structure to add or remove content."],"supporting_citations":[{"why":"Supplies the RTV edge-preserving smoothing method used to produce structure labels for training.","marker":"[31]"},{"why":"Edge-guided inpainting baseline whose structure representation motivates the use of smoother structure images.","marker":"[21]"},{"why":"Contextual-attention baseline that motivates the appearance-flow approach to long-range texture correlation.","marker":"[33]"},{"why":"Provides the irregular mask dataset and the partial-convolution baseline used in comparisons.","marker":"[16]"},{"why":"Introduces appearance flow, which the paper adapts to sample texture features for inpainting.","marker":"[35]"},{"why":"Provides the large scene dataset used for training and evaluation.","marker":"[34]"},{"why":"Defines FID, one of the objective perceptual-quality metrics used in the comparison.","marker":"[10]"},{"why":"Introduces the generative adversarial objective used by both the structure and texture stages.","marker":"[7]"}],"fun_headline_variants":["Inpainting: rebuild structure first, then texture via flow","Two-stage model: structure completion then appearance flow","Edge-preserving structure guides texture inpainting","Gaussian sampling fixes appearance flow in inpainting","Structure-aware flow yields better image inpainting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim rests on the assumption that a fixed scale of edge-preserving smoothing separates texture from structure uniformly across images and mask shapes, rather than being a preprocessing choice tuned to the evaluation data.","fun_headline_variants_meta":{"raw":{"variants":["Inpainting: rebuild structure first, then texture via flow","Two-stage model: structure completion then appearance flow","Edge-preserving structure guides texture inpainting","Gaussian sampling fixes appearance flow in inpainting","Structure-aware flow yields better image inpainting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00018,"raw_usage":{"total_tokens":1233,"prompt_tokens":801,"completion_tokens":432,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":417,"completion_tokens_details":{"reasoning_tokens":357}},"tokens_in":417,"tokens_out":432,"duration_ms":4830,"temperature":1.0,"reasoning_tokens":357,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:00:04.484231+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same two-stage architecture with structure labels produced by a plain Gaussian low-pass filter instead of RTV, and compare on a held-out dataset: if there is no performance drop, the edge-preserving properties of RTV are not the cause of the gain. Alternatively, vary the RTV scale $\\sigma$ on a held-out set; if performance is flat across $\\sigma$, the claim that $\\sigma \\approx 3$ is the right separation scale is a tuning artifact.","supporting_citations":[{"cited_title":"Structure extrac- tion from texture via relative total variation","cited_arxiv_id":null,"evidence_quote":"Supplies the RTV edge-preserving smoothing method used to produce structure labels for training."},{"cited_title":"Generative image inpainting with con- textual attention","cited_arxiv_id":null,"evidence_quote":"Contextual-attention baseline that motivates the appearance-flow approach to long-range texture correlation."},{"cited_title":"Image inpainting for irregular holes using partial convolutions","cited_arxiv_id":null,"evidence_quote":"Provides the irregular mask dataset and the partial-convolution baseline used in comparisons."},{"cited_title":"View synthesis by appearance flow","cited_arxiv_id":null,"evidence_quote":"Introduces appearance flow, which the paper adapts to sample texture features for inpainting."},{"cited_title":"Places: A 10 million image database for scene recognition","cited_arxiv_id":null,"evidence_quote":"Provides the large scene dataset used for training and evaluation."}],"review_version":1}