{"id":"41a15393-d734-4bc2-9ad3-1172d5ffe2cc","arxiv_id":"2412.07751","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper introduces the Blurry Places benchmark and an evaluation showing that deblurring, especially DeblurGANv2, improves visual place recognition accuracy under severe synthetic motion blur.","lead":"This paper introduces a new benchmark that simulates motion blur by averaging high-frame-rate video frames, and it tests how nine visual place recognition methods and three deblurring methods perform as blur increases. The results show that deblurring helps under severe blur, and the paper proposes an adaptive deblurring strategy that can save energy while keeping accuracy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Synthetic motion blur in Blurry Places (Eq. 3) is never validated against real motion blur; the benchmark's external validity, and thus the paper's main rankings, rest on an untested transfer assumption.","rationale":"After reading the paper, the central contribution is the Blurry Places benchmark and the empirical findings drawn from it. The most load-bearing assumption is that the synthetic blur generation in Eq. 3 is representative of the motion blur a robot would encounter. This assumption is not tested anywhere in the paper; the authors explicitly market the benchmark as 'without field image acquisition,' which is an advantage for convenience but also a threat to external validity. The concrete test above would settle the transfer question directly.\n\nI considered whether an internal inconsistency in the adaptive deblurring section should be the primary concern. Table IV reports Detect+Deblur time per query of 43.65 ms on CSN-Mixed, close to the 44.25 ms of All Deblur, even though the shuffled dataset is about 50% sharp frames. If the Laplacian-variance detector were actually skipping deblurring on sharp frames, the time should be much lower than all-deblur unless the detector itself costs ~18 ms/query, which is implausible for Laplacian variance. This suggests either the detection stage is not saving the expected compute or the reported timings are measured differently. However, this issue affects only the third contribution (adaptive deblurring); the synthetic-blur validity issue affects the entire benchmark and the main deblurring ranking claim. Thus I agree with the reader's weakest assumption and recommend keeping the verdict conditional.","tokens_in":11903,"tokens_out":7411,"duration_ms":85390,"concrete_test":"Run the paper's exact VPR + deblurring evaluation on a set of real motion-blurred images with known exposure times (e.g., using the RealBlur dataset or a beam-splitter/long-exposure field rig), and compare per-method AUC curves against the synthetic blur results in Figs. 3-5. Specifically, check whether DeblurGANv2 remains the best deblurrer and whether the average AUC gain over no-deblur is within ~0.03 of the values reported for the corresponding blur level. If the ranking changes or the gain magnitude differs materially, the Blurry Places benchmark is not a valid proxy for real-world motion blur and the conclusions should be re-scoped as synthetic-only.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The benchmark's core assumption is that blur generated by averaging L consecutive 240fps frames (Eq. 3) faithfully represents real motion blur. This is asserted in Section III-A with no validation against physically captured blurred images. Eq. 1 models an ideal global-shutter continuous exposure, but the GoPro 11's 240fps frames are themselves captured with a rolling shutter and finite per-frame exposure; averaging L such frames is not numerically identical to a single exposure of duration L/240 s. Real motion blur also includes sensor noise, non-uniform camera motion, and 3D scene structure, none of which are modeled. Because the benchmark is the sole evidence for the paper's main conclusions (which VPR methods are robust, and that DeblurGANv2 is the most effective deblurrer), the external validity of every reported ranking depends on this unvalidated transfer. A real-blur test could easily change the relative ordering of deblurring methods, especially since DeblurGANv2 and GShift-Net were additionally trained on 240fps-averaged data, making them artificially in-domain for this benchmark.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Blurry Places, a new benchmark for evaluating visual place recognition (VPR) under motion blur. Blurred query images are generated by averaging L consecutive frames from 240 fps videos recorded along three outdoor routes in Italy, yielding nine blur levels (L=1 to L=240) and both same-traverse and mixed-condition (weather, illumination, viewpoint) query-reference pairs. The authors evaluate nine VPR methods under increasing blur and three deblurring methods (DeblurGANv2, FFTFormer, GShift-Net), reporting that motion blur degrades VPR performance and that DeblurGANv2 is the most effective and robust deblurring method. They also propose an adaptive deblurring strategy that detects blurred queries and deblurs selectively, claiming the same performance gain as full deblurring at lower energy cost. A public repository for the blur generator is provided.","tokens_in":12095,"tokens_out":6361,"duration_ms":56432,"significance":"If the benchmark is valid, it fills a genuine gap: motion blur is an underexplored but practically important challenge for VPR, and existing datasets are limited in blur intensity and scene diversity. The paper's systematic sweep over blur levels combined with other appearance changes is a useful resource, and the code release supports reproducibility. The observation that global-descriptor VPR methods benefit most from deblurring and the energy analysis of adaptive deblurring are practically relevant. However, the significance is conditional on the synthetic blur being representative of real-world motion blur, which is not validated, and on the deblurring comparison not being confounded by training-domain overlap.","major_comments":[{"comment":"The synthetic blur is generated by averaging L consecutive frames from 240fps video, but this process is not validated against real motion-blurred images. The paper's central contribution is a benchmark that 'enables comprehensive analysis without field image acquisition,' which depends on the synthetic blur faithfully representing real-world motion blur. However, real cameras exhibit rolling-shutter distortion, finite per-frame exposure, sensor noise, and non-uniform motion, so averaging frames is not mathematically equivalent to a single exposure of duration L/240 s as implied by Eq. (4). Please provide a validation study comparing the synthetic blur against real blurred images (e.g., captured with longer exposure or with known blur kernels), or at minimum a qualitative and quantitative discussion of the domain gap. The absence of this validation weakens the external validity of all subsequent rankings and conclusions.","section":"Section III-A, Eq. (3)"},{"comment":"The comparison of deblurring methods is confounded by training-data distribution. DeblurGANv2 and GShift-Net were additionally trained on DVD (and DeblurGANv2 on NFS), both containing 240fps video data, while FFTFormer was trained on RealBlur and HIDE. Since the Blurry Places benchmark is generated by averaging 240fps frames, DeblurGANv2 and GShift-Net are effectively in-domain for this synthetic blur. The conclusion that 'DeblurGANv2 proves to be the most effective and robust choice of the three when applied to VPR' may be an artifact of this overlap. Please control for training distribution, e.g., by fine-tuning all methods on the same data, or by evaluating on real blurred images, or by reporting performance with held-out synthetic blur and analyzing the domain gap.","section":"Section V-C, Table III"},{"comment":"The evaluation protocol is under-specified, which hampers reproducibility. The paper does not state which similarity measure is used (e.g., cosine similarity, Euclidean distance), how the reference set is defined, or how AUC is computed (e.g., area under a precision-recall curve, average top-N recall, or rank-based metric). It also omits descriptor normalization and key hyperparameters for each VPR method. Please provide a complete protocol description, including any preprocessing steps, so that the reported figures can be independently reproduced and compared with future work.","section":"Section IV, Experimental Setup"},{"comment":"The adaptive deblurring analysis lacks details needed to interpret the results. No threshold or calibration procedure is given for the Laplacian-variance blur detector, so the operating point of 'Detect+Deblur' is not reproducible. Furthermore, the efficiency numbers are inconsistent: in Table IV(b), Detect+Deblur has a higher total time than All Deblur (16.46 s vs 15.40 s) yet a lower energy consumption (5.25 kJ vs 5.42 kJ), which is not explained. Please specify the detection threshold, report the false-positive/negative rates of the detector, and clarify how energy and time are measured.","section":"Section V-D, Table IV"}],"minor_comments":[{"comment":"The summation notation is incorrect: the sum should be over I_i(x) for i = j to j+L-1, not I_j(x), and the current expression implies L+1 terms while dividing by L.","section":"Eq. (3)"},{"comment":"The phrase 'without field image acquisition' overstates the case, because the dataset is built from field-recorded videos; it would be more accurate to say 'without dedicated acquisition of blurred images on the field.'","section":"Section I, Contributions"},{"comment":"The model name is written inconsistently as 'P-NetVLAD' and 'PNet-VLAD'; please use a single spelling throughout.","section":"Section V-C"},{"comment":"The 'Frames' column lists raw counts without stating whether these are total sharp frames per video or the number used after processing; please clarify the unit and selection.","section":"Table I"},{"comment":"The tables and figures report single AUC values without variance across runs. Given that some methods involve stochastic components, providing confidence intervals or repeated evaluations would strengthen the reliability of the observed rankings.","section":"Section V, Results"}],"recommendation":"major_revision","confidential_remarks":"The benchmark and code release are valuable, but the two critical issues are the unvalidated synthetic blur and the training-domain confound in the deblurring comparison. Both are addressable with additional experiments. If the authors demonstrate transferability to real blurred images and re-analyze the deblurring results controlling for domain, the paper would be a solid contribution. The paper is not fully publishable in its current form, but the core idea is sound and the required revisions are within the scope of a major-revision request."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read the Blurry Places paper. It does something genuinely useful: builds a motion-blur benchmark for VPR from 240fps GoPro footage by averaging L frames, with three routes, nine blur levels, and mixed appearance-change traverses. The systematic comparison of nine VPR methods and three deblurring methods is the most complete treatment of this specific problem I've seen, and the main finding—that deblurring helps most for severe blur and for descriptor-based models like CosPlace and EigenPlaces—is plausible and backed by a lot of tables. The adaptive deblurring analysis is a nice bonus, even if it's preliminary.\n\nThe soft spots are real but not disqualifying. The synthetic blur is never validated against physically captured blurred images. Averaging 240fps frames is the standard trick used to make GoPro and HIDE, so it's not a bad proxy, but the paper claims the benchmark enables analysis 'without field image acquisition' and then draws robust conclusions about how VPR methods will behave under real motion blur. That transfer is assumed, not shown. A small validation set with real blurred captures would have greatly strengthened the claims.\n\nThe deblurring comparison is also partially confounded: DeblurGANv2 and GShift-Net were trained on 240fps-averaged data (DVD, NFS), which is exactly the same synthetic degradation used to build Blurry Places. FFTFormer, trained on RealBlur and HIDE, is at a domain disadvantage. The authors acknowledge the training data difference in Section V-C, but they don't treat it as a limitation when they conclude DeblurGANv2 is 'most effective and robust.' That conclusion should be softened.\n\nAlso minor: no error bars or statistical tests across runs—AUC differences of 0.02-0.04 are treated as meaningful. And the adaptive deblurring section reports same-AUC for detect+deblur as all-deblur without reporting detection accuracy, so the reader can't tell if the detector is near-perfect or if the test is too easy.\n\nOverall, the paper is honest, the benchmark is a reusable resource, and the analysis is more thorough than typical for this area. The central results are conditional on the synthetic-blur transfer assumption, but that's a standard limitation for benchmark papers and not a fatal one. The paper deserves a serious referee; the main requests should be a real-blur validation or explicit caveats, error bars, and reporting detection metrics in the adaptive section.","headline":"A genuinely useful VPR motion-blur benchmark with a thorough method comparison, but the central rankings rest on an unvalidated synthetic-blur transfer assumption and a deblurring comparison that is partly confounded by training data overlap.","tokens_in":12643,"tokens_out":2228,"would_cite":true,"duration_ms":20165,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new synthetic-blur benchmark shows how motion blur degrades visual place recognition and how deblurring can recover most of the lost accuracy.","keywords":["visual place recognition","motion blur","deblurring","benchmark","adaptive deblurring","image formation","mobile robotics"],"falsifier":"Collect real motion-blurred images from a camera moving at known angular velocity and compare VPR accuracy against synthetic blur at the corresponding L value; if method rankings or deblurring gains systematically differ, the benchmark's conclusions do not transfer to real robots.","tokens_in":11723,"feed_emoji":"🤖","tokens_out":3037,"duration_ms":26352,"temperature":0.7,"pith_summary":"This paper introduces the Blurry Places benchmark, which creates controlled motion blur by averaging up to 240 consecutive frames from 240fps videos, to study how blur affects Visual Place Recognition (VPR) and whether deblurring restores performance. Evaluating nine VPR methods across three routes and nine blur levels, it shows that all methods lose accuracy as blur increases, with FloppyNet most robust in blur-only conditions and global-descriptor models benefiting most from deblurring. Among three deblurring methods, DeblurGANv2 gives the largest and most consistent improvements. The paper further shows that an adaptive strategy—detecting blur first and deblurring only blurred queries—matches the accuracy gain of deblurring every frame while consuming less energy.","feed_headline":"Synthetic blur benchmark shows how motion blur breaks place recognition and how to fix it","feed_subtitle":"Averaging 240fps frames creates realistic blur; DeblurGANv2 and adaptive deblurring restore most accuracy at far lower energy.","key_machinery":"The benchmark generation is carried by the blur formation model in Eq. 3: a blurred image is the average of L consecutive sharp frames from a 240fps video, approximating the temporal integral of the physical image-formation process. This makes blur intensity directly controllable through L, with an implied exposure time of τ = L/240 seconds, and avoids filter-based blur that would not embed real scene motion. The adaptive-deblurring experiments additionally rely on Laplacian variance as a lightweight blur detector to decide which queries warrant deblurring.","core_discovery":"The paper's central claim is that motion blur, which VPR research has largely ignored, measurably degrades place recognition and that deblurring can offset much of this degradation, but only when the blur is severe enough and the VPR model relies on global image descriptors. The authors establish this by constructing the first benchmark purpose-built for motion blur together with VPR-specific appearance changes, and by systematically showing that deblurring's benefit depends on the VPR method, the blur level, and the scene variations present. They further claim that an adaptive scheme—Laplacian-variance blur detection followed by deblurring of only blurred queries—achieves the same accuracy as deblurring all frames while reducing energy consumption.","pith_inferences":["The benchmark's frame-averaging technique could be extended to simulate low-light long-exposure blur, where motion blur and sensor noise interact, to test VPR in night conditions.","The results imply that VPR systems should treat deblurring as a conditional module rather than a fixed preprocessing step, with the blur-detection threshold tuned per model and deployment environment.","Because the video-deblurring method GShift-Net produced the best extreme-blur scores with some models, temporal information from adjacent frames may be valuable for VPR at very high blur levels—a direction the authors explicitly flag for future work."],"forward_implications":["VPR evaluations can now include well-controlled motion blur without field acquisition, using the released Blurry Places data and generation scripts.","Deblurring should be applied selectively: it helps most at high blur levels (L ≥ 120) and with global-descriptor models such as CosPlace and EigenPlaces, while giving little or no benefit to already blur-tolerant models like AnyLoc at low blur.","DeblurGANv2 is the recommended deblurring method among the three tested for VPR, producing larger and more consistent AUC improvements than GShift-Net or FFTFormer.","Adaptive deblurring with a Laplacian-variance detector matches the accuracy of full deblurring while cutting energy and processing time when sharp images are common."],"supporting_citations":[{"why":"Supplies the image-formation integral that justifies averaging sharp frames as a realistic model of motion blur.","marker":"[14]"},{"why":"The GoPro deblurring dataset established the practice of averaging 240fps frames to synthesize blur, which this benchmark extends.","marker":"[9]"},{"why":"DeblurGANv2 is the deblurring method found most effective and robust; its design and training data are central to the comparison.","marker":"[25]"},{"why":"MixVPR is one of the top-performing VPR methods in the benchmark and a key baseline for measuring deblurring gains.","marker":"[17]"},{"why":"AnyLoc provides the robustness comparison for large-scale pretrained features, showing limited deblurring benefit.","marker":"[18]"},{"why":"Prior work on motion-blur-resistant VSLAM motivates the use of Laplacian variance for blur detection on resource-constrained devices.","marker":"[5]"}],"fun_headline_variants":["Blur benchmark reveals when deblurring helps place recognition","Adaptive deblurring saves energy while restoring VPR accuracy","First motion-blur benchmark for visual place recognition","Motion blur hurts VPR; adaptive deblurring recovers accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results depend on synthetic motion blur produced by averaging frames faithfully representing real-world motion blur, including its rolling-shutter distortion, non-uniform camera motion, and sensor noise.","fun_headline_variants_meta":{"raw":{"variants":["Blur benchmark reveals when deblurring helps place recognition","Adaptive deblurring saves energy while restoring VPR accuracy","First motion-blur benchmark for visual place recognition","Motion blur hurts VPR; adaptive deblurring recovers accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000582,"raw_usage":{"total_tokens":2715,"prompt_tokens":898,"completion_tokens":1817,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":514,"completion_tokens_details":{"reasoning_tokens":1746}},"tokens_in":514,"tokens_out":1817,"duration_ms":12183,"temperature":1.0,"reasoning_tokens":1746,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:30:32.452359+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect real motion-blurred images from a camera moving at known angular velocity and compare VPR accuracy against synthetic blur at the corresponding L value; if method rankings or deblurring gains systematically differ, the benchmark's conclusions do not transfer to real robots.","supporting_citations":[{"cited_title":"Ikeuchi, Ed., Computer Vision: A Reference Guide","cited_arxiv_id":null,"evidence_quote":"Supplies the image-formation integral that justifies averaging sharp frames as a realistic model of motion blur."},{"cited_title":"Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring","cited_arxiv_id":"1612.02177","evidence_quote":"The GoPro deblurring dataset established the practice of averaging 240fps frames to synthesize blur, which this benchmark extends."},{"cited_title":"Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better,","cited_arxiv_id":null,"evidence_quote":"DeblurGANv2 is the deblurring method found most effective and robust; its design and training data are central to the comparison."},{"cited_title":"MixVPR: Feature Mixing for Visual Place Recognition,","cited_arxiv_id":null,"evidence_quote":"MixVPR is one of the top-performing VPR methods in the benchmark and a key baseline for measuring deblurring gains."},{"cited_title":"A novel motion blur resistant vslam framework for micro/nano-uavs,","cited_arxiv_id":null,"evidence_quote":"Prior work on motion-blur-resistant VSLAM motivates the use of Laplacian variance for blur detection on resource-constrained devices."}],"review_version":1}