{"id":"b478f998-72d0-4ef2-b0e6-2a5792d4912a","arxiv_id":"2506.12032","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper reports 100% decoding accuracy for a UNet-based scientific data watermarker, but the evaluation is undermined by using a fixed message for both training and testing and by not testing the robustness claimed in the abstract.","lead":"A single-author preprint describes a neural watermarking system that hides binary messages inside climate and fluid simulation data using a UNet autoencoder, and reports perfect message recovery. The paper's main claims about robustness to noise, cropping, and compression are contradicted by its own limitations section, and the reported decoding accuracy is suspect because the message used for testing is the same one used for training.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported 100% decoder accuracy may be an artifact of training and testing on the same fixed message ('Hello World!'); with no held-out-message evaluation, the central watermarking claim is unsupported.","rationale":"I independently arrived at the same load-bearing concern as the reader: the paper trains and evaluates on one fixed message, making the reported 100% decoding accuracy a weak signal that is consistent with memorization. The proposed test of decoding random held-out messages injected through the trained encoder would settle whether the decoder performs general watermark recovery or simply outputs the training message. The reader's secondary concern about the unsupported robustness claims is also valid and is explicitly acknowledged in the paper's limitations section, but the fixed-message issue is the most fundamental because it vitiates the primary quantitative result. I therefore agree with the reader's REJECT verdict and recommend no change to that verdict.","tokens_in":9312,"tokens_out":3890,"duration_ms":42588,"concrete_test":"Using the released checkpoints (e.g., the ERA5 or Fluid Flow UNet from https://github.com/KrtiT/scientific-data-watermarker), generate 100 random 100-bit messages. For each message, feed it into the trained encoder's latent injection point (args.forward_version=2) and watermark 100 held-out test images; decode with the trained decoder and compute bit accuracy versus the injected message. Also compute the decoder's output on the original, unwatermarked images to verify the claimed inverse. If bit accuracy on unseen random messages is near 50% (chance), the reported 100% reflects fixed-message memorization, and the decoder does not implement general watermark recovery.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline result is 100% decoder accuracy across all three datasets (Section 4, Tables 1-3). Section 3.2 states that the embedded payload is always the message 'Hello World!' converted to a 1x100 binary vector, and all training uses this same message. The decoder is co-optimized with a message loss on exactly this vector (Section 3.3), and the accuracy metric in Section 4 simply compares the decoder's binarized output to the same fixed vector. Consequently, a decoder that always outputs the 'Hello World!' bit string would score 100% on every watermarked test image without reading any image-specific information. The paper's reported 'inverse' behavior on non-watermarked images does not affect this metric, because accuracy is measured only on watermarked inputs. No experiment uses a held-out, random, or per-image message, so the 100% figure is consistent with memorization rather than general message embedding. This invalidates the primary quantitative evidence for the central claim. The abstract additionally claims robustness to noise, cropping, and compression, which Section 6.1 explicitly says was not evaluated; this is a second unsupported overclaim, but the fixed-message issue alone is sufficient to undermine the headline accuracy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a neural watermarking framework for scientific imaging data, using a UNet-style convolutional autoencoder to embed a fixed 100-bit message into ERA5, Fluid Flow, and Cosmology datasets from the SuperBench benchmark. The authors compare against an SVD-based watermarker and report 100% decoder accuracy across all datasets, high PSNR/SSIM, lower physics loss than SVD, and visually imperceptible outputs. The central claim is that this framework provides robust, fidelity-preserving watermarking for scientific data provenance and AI security workflows.","tokens_in":9572,"tokens_out":4703,"duration_ms":35542,"significance":"If the reported results were valid, the framework would be a practical contribution to data provenance in scientific machine learning, and the public code/notebooks plus the use of standard SuperBench data are useful starting points. However, the headline 100% decoding accuracy is not supported by the evidence as presented: the decoder is trained and evaluated on the same fixed 'Hello World!' message, so the accuracy is consistent with memorization rather than with general message-embedding capability. The abstract also claims robustness to noise, cropping, and compression that Section 6.1 explicitly states was not evaluated. These issues undermine the central quantitative evidence, leaving the paper as a preliminary demonstration rather than a validated watermarking system.","major_comments":[{"comment":"The fixed-message problem invalidates the headline accuracy claim. Section 3.2 states that the embedded payload is always 'Hello World!' converted to a 1x100 binary vector, and that early experiments using random messages per epoch were discontinued. The decoder is trained with a message loss on exactly this vector (Section 3.3), and the accuracy metric defined in Section 4 simply compares the decoder's binarized output to the same fixed vector on watermarked test images. A decoder that always outputs the 'Hello World!' bit string would achieve 100% accuracy on every watermarked test image without extracting any image-dependent information. No held-out-message or per-image random-message experiment is reported, so the 100% accuracy in Tables 1-3 does not demonstrate general watermark embedding or recovery. This is a load-bearing flaw in the paper's central claim.","section":"Section 3.2 and Tables 1-3"},{"comment":"The abstract and introduction claim that the method 'ensures watermark persistence under lossy transformations - including noise injection, cropping, and compression,' yet Section 6.1 explicitly states 'We did not yet evaluate robustness to perturbations such as noise, cropping, or compression.' The robustness claim is therefore unsupported and should be removed or substantiated with the missing experiments.","section":"Abstract and Section 6.1"},{"comment":"The training protocol is described inconsistently. Section 3.2 reports a two-phase schedule of 100 epochs with image and message losses followed by 20 epochs with added physics loss for ERA5 and Fluid Flow, and 50 epochs for Cosmology. Section 3.4 instead describes a 50-epoch curriculum in which the first 30 epochs use only image and physics losses and the message loss is introduced after epoch 30. Section 5.7 reports specific early-stopping epochs (e.g., 81/19 for Fluid Flow, 90/16 for ERA5) that do not match Section 3.4, and Appendix A.2 says 'up to 100 pre-physics + 20 with physics loss.' As a result, the exact training regime behind Tables 1-3 and 4-5 is unclear, which is a serious reproducibility problem.","section":"Sections 3.2, 3.4, 5.7, and A.2"},{"comment":"The message loss is defined inconsistently. Section 3.3 gives the total loss as L = MSE(X, X') + MSE(M, M'), whereas Section 3.2 states that the message loss is binary cross-entropy between the decoded watermark and the true message, plus the negative BCE of the decoded inverse from the original image. Since decoder training is central to the claims, the actual loss function needs to be stated unambiguously.","section":"Sections 3.2 and 3.3"},{"comment":"The physics loss is used both as a training objective and as the evaluation metric for scientific fidelity. Tables 1 and 2 report physics loss (or 1 - ACC) as evidence that the UNet preserves scientific structure, but because the model was explicitly trained to minimize exactly that quantity, these numbers reflect optimization fit rather than independent validation. An independent metric, such as downstream forecast skill or a held-out physics diagnostic not used in training, is needed to support the fidelity claim.","section":"Tables 1-2 and Section 3.4"}],"minor_comments":[{"comment":"The UNet row in Table 1 appears to concatenate values ('0.910.02100%'); please reformat the table so that all five metric columns are legible.","section":"Table 1"},{"comment":"The bullet 'Scientific V alidity' contains a typo; it should read 'Validity'.","section":"Section 5.5"},{"comment":"Figure 3 is referenced twice: once in Section 5.4 for ERA5 and again in Appendix A.1 as part of the training diagnostics list. Please renumber figures to avoid duplicate numbering.","section":"Figures 3, 7, and 8"},{"comment":"No error bars, confidence intervals, or multiple-seed results are reported. Since training is stochastic and early stopping depends on a 98% decoder-accuracy threshold, single-run values are difficult to assess for statistical significance.","section":"Tables 1-3 and 4-5"},{"comment":"The abstract's 'sub-1% MSE' is not directly reported in the evaluation; the paper primarily reports PSNR and SSIM. Please state the fidelity metric explicitly and consistently.","section":"Abstract"}],"recommendation":"reject","confidential_remarks":"The paper appears to be an early preprint with substantial claims relative to the evidence. The fixed-message issue is a methodological flaw that a revised version could potentially address by retraining and evaluating with held-out or per-image random messages and by adding the missing robustness experiments, but as submitted the central quantitative claim is not supported. I also note that the reference list includes several self-citations to unpublished preprints ([3], [4], [5], [8], [16], [17], [18]); please verify that these are necessary and that no prior-publication overlap exists."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know: this is an application of existing neural watermarking to SuperBench scientific data with a physics-loss term. The central quantitative claim—100% decoding accuracy on all three datasets—is not supported because the decoder is trained and tested on the same fixed payload 'Hello World!'. A decoder that always outputs that message would score 100% without reading any image-specific information. That alone sinks the headline.\n\nTo the paper's credit, it ships code and uses public datasets (ERA5, Fluid Flow, Cosmology), and the ablation in Tables 4–5 shows that adding the physics/ACC loss improves fidelity metrics substantially over baselines. The motivation—provenance and traceability for scientific data—is real and worth pursuing. The idea of a physics-aware loss for watermark embedding is a reasonable extension, though not a deep one.\n\nBut the soft spots are serious. The abstract claims robustness to noise, cropping, and compression, while Section 6.1 explicitly says these were not evaluated. The training schedule is internally inconsistent: Section 3.2 describes a two-phase 100+20 epoch scheme for ERA5 and Fluid Flow; Section 3.4 says 50 epochs total with 30+20; the appendix says up to 100+20. The message loss is defined as MSE in Equation (1) but described as binary cross-entropy in Section 3.2. There are no error bars, no multi-seed runs, and the decoder's claimed 'inverse' behavior on non-watermarked images is never quantified. These are not minor cosmetic issues—they mean the fidelity numbers and decode rates cannot be trusted.\n\nThis is a workshop-level application note with a useful kernel but not a defensible research claim as written. A serious referee would need a held-out-message evaluation, proper robustness tests under the asserted distortions, consistent training details, and error bars before the core claim could be taken seriously. As is, the evidence does not support the paper's framing as a robust, verifiable watermarking system.\n\nI would not send this to peer review in its current state; desk reject with an invitation to resubmit after fixing the evaluation would be appropriate. For a reading group, it could serve as a cautionary example of how a fixed test message can invalidate a watermarking result.","headline":"This paper applies a known UNet watermarking recipe to scientific imagery with a physics-loss twist, but the fixed-message evaluation makes the headline 100% decoding accuracy meaningless.","tokens_in":10046,"tokens_out":2120,"would_cite":false,"duration_ms":18061,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A UNet encoder-decoder pair is the load-bearing object: it embeds a fixed binary message into a deep latent layer of scientific images and retrieves it with 100% reported accuracy, while a physics loss is meant to keep the reconstructed…","keywords":["watermarking","scientific data provenance","UNet autoencoder","physics loss","ERA5","Navier-Stokes","decoder accuracy","data integrity"],"falsifier":"Re-train or evaluate the same decoder on random, held-out binary messages that were never seen during training, and measure bit accuracy on watermarked images; if accuracy stays at the level of an always-decodes-'Hello World!' baseline, the central claim collapses. A second test is to apply noise, cropping, and JPEG-style compression to watermarked test images and check whether 100% decoding survives, which the paper identifies as future work.","tokens_in":9106,"feed_emoji":"🔏","tokens_out":4884,"duration_ms":33140,"temperature":0.7,"pith_summary":"The paper claims that a UNet-based autoencoder can embed a 100-bit binary watermark into high-dimensional scientific fields such as temperature, vorticity, and geopotential, and that the decoder recovers the full message from every test image. Across ERA5 climate reanalysis, Navier-Stokes fluid flow, and cosmology datasets, the method reports perfect decoding accuracy and substantially lower distortion than a classical SVD watermarker. If the claim holds, scientific datasets could carry verifiable provenance markers without visible or numerical degradation, enabling auditability and traceability in data pipelines. The paper positions this as a step toward securing AI systems that train or infer on scientific data.","feed_headline":"Watermark decoder hits 100% accuracy on climate and fluid fields","feed_subtitle":"A UNet hides a binary message in temperature and vorticity fields while keeping them scientifically usable.","key_machinery":"The load-bearing object is the co-trained UNet encoder-decoder pair. The message, converted from the string 'Hello World!' into a 1 by 100 binary vector, is injected into the last downsampling layer of the UNet rather than the input. Training combines image mean squared error, message binary cross-entropy (including the inverse-from-original term), and a physics loss that changes per dataset, with a staged schedule that introduces the message loss after the model has learned to reconstruct. The decoder's inverse-output property from unwatermarked images is what turns plain retrieval into an authentication mechanism.","core_discovery":"The central claim is that physics-aware neural watermarking can hide a fixed binary string in structured scientific data while keeping the data scientifically usable. The encoder is a UNet that reconstructs the input with a message injected into a deep latent layer; the decoder maps the watermarked image back to the message. A distinctive property is introduced: from an unwatermarked image the same decoder outputs the bitwise inverse of the embedded message, giving a built-in authentication check. Training minimizes reconstruction mean squared error, message cross-entropy, and a domain-specific physics loss (anomaly correlation for ERA5, divergence for fluid flow), and the paper reports 100% decoder accuracy with sub-1% MSE across all three test sets. The paper also argues that the method generalizes beyond weather and fluid data to other structured domains.","pith_inferences":["The reported 100% accuracy is not yet evidence of true watermark embedding until the decoder is tested on messages it did not train on; a random held-out message test would separate genuine embedding from fixed-message memorization.","The abstract's robustness claim is best read as a design goal rather than a demonstrated result, since the paper states it has not evaluated noise, cropping, or compression; applying those transformations to watermarked test images is a direct testable extension.","The inverse-output property could be used as a false-positive detector: if an image is unwatermarked, the decoder is expected to output the complement, which could verify the absence of watermarking.","A practical path toward scalability would be a single encoder-decoder trained across multiple scientific domains with shared physics-informed losses, rather than a separate model per dataset."],"forward_implications":["Scientific datasets such as ERA5 reanalysis and fluid-flow simulations could carry verifiable provenance markers without visible or numerical artifacts, since reported PSNR reaches 49.4 dB on ERA5 and SSIM exceeds 0.99 where the UNet is used.","Downstream models that ingest watermarked fields could, in principle, have their training data audited by decoding the embedded message from any copy, because the decoder retrieves the full message from every test image.","The inverse-decoding property means an unwatermarked original yields the complement of the message, giving a built-in authenticity check that distinguishes genuine watermarked data from unmarked data.","Adopting the method in a new domain requires training a fresh UNet and defining a domain-appropriate physics loss, so deployment cost scales with the number of dataset types."],"supporting_citations":[{"why":"Supplies the ERA5, Fluid Flow, and Cosmology datasets and the per-domain physics-loss definitions (divergence, 1-ACC) used in evaluation.","marker":"[11]"},{"why":"Provides the dwtDctSvd method that the SVD baseline adapts and compares against.","marker":"[12]"},{"why":"Establishes the prior encoder-decoder watermarking architecture that the UNet design follows.","marker":"[13]"},{"why":"Grounds the physics-informed loss concept that motivates the Physics Loss term.","marker":"[9]"},{"why":"Supports the use of physics-constrained loss terms in learned generative models.","marker":"[14]"},{"why":"Provides additional precedent for physics-aware latent sequence generation used to justify the physics-loss approach.","marker":"[15]"}],"fun_headline_variants":["Neural watermark hides messages in climate and fluid data","Physics-aware watermark achieves 100% decode on scientific fields","Invisible watermarking secures climate and fluid datasets","UNet watermark embeds verifiable IDs in physical data","Watermarking meets physics: 100% accuracy on ERA5 and flows"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that 100% decoding accuracy comes from the watermark actually being embedded in the image, rather than from the decoder having memorized the single fixed message 'Hello World!' during training.","fun_headline_variants_meta":{"raw":{"variants":["Neural watermark hides messages in climate and fluid data","Physics-aware watermark achieves 100% decode on scientific fields","Invisible watermarking secures climate and fluid datasets","UNet watermark embeds verifiable IDs in physical data","Watermarking meets physics: 100% accuracy on ERA5 and flows"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000209,"raw_usage":{"total_tokens":1374,"prompt_tokens":880,"completion_tokens":494,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":496,"completion_tokens_details":{"reasoning_tokens":425}},"tokens_in":496,"tokens_out":494,"duration_ms":3810,"temperature":1.0,"reasoning_tokens":425,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:48:54.519201+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-train or evaluate the same decoder on random, held-out binary messages that were never seen during training, and measure bit accuracy on watermarked images; if accuracy stays at the level of an always-decodes-'Hello World!' baseline, the central claim collapses. A second test is to apply noise, cropping, and JPEG-style compression to watermarked test images and check whether 100% decoding survives, which the paper identifies as future work.","supporting_citations":[{"cited_title":"SuperBench: A Super-Resolution Benchmark Dataset for Scientific Machine Learning","cited_arxiv_id":"2306.14070","evidence_quote":"Supplies the ERA5, Fluid Flow, and Cosmology datasets and the per-domain physics-loss definitions (divergence, 1-ACC) used in evaluation."},{"cited_title":"Invisible image watermarks are provably removable using generative ai, 2023","cited_arxiv_id":null,"evidence_quote":"Provides the dwtDctSvd method that the SVD baseline adapts and compares against."},{"cited_title":"Supervised gan watermarking for intellectual property protection","cited_arxiv_id":null,"evidence_quote":"Establishes the prior encoder-decoder watermarking architecture that the UNet design follows."},{"cited_title":"Towards Conditional Generation of Minimal Action Potential Pathways for Molecular Dynamics","cited_arxiv_id":"2111.14053","evidence_quote":"Supports the use of physics-constrained loss terms in learned generative models."},{"cited_title":"Turaga, Ross Maciejewski, and Abhishek Singharoy","cited_arxiv_id":null,"evidence_quote":"Provides additional precedent for physics-aware latent sequence generation used to justify the physics-loss approach."}],"review_version":1}