{"id":"956370e8-85ce-4ac1-b3bf-22a5f39fbfaf","arxiv_id":"2411.11562","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A new six-sensor synthetic raw-domain denoising benchmark and a consistency-plus-adversarial training method that yield modest but consistent transfer gains on held-out sensors.","lead":"This paper introduces MSSIDD, a benchmark of 60,000 raw image pairs spanning six simulated camera sensors, built by reversing the ISP pipeline on clean SIDD photos. It also proposes a sensor-consistency training framework that improves denoising transfer to unseen sensors by roughly 0.1 to 0.2 dB PSNR.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The inverse-ISP synthetic proxy is the load-bearing assumption, and Table 3's real-sensor validation cannot test it because no clean references for Sony/Canon are described, leaving the real-world transfer claim unsupported.","rationale":"The reader's weakest assumption and my load-bearing concern are the same root issue: the inverse-ISP pipeline is assumed to be a faithful proxy for real raw sensor data, and the paper's only attempt to validate that assumption on real cameras is not described with enough detail to rule out circularity. Tables 1-2 are entirely internal to the synthetic benchmark, so they cannot establish that MSSIDD predicts real sensor transfer. Table 3 is the designated bridge to reality, but the text omits how clean raw ground truth was obtained for Sony and Canon, how PSNR was computed, and whether the same inverse-transformed references were reused. Because the claim of real-world transferability depends on this experiment, it must be clarified or the claims should be softened. This does not change the reader's conditional verdict; it strengthens the conditions under which acceptance is appropriate: release the real-sensor data/protocol, provide multiple-seed statistics, and either justify the inverse-ISP realism or retract the strong real-sensor language.","tokens_in":21409,"tokens_out":13200,"duration_ms":136357,"concrete_test":"Re-run the real-sensor experiment with a documented protocol: acquire genuine clean raw references for Sony and Canon (e.g., multi-frame averaging of the same scene), train on the five synthetic sensors only, and report PSNR/SSIM deltas for MS-NAFNet vs NAFNet and MS-Restormer vs Restormer. If the Table 3 deltas shrink to within training noise (no error bars are reported anywhere), the real-sensor claim is unsupported. If the authors cannot produce such references, Table 3 as currently written is uninterpretable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The benchmark's validity hinges on the claim that inverse-ISP synthesis (Section 3.1) produces clean raw images whose cross-sensor differences (color, noise, black level) match real cameras. This is the load-bearing assumption: if it fails, Tables 1-2 rank models on simulated sensor shifts that need not predict real transfer. The intended real-sensor check (Section 5.4) cannot close the gap as written. The paper never states how PSNR/SSIM references were obtained for the real Sony and Canon raw images--no multi-frame-averaged clean raw, no alignment procedure, no ISP description. It only says a Sony sRGB image was inverse-transformed with sensor6 parameters to create one of the three test domains. If the same inverse-ISP outputs are reused as the reference for Sony/Canon, Table 3 is circular; if genuine clean references exist, they are not documented. Either way, the only evidence connecting the synthetic benchmark to real sensors is missing, so the central claim that MSSIDD measures and enables real sensor transferability is not yet supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces MSSIDD, a synthetic raw-domain benchmark for evaluating the sensor transferability of denoising models. Starting from clean sRGB ground-truth images from SIDD, the authors apply an inverse ISP pipeline (inverse tone mapping, inverse gamma, inverse color correction, inverse white balance, and mosaicing) to generate clean raw images for six simulated sensors, then add shot and read noise using calibration parameters from each sensor. The resulting 60,000 training and 6,000 validation paired noisy/clean raw images are used to benchmark several existing denoisers under leave-one-sensor-out transfer. The paper also proposes a multi-sensor consistency training framework that combines intra-image and inter-image relation-based feature consistency losses with adversarial sensor classification, and shows consistent improvements over base models on the synthetic benchmark and on a small real-sensor evaluation. The dataset and code are released publicly.","tokens_in":21608,"tokens_out":5369,"duration_ms":52426,"significance":"MSSIDD addresses a real gap: there is no widely used raw-domain benchmark specifically designed to measure cross-sensor generalization of denoising models. The dataset itself, if its synthetic generation is trustworthy, would be a useful resource, and the proposed consistency losses are simple, model-agnostic, and show consistent gains of roughly 0.1–0.2 dB across all held-out sensors in Tables 1–2. The authors also provide code, a datasheet, and a public release, which is commendable. However, the benchmark's central validity rests on the assumption that the inverse-ISP synthesis produces raw images whose sensor-to-sensor differences faithfully mimic real cameras. The real-sensor validation in Section 5.4 is not documented sufficiently, and the evaluation protocol in the Raw2Raw setting is not fully described in the main text. These are load-bearing issues that must be resolved before the dataset and the transferability claims can be accepted.","major_comments":[{"comment":"The real-sensor validation is not documented. The text says that Sony and Canon raw images were captured and that \"we subsequently use Sony sRGB image to perform inverse transformation according to the camera parameters of sensor6, resulting in three domains, namely sensor6, Sony, and Canon.\" It is never explained how the clean reference images for the Sony and Canon raw domains were obtained. If the references are also produced by the same inverse-ISP transformation, then the evaluation is circular and only tests within the synthetic pipeline. If genuine clean raw references exist (e.g., from multi-frame averaging), the acquisition and alignment procedure, the ISP parameters, and the way the references were validated must be described. As written, Table 3 cannot support the claim that the proposed supervision improves generalization to real sensors.","section":"Section 5.4, Table 3"},{"comment":"The benchmark's validity rests on the assumption that the inverse-ISP pipeline yields clean raw images whose sensor-to-sensor differences (color, noise, black level) match real cameras. The paper provides no validation of this assumption. It relies on the general unprocessing idea of [11] but introduces modifications (random digital gain drawn from a Gaussian, random convex combinations of AWB and CCM parameters) and uses sensor parameters from a public webpage without any comparison to real raw captures. The authors should provide evidence, for example, by comparing the statistics of the synthesized raw images against real raw data from the same sensors, or by demonstrating that models trained on MSSIDD transfer to real raw data in a protocol with clearly documented ground-truth references.","section":"Section 3.1–3.2, Eq. (1)"},{"comment":"The Raw2Raw evaluation protocol is not described accurately in the main text. Section 5.1 says PSNR and SSIM are reported on raw and sRGB outputs, but the supplement states that \"under the RAW2RAW setting, we will conduct the digital gain and gamma correction on the predicted clean raw images output by the network\" with a gain of 2.0 and gamma of 1/2.2 before computing metrics. This post-processing is part of the protocol and must be stated in the main text, otherwise the benchmark results are not reproducible from the paper alone. The authors should also analyze sensitivity to the chosen gain and gamma values, since the ranking of methods could depend on these choices.","section":"Section 5.1 and Supplemental Section 7.2"},{"comment":"The proposed intra- and inter-image consistency losses assume that the same scene is available across sensors with pixel-perfect alignment. This assumption holds trivially in the MSSIDD benchmark because all six sensor images are generated from the same sRGB image via the inverse pipeline. For real multi-sensor data, capturing the same scene with different cameras introduces parallax, registration errors, and different lens/ISP characteristics, which would break the assumption. The paper should clarify how the proposed supervision is intended to be applied to real sensor data, or state explicitly that the method is designed for synthetic multi-sensor training data where such alignment is available.","section":"Section 4.1, Eqs. (5)–(7)"}],"minor_comments":[{"comment":"The abstract states the dataset consists of \"60,000 raw images of six distinct sensors,\" but Section 3.3 and Table 4 describe 60,000 training pairs plus 6,000 validation pairs, totaling 66,000. Please correct the abstract or clarify that the number refers only to training pairs.","section":"Abstract and Section 3.3"},{"comment":"The datasheet says \"All raw images are 4 channels with RGGB bayer pattern,\" while Section 3.2 says the raw images follow the BGGR Bayer pattern. One of these is inconsistent and should be fixed.","section":"Datasheet, Section 8 vs. Section 3.2"},{"comment":"The normalization constants in Eqs. (5)–(7) are written with ambiguous notation (e.g., \"N (N − 1)/2|D|\"). Use explicit fractions such as \\frac{N(N-1)}{2|D|L} to avoid confusion about the placement of the dataset-size and scale factors.","section":"Section 4.1, Eqs. (5)–(7)"}],"recommendation":"major_revision","confidential_remarks":"The paper's strongest contribution is the dataset, but its validity hinges on the inverse-ISP proxy. The real-sensor validation in Section 5.4 is not adequately documented and, as written, risks being circular. The method itself is a fairly straightforward application of relation-based consistency and adversarial domain confusion to raw denoising; the novelty is moderate. If the authors can provide a credible real-sensor validation (with proper ground-truth references and protocol details) and fix the metric documentation, the paper would be suitable for publication. Otherwise, the central claim about enabling real sensor transferability remains unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a genuinely new public resource, the first raw-domain benchmark explicitly partitioned for cross-sensor transferability, and the proposed consistency supervision (relation-based intra- and inter-image constraints plus gradient-reversal adversarial training) is a plausible plug-in that delivers small but consistent gains, roughly 0.1 to 0.2 dB, on all six held-out synthetic sensors in both Raw2Raw and Raw2RGB settings. The dataset ships with code, a datasheet, and per-image metadata, and the ablation in Fig. 7 shows each loss component contributes. That is honest, reproducible work.\n\nThe soft spot is proportional to the claim. The benchmark is fully synthetic: clean raw images are produced by inverse ISP from SIDD sRGB ground truth, and the six sensors differ only through simulated white balance, color correction, and noise parameters. That follows the established Unprocessing recipe, so it is a reasonable first-order proxy. But the real-sensor validation in Section 5.4 is under-specified. The paper never describes how the PSNR/SSIM references for the real Sony and Canon raw images were obtained, with no multi-frame averaging, alignment, or rendering pipeline mentioned. The sentence saying a Sony sRGB image was inverse-transformed with sensor6 parameters to produce three domains only adds confusion. If the same inverse-ISP outputs are reused as the references for Sony and Canon, Table 3 is close to circular; if genuine clean references exist, they are not documented. The stress-test note is right about this. Either way, the only evidence bridging the synthetic benchmark to real sensors is missing, so the real-world transferability claim is not yet supported. What is supported is transfer among the six synthetic sensors.\n\nTwo smaller issues. First, the Raw2Raw PSNR/SSIM numbers in Tables 1 through 3 are computed after applying a fixed digital gain and gamma to the predicted raw image, disclosed only in the supplementary. The main text should say so, or readers will read the table as raw-domain PSNR. Second, the paper reports single runs, and 0.1 dB margins need error bars; the consistency across two architectures and all six sensors mitigates this, but multiple seeds are needed.\n\nWho is this for: raw-denoising researchers who want a ready-made transferability benchmark, and teams that must ship one denoiser across many camera modes. The paper deserves a serious referee. The resource is useful, the method is a reasonable candidate baseline, and the weaknesses are fixable with more analysis, namely multiple seeds, a documented real-sensor protocol, and either a realism check on the inverse ISP or softened claims.","headline":"A genuinely new public benchmark for raw-domain sensor transferability with a plausible consistency-training plug-in, but the synthetic proxy and the under-documented real-sensor table mean the real-world claims outrun the evidence.","tokens_in":22148,"tokens_out":6241,"would_cite":true,"duration_ms":56787,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Denoising models can be trained to transfer across camera sensors using multi-sensor consistency supervision and a new six-sensor raw benchmark, MSSIDD.","keywords":["multi-sensor denoising","raw-domain benchmark","sensor transferability","inverse ISP synthesis","sensor consistency supervision","adversarial training","domain generalization","image denoising"],"falsifier":"Collect raw noise/clean pairs from a new real camera with its own calibration, train a model on MSSIDD's five synthetic sensors, and evaluate on that real camera without any inverse-transformed data; if the synthetic-trained model does not outperform a model trained only on a single unrelated sensor (or if the performance gap reverses), the inverse-ISP synthesis assumption fails.","tokens_in":21180,"feed_emoji":"📷","tokens_out":4807,"duration_ms":43183,"temperature":0.7,"pith_summary":"This paper introduces MSSIDD, the first raw-domain benchmark for evaluating whether image-denoising models transfer across camera sensors. It generates 60,000 noisy/clean raw image pairs for six distinct sensors by applying the inverse of a camera ISP pipeline (inverse tone mapping, gamma, color correction, white balance, digital gain, and mosaicing) to clean sRGB images from SIDD. The paper then proposes training with multi-sensor consistency supervision: intra-image constraints that align feature differences across sensors of the same scene, inter-image constraints that do the same across image pairs, and an adversarial sensor classifier that pushes features to be sensor-invariant. Applied to NAFNet and Restormer, this supervision consistently improves performance on all six held-out sensor tasks and on two real cameras, suggesting a general recipe for sensor-invariant raw denoising.","feed_headline":"Denoisers can transfer to unseen sensors with consistency training","feed_subtitle":"A new six-sensor raw benchmark shows consistency supervision beats baselines on held-out cameras.","key_machinery":"The central object is the multi-scale, multi-sensor consistency loss $L_{MS} = \\lambda_1 L_{inter} + \\lambda_2 L_{intra} + \\lambda_3 L_{adv}$. $L_{intra}$ constrains, for each image and each scale of a designated sub-network, the difference between feature maps of different sensors to match the difference between their noisy inputs; $L_{inter}$ applies the same relation-based constraint across different images in a batch; and $L_{adv}$ trains a sensor classifier on the sub-network features with a gradient reversal layer so the features become sensor-invariant. The loss is a plug-in that can be added to any encoder-decoder denoising architecture, and the paper's data-generation pipeline (inverse ISP starting from SIDD sRGB) provides the six-sensor paired raw data needed to compute it.","core_discovery":"The core claim is that a denoising network can learn to ignore which sensor produced an image, so that a model trained on a handful of known sensors works on a new, unseen sensor without fine-tuning. The authors argue this with the MSSIDD dataset, where the only differences between the six sensor domains are the calibrated ISP parameters (white balance, color correction, digital gain, black level) and the shot/read noise model. They then show that enforcing consistency between features extracted from different sensors of the same image, and between different images, while adversarially confusing a sensor classifier, yields quantitative gains on every held-out sensor under both raw-domain and sRGB-domain supervision. The same supervision transfers to Sony and Canon real-camera data, which supports the claim that the learned features are sensor-invariant rather than just tuned to synthetic noise levels.","pith_inferences":["Because the real-sensor validation in Section 5.4 generates the sensor6 domain by the same inverse transformation used to build the benchmark, that validation may not be fully independent of the synthetic-data assumption; a cleaner test would evaluate only on raw data captured natively by an entirely unseen camera.","The relation-based consistency losses resemble contrastive and relative-learning constraints, suggesting the recipe could extend to other sensor-dependent restoration tasks such as demosaicing, color correction, or super-resolution, and perhaps to domain generalization beyond cameras.","The dataset's noise statistics and ISP parameters are sampled rather than calibrated per image, so transferability scores may depend on the chosen ranges (for instance ISO 2400-12800 and the inverse-gain Gaussian); ablating these ranges would reveal how sensitive the benchmark is to the synthesis choices."],"forward_implications":["On the MSSIDD benchmark, MS-NAFNet and MS-Restormer outperform all baselines on all six transfer-to-sensor tasks in both Raw2Raw and Raw2RGB settings, with gains over the base models reaching about 0.2 dB.","The consistency supervision can be added to existing denoising architectures with negligible computational overhead, making it a lightweight way to improve sensor portability in mobile terminals.","A model trained on five of the six MSSIDD sensors transfers to the sixth without seeing its training data, establishing a reproducible evaluation protocol for sensor transferability research.","The real-sensor experiments with Sony and Canon cameras show that the same supervision improves performance on genuinely captured raw data, not only on the synthetic dataset."],"supporting_citations":[{"why":"Supplies the inverse-ISP pipeline and noise model on which MSSIDD data generation is based.","marker":"[11]"},{"why":"Provides the clean sRGB ground-truth images from SIDD that are inversely transformed into all MSSIDD raw pairs.","marker":"[3]"},{"why":"NAFNet is the base architecture into which the consistency supervision is plugged to produce MS-NAFNet.","marker":"[16]"},{"why":"Restormer is the second base architecture that receives the same supervision to produce MS-Restormer.","marker":"[58]"},{"why":"Contributes the gradient reversal layer and adversarial classification procedure used for sensor confusion.","marker":"[21]"},{"why":"Source of the six sensor calibration parameters (white balance, CCM, noise coefficients) that define the sensor domains.","marker":"[1]"}],"fun_headline_variants":["Consistency training lets denoisers handle unseen sensors","MSSIDD: New benchmark for multi-sensor denoising","Train on six sensors, denoise any camera","Sensor-invariant denoising via consistency training","Across-sensor denoising: one model for all cameras"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that taking a clean sRGB image and running it backward through an inverse ISP pipeline faithfully produces raw images that behave like real camera raw data for the purpose of training and testing denoising transfer.","fun_headline_variants_meta":{"raw":{"variants":["Consistency training lets denoisers handle unseen sensors","MSSIDD: New benchmark for multi-sensor denoising","Train on six sensors, denoise any camera","Sensor-invariant denoising via consistency training","Across-sensor denoising: one model for all cameras"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000158,"raw_usage":{"total_tokens":1208,"prompt_tokens":909,"completion_tokens":299,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":219}},"tokens_in":525,"tokens_out":299,"duration_ms":3377,"temperature":1.0,"reasoning_tokens":219,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:22:14.089796+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect raw noise/clean pairs from a new real camera with its own calibration, train a model on MSSIDD's five synthetic sensors, and evaluate on that real camera without any inverse-transformed data; if the synthetic-trained model does not outperform a model trained only on a single unrelated sensor (or if the performance gap reverses), the inverse-ISP synthesis assumption fails.","supporting_citations":[{"cited_title":"Unpro- cessing images for learned raw denoising","cited_arxiv_id":null,"evidence_quote":"Supplies the inverse-ISP pipeline and noise model on which MSSIDD data generation is based."},{"cited_title":"A high-quality denoising dataset for smartphone cameras","cited_arxiv_id":null,"evidence_quote":"Provides the clean sRGB ground-truth images from SIDD that are inversely transformed into all MSSIDD raw pairs."},{"cited_title":"Simple baselines for image restoration","cited_arxiv_id":null,"evidence_quote":"NAFNet is the base architecture into which the consistency supervision is plugged to produce MS-NAFNet."},{"cited_title":"Restormer: Efficient transformer for high- resolution image restoration","cited_arxiv_id":null,"evidence_quote":"Restormer is the second base architecture that receives the same supervision to produce MS-Restormer."},{"cited_title":"Unsupervised domain adaptation by backpropagation","cited_arxiv_id":null,"evidence_quote":"Contributes the gradient reversal layer and adversarial classification procedure used for sensor confusion."},{"cited_title":"https:// www.sony-semicon.com/en/products/is/ camera/index.html","cited_arxiv_id":null,"evidence_quote":"Source of the six sensor calibration parameters (white balance, CCM, noise coefficients) that define the sensor domains."}],"review_version":1}