{"id":"2bd60260-890c-4177-b34f-d616ee441d8e","arxiv_id":"2506.06890","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-stage pipeline of Pix2PixHD image translation followed by NeRF or 3DGS produces color novel views from simulated binary single-photon camera images.","lead":"The authors propose translating binary single-photon camera images into color images with Pix2PixHD, then feeding those into NeRF or 3D Gaussian Splatting to synthesize new views. The value is a possible route to 3D reconstruction from extremely low-light binary sensors, but it is validated only on simulated data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The simulated-SPC operating point is not fixed by the paper: Eq. (1) depends on the product q·phi·T, but pixel values are used directly as flux with no exposure time or absolute units, so the reported ranking may be an artifact of one arbitrary simulation setting.","rationale":"The reader identified simulator fidelity to real hardware as the weakest assumption, and I agree that is a genuine limitation, but the sharper load-bearing issue is internal to the simulator: the free parameter T and the uncalibrated flux mapping mean the simulated task itself is not well-defined. Even before considering real-sensor transfer, the reported ranking could depend on an arbitrary choice of operating point. This reinforces the reader's CONDITIONAL verdict rather than overturning it: the paper needs either a calibration study within the simulator or real-sensor validation to make the central claim robust. I did not find a reason to move to REJECT, because the two-stage pipeline is a plausible application of existing components and the reported numbers, while under-supported, are not internally contradictory in a way that invalidates the method outright. The absence of code and error bars further weakens reproducibility, but the most load-bearing check is to vary the simulator's operating point and observe whether the ranking survives. Therefore the appropriate verdict remains CONDITIONAL, and my read does not change the reader's verdict.","tokens_in":6493,"tokens_out":4174,"duration_ms":52278,"concrete_test":"Rerun all eight Table 1 pipelines at two physically distinct operating points by choosing exposure time T so that the mean detected photons per pixel per frame from Eq. (1) is 0.1 and 10, while keeping q_SPAD = 0.45 and tau_d = 150 ns. If the rank ordering of approaches changes between these settings, the claim that Pix2PixHD+NeRF/3DGS is best is simulator-dependent rather than robust. Additionally, report per-scene means and standard deviations over at least three training seeds for the top-row comparison; if Pix2PixHD+NeRF versus Restormer+NeRF falls within one standard deviation, the headline comparison is not statistically meaningful.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Section 5) is that Pix2PixHD + NeRF/3DGS achieves notable improvements over other combinations, supported entirely by Table 1. However, the SPC simulator in Section 3.1 never fixes the physical operating point. Equation (1), E[N_T^SPC] = q_SPAD·phi·T / (1 + q_SPAD·phi·tau_d), shows that the binary occupancy is governed by the product q·phi·T, yet the text says 'photon flux phi represents the pixel intensity values' without specifying absolute units or exposure time T. With T free, the same RGB image can produce almost all-zero or almost all-one binary frames, and the difficulty of the I2I translation task changes accordingly. A method that wins at one occupancy level may lose at another, so the Table 1 ranking is not shown to be robust even within the simulator. The conclusion explicitly defers real-sensor evaluation to future work, and no sweep over T or flux is reported. This is compounded by an internal inconsistency: Section 4 states CycleGAN failed to translate SPC images, yet Table 1 lists CycleGAN+NeRF with PSNR 16.49. No error bars, per-scene breakdowns, or seeds are reported, so the margin between Pix2PixHD+NeRF (22.70) and Restormer+NeRF (20.42) could be within run-to-run variation. The concern is about correctness risk: if the uncalibrated operating point is arbitrary, the headline ranking is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a modular two-stage pipeline for novel view synthesis from binary Single Photon Camera (SPC) images. Stage 1 translates simulated binary SPC images into RGB-like images using image-to-image translation models (Pix2PixHD, Pix2Pix, CycleGAN, LookatMe, Restormer, DiffBIR), and Stage 2 feeds the translated RGB images into NeRF or 3D Gaussian Splatting for novel view rendering. A physics-inspired SPC simulator based on Poisson photon arrivals and SPAD dead time is used to generate paired training and evaluation data. Quantitative results on simulated IBRNet scenes (Table 1) and qualitative comparisons (Fig. 2) lead the authors to claim that Pix2PixHD + NeRF/3DGS outperforms other combinations in perceptual quality and geometric consistency, with single-stage NeRF/3DGS baselines performing worse (Table 2).","tokens_in":6993,"tokens_out":2493,"duration_ms":29810,"significance":"If established, the result is practically relevant: it offers a modular recipe for turning a severely quantized, binary sensing modality into a form usable by standard radiance-field pipelines, and it systematically compares several I2I backbones for this task. The strengths of the paper are its clearly stated modular framework, the inclusion of a statistically motivated SPC simulator, and the breadth of the combination table (eight two-stage pipelines plus single-stage ablations). However, the central quantitative claim rests entirely on a synthetic benchmark generated by the authors' own simulator, with no fixed, reported physical operating point, no error bars or per-scene breakdowns, and no validation against real SPC hardware. The claimed ranking is therefore plausible but not yet demonstrated to be robust, which is what limits the current significance.","major_comments":[{"comment":"The SPC simulator's physical operating point is not fixed. Equation (1) shows that the mean detected count E[N_T^SPC] = q_SPAD * phi * T / (1 + q_SPAD * phi * tau_d) depends on the product q*phi*T, yet the text states only that 'photon flux phi represents the pixel intensity values' and does not report absolute flux units or exposure time T. The paper states that q_SPAD = 0.45 and tau_d = 150 ns are fixed, but T is left unspecified and the simulation procedure says T and q are 'adjusted' to ensure visibility. With T free, the same RGB image can be rendered as almost-all-zero or almost-all-one binary frames, which changes the difficulty of the I2I translation task dramatically. The headline ranking in Table 1 is therefore not shown to be robust even within the simulator, because no sweep over T or flux is reported. This is a load-bearing issue: the central claim 'Pix2PixHD + NeRF/3DGS outperforms other combinations' is only established for one uncalibrated operating point.","section":"§3.1, Eq. (1)"},{"comment":"There is an internal inconsistency between the text and Table 1. Section 4 states that CycleGAN and LookatMe 'failed to successfully translate SPC binary images (3-channel)', yet Table 1 lists CycleGAN + NeRF with PSNR 16.49. Similarly, Restormer and DiffBIR are described as 'inefficient', yet they appear as rows in Table 1. The manuscript never explains how these rows were produced or whether the listed numbers represent a failure mode or a valid baseline. In addition, Table 1 reports no error bars, no seeds, no per-scene breakdown, and no statistical significance testing; the margins between Pix2PixHD+NeRF (PSNR 22.70) and Restormer+NeRF (PSNR 20.42), and between Pix2PixHD+3DGS (22.21) and DiffBIR+NeRF (19.35), could be within run-to-run variation. The paper should either remove the failed baselines from Table 1 or explain what the numbers mean, and should provide variance information.","section":"§4, Table 1"},{"comment":"The evaluation is entirely synthetic: SPC images for both training and testing are generated by the authors' own simulator, and the conclusion explicitly defers real-sensor evaluation to future work. The central claim of practical relevance for binary SPC imagery is therefore not directly supported by the presented experiments. The simulator is grounded in a Poisson/dead-time model, but no comparison to real SPC measurements, no parameter-calibration procedure, and no sensitivity analysis are provided. A concrete test would be to evaluate the trained two-stage pipeline on real SPC captures (even a single scene) or, at minimum, to validate that the simulator's binary statistics match a real sensor at the chosen operating point. Without this, the reported ranking may be an artifact of the simulator's assumptions rather than a property of real SPC data.","section":"§5, Conclusion"}],"minor_comments":[{"comment":"The title contains a typo: 'TRANSLA TION' should be 'TRANSLATION'.","section":"Title"},{"comment":"Equation (2) is not derived or cited precisely; the reference list includes Grimmett and Stirzaker for renewal processes and cites 'Ingle and Proakis' in the text, but the cited reference [21] is by Ingle, Velten, and Gupta, not Proakis. Please correct the attribution.","section":"§3.1, Eq. (1)-(2)"},{"comment":"The text uses inconsistent notation: the exposure time is introduced as 'T', then written as 'NT SP C', and q_SPAD appears in both the equations and text. Please unify the notation and define all variables at first use.","section":"§3.1"},{"comment":"The phrase 'approximately 8 feasible working combinations' is unclear: Table 1 contains exactly eight rows, but the text says CycleGAN failed and denoising was inefficient. Please clarify which combinations are considered 'working' and which are reported only as negative baselines.","section":"§4"},{"comment":"The qualitative discussion relies heavily on visual inspection ('please zoom in'), but the figure is not available in the manuscript text. Please ensure the figure is legible at publication resolution and that the scene names and view indices are labeled.","section":"§4, Fig. 2"},{"comment":"No training details are given for the I2I models (learning rate, batch size, number of epochs, resolution, train/test split). Without these, the comparison in Table 1 is not reproducible. At minimum, please add a supplementary table of hyperparameters and dataset splits.","section":"§4.1"},{"comment":"The paper would benefit from a code/data release statement and from reporting the number of scenes used for evaluation. The claim that Pix2PixHD's feature-matching loss is 'pivotal' is plausible but is not directly ablated; an ablation removing the feature-matching loss would strengthen the claim.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a short conference-style paper whose central quantitative claim depends on a synthetic simulator with an unspecified exposure time and no error bars. The idea is reasonable and the modular comparison is useful, but the evaluation is too fragile to support the headline conclusion as written. I would advise the editor that the revision needs either (a) a sensitivity analysis over the simulator operating point with error bars, or (b) validation on at least one real SPC capture, before the paper can be accepted. The internal inconsistency about CycleGAN's status in Table 1 also needs to be resolved. These are fixable within the scope of a revision, so I do not recommend rejection at this stage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Plainly: this is an application paper that benchmarks a two-stage I2I + NeRF/3DGS pipeline on simulated binary SPC images. The components are all prior art, and the only new element is the combination for this input modality—an incremental extension of the same group's earlier 2D colorization work [17]. That said, the paper does a few things honestly: the SPC simulator is physically grounded (Poisson arrivals, dead-time renewal), the evaluation covers eight pipeline combinations on a shared benchmark, and the qualitative figures do show Pix2PixHD's feature-matching loss producing more plausible color than denoising-first baselines.\n\nThe soft spots are not minor. The entire quantitative case rests on Table 1, which has no error bars, no per-scene breakdown, and no code or data release. The internal inconsistency is real: Section 4 states CycleGAN and LookatMe failed to translate, yet Table 1 lists CycleGAN+NeRF at 16.49 PSNR. Either the 'failed' claim or the table entry has to be explained. More importantly, the stress-test concern about the operating point holds up. Eq. (1) depends on the product q·phi·T, but the paper never reports absolute flux units or exposure time T; it says T is 'adjusted' to ensure visibility. Without a sweep over occupancy levels—from sparse to saturated—there is no reason to believe the ranking survives changes in the simulator's operating point. The 2.3 dB margin between Pix2PixHD+NeRF and Restormer+NeRF could easily flip under different photon-starved conditions, which is the regime SPC is actually used in.\n\nThe paper is transparent that real-sensor evaluation is future work. That is acceptable for a workshop-style benchmark, but it means the central claim is currently unverified outside the authors' own simulator.\n\nWho should read this: people in the SPC/computational imaging community who want a quick baseline for 3D synthesis from binary sensors. It is not a major advance, but it is a legitimate empirical question worth answering.\n\nMy recommendation: send it to peer review, but only with the expectation of heavy revision. Require code and data, an operating-point sweep over T and flux, and a direct confrontation with the CycleGAN inconsistency. If those are provided, the benchmark could be a useful reference point.","headline":"A plausible but under-supported benchmark: two-stage I2I+NeRF/3DGS is new for binary SPC, but the uncalibrated simulator and missing validation leave the ranking unproven.","tokens_in":7355,"tokens_out":3462,"would_cite":false,"duration_ms":35269,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two-stage translation turns binary single-photon images into color 3D views.","keywords":["single-photon cameras","SPAD","binary image translation","Pix2PixHD","Neural Radiance Fields","3D Gaussian Splatting","novel view synthesis","image-to-image translation"],"falsifier":"Train and evaluate the same pipelines on real SPC recordings with ground-truth RGB captures from the same scene geometry; if Pix2PixHD + NeRF/3DGS no longer beats the denoising baselines or direct reconstruction on PSNR/SSIM/LPIPS, the paper's central claim is refuted. A less expensive check is to compare the photon-count statistics of the simulator against real sensor histograms under controlled flux.","tokens_in":6322,"feed_emoji":"📸","tokens_out":5336,"duration_ms":48079,"temperature":0.7,"pith_summary":"Single-photon cameras capture light with extreme sensitivity but output only binary images, stripping away the texture and color that NeRF and 3D Gaussian Splatting need. This paper argues that a two-stage pipeline fixes the problem: first translate the binary SPC frames into plausible RGB images with an image-to-image model, then run NeRF or 3D Gaussian Splatting on those RGB images to synthesize novel views. Across an eight-way comparison on simulated SPC data, the paper reports that Pix2PixHD followed by NeRF or 3DGS gives the highest PSNR, SSIM, and LPIPS scores, with Pix2PixHD + NeRF reaching 22.70 PSNR and 0.6843 SSIM. The paper also contributes a physics-inspired SPC simulator, based on Poisson photon arrivals and SPAD dead time, to produce paired binary/RGB training data.","feed_headline":"Two-stage pipeline turns one-bit camera images into 3D views","feed_subtitle":"Pix2PixHD translation plus NeRF or 3DGS recovery beats direct 3D on simulated single-photon scenes.","key_machinery":"The load-bearing component is the two-stage modular pipeline. Stage one is image-to-image translation: Pix2PixHD, whose multi-scale generator and discriminator with feature-matching loss convert binary SPC inputs into RGB images. Stage two is radiance-field reconstruction: NeRF (a fully connected network mapping position and viewing direction to color and density) or 3D Gaussian Splatting (a rasterized collection of 3D Gaussians) renders novel views from the translated RGB frames. Supporting this is the SPC simulator: it treats photon arrivals as a Poisson renewal process with a dead-time model, using equations for the mean and variance of detected photon counts (with quantum efficiency 0.45 and dead time 150 ns), then thresholds counts to binary detections per color channel.","core_discovery":"The central claim is that binary single-photon-camera images, which look like black-and-white noise, can be turned into high-quality color novel views by decoupling the problem into two stages rather than forcing 3D reconstruction to operate directly on the binary data. The first stage uses Pix2PixHD, a conditional GAN with a feature-matching adversarial loss, to hallucinate color and texture from the binary input; the second stage feeds those RGB images to NeRF or 3D Gaussian Splatting. The paper reports that this combination outperforms direct single-stage NeRF/3DGS, denoise-then-reconstruct baselines (Restormer and DiffBIR), and other translation models (Pix2Pix, CycleGAN, LookatMe), on simulated scenes from the IBRNet-Collected-2 dataset.","pith_inferences":["If the simulator's Poisson/dead-time model closely matches real SPC sensors, the same two-stage pipeline should transfer to hardware; the paper lists real-sensor validation as future work, so this remains a testable prediction rather than a demonstrated result.","The success of a GAN-based translator suggests that other powerful conditional generation models could further close the gap with ground-truth RGB, potentially improving the downstream 3D stage without changing the reconstruction module.","The modularity invites end-to-end joint training of translation and reconstruction, which the paper names as future work; a single loss could encourage translation outputs that are optimized for geometric consistency, not just visual similarity.","Because the simulator thresholds photon counts at zero, extremely low-flux regions become uniformly black bits; a real sensor would exhibit thermal noise and non-uniform quantum efficiency, which could change which translation model wins."],"forward_implications":["Pix2PixHD + NeRF reaches PSNR 22.70 and SSIM 0.6843 on simulated SPC novel-view synthesis, the best of the eight compared combinations.","Pix2PixHD + 3D Gaussian Splatting attains the best LPIPS (0.4251), meaning perceptual quality is strongest when the second stage is 3DGS.","Direct single-stage reconstruction (NeRF or 3DGS on raw binary SPC images) produces washed-out or noisy renders and lower scores, showing that translation before reconstruction is necessary.","Denoising binary SPC images with Restormer or DiffBIR before the 3D stage does not match the translation-first pipeline, producing lighter colors and added noise.","The two-stage design is modular, so each stage can be swapped and optimized independently."],"supporting_citations":[{"why":"Supplies Pix2PixHD, the first-stage translator whose feature-matching adversarial loss recovers color from binary SPC inputs.","marker":"[12]"},{"why":"Supplies NeRF, the second-stage reconstruction model for novel-view synthesis from translated RGB images.","marker":"[13]"},{"why":"Supplies 3D Gaussian Splatting, the alternative second-stage model with real-time rasterization rendering.","marker":"[16]"},{"why":"Provides prior SPC-to-color conversion insights that inform the simulator's design.","marker":"[17]"},{"why":"Provides the mean/variance photon-count equations and dead-time modeling used by the simulator.","marker":"[19]"},{"why":"Restormer, the denoising baseline compared against translation-first pipelines.","marker":"[24]"},{"why":"DiffBIR, the diffusion-based denoising baseline compared against translation-first pipelines.","marker":"[25]"},{"why":"NeRF-LLFF dataset, source of training images for the I2I models after augmentation.","marker":"[26]"},{"why":"IBRNet-Collected-2 scenes, the evaluation dataset on which quantitative metrics are measured.","marker":"[27]"}],"fun_headline_variants":["One-bit camera noise to colorful 3D views via two stages","Binary SPC images become 3D scenes using Pix2PixHD and NeRF","Two-step SPC to 3D: translate first, reconstruct later","From single-photon noise to novel views in two stages","SPC images get color and depth via I2I and 3D synthesis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire evaluation rests on the simulated SPC images faithfully matching what a real single-photon camera would output; if the simulator's noise, dead-time, and quantum-efficiency assumptions are off, the reported ranking of pipelines may not carry over to actual hardware.","fun_headline_variants_meta":{"raw":{"variants":["One-bit camera noise to colorful 3D views via two stages","Binary SPC images become 3D scenes using Pix2PixHD and NeRF","Two-step SPC to 3D: translate first, reconstruct later","From single-photon noise to novel views in two stages","SPC images get color and depth via I2I and 3D synthesis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000235,"raw_usage":{"total_tokens":1496,"prompt_tokens":940,"completion_tokens":556,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":458}},"tokens_in":556,"tokens_out":556,"duration_ms":5739,"temperature":1.0,"reasoning_tokens":458,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:46:19.072270+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train and evaluate the same pipelines on real SPC recordings with ground-truth RGB captures from the same scene geometry; if Pix2PixHD + NeRF/3DGS no longer beats the denoising baselines or direct reconstruction on PSNR/SSIM/LPIPS, the paper's central claim is refuted. A less expensive check is to compare the photon-count statistics of the simulator against real sensor histograms under controlled flux.","supporting_citations":[{"cited_title":"High-resolution image synthesis and semantic manipulation with con- ditional gans,","cited_arxiv_id":null,"evidence_quote":"Supplies Pix2PixHD, the first-stage translator whose feature-matching adversarial loss recovers color from binary SPC inputs."},{"cited_title":"Scene completion using millions of photographs,","cited_arxiv_id":null,"evidence_quote":"Supplies NeRF, the second-stage reconstruction model for novel-view synthesis from translated RGB images."},{"cited_title":"Holistically-nested edge detection,","cited_arxiv_id":null,"evidence_quote":"Supplies 3D Gaussian Splatting, the alternative second-stage model with real-time rasterization rendering."},{"cited_title":"Photographic image synthesis with cascaded refinement networks,","cited_arxiv_id":null,"evidence_quote":"Provides prior SPC-to-color conversion insights that inform the simulator's design."},{"cited_title":"Nerf: Representing scenes as neural radiance fields for view synthesis,","cited_arxiv_id":null,"evidence_quote":"Provides the mean/variance photon-count equations and dead-time modeling used by the simulator."},{"cited_title":"However, this method resulted in ren- dered images that were lighter in color, lacked texture details, and exhibited increased noise","cited_arxiv_id":null,"evidence_quote":"Restormer, the denoising baseline compared against translation-first pipelines."},{"cited_title":"Single-photon camera guided extreme dynamic range imaging,","cited_arxiv_id":null,"evidence_quote":"DiffBIR, the diffusion-based denoising baseline compared against translation-first pipelines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"NeRF-LLFF dataset, source of training images for the I2I models after augmentation."},{"cited_title":"High flux passive imaging with single-photon sensors,","cited_arxiv_id":null,"evidence_quote":"IBRNet-Collected-2 scenes, the evaluation dataset on which quantitative metrics are measured."}],"review_version":1}