{"id":"84b06353-42be-4b5f-8e2f-1c12cf15af9a","arxiv_id":"2501.12150","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"DNRSelect trains a reinforcement-learning view selector on cheap rasterized images and a depth/normal/UV texture aggregator, so deferred neural rendering needs ray-traced images only for the selected views.","lead":"This paper proposes DNRSelect, a system that uses reinforcement learning to choose a small set of camera views for deferred neural rendering, then ray-traces only those views. The goal is to cut the cost of generating ray-traced training images while keeping rendering quality high.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"View selector is trained on rasterized images (Eq. 5) but evaluated after fine-tuning on ray-traced images; no experiment tests whether rasterized loss ranks views the same as ray-traced loss.","rationale":"The reader's weakest assumption is the rasterized-to-ray-traced proxy in the selector, and I agree this is the most load-bearing concern. The paper's main contribution is reducing expensive ray-traced data by training the selector on cheap rasterized images; if this transfer fails, the method's core advantage disappears even though the 3D texture aggregator and multi-loss fine-tuning could still help. The ablations in Table I and Fig. 8 show that individual components improve performance relative to internal baselines, but they never vary the image modality used to train the selector, so the proxy is not empirically grounded. I also note that the 'theoretical upper limit' language in Sec. IV-B is undefined, which makes the headline overclaim impossible to verify; this is secondary to the proxy issue but should be corrected. The reader's conditional verdict is appropriate: the idea is plausible and the experiments are directionally consistent, but the proxy assumption, missing architecture-matched DNR baseline, and absence of code/error bars leave the strong claims unproven. My stress-test does not move the verdict.","tokens_in":9809,"tokens_out":6062,"duration_ms":62975,"concrete_test":"Re-run Step 1 selector training on the NeRF-Synthetic dataset with ray-traced images instead of rasterized images, keeping the RL algorithm, Step-2 fine-tuning, and evaluation protocol identical; compare final PSNR/SSIM/LPIPS at M=25 and M=50 selected views, with at least 3 seeds for error bars. Also report Jaccard overlap of the selected view sets. If the ray-traced-trained selector outperforms the rasterized-trained selector by more than the run-to-run variation, Eq. 5's proxy assumption is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central cost-saving claim depends on an untested proxy equivalence. In Step 1 (Sec. III-C, Eq. 5), the RL view selector is optimized using reward -L_DNR^c computed from DNR's photometric loss on rasterized images; in Step 2 the selected views are used to acquire ray-traced images and fine-tune DNR. For the method to deliver its advertised savings, the ranking of candidate views induced by rasterized-image loss must be the same as, or at least highly correlated with, the ranking induced by ray-traced-image loss after fine-tuning. The paper never tests this: Fig. 8c compares reward strategies, not selector training on rasterized vs. ray-traced images. Because ray-traced appearance includes global illumination, shadows, and reflections that rasterized images lack, the optimal views for minimizing final ray-traced rendering loss could differ systematically. If so, the selector could choose views that are informative for coarse rasterized geometry but miss critical ray-traced lighting complexity, and the claimed quality-vs-ray-trace-budget advantage would shrink or disappear. The Sec. IV-B statement that DNRSelect 'nearly reaches the theoretical upper limit' and can 'sometimes exceed this limit' is also never defined; no oracle or upper-bound baseline is identified, so this part of the headline result is not evaluable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DNRSelect, a two-stage deferred neural rendering framework. In Step 1, a Q-learning-based view selector (Sec. III-C) is trained jointly with DNR on cheap rasterized images, using the negative DNR photometric loss as the reward in Eq. (5). In Step 2, the coarse DNR model is fine-tuned on ray-traced images only at the M views selected in Step 1, under a six-term loss (Eq. (9)). A 3D texture aggregator (Sec. III-D) fuses depth, normal, and UV maps into neural textures to improve geometric consistency under sparse views. Experiments on NeRF-Synthetic compare DNRSelect with NeRF-based view-selection baselines (ActiveNeRF, Density-aware Ensembles, uncertainty sampling, InstantNGP) across view budgets from 5 to 100. The paper claims that DNRSelect outperforms these baselines, 'nearly reaches the theoretical upper limit,' and 'can sometimes exceed this limit' with fewer views (Sec. IV-B, paragraph after Fig. 6).","tokens_in":10023,"tokens_out":3341,"duration_ms":35432,"significance":"If the central claims hold, DNRSelect would be a practically useful way to cut DNR's dependence on expensive ray-traced training images: it uses rasterized images for view selection, acquires ray-traced data only at selected views, and reports quality gains over NeRF-based competitors. The work is timely given DNR's data cost and the growing interest in active view selection for neural rendering. Credit should be given for testing several design choices (aggregator strategy, reward strategies, data modalities) and for identifying a real gap: no prior work specifically tackles active view selection for deferred neural rendering. However, the supporting evidence is currently thin: no error bars or repeated seeds, no vanilla DNR baseline under identical view budgets, and no experiment validating the rasterized-image proxy that underpins the view selector. The 'theoretical upper limit' claim is also undefined, so the headline quantitative result is not evaluable as written.","major_comments":[{"comment":"The paper states that DNRSelect 'nearly reaches the theoretical upper limit' and 'with fewer selected views, it can sometimes exceed this limit,' but no oracle, all-views baseline, or mathematical definition of this limit is given anywhere in the manuscript. Without a specified upper-bound curve, this load-bearing quantitative claim cannot be checked. The authors should define the limit explicitly (e.g., DNR trained on all 100 training views with the same architecture and losses) and plot the corresponding curve in Fig. 6.","section":"Sec. IV-B, paragraph after Fig. 6"},{"comment":"The view selector is trained in Step 1 solely on rasterized images with reward -L_DNR^c, but the selected views are then used to acquire ray-traced images for Step 2 fine-tuning. The paper never tests whether the ranking of views induced by the rasterized-image loss correlates with the ranking induced by the ray-traced-image loss after fine-tuning. Since ray-traced images contain global illumination, shadows, and reflections absent from rasterized images, the selected views could be systematically suboptimal for the final rendering task. A direct experiment comparing (i) a selector trained on rasterized loss and (ii) a selector trained on ray-traced loss at the same view budgets, or reporting the correlation between the two loss rankings, is needed to support the central cost-saving claim.","section":"Sec. III-C/E, Eq. (5), Fig. 8c"},{"comment":"The main quantitative comparison is against NeRF-based methods (ActiveNeRF, Density-aware Ensembles, uncertainty sampling variants, InstantNGP), not against vanilla DNR or DNR with simple view-selection baselines such as random or uniform sampling. Because DNRSelect changes both the rendering architecture and the view-selection strategy, the curves in Fig. 6 do not isolate the contribution of the RL selector. The authors should add vanilla DNR trained on the same number of randomly or uniformly selected views, vanilla DNR trained on all views, and DNR with the 3D texture aggregator but without RL selection. In addition, no error bars or repeated-seed results are reported, so 'consistently outperforms' is not substantiated.","section":"Sec. IV-B, Fig. 6"},{"comment":"The ablation table reports results at a single view count (25 views) and the checkmark layout is visually ambiguous: it is difficult to tell which components are enabled in each row. Moreover, the table reports only point estimates without variance. Since the RL selector and texture aggregator are the two main contributions, their ablations should be shown across several view budgets (e.g., 10, 25, 50 views) and with multiple evaluation seeds to demonstrate that the gains are consistent and not specific to one configuration.","section":"Sec. IV-C, Table I"}],"minor_comments":[{"comment":"The notation 'scam t' and 'sobs t' appears to be a typesetting error for s_t^cam and s_t^obs; please fix the superscript/subscript formatting throughout.","section":"Sec. III-C"},{"comment":"The symbol T is used both for the neural texture in Sec. III-A and for the trajectory length in Eq. (4); please use separate symbols (e.g., T_tex and T_horizon) to avoid ambiguity.","section":"Eq. (4)"},{"comment":"Implementation details list many hyperparameters but omit the RL discount factor γ, the epsilon-greedy exploration schedule, and the number of action/state dimensions; these are needed for reproducibility.","section":"Sec. IV-A"},{"comment":"The reward-strategy comparison would be easier to interpret if the text described which bars correspond to which reward and whether 'Raster' and 'Ray Tracing' refer to the data modality used for training or for evaluation; the current caption is ambiguous.","section":"Fig. 8c"},{"comment":"The phrase 'the code will be released' is not a substitute for an available implementation; providing a link or supplementary material would help reviewers and readers reproduce the reported numbers.","section":"Abstract and Sec. I"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [name],\n\nYou can skip the desk-reject instinct on this one—it's a real paper with a genuinely new combination—but don't send it to review without demanding fixes. The new bits: first RL-based view selector for deferred neural rendering, and a 3D texture aggregator that fuses depth, normal, and UV neural textures. The selector is trained on rasterized images (cheap) and picks M < N views for ray-traced acquisition. The idea is sensible, and the reward being the DNR loss is not circular; it's just active learning. The code isn't out but the method is described well enough to reimplement.\n\nWhat's good: the ablations show each component contributes, and the reward-strategy comparison in Fig. 8c is useful. The motivation (ray-traced data is the bottleneck) is real.\n\nSoft spots, in order of severity. First, the central transfer assumption is untested: the selector is trained on rasterized-image loss, but evaluated after fine-tuning on ray-traced images. If the ranking of candidate views under rasterized loss doesn't match the ranking under ray-traced loss, the savings shrink. No experiment compares selection trained on rasterized vs. ray-traced images. That's the load-bearing assumption of the whole cost-saving claim, and it's just asserted.\n\nSecond, the \"theoretical upper limit\" is never defined. The sentence \"nearly reaches the theoretical upper limit\" and \"can sometimes exceed this limit\" appears after Fig. 6 with no oracle or baseline identified. As written, that's not an evaluable claim. They need to either define the limit (e.g., DNR trained on all 100 views) and show the curves, or drop the sentence.\n\nThird, the comparison set is thin for what they claim. They compare against NeRF-based view selection methods, but not against vanilla DNR with random selection or all views under identical view budgets. Without an architecture-matched DNR baseline, you can't tell whether the gain comes from view selection or just from the extra texture aggregator and loss functions. Also, no error bars or repeated seeds anywhere; the ablations all appear to be single runs.\n\nMinor: the ablation table's checkmarks are garbled, and \"sometimes exceed the limit\" remains unexplained. The related work is fine, and the citation of [28] as the RL view selector source is appropriate.\n\nBottom line: the paper is worth engaging. The core idea is plausible and the problem is real. But the headline claims outrun the evidence. I would send it to review, but with a strong request for an oracle/upper-bound definition, a DNR-with-random-views baseline, error bars, and ideally the rasterized-vs-ray-traced selection experiment.\n\nWho it's for: people working on active view selection in neural rendering, especially for robotics with DNR. Not a field-changer. Deserves a serious referee.\n\nRecommendation: conditionally accept after major revision; if the missing experiments don't come, reject.","headline":"Genuinely new combination of RL view selection and DNR, but the central rasterized-to-ray-traced transfer is untested and the headline claim is undefined.","tokens_in":10613,"tokens_out":2846,"would_cite":true,"duration_ms":26966,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A reinforcement-learning view selector trained on cheap rasterized images can pick the few camera views worth ray tracing, letting deferred neural rendering reach near-full-data quality with far fewer ray-traced images.","keywords":["deferred neural rendering","view selection","reinforcement learning","next best view","ray-traced images","3D texture aggregator","neural textures","novel view synthesis"],"falsifier":"Train the identical two-step pipeline but compute the Step-1 reward on ray-traced images instead of rasterized images for the same candidate views. If the ray-traced selector picks a different set of $M$ views and the resulting fine-tuned renderer scores measurably higher PSNR at the same $M$, then rasterized images are not a sufficient proxy for choosing views, and the paper's cost-saving argument weakens.","tokens_in":9541,"feed_emoji":"🎨","tokens_out":8706,"duration_ms":83480,"temperature":0.7,"pith_summary":"Deferred neural rendering produces controllable, photorealistic images but typically needs a large set of expensive ray-traced training views. DNRSelect claims that the number of ray-traced views can be cut sharply by first training a reinforcement-learning view selector on cheap rasterized images; the selector learns which camera positions carry the most information for the renderer, and ray-traced images are acquired only for those positions. On the NeRF-Synthetic benchmark the authors report that this two-step scheme matches or exceeds uncertainty-based view-selection baselines at the same small view counts and that it nearly reaches, sometimes surpassing, the quality of training on the full set of ray-traced views. A complementary 3D texture aggregator fuses depth, normal, and UV information so geometry stays consistent when views are sparse. If the claim holds, high-fidelity deferred neural rendering becomes affordable with a fraction of the ray-traced data, which matters for robotics and interactive applications.","feed_headline":"View-selection agent cuts ray-traced data for neural rendering","feed_subtitle":"Training a view picker on cheap rasterized images tells the renderer which viewpoints are worth expensive ray tracing.","key_machinery":"Two components carry the argument. The first is the reinforcement-learning view selector: the state at step $t$ is a camera embedding of previously selected views plus a CNN feature of the current spatial neural texture; Q-learning with temporal-difference targets (Eqs. 2--4) scores candidate next views, and the next action is chosen greedily with $\\epsilon$-greedy exploration. Its reward is the negative coarse DNR loss, so the selector and the renderer are optimized toward the same objective. The second is the 3D texture aggregator: hierarchical sampling $\\pi(\\cdot)$ separately extracts neural textures $\\{T^{ex}_D, T^{ex}_N, T^{ex}_U\\}$ from depth, normal, and UV maps; these are concatenated and processed by a CNN into the spatial neural texture $T$ that feeds the U-Net renderer and also serves as the selector's observation. This separate-per-modality fusion is what the paper credits for preserving geometric consistency that early channel-wise concatenation loses.","core_discovery":"The paper's central claim is that optimal views for deferred neural rendering can be discovered without ray tracing. DNRSelect trains a Q-learning view selector in Step 1 on rasterized images, rewarding it with the negative current DNR loss, so the selector outputs a sequence of $M<N$ camera views that maximize expected future rendering quality. Step 2 then fine-tunes the coarse DNR model on ray-traced images only for those $M$ views, using a six-loss supervision (photometric, SSIM, perceptual, frequency, total variation, and texture regularization). The authors further claim that a 3D texture aggregator, which processes depth, normal, and UV maps separately and then fuses their neural textures, reduces the artifacts that sparse views cause, and that the complete system on NeRF-Synthetic outperforms uncertainty-based NeRF view-selection methods and nearly reaches, sometimes exceeds, full-data quality.","pith_inferences":["The paper does not compare rasterized-trained versus ray-traced-trained selectors, so a direct test of the proxy assumption remains open; if the two rankings diverge, view selection could be improved further by using a small amount of ray-traced data during training.","Exceeding the 100-view upper limit with fewer selected views implies the upper limit is not a hard ceiling; it may reflect that random full training includes redundant views, so a well-chosen subset can be a better training set, not just a cheaper one.","The same two-step pattern---cheap proxy renders for selection, expensive renders for fine-tuning---could transfer to other neural rendering stacks, such as Gaussian-splatting pipelines that need high-quality shading only at chosen viewpoints.","For robotics, the selector could be run online: an exploring agent could use rasterized proxies to decide where to take the next high-fidelity measurement, extending the next-best-view idea from reconstruction to deferred rendering."],"forward_implications":["At a fixed number of training views, DNRSelect reports higher PSNR and SSIM than ActiveNeRF, density-aware NeRF ensembles, and other uncertainty-based information-gain view selectors on NeRF-Synthetic.","Because the selector is trained on rasterized images, the expensive ray-traced acquisition can be deferred until after the useful viewpoints are known, so the cost of data collection is concentrated on the views that matter.","Each of the three additions---data augmentation, multi-loss supervision, and the RL selector plus aggregator---improves the ablation results, with the full system reaching 28.12 PSNR versus 26.64 without the proposed components at 25 views.","Fusing depth, normal, and UV maps with the separate-texture aggregator beats using UV maps alone, UV plus depth, or UV plus normal, and it beats concatenating all three into a single 9-channel input.","The method's compatibility with DNR suggests deferred rendering can be made active, letting an agent choose views rather than passively consuming a fixed dataset."],"supporting_citations":[{"why":"Defines the deferred neural rendering architecture with neural textures and a U-Net renderer that DNRSelect extends and fine-tunes.","marker":"[5]"},{"why":"Provides the learning-to-select-camera-views formulation that the paper adapts into an RL view selector for DNR.","marker":"[28]"},{"why":"Supplies the Q-learning and temporal-difference target used to train the view selector's action-value function.","marker":"[32]"},{"why":"Contributes the NeRF-Synthetic dataset used for all experiments and the full-view training regime that serves as the quality ceiling.","marker":"[1]"},{"why":"ActiveNeRF is a next-best-view baseline based on uncertainty estimation that DNRSelect compares against.","marker":"[3]"},{"why":"Density-aware NeRF Ensembles is a second uncertainty-based baseline in the view-count comparison.","marker":"[29]"},{"why":"InstantNGP is a fast neural-rendering baseline included in the comparison across training view counts.","marker":"[34]"}],"fun_headline_variants":["RL view selection cuts ray-traced data for high-fidelity neural rendering","Pick views with reinforcement learning, then ray-trace only those","DNRSelect: Learn view importance from cheap images, render rich with few ray traces","View selector trained on rasterized images minimizes need for ray tracing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that cheap rasterized images reveal which viewpoints are most valuable for final ray-traced rendering quality; if the two rankings diverge, the selected views are not the informative ones.","fun_headline_variants_meta":{"raw":{"variants":["RL view selection cuts ray-traced data for high-fidelity neural rendering","Pick views with reinforcement learning, then ray-trace only those","DNRSelect: Learn view importance from cheap images, render rich with few ray traces","View selector trained on rasterized images minimizes need for ray tracing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000441,"raw_usage":{"total_tokens":2245,"prompt_tokens":962,"completion_tokens":1283,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":1205}},"tokens_in":578,"tokens_out":1283,"duration_ms":10635,"temperature":1.0,"reasoning_tokens":1205,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:27:33.816786+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the identical two-step pipeline but compute the Step-1 reward on ray-traced images instead of rasterized images for the same candidate views. If the ray-traced selector picks a different set of $M$ views and the resulting fine-tuned renderer scores measurably higher PSNR at the same $M$, then rasterized images are not a sufficient proxy for choosing views, and the paper's cost-saving argument weakens.","supporting_citations":[{"cited_title":"Deferred neural rendering: image synthesis using neural textures,","cited_arxiv_id":null,"evidence_quote":"Defines the deferred neural rendering architecture with neural textures and a U-Net renderer that DNRSelect extends and fine-tunes."},{"cited_title":"Learning to Select Camera Views: Efficient Multiview Understanding at Few Glances","cited_arxiv_id":"2303.06145","evidence_quote":"Provides the learning-to-select-camera-views formulation that the paper adapts into an RL view selector for DNR."},{"cited_title":"Nerf: Representing scenes as neural radiance fields for view synthesis,","cited_arxiv_id":null,"evidence_quote":"Contributes the NeRF-Synthetic dataset used for all experiments and the full-view training regime that serves as the quality ceiling."},{"cited_title":"Activenerf: Learning where to see with uncertainty estimation,","cited_arxiv_id":null,"evidence_quote":"ActiveNeRF is a next-best-view baseline based on uncertainty estimation that DNRSelect compares against."},{"cited_title":"Density-aware nerf ensembles: Quantifying predictive uncertainty in neural radiance fields,","cited_arxiv_id":null,"evidence_quote":"Density-aware NeRF Ensembles is a second uncertainty-based baseline in the view-count comparison."},{"cited_title":"Instant neural graphics primitives with a multiresolution hash encoding,","cited_arxiv_id":null,"evidence_quote":"InstantNGP is a fast neural-rendering baseline included in the comparison across training view counts."}],"review_version":1}