{"id":"8f46add3-aa56-47d1-a67b-9c3ba87a0bed","arxiv_id":"2504.19749","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"STCOcc improves 3D occupancy and scene flow prediction with explicit occupancy-state-guided sparse attention, achieving state-of-the-art RayIoU (41.7%) and mAVE (0.44) on nuScenes-based benchmarks.","lead":"This paper presents STCOcc, a camera-only 3D occupancy and scene flow prediction method that uses predicted occupied states to refine spatial and temporal features. It reports top results on the Occ3D and OpenOcc benchmarks while cutting training memory to about 8 to 9 GB, making strong perception models cheaper to train.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim rests on single-run margins smaller than typical seed variance and on an unresolved camera-mask discrepancy between Table 1 and Table 2; both need checking before the headline numbers can be trusted.","rationale":"The reader's weakest assumption is that reported performance gaps are real rather than run-to-run noise and that the comparison protocol is fair. My reading of Tables 1-3 confirms that the decisive margins are small and that the camera-mask footnote creates a concrete ambiguity: the same STCOcc configuration appears without the mask symbol in Table 1 and with it in Table 2. That inconsistency is not merely cosmetic because the camera mask changes the training objective and can plausibly shift results by more than the claimed 0.1-0.5 point advantage. The ablation protocol (Section 4.3) uses only the first 300 sequences and no error bars, so the component contributions in Table 4 are also single-run estimates. I agree with the conditional disposition: the architecture and ablations are promising, but the headline SOTA claim should not be accepted without clarifying the protocol, releasing exact configs, and providing seed variance. I do not see an internal mathematical contradiction in the method itself; the concern is empirical robustness and comparability.","tokens_in":14431,"tokens_out":9804,"duration_ms":95804,"concrete_test":"Run STCOcc (R50, 704x256) on Occ3D-nus in both camera-mask and no-camera-mask modes with three seeds each, and rerun OPUS-L and COTR under the same schedule, batch size, and optimizer; report mean and standard deviation for RayIoU and mIoU along with the exact config used for Table 1 and Table 2. Also compute OpenOcc mA VE using the same true-positive selection rule as FB-Occ. If any of the 0.1-0.5 point margins falls within one standard deviation, or if toggling the camera mask changes RayIoU by more than 0.5, the SOTA wording should be softened.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is empirical SOTA on two benchmarks. The decisive margins are small: Table 1 gives STCOcc RayIoU 41.7 vs OPUS-L 41.2 and COTR 41.0; Table 2 gives mIoU 44.6 vs COTR 44.5; Table 3 gives RayIoU 40.8 vs Let Occ Flow 40.5 and mA VE 0.44 vs FB-Occ 0.83 and SparseOcc 0.87. Every number is a single run with no error bars or seed information, so if run-to-run variance is comparable to these 0.1-0.5 point gaps, the SOTA claim is not established. More specific to the protocol: Table 1 lists STCOcc without the camera-mask symbol, while Table 2 lists STCOcc with it for the same R50 704x256 configuration and the same 7.7 GB memory footnote. The footnote says the symbol means 'trained with camera mask.' Unless the authors used two different training protocols and did not say so, the 41.7 RayIoU and 44.6 mIoU cannot both come from the same model, and the comparison against unstarred baselines in Table 1 is not well-defined. The OpenOcc mA VE improvement is also large (0.44 vs 0.83/1.45) and is not accompanied by an analysis of how the true-positive set for mA VE changes with occupancy quality, so part of the gain could reflect a different denominator rather than better flow. These are correctness and comparability risks, not an attack on the method's value.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"STCOcc proposes an explicit state-based modeling framework for camera-only 3D occupancy prediction and scene flow prediction. The method uses a cascade decoder with a Self-Recursive Occupancy Predictor (SROP), Occupancy-Aware Spatial Cross-Attention (OA-SCA), Occupancy-Aware Temporal Self-Attention (OA-TSA), and a sparse temporal fusion module that separates non-empty and empty regions into long-term and short-term streams. The paper reports state-of-the-art RayIoU/mIoU on Occ3D-nus and RayIoU/mAVE on OpenOcc, with substantially reduced training memory, and releases code.","tokens_in":14816,"tokens_out":5896,"duration_ms":59226,"significance":"If the reported results are robust, STCOcc is a meaningful advance: the explicit use of predicted occupancy state to renovate spatial and temporal features is a plausible and well-motivated alternative to purely implicit or depth-only feature fusion, and the sparse temporal fusion is a computationally attractive design. The paper also ships code, which supports reproducibility. However, the central claim is an empirical benchmark measurement, and the decisive margins over prior methods are small (0.1–0.5 RayIoU/mIoU points in Tables 1–3), while every reported number is a single run without error bars or seed information. The protocol inconsistency about camera-mask training between Tables 1 and 2 is load-bearing for the comparison, and the OpenOcc mAVE gain may be affected by a changing true-positive evaluation set. These issues need to be addressed before the headline SOTA claim can be accepted as established.","major_comments":[{"comment":"The camera-mask notation is directly inconsistent for the same configuration. Table 1 lists STCOcc (R50, 704×256) without the ∗ symbol, while Table 2 lists STCOcc (R50, 704×256) with ∗, and the footnote says ∗ means \"trained with camera mask.\" Both entries report the same 7.7 GB memory and the same backbone/input size, but the RayIoU 41.7 and mIoU 44.6 cannot both come from the same training protocol unless the authors used two different protocols and did not say so. If the Occ3D-nus RayIoU result was produced with camera-mask supervision, its comparison against the unstarred baselines in Table 1 is not well-defined. Please clarify which setting produced each number and justify the comparison.","section":"§4.2, Tables 1 and 2"},{"comment":"All headline results are single runs with no error bars, seed information, or variance estimates. The reported SOTA margins are small: Table 1 gives 41.7 vs. 41.2 (OPUS-L) and 41.0 (COTR); Table 2 gives 44.6 vs. 44.5 (COTR); Table 3 gives 40.8 vs. 40.5 (Let Occ Flow). If run-to-run variance is comparable to these gaps, which is typical in this setting, the SOTA claim is not established. Please report mean and standard deviation over at least three seeds for the main comparison tables, or otherwise provide evidence that the differences are outside noise.","section":"§4.2, Tables 1–3"},{"comment":"The ablations are performed on only the first 300 training sequences, while the main results are on the full 600-sequence training set, so the component contributions in Table 4 may not transfer to the full-data setting. More importantly, each ablation is a single run with no variance. Table 7, for example, shows OA-SCA at 38.3 vs. DA-SCA at 37.7 and SCA at 37.5; the 0.6–0.8 point differences, on half the training data and without repetitions, are within plausible seed noise and do not support the statement that SCA and DA-SCA \"do not significantly improve performance.\" Please rerun these comparisons with multiple seeds or report variance, and state the training-set size in the table captions.","section":"§4.3, Tables 4–7"},{"comment":"The mAVE metric is computed only over true positives within a query-ray threshold of 2 meters, as stated in Section 4.1. Since STCOcc also changes occupancy predictions (RayIoU 40.8 vs. 33.4 for SparseOcc), the set of objects over which mAVE is averaged can differ between methods. The large mAVE improvement from 0.83–0.87 to 0.44 could partly reflect a different denominator rather than better scene flow. Please provide a matched evaluation that conditions on the same true-positive set, or report mAVE with recall/coverage statistics so the reader can separate flow quality from occupancy-quality effects.","section":"§4.1 and Table 3"}],"minor_comments":[{"comment":"The abstract says training memory is reduced to 8.7 GB, but Table 1 reports 7.7 GB for the main Occ3D-nus setting and Table 3 reports 8.7 GB for OpenOcc; please clarify which configuration the abstract refers to.","section":"Abstract and Tables 1/3"},{"comment":"CTF-Occ is cited as [40], but reference [40] is the Occ3D dataset paper; please provide the correct source for CTF-Occ.","section":"Table 2"},{"comment":"The phrases \"3.7% increase in mA VE\" and \"45.5% increase in mA VE\" mix relative improvements with percentage points; please report changes consistently (e.g., absolute differences or explicit relative percentages) to avoid confusion.","section":"§4.3 text"},{"comment":"The description of d_r and d'_r as \"analogous to z and d in Eq. (4)\" is unclear because z is a world coordinate while d is an image-plane depth; please define both quantities in a self-contained way.","section":"§3.2.3, Eq. (7)"},{"comment":"There are several minor typos and grammatical issues, including \"corrreponds\" (page 5), \"paradism\" (Section 2.2), \"offical\" (Table 3 caption), and \"V ora\" in reference [4]; a careful proofread is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The method is interesting and potentially publishable, but the empirical SOTA claim currently rests on a resolvable protocol inconsistency (camera-mask notation) and on single-run margins that are smaller than typical seed variability. The authors should be asked to rerun the critical comparisons with multiple seeds and to document the exact training protocol for each table before the paper can be recommended for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick read of STCOcc. It deserves a serious look. The concrete contribution is using the model's own occupancy predictions as an explicit geometric state to reweight spatial cross-attention and to route sparse temporal fusion into long/short streams. That combination is new, and it buys real things: a large scene-flow gain on OpenOcc (mAVE 0.44 vs 0.83 for FB-Occ) and a training-memory cut to about 8 GB, which is not trivial. The writing is clear, the ablations are structured, and releasing code helps.\n\nThe soft spots are genuine but not disqualifying. First, every headline number is a single run. The gaps over the next-best methods are 0.1–0.5 RayIoU points in Table 1 and 0.1 mIoU in Table 2. That is a common weakness in this literature, and I would not reject for that alone, but the SOTA claim is simply not established without variance information. Second, the table footnotes conflict: Table 1 lists STCOcc without the camera-mask asterisk, while Table 2 lists the same configuration with it. Since the asterisk means a different training protocol, the two numbers cannot both describe the same model unless the authors silently used two protocols. That needs fixing. Third, the ablations run on half the training data, and some component margins (0.6–0.8% RayIoU) are in the range of expected seed noise. Fourth, the mAVE improvement is large enough that I would want an analysis of how the true-positive set changes with occupancy quality; if flow error is measured only on predicted positives, better occupancy alone can reduce mAVE without better flow.\n\nOn balance: the architecture is sensible, the paper is honest in what it reports, and the central engineering claim—explicit state-based renovation works—is credible. It is a conditional-accept systems paper, not a done deal. A serious referee should engage, and the authors should add seeds or error bars, resolve the camera-mask discrepancy, and justify or replace the half-data ablations.","headline":"A plausible, well-engineered occupancy/flow system whose headline SOTA numbers are single runs and whose two main tables disagree on the camera-mask training protocol.","tokens_in":15364,"tokens_out":3042,"would_cite":true,"duration_ms":32389,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"STCOcc claims that camera-only 3D occupancy and scene flow prediction improves by explicitly renovating features with the occupied state of 3D space, reporting 41.7% RayIoU on Occ3D-nus and 0.44 mAVE on OpenOcc while cutting training…","keywords":["3D occupancy prediction","scene flow prediction","camera-only perception","sparse attention","temporal fusion","occupancy state","RayIoU","autonomous driving"],"falsifier":"Run the published Occ3D-nus recipe (ResNet-50, 704×256, 36 epochs) several times and measure the spread of RayIoU; if seed-to-seed variation approaches the 0.5–1.7 point margins over the strongest prior methods, the claimed lead is not established. Also verify whether the Table 1 row without a camera-mask marker and the Table 2 row with a camera-mask marker use identical training data, since the mask changes supervision.","tokens_in":14239,"feed_emoji":"🚗","tokens_out":6308,"duration_ms":58787,"temperature":0.7,"pith_summary":"The paper is trying to establish that the missing ingredient in camera-only 3D occupancy and scene flow prediction is explicit use of the occupied state of space, rather than reliance on implicit loss-driven feature learning. It proposes STCOcc, a cascade decoder that repeatedly predicts occupancy weights and uses them to reweight spatial and temporal attention, plus a sparse long-term temporal fusion. The reported payoff is a RayIoU of 41.7% on Occ3D-nus and 40.8% RayIoU with 0.44 mAVE on OpenOcc, exceeding prior camera-only methods, while training memory drops to 7.7–8.9GB. If the numbers hold, the paper offers a recipe for more geometric, memory-lean 3D perception from cameras alone.","feed_headline":"Camera-only 3D occupancy hits 41.7 RayIoU at 7.7GB training","feed_subtitle":"Sparse occupancy-state renovation also cuts GPU memory to 8.7GB while predicting scene flow.","key_machinery":"The load-bearing object is the occupied state $W$, a per-voxel occupancy weight produced by the Self-Recursive Occupancy Predictor (SROP), which iteratively refines $W$ across transformer layers via $W^i_{t,l} = f^i(B^i_{t,l}) + \\alpha^i_l W^i_{t,l-1}$. This state drives three components: Occupancy-Aware Spatial Cross-Attention (OA-SCA) reweights sampled 3D reference points by $\\Omega_x = w_x \\beta_x$, combining occupancy with a Gaussian depth-matching factor $\\beta_x$, to resolve the ambiguity in ordinary spatial cross-attention; Occlusion-Aware Temporal Self-Attention (OA-TSA) reweights deformable temporal attention by occupancy; and Sparse Temporal Fusion samples top-k occupied seeds into long-term and short-term streams fused by MLP and written back into the voxel feature. The same occupancy state also supplies per-stage supervision through the occupancy loss.","core_discovery":"The central claim is that replacing implicit, loss-supervised refinement with explicit renovation guided by an iterated occupancy state makes camera-only 3D perception both more accurate and cheaper. On Occ3D-nus the method reports RayIoU 41.7% with a ResNet-50 at 704×256 input, and 42.1% at 1408×512; on OpenOcc it reports RayIoU 40.8% and mAVE 0.44, all while training in 7.7–8.9GB. The paper attributes the gain to using occupied-state guidance to resolve the ray-level ambiguity in spatial cross-attention and to focus temporal modeling on non-empty regions.","pith_inferences":["Inference: The same occupancy-weight-gated attention could be applied to other sparse 3D tasks such as object detection or motion forecasting, where ray ambiguity and empty-space redundancy also dominate.","Inference: Because SROP is recursive and shares weights across layers, it resembles an unrolled refinement; one testable extension is whether more stages or shared weights across stages yield further gains or saturate.","Inference: The memory reduction depends on top-k sparsity, so the 8.7GB figure is a point on a curve rather than a fixed property; varying the sampling ratio should trade memory against RayIoU.","Inference: The volume-rendering analogy suggests fusing occupancy and depth into a single transmittance-like weight could unify depth estimation and occupancy prediction in one decoder."],"forward_implications":["Camera-only occupancy prediction can reach RayIoU above 41% on Occ3D-nus with a ResNet-50 at 704×256 input, a regime previously dominated by heavier backbones and larger images.","Training memory below 9GB means the method fits on a single consumer-grade GPU, making occupancy pretraining and longer temporal histories accessible without large-scale hardware.","Scene flow can be predicted jointly with occupancy at mAVE 0.44 under 3D supervision only, without lidar or rendering supervision.","Explicit occupancy-guided reweighting, not just depth, is what resolves ray-level ambiguity in spatial cross-attention, so the volume-rendering analogy transfers to occupancy perception."],"supporting_citations":[{"why":"Defines RayIoU, the headline metric, and the fully sparse occupancy baseline whose selection of non-empty voxels STCOcc extends with occupancy-gated attention.","marker":"[21]"},{"why":"Provides the spatial cross-attention and temporal self-attention that OA-SCA and OA-TSA are built on and ablated against.","marker":"[15]"},{"why":"Supplies the depth-aware spatial cross-attention baseline that OA-SCA is compared with.","marker":"[17]"},{"why":"Defines the OpenOcc dataset, the scene flow task, and the mAVE metric used in the main OpenOcc experiments.","marker":"[38]"},{"why":"Defines the Occ3D-nus benchmark and its occupancy evaluation protocol.","marker":"[40]"},{"why":"Supplies the Scene-Class Affinity Loss used in the per-stage occupancy loss.","marker":"[5]"},{"why":"Supplies the depth network used as the feature extractor's depth branch.","marker":"[12]"},{"why":"Supplies the lift-splat voxel pooling method that builds the initial coarse 3D voxel features.","marker":"[36]"}],"fun_headline_variants":["Explicit renovation hits 41.7 RayIoU, trains in 7.7GB","Camera-only occupancy and flow: 41.7 RayIoU at 7.7GB","Sparse cascade renovation cuts GPU memory to 7.7GB","Occupancy-state guidance boosts RayIoU to 41.7 on a budget","Cascade renovation: camera-only 3D with scene flow, low memory"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported wins over prior methods are real rather than run-to-run noise, which matters because the main tables give single runs and the training protocol differs across comparison settings.","fun_headline_variants_meta":{"raw":{"variants":["Explicit renovation hits 41.7 RayIoU, trains in 7.7GB","Camera-only occupancy and flow: 41.7 RayIoU at 7.7GB","Sparse cascade renovation cuts GPU memory to 7.7GB","Occupancy-state guidance boosts RayIoU to 41.7 on a budget","Cascade renovation: camera-only 3D with scene flow, low memory"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000357,"raw_usage":{"total_tokens":1910,"prompt_tokens":897,"completion_tokens":1013,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":513,"completion_tokens_details":{"reasoning_tokens":905}},"tokens_in":513,"tokens_out":1013,"duration_ms":9265,"temperature":1.0,"reasoning_tokens":905,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:44:25.204853+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the published Occ3D-nus recipe (ResNet-50, 704×256, 36 epochs) several times and measure the spread of RayIoU; if seed-to-seed variation approaches the 0.5–1.7 point margins over the strongest prior methods, the claimed lead is not established. Also verify whether the Table 1 row without a camera-mask marker and the Table 2 row with a camera-mask marker use identical training data, since the mask changes supervision.","supporting_citations":[{"cited_title":"Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers","cited_arxiv_id":null,"evidence_quote":"Provides the spatial cross-attention and temporal self-attention that OA-SCA and OA-TSA are built on and ablated against."},{"cited_title":"Scene as occupancy","cited_arxiv_id":null,"evidence_quote":"Defines the OpenOcc dataset, the scene flow task, and the mAVE metric used in the main OpenOcc experiments."},{"cited_title":"Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving","cited_arxiv_id":null,"evidence_quote":"Defines the Occ3D-nus benchmark and its occupancy evaluation protocol."},{"cited_title":"Monoscene: Monoc- ular 3d semantic scene completion","cited_arxiv_id":null,"evidence_quote":"Supplies the Scene-Class Affinity Loss used in the per-stage occupancy loss."},{"cited_title":"Bevstereo: Enhancing depth estimation in multi-view 3d object detection with temporal stereo","cited_arxiv_id":null,"evidence_quote":"Supplies the depth network used as the feature extractor's depth branch."}],"review_version":1}