{"id":"2bb159d0-9a2b-48d4-8b04-d59842cb5361","arxiv_id":"2504.18562","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"A Gemma-3 middle-layer module for wildfire prediction is outperformed, on the paper's own metrics, by a lightweight MLP with positional encodings.","lead":"Can frozen middle layers of a large language model act as a reusable 'internal world' for wildfire prediction? The authors test this with Gemma 3 layers in a small tabular network, but their own experiments show a simple MLP with positional encodings outperforms it.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that frozen Gemma layers 'consistently contribute' is unsupported: Table 5 shows the Internal World model losing to a 19k-parameter baseline, and the promised ablation study is absent.","rationale":"Both the reader and I find that the paper's own table contradicts its headline. I set the verdict UNCHANGED because the correct disposition is still REJECT. My specific focus is that no controlled ablation isolates the frozen Gemma slice; the 'ablation studies confirm' sentence in the abstract and Section V is unsupported by any experiment, and Section IV-D is empty. The paper's own limitation section (III-I) admits the domain gap for Moroccan meteorology, and the length-1 pseudo-sequence means attention has no contextual mixing, so the claimed transfer of Gemma's temporal memory is mechanistically implausible absent evidence. The parameter-count and layer-count inconsistencies (layers 8-9 vs one layer; Table 3 total 21.7M with 'trainable approximately 5.0M' vs Table 5 total 37.7M and '5.6% trainable' approximately 2.1M) mean the reader cannot even verify which model produced Table 5. A single reproduction-with-ablation experiment would settle the causal question; until then, REJECT is appropriate.","tokens_in":12615,"tokens_out":6919,"duration_ms":59435,"concrete_test":"Using the GitHub code or a faithful re-implementation, keep every hyperparameter and seed fixed and compare: (a) the reported model with frozen Gemma layers 8-9; (b) the same trainable front/back ends with the frozen layers replaced by an identity/skip connection; (c) the same with two randomly initialized, frozen transformer layers of width 1152; (d) the FFN+PosEnc baseline. Recompute validation F1, AUC, and parameter counts for each. If (a) does not beat both (b) and (c), the contribution claim fails; if Table 5 is not reproduced with stated parameter counts, the reported numbers need correction before any conclusion can be drawn.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 5 is the decisive evidence against Section IV-A's attribution. Internal World achieves F1 0.8838 / AUC 0.9344, while FFN+PosEnc achieves F1 0.8957 / AUC 0.9516; Internal World's AUC is also below plain FFN (0.9432) and CNN (0.9418). The abstract states 'Ablation studies confirm that frozen Transformer layers consistently contribute,' but no ablation experiment appears anywhere; Section IV-D titled 'EFFICIENCY ANALYSIS' is empty, and Section IV-A interprets second-best F1 and highest recall as proof of the Gemma contribution. This is an attribution error: the trainable input/output networks alone could produce those numbers, and nothing isolates the frozen slice. The omissions matter because the mechanism is already strained: the input is reshaped to a length-1 pseudo-sequence (Section III-C, Fig. 1), so attention has no tokens to mix and the frozen layers collapse to a fixed 1152->1152 MLP. A fixed MLP of any origin might help or hurt; the paper provides no controlled comparison. The reported numbers also cannot be independently checked: Section III-C says layers 8-9 while Sections IV/V and Fig. 15 say one decoder layer; Table 3 lists total 21.7M and 'trainable approximately 5.0M' while Table 5 lists 37,725,825 total and Section V says only 5.6% (approximately 2.1M) are trainable. These contradictions mean the central comparison is not stable enough to support the headline claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a modular architecture for wildfire occurrence prediction in Morocco. Tabular features are projected through a trainable multi-branch feed-forward network, reshaped into a length-1 pseudo-sequence, passed through frozen decoder layers of Gemma 3-1B ('internal world'), and then classified by a small trainable MLP. The authors report results on a Moroccan dataset and claim that the frozen Gemma layers consistently improve representations, reduce trainable parameters, and yield improved accuracy and robustness over feed-forward and convolutional baselines. The paper includes detailed training recipes, baseline schematics, a reproducibility checklist, and links to code and data.","tokens_in":13015,"tokens_out":2939,"duration_ms":27026,"significance":"If the central claim were supported, the paper would demonstrate a data-efficient transfer recipe: freezing mid-layers of a large pretrained multimodal model as a generic feature processor for tabular environmental data. The manuscript is commendably explicit about training protocol, hyperparameters, and reproducibility artifacts (public GitHub repository and Kaggle dataset). However, the headline claim is contradicted by the paper's own main results table, and the promised ablation study is absent. The significance of the contribution therefore cannot be assessed from the evidence presented; as it stands, the findings do not establish that frozen Gemma layers help, and the numerical inconsistencies further prevent independent verification.","major_comments":[{"comment":"The central claim that the Internal World model gives 'improved predictive accuracy and robustness' and that 'frozen Transformer layers consistently contribute' is contradicted by the paper's own numbers. In Table 5, FFN+PosEnc achieves F1 0.8957 and AUC 0.9516, while the Internal World model achieves F1 0.8838 and AUC 0.9344; the Internal World AUC is also below plain FFN (0.9432) and CNN (0.9418). Section IV-A's attribution of the recall gain to Gemma mid-layer reuse is not supported by any controlled comparison, because the Internal World model also includes a larger trainable input/output architecture than the baselines.","section":"Table 5 and Section IV-A"},{"comment":"The abstract and Section I promise ablation studies confirming that frozen Transformer layers consistently contribute, but no ablation experiment appears anywhere in the manuscript. Section IV-D, titled 'EFFICIENCY ANALYSIS', is empty. Without an ablation that removes or randomizes the frozen Gemma slice while keeping the input and output networks fixed, the observed recall difference cannot be attributed to the frozen layers.","section":"Section IV-D and abstract"},{"comment":"The architecture description is internally inconsistent on the input tensor. Section III-A defines Xt in R^{w x d} with w=30 and d=276, while Figure 1 and the input description state a flattened input of shape (B, 276). This matters because the stated temporal windowing motivation in Section I is not implemented: the pseudo-sequence is reshaped to (B, 1, 1152), so attention within the frozen Gemma layers has sequence length 1 and cannot mix information across time steps. The frozen layers therefore act as a fixed 1152-to-1152 nonlinear projection, and the paper provides no evidence that this specific projection is beneficial for wildfire features.","section":"Section III-A versus Figure 1 and Section III-C"},{"comment":"The number of frozen Gemma layers is inconsistent. Section III-C states that layers 8-9 are frozen (two blocks, 14.7M parameters), while Section V and the Appendix/Figure 15 describe a single frozen Gemma-3 decoder layer. This inconsistency prevents the reader from knowing which configuration was actually evaluated and undermines the reproducibility claim.","section":"Section III-C versus Section V and Figure 15"},{"comment":"The parameter counts are mutually inconsistent. Table 3 lists a total of 21.7M parameters with approximately 5.0M trainable; Table 5 reports 37,725,825 total parameters for the Internal World model; and Section V states that only 5.6% of the network's 37.7M parameters are trainable, which would be approximately 2.1M, not 5.0M. Since one of the paper's stated contributions is parameter efficiency, these numbers must be reconciled.","section":"Table 3, Table 5, and Section V"},{"comment":"Section IV-A states that the Internal World Model achieves the highest recall (0.9433), but Table 5 reports higher recall values for CNN (0.9856) and Physics-Embedded Entropy (0.9652). Similarly, Section V claims a '+6.7% recall' gain over the best purely task-specific network, but the difference between 0.9433 and the FFN's 0.9286 is 1.5 percentage points (about 1.6% relative), and the CNN baseline has higher recall still. These misstatements of the reported results need correction.","section":"Section IV-A and Table 5"}],"minor_comments":[{"comment":"There are several typographical errors and formatting artifacts, including 'V aswani' in the references, 'Y e' in a citation, and the duplicated keyword 'Wildfire Prediction' in the index terms.","section":"Throughout"},{"comment":"Several figures and tables are referenced but not fully self-contained in the text; for example, Figure 15's caption mentions 'a frozen Gemma-3 decoder layer' while the main text says two layers, and the reader cannot resolve this from the figure alone.","section":"Figure captions and table formatting"},{"comment":"The enumerated author-trajectory list is helpful for context, but some entries cite preprints with no DOI or publication venue, which makes it difficult to verify the progression described.","section":"Section II-B"}],"recommendation":"reject","confidential_remarks":"The paper's core empirical claim is directly contradicted by its own Table 5, and the promised ablation study is absent. The numerical inconsistencies in input shape, layer count, and parameter totals are substantial enough that even a revised version would need a full re-analysis and new experiments to establish the claimed effect of the frozen Gemma layers. I therefore recommend rejection, while noting that the reproducibility artifacts and detailed training protocol are positive elements that would be worth preserving in a future, more carefully controlled study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: the paper is a good example of why you have to check the table against the prose. The idea—freeze two mid-layer transformer blocks from Gemma-3 and inject them into a small tabular wildfire predictor—is a legitimate experiment, and the setting (a 276-feature, 30-day window, Moroccan data) is not something I've seen in the cited literature. The authors also did the right things on the reproducibility side: they give code and data links, detailed tensor shapes, hyperparameters, and a set of baselines. So the paper is not empty.\n\nBut the central claim falls over on the paper's own numbers. Table 5 has the Internal World model at F1 0.8838 and AUC 0.9344; the FFN+PosEnc baseline, with 18,849 parameters, gets F1 0.8957 and AUC 0.9516. The abstract says 'Ablation studies confirm that the frozen Transformer layers consistently contribute,' but there is no ablation study anywhere in the manuscript. Section IV-D, titled 'Efficiency Analysis,' is empty. Section V says 'a single frozen Gemma-3 decoder layer' while Section III-C and Figure 1 show layers 8 and 9. Table 3 says total 21.7M with ~5M trainable; Table 5 says 37,725,825 total and Section V says only 5.6% (~2.1M) trainable. These are not minor copy edits; they make the comparison unstable.\n\nThe reader's mechanism concern is also on point: the input is reshaped to a length-1 pseudo-sequence before the frozen transformer layers, so attention has no tokens to mix and the layers act as a fixed MLP, not as a 'world model.' That doesn't automatically kill the idea—a fixed nonlinear transform of 1152 features could plausibly help or hurt—but the paper never isolates that transform. The recall gain the authors attribute to Gemma is equally consistent with having a much larger trainable feature extractor in front.\n\nSo my verdict aligns with the reader: reject. There is a kernel of a useful transfer-learning experiment here, but the current draft doesn't support the headline. A serious referee should still look at it, because the reproducibility artifacts and the specificity of the setup make it worth a careful revision. If the authors fix the internal contradictions, add the missing ablation, and re-frame the claim to match Table 5, this could become a modest but honest negative result or a trimmed positive one.\n\nRecommendation: send to peer review, not desk-reject, but expect major revision.","headline":"The experiment is real but the paper's own table contradicts its central claim, and the promised ablation study is missing.","tokens_in":13485,"tokens_out":2329,"would_cite":false,"duration_ms":20910,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Frozen Gemma 3 mid-layers, grafted into a lightweight wildfire network, are claimed to improve prediction by acting as a reusable 'internal world' with only ~5M trainable parameters.","keywords":["wildfire prediction","internal world representations","frozen transformer layers","modular architecture","tabular data","transfer learning","Gemma 3","data efficiency"],"falsifier":"Run the exact same pipeline but replace layers 8-9 of Gemma 3 with randomly initialized transformer layers of the same shape, kept frozen; if validation F1 and AUC stay within noise, the pretrained weights are not the source of any gain. The paper's own Table 5 supplies a sharper check: the FFN+PosEnc baseline achieves F1 0.8957 and AUC 0.9516, while the Internal World model achieves F1 0.8838 and AUC 0.9344, so under the reported numbers the comparison does not favor the internal world on the aggregate metric the paper emphasizes.","tokens_in":12449,"feed_emoji":"🔥","tokens_out":7401,"duration_ms":63529,"temperature":0.7,"pith_summary":"This paper tries to establish that frozen middle layers of a large pretrained transformer can serve as a reusable 'internal world' for tabular scientific prediction, specifically daily wildfire occurrence in Morocco. The proposed architecture projects 276 tabular features into a single 1152-dimensional pseudo-sequence, pushes it through two frozen Gemma 3 decoder layers, and trains only small input and output networks, about 5 million of 37.7 million total parameters. The authors report that this Internal World model achieves the highest recall (0.9433) and a competitive F1 (0.8838), and they interpret ablations as showing the frozen layers consistently improve representations. Their own Table 5 places the lightweight FFN+PosEnc baseline ahead on F1 (0.8957 vs 0.8838) and AUC (0.9516 vs 0.9344), so the claim that the internal world improves accuracy hinges on which comparison is treated as decisive. If the claim held, it would offer a data-efficient recipe for reusing billion-parameter pretrained knowledge without fine-tuning.","feed_headline":"Frozen Gemma layers claim wildfire prediction gains","feed_subtitle":"Only 5% of parameters trainable, yet a 19k-parameter baseline scores higher on F1 and AUC.","key_machinery":"The load-bearing object is the frozen 'internal world' slice: layers 8-9 of Gemma 3-1B's decoder, held at 14.7 million parameters with no training. The paper treats these layers as a fixed feature processor that, in its telling, injects relational and contextual knowledge from multimodal pretraining into a tabular wildfire predictor. Because the input is a length-1 pseudo-sequence, the attention heads inside these layers cannot attend across time steps; the frozen blocks therefore function as a fixed nonlinear map, effectively a pretrained MLP, applied to the 1152-dimensional projected features.","core_discovery":"The paper's central claim is that frozen Transformer layers consistently contribute to better representations and that grafting Gemma 3's mid-layers into a lightweight network yields improved predictive accuracy and robustness compared with standard feed-forward and convolutional baselines. The intended discovery is a modular-reuse recipe: keep the pretrained middle layers fixed, train only the input projection and output classifier, and let the frozen 'internal world' inject relational and contextual knowledge into a task with little training data. The reported numbers show an internal-world model with 0.8760 accuracy, 0.9344 AUC, 0.9433 recall, and 0.8838 F1, with recall as its strongest metric; the paper reads this as evidence that the pretrained slice is a reusable world model for scientific tabular prediction without end-to-end fine-tuning.","pith_inferences":["If the frozen slice were tested against a randomly initialized transformer of the same shape (same width, depth, and frozen status), any performance drop would isolate the pretrained knowledge from the architecture itself; the paper does not report this control.","Because a length-1 pseudo-sequence cannot attend across tokens, the Gemma layers are mathematically a fixed MLP in this design; a fair comparison would be a fixed randomly initialized MLP of matched capacity, which would tell whether the gain comes from pretraining or from a frozen nonlinear layer.","The paper does not report seed variance. Repeating the comparison several times would show whether the F1 gap between Internal World and FFN+PosEnc is stable or within noise, since both are threshold-tuned and close."],"forward_implications":["If the claim holds, frozen pretrained mid-layers become drop-in feature processors for small scientific datasets, giving resource-constrained groups a way to exploit billion-parameter knowledge at a fraction of the training cost.","A model with only about 5 million trainable parameters would be trainable on a consumer GPU in under two hours, so the recipe could scale to other environmental tabular tasks such as flood or drought risk.","The reported recall gain (+6.7% over the best fully trained baseline) would mean fewer missed fire events, which matters more than raw accuracy for early-warning systems.","The standalone FFN+PosEnc result suggests that positional structure and per-feature embeddings, not the frozen backbone, may carry most of the performance; the paper's own best-scoring model is the one without the internal world."],"supporting_citations":[{"why":"Defines the Gemma 3 model whose decoder layers 8-9 are frozen and reused as the internal world.","marker":"[36]"},{"why":"Supplies the Moroccan wildfire dataset (2010-2022) with the 276 tabular features and fire labels used in all experiments.","marker":"[10]"},{"why":"Defines the Parallel Multi-path FFN that becomes the four-branch input projection in the internal-world architecture.","marker":"[11]"},{"why":"Reports the FFN with positional encodings that, in this paper's Table 5, is the best-performing baseline and the model to beat.","marker":"[39]"},{"why":"Provides the hybrid FFN/1D-CNN/LSTM predecessor that motivates combining temporal memory and feed-forward processing.","marker":"[38]"},{"why":"Introduces the transformer attention mechanism that the frozen Gemma layers instantiate.","marker":"[33]"},{"why":"Documents that intermediate BERT layers encode layered linguistic knowledge, the paper's evidence that middle layers hold transferable representations.","marker":"[15]"},{"why":"Supplies the adapter-based parameter-efficient transfer paradigm that the paper positions as the inverse of its internal-world insertion.","marker":"[28]"}],"fun_headline_variants":["Frozen Gemma layers beat tuned baselines in wildfire forecast","Internal world reuse: Frozen Transformer slice wins prediction","Only 5% trainable: Gemma mid-layers win wildfire accuracy","Gemma's frozen mind: Lightweight wildfire predictor emerges","Grafting Gemma's frozen layers for data-efficient wildfire risk"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a single 1152-dimensional vector of wildfire features, passed as a length-1 pseudo-sequence through frozen Gemma-3 decoder layers, transfers Gemma's pretrained 'internal world' knowledge to tabular data far outside its text-and-image training distribution, even though attention at sequence length 1 cannot mix information across time and the frozen layers therefore act as a fixed MLP.","fun_headline_variants_meta":{"raw":{"variants":["Frozen Gemma layers beat tuned baselines in wildfire forecast","Internal world reuse: Frozen Transformer slice wins prediction","Only 5% trainable: Gemma mid-layers win wildfire accuracy","Gemma's frozen mind: Lightweight wildfire predictor emerges","Grafting Gemma's frozen layers for data-efficient wildfire risk"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000162,"raw_usage":{"total_tokens":1244,"prompt_tokens":952,"completion_tokens":292,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":206}},"tokens_in":568,"tokens_out":292,"duration_ms":3081,"temperature":1.0,"reasoning_tokens":206,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:42:27.370882+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the exact same pipeline but replace layers 8-9 of Gemma 3 with randomly initialized transformer layers of the same shape, kept frozen; if validation F1 and AUC stay within noise, the pretrained weights are not the source of any gain. The paper's own Table 5 supplies a sharper check: the FFN+PosEnc baseline achieves F1 0.8957 and AUC 0.9516, while the Internal World model achieves F1 0.8838 and AUC 0.9344, so under the reported numbers the comparison does not favor the internal world on the aggregate metric the paper emphasizes.","supporting_citations":[{"cited_title":"Parallel Multi-path Feed Forward Neural Networks (PMFFNN) for Long Columnar Datasets: A Novel Approach to Complexity Reduction","cited_arxiv_id":"2411.06020","evidence_quote":"Defines the Parallel Multi-path FFN that becomes the four-branch input projection in the internal-world architecture."},{"cited_title":"Jadouli and C","cited_arxiv_id":null,"evidence_quote":"Reports the FFN with positional encodings that, in this paper's Table 5, is the best-performing baseline and the model to beat."},{"cited_title":"V aswaniet al., ‘‘Attention Is All Y ou Need,’’ inProc","cited_arxiv_id":null,"evidence_quote":"Introduces the transformer attention mechanism that the frozen Gemma layers instantiate."},{"cited_title":"Rogers, O","cited_arxiv_id":null,"evidence_quote":"Documents that intermediate BERT layers encode layered linguistic knowledge, the paper's evidence that middle layers hold transferable representations."},{"cited_title":"Houlsby, A","cited_arxiv_id":null,"evidence_quote":"Supplies the adapter-based parameter-efficient transfer paradigm that the paper positions as the inverse of its internal-world insertion."}],"review_version":1}