{"id":"98ee9ad2-56e9-4b98-acb1-9639a08f0f8e","arxiv_id":"2412.20329","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"Two DQN variants, a reservoir FFNN and an LSTM with attention, reach best-known energies on short 3D HP benchmark sequences but miss several longer targets that the abstract claims to match.","lead":"This paper trains two deep reinforcement learning agents, a reservoir-based network and an LSTM with attention, to fold short proteins on a 3D lattice model. It reports that the agents often match known best energy values on small benchmarks, but the paper's own tables show several longer sequences fall short of the claimed optimal values.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own Table 3 shows LSTM-A misses best-known values on 3d5–3d9 and FFNN-R misses on A6/A7, directly contradicting the abstract's claim of consistent optimal conformations; the 25% fewer-episodes claim has no logged convergence data.","rationale":"The reader's REJECT verdict is correct. My independent read identifies an even more direct problem than the reader's stated weakest assumption: the empirical record reported in the paper itself contradicts the central claim. Because the incorrect claim is stated in the abstract and conclusion, it cannot be passed as a minor overstatement. I also checked whether the '25% fewer episodes' claim could be verified from Table A.1; it cannot, since total episode counts are identical and convergence episodes are not reported. I do not see a need to move the verdict; REJECT/UNCHANGED remains appropriate. I agree partially with the reader because their non-Markovian-state concern is real but not the decisive failure. No code or data artifacts were provided, so independent reproduction is not possible from the manuscript; the Table 3 inconsistency is sufficient on its own. The prose claims about reservoir and LSTM-A benefits, e.g. 'state-of-the-art results' and 'optimal conformations that align with known energy minima' in Section 4.1, are also contradicted by the same table.","tokens_in":11689,"tokens_out":6975,"duration_ms":64283,"concrete_test":"Run the released training code with the paper's stated hyperparameters (e.g., LSTM-A, 5 layers, hidden 512, batch 32, 750K episodes for 3d9; FFNN-R, 4 layers, 200K episodes for A7) across at least 10 random seeds, logging per-episode minimum energy and the first episode that reaches BKV. If the best energies reproduce Table 3 (-51 for 3d9, -12 for A7), the matching-BKV claim is disproven; if they reproduce BKV (-55, -13), Table 3 contains a reporting error and the efficiency claim can then be tested against the logged convergence episodes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing claim is the abstract's 'Both architectures consistently achieve optimal conformations matching the best known energy values.' This is contradicted by Table 3, the paper's own reported final energies. For the longer benchmark sequences, LSTM-A attains -33 instead of BKV -35 (3d5), -30 instead of -31 (3d6), -32 instead of -34 (3d7), -40 instead of -44 (3d8), and -51 instead of -55 (3d9). For the shorter A-set, FFNN-R attains -11 instead of -12 (A6) and -12 instead of -13 (A7). Thus 7 of the 19 benchmark entries are not optimal by the paper's own standard, and the claimed 'consistently' is false. This is an internal inconsistency, not a disagreement with an external consensus. The efficiency claim is likewise unsupported: Section 3.1 asserts FFNN-R converges in about 200,000 episodes and 'approximately 25% fewer training episodes,' but Table A.1 records 200,000 episodes for both FFNN-VANILLA and FFNN-RESERVOIR on A1-A10 and gives no episode at which each run first reached its best energy. The non-Markovian state-representation issue raised by the reader is plausible but secondary; even if the state were fully informative, the reported energies would not support the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two deep reinforcement learning architectures for protein structure prediction in the 3D Hydrophobic-Polar (HP) lattice model: a reservoir-based feedforward network (FFNN-R) for short sequences (N <= 36) and an LSTM with multi-head attention (LSTM-A) for longer sequences. The authors formulate protein folding as a self-avoiding walk and frame it as a Markov decision process solved with stabilized Deep Q-Learning. They report energy values on the 3d and A benchmark sets, claim that both architectures consistently match best-known energy values (BKV), and claim that FFNN-R reaches optimal conformations with approximately 25% fewer training episodes than the vanilla FFNN. The paper also discusses attention patterns, parameter counts, and training times, and releases code via GitHub.","tokens_in":12044,"tokens_out":3782,"duration_ms":35070,"significance":"If the central claims were correct, this would be a noteworthy contribution: it would introduce reservoir computing to 3D HP protein folding, demonstrate an attention-enhanced LSTM that matches BKVs on long benchmark sequences, and provide an unusually sample-efficient DQN variant, with code released for reproducibility. However, the headline claim of consistent optimality is directly contradicted by the paper's own Table 3, and the efficiency claim is not supported by the logged experimental data in Table A.1. The architectural ideas are interesting, but the reported results do not establish the stated contribution.","major_comments":[{"comment":"The abstract's claim that 'both architectures consistently achieve optimal conformations matching the best known energy values' is contradicted by Table 3. For the long sequences, LSTM-A reports -33, -30, -32, -40, and -51 for 3d5, 3d6, 3d7, 3d8, and 3d9, respectively, versus BKVs of -35, -31, -34, -44, and -55. For the short A-set, FFNN-R reports -11 and -12 for A6 and A7 versus BKVs of -12 and -13. By the paper's own standard, 7 of the 19 benchmark entries are not optimal, so the 'consistently' claim is internally inconsistent with the reported data.","section":"Abstract and Section 1 vs. Table 3"},{"comment":"The right-most column of Table 3, described as 'the least energy conformation found by our models,' is inconsistent with Table A.1. For A4, Table A.1 lists FNN-RESERVOIR's Best E_min as -14 while Table 3 reports -15; for A9, Table A.1 lists Best E_min as -6 while Table 3 reports -7. This indicates that the reported column mixes different statistics (E_Min versus Best E_min) across rows, so the reader cannot determine what quantity is actually being reported.","section":"Table A.1 vs. Table 3"},{"comment":"The claim that FFNN-R converges in about 200,000 episodes and requires 'approximately 25% fewer training episodes' is unsupported by the logged data. Table A.1 records 200,000 training episodes for both FNN-VANILLA and FNN-RESERVOIR on A1-A10, and it gives no information about the episode at which each run first reached its best energy. Without first-convergence episode counts, the 25% reduction claim has no evidentiary basis.","section":"Section 3.1, Efficiency"},{"comment":"The discussion states that 'both FFNN and FFNN-R demonstrate robust performance across multiple independent training runs' and that the architectures 'show consistent performance with lower variance in final energy values,' but no variance statistics, standard deviations, or repeated-run results are reported anywhere in the paper. The reported energies appear to be best-of-run values, so the consistency claim is not established.","section":"Section 4, Discussion"},{"comment":"The 8-dimensional one-hot state encodes only six action indicators and the H/P type of the current bead; it does not directly expose lattice coordinates or an occupancy map. The paper asserts that the network 'deduces this information from actions taken thus far,' but no experiment verifies that the implicit action-history memory is sufficient to reconstruct the self-avoiding path. Without such a verification, the claim that the architectures learn general folding behavior cannot be separated from memorization of the specific benchmark sequences.","section":"Section 2.2.3, State Representation"}],"minor_comments":[{"comment":"The title contains a formatting error: 'M ODEL' should be 'MODEL'.","section":"Title"},{"comment":"The header 'FNN-V ANILLA' contains a typo and should read 'FNN-VANILLA'.","section":"Table A.1"},{"comment":"Several rows in Table A.1 are missing parameter counts or training times (e.g., LSTM-OLH 3d2, FNN-VANILLA rows), making the efficiency comparison incomplete.","section":"Table A.1"},{"comment":"The reservoir input weight matrix is described as 'trainable' in Eq. (12), while the text says the reservoir weight matrix W is 'randomly initialized and remains fixed'; this apparent inconsistency in terminology should be clarified.","section":"Section 2.3.1"},{"comment":"The statement in the Limitations section that 'our implementation achieves state-of-the-art results' is not supported by Table 3, which shows several entries below the quoted BKVs and below the GA baseline for long sequences.","section":"Section 4.1"}],"recommendation":"reject","confidential_remarks":"The manuscript's abstract and conclusions appear to overstate the experimental results relative to the paper's own tables. The internal inconsistencies in Tables 3 and A.1, and the absence of convergence data for the efficiency claim, would require substantial additional experiments and a major re-framing of the contribution before the paper could be considered for publication. The architectural ideas may have merit, but the current evidence does not support the central claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful, reasonably honest benchmark paper that applies two fairly standard DQN variants to the 3D HP lattice model and reports new numbers on the 3d and A sequence sets. The first reservoir-computing DQN for 3D HP and the LSTM-with-multi-head-attention DQN for longer sequences are legitimate extensions of Yang et al. 2023, and the software/data availability statement is a plus, though no URL is given.\n\nWhat the paper does well: the appendix is unusually transparent. Table A.1 lists per-run best energies, training episodes, wall-clock time, and parameter counts. The text in Section 3.1 also admits that LSTM-A only 'approached' BKVs for 3d6-3d9. That level of detail is rare and gives a referee something to work with.\n\nWhere it falls down: the abstract and conclusion claim both architectures 'consistently achieve optimal conformations matching the best known energy values.' Table 3 contradicts that. LSTM-A misses BKV on all five long sequences (3d5-3d9: -33 vs -35, -30 vs -31, -32 vs -34, -40 vs -44, -51 vs -55), and FFNN-R misses A6 and A7 (-11 vs -12, -12 vs -13). That is 7 of 19 benchmark entries below BKV. This is an internal inconsistency, not a disagreement with an external benchmark. The 25% fewer-episodes claim for FFNN-R is also unsupported: Table A.1 records 200K episodes for both vanilla and reservoir variants on A1-A10, with no first-hit episode logged. The non-Markovian state concern is plausible but secondary; even if the state were fully informative, the reported energies would not support the headline.\n\nNone of this kills the underlying work. The short-sequence results are credible, the attention-weight analysis is interesting, and the authors have the good sense to include the data that undermines their own abstract. That asymmetry suggests the claims could be fixed with a major revision.\n\nWho should read it: people working on HP-lattice search algorithms and RL-based folding methods. It is not a serious contribution to real protein structure prediction, and the authors largely avoid claiming that. I would send this to peer review rather than desk reject, with the clear expectation that the authors revise the abstract and conclusion to match Table 3 and provide actual convergence data for the efficiency claim.","headline":"A competent empirical application of standard DQN variants to the 3D HP model, undone by an abstract that overstates its own Table 3.","tokens_in":685,"tokens_out":827,"would_cite":false,"duration_ms":33252,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Deep Q-learning with reservoir and attention networks matches or approaches best-known 3D HP folding energies while using fewer training episodes.","keywords":["3D HP model","protein structure prediction","deep reinforcement learning","reservoir computing","LSTM","multi-head attention","deep Q-learning","self-avoiding walk"],"falsifier":"Train FFNN-R and LSTM-A on held-out HP sequences not in the published benchmark, using the same one-hot state and hyperparameters, and compare final energies to exhaustive or best-known values. A clear failure on held-out sequences, or a measurable change in performance when the state is augmented with an explicit occupancy map, would show that the claimed optimality depends on implicit path memory rather than on a general folding policy.","tokens_in":11506,"feed_emoji":"🧬","tokens_out":9596,"duration_ms":84357,"temperature":0.7,"pith_summary":"This paper tries to establish that deep Q-learning, equipped with architectures chosen for temporal memory, can solve instances of the NP-complete 3D Hydrophobic-Polar (HP) protein folding problem as well as the best known specialized solvers while training more efficiently. For chains up to 36 residues, a reservoir-based feedforward network (FFNN-R) is claimed to reach the best-known energy values with about 25% fewer training episodes than a plain fully connected network. For longer chains, an LSTM with multi-head attention (LSTM-A) is claimed to match or closely approach best-known energies. If the claims hold, deep reinforcement learning becomes a practical, sample-efficient alternative to genetic algorithms and Monte Carlo methods on this benchmark.","feed_headline":"Reservoir and attention nets fold 3D HP proteins at best-known energies","feed_subtitle":"For short chains, a reservoir network needs 25% fewer training episodes; an attention-LSTM handles longer sequences.","key_machinery":"The load-bearing objects are the two Q-network architectures. FFNN-R applies a fixed random reservoir update $r(t) = f(W_{\\mathrm{in}} x(t) + W r(t-1))$ with a tanh activation and sparse random connectivity, projecting the 8-dimensional one-hot state into a high-dimensional temporal representation that is then passed through fully connected layers. LSTM-A runs the state sequence through an LSTM of hidden size 512 and pools the hidden states with four-head attention, $\\mathrm{MultiHead}(H) = \\mathrm{Concat}(\\mathrm{head}_1,\\dots,\\mathrm{head}_4)W_O$, before a final layer outputs per-action Q-values. Both are trained as DQNs with experience replay, a target network, and a smooth L1 loss, so the claimed gains come from the state representation and architecture rather than new reinforcement-learning machinery.","core_discovery":"The central claim is that the two architectures make Deep Q-Networks competitive with dedicated HP-model solvers: FFNN-R, a fully connected network fed by a fixed random reservoir, reaches best-known energies on most short benchmark sequences (A1–A5 and A8–A10) with roughly 25% fewer episodes than the vanilla FFNN baseline, and LSTM-A, an LSTM with multi-head attention, matches best-known energies on the 3d1–3d4 sequences and comes within a few hydrophobic contacts of best-known values on 3d5–3d9. The paper takes this as evidence that implicit temporal memory (reservoir) and long-range attention (LSTM-A) are the ingredients that let DQN scale on the 3D HP model.","pith_inferences":["The reported 25% episode reduction is measured against the paper's own vanilla FFNN, not against genetic algorithms, PERM, or other solvers, so the sample-efficiency claim should be read as architecture-to-architecture rather than method-to-method.","Because the state encoder omits lattice coordinates and the occupancy map, the learned policy may be capturing sequence-specific action statistics rather than general 3D geometry; adding an explicit occupancy-map channel is a natural test of this.","The observed specialization of attention heads into periodic and position-specific patterns hints that LSTM-A is learning secondary-structure-like motifs and long-range contacts separately, which could be exploited to seed folding on even longer chains.","A hybrid that lets FFNN-R propose short local fragments and LSTM-A assemble them could sidestep FFNN-R's degradation beyond length 36 and LSTM-A's quadratic memory cost, though the paper does not explore this."],"forward_implications":["For proteins under 36 residues, FFNN-R reaches best-known 3D HP energies with about 25% fewer training episodes than a vanilla fully connected DQN, making deep RL a faster route to optimal short-chain folds.","For chains of 46–60 residues, LSTM-A outperforms the plain FFNN and the LSTM that uses only its last hidden state, suggesting multi-head attention is what lets DQN capture long-range interactions.","The attention weights start specializing only after about 100,000 episodes, which implies the model first learns local folding patterns and then uses attention for global assembly.","Both architectures use the same stabilized DQN framework, so the efficiency difference is attributable to the two architectural changes, not to replay or target-network tricks.","Across the standard benchmark, reported final energies match best-known values on the short sequences and approach them on the longest ones, establishing a new sample-efficiency point for DRL-based HP folding."],"supporting_citations":[{"why":"Supplies the stabilized DQN recipe (experience replay, target network, epsilon-greedy) that both architectures train with.","marker":"[Mnih et al., 2015]"},{"why":"Introduces echo-state/reservoir computing, the basis of FFNN-R's fixed random projection layer.","marker":"[Jaeger, 2007]"},{"why":"Provides the multi-head attention mechanism used by LSTM-A to pool LSTM hidden states.","marker":"[Vaswani et al., 2017]"},{"why":"Formulates 3D HP folding as a self-avoiding-walk MDP and provides the prior DRL setup this work extends.","marker":"[Yang et al., 2023]"},{"why":"Supplies the benchmark sequences and best-known energy values used for all evaluations.","marker":"[Boumedine and Bouroubi, 2022]"},{"why":"Establishes the prior FoldingZero DRL approach in the HP model, the main deep-RL point of comparison.","marker":"[Li et al., 2018]"}],"fun_headline_variants":["Deep RL folds 3D HP proteins 25% faster with reservoir and attention nets","Reservoir and attention nets match best-known folds in 3D HP model","DQN folds 3D HP proteins with 25% fewer episodes and matching energies","Attention-LSTM and reservoir nets hit optimal 3D HP protein folds","Stabilized DQN with reservoir and attention matches best-known energy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 8-dimensional one-hot state, which records only the available actions and the current bead's H/P type, conveys enough information for the network to reconstruct the self-avoiding path from the history of actions; if this implicit memory is ambiguous, the reported optimality is tied to the specific benchmark sequences and will not transfer to new HP instances.","fun_headline_variants_meta":{"raw":{"variants":["Deep RL folds 3D HP proteins 25% faster with reservoir and attention nets","Reservoir and attention nets match best-known folds in 3D HP model","DQN folds 3D HP proteins with 25% fewer episodes and matching energies","Attention-LSTM and reservoir nets hit optimal 3D HP protein folds","Stabilized DQN with reservoir and attention matches best-known energy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000668,"raw_usage":{"total_tokens":2972,"prompt_tokens":795,"completion_tokens":2177,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":411,"completion_tokens_details":{"reasoning_tokens":2073}},"tokens_in":411,"tokens_out":2177,"duration_ms":14016,"temperature":1.0,"reasoning_tokens":2073,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:24:17.116234+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train FFNN-R and LSTM-A on held-out HP sequences not in the published benchmark, using the same one-hot state and hyperparameters, and compare final energies to exhaustive or best-known values. A clear failure on held-out sequences, or a measurable change in performance when the state is augmented with an explicit occupancy map, would show that the claimed optimality depends on implicit path memory rather than on a general folding policy.","supporting_citations":[],"review_version":1}