{"id":"aa8c860d-f757-469f-bae2-0477d853d256","arxiv_id":"2411.11318","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Syllabus provides a portable curriculum learning library with a unified API, reproduces prior baselines, and shows that standard automatic curricula do not transfer to NetHack and Neural MMO.","lead":"Syllabus is a portable library for adding curriculum learning to reinforcement learning agents, with a unified API and process-safe synchronization. The paper shows the library reproduces known curriculum results and applies it for the first time to NetHack and Neural MMO, where existing methods mostly fail to beat random task sampling.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"LP/SFL 'do not transfer' conclusion rests on a manually scaled episodic-return success proxy; if that proxy saturates, the negative-transfer claim is an artifact.","rationale":"The reader identified the same weakest assumption. I find it the most load-bearing because it targets the paper's novel empirical contribution, namely that existing methods do not transfer to NetHack/Neural MMO, rather than the library design itself. The portability claim is independently supported by the reproduction experiments (Fig. 3 matching reference PLR/LP/OMNI curves), the multi-library integrations, and the optimization measurements, so I would not reject or weaken the library claim. However, the 'do not transfer' conclusion is central to the abstract and contribution list, and it depends entirely on a success proxy that LP/SFL were never designed for. The manual scale makes the proxy unprincipled: in NetHack, scores below 1000 all map to sub-1 values and above 1000 all saturate at 1; in Neural MMO, the max agent return is chosen without exploring alternatives. Because LP's EMA-difference and SFL's p(1-p) both require a meaningful binary signal, the proxy can manufacture failure. The paper even flags this in Sec. 8, where it says value predictions might be a better approximation than the return-based success metric and leaves the investigation to future work. Thus the appropriate verdict remains CONDITIONAL: the portability contribution is credible, but the negative-transfer finding should be re-examined with a binary task-success signal before being accepted. The concrete test would settle whether the concern actually lands.","tokens_in":26458,"tokens_out":5205,"duration_ms":53386,"concrete_test":"Recompute the LP and SFL curves in Fig. 4c/4d using a true binary success signal, holding all other hyperparameters fixed: for NetHack, define success as reaching a specified dungeon depth or gaining an experience level within the episode; for Neural MMO, use the event/achievement thresholds already defined in Supp. E.6 (e.g., harvest/equip counts). If the relative ordering of LP/SFL versus DR changes, the negative-transfer claim is an artifact of the clipped-return proxy. As a cheaper preliminary check, use the existing W&B logs to measure the fraction of episode returns clipped to exactly 1.0 over training in each environment.","verdict_should_be":"UNCHANGED","load_bearing_attack":"LP (Kanitscheider et al. 2021) and SFL (Rutherford et al. 2024) require a per-task binary success signal; Syllabus substitutes scaled, clipped episodic return: min(max(R/1000,0),1) in NetHack and min(max(max_a(R)/10,0),1) in Neural MMO (D.4, D.3), with scales hand-picked from the environment's 'typical reward range.' This proxy is not a pass/fail indicator. LP's learning progress is the absolute difference between fast/slow EMAs of success; once mean episode return exceeds the hand scale, every task's proxy saturates at 1, so LP sees zero progress and SFL's p(1-p) collapses. That exactly matches the observed NetHack pattern of early acceleration followed by lower asymptotic performance, and it means the Sec. 8 conclusion that 'popular curriculum methods are far less effective outside their original environments' can be explained by metric saturation rather than by a property of the algorithms. The paper itself concedes in Sec. 8 that the return-based metric may be worse than value predictions and leaves investigation to future work, yet the negative-transfer contribution is already drawn from it. The portability/infrastructure claim is not affected, but the paper's main empirical contribution is.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Syllabus, a library that separates curriculum-learning logic from RL training code by providing a minimal API for task spaces, curricula, and environment wrappers, together with a separate multiprocessing synchronization pathway. The authors validate their implementations by reproducing PLR on Procgen and LP/OMNI on Crafter against reference codebases, then present new baselines across Procgen, Crafter, NetHack, and Neural MMO, concluding that popular automatic curriculum methods do not transfer well to complex new environments. The portability/infrastructure claim is central and supported by the reproduction experiments; the empirical negative-transfer claim is the paper's main new scientific conclusion.","tokens_in":26741,"tokens_out":3275,"duration_ms":33654,"significance":"If the portability claims hold, Syllabus fills a genuine gap: it is the first curriculum-learning library that works with multiple RL frameworks without entangling curriculum code into the training loop, and the authors provide machine-checked reproductions against external reference implementations, including exact matches for LP/OMNI on Crafter and close matches for PLR on Procgen. The new baselines and the first ACL experiments on NetHack and Neural MMO are useful resources for the community. However, the negative-transfer conclusion depends critically on a success proxy (scaled, clipped episodic return) that is not validated for LP and SFL, and the paper itself concedes that the proxy may be worse than value predictions. This makes the paper's main empirical contribution, as stated in Contribution 3, currently unsupported, even though the infrastructure contribution remains valuable.","major_comments":[{"comment":"The success metric used for LP and SFL on all environments except Crafter is defined as scaled, clipped episodic return rather than a binary task-completion signal. In NetHack the metric is min(max(R/1000,0),1), in Neural MMO min(max(max_a(R)/10,0),1), and in Procgen min(max(r_N,0),1). LP and SFL were designed for per-task pass/fail success rates: LP computes learning progress as the absolute difference between fast and slow EMAs of success, and SFL samples according to p(1-p). If the mean episode return exceeds the manually chosen scale, every task's proxy success saturates to 1, so LP reports zero progress and SFL's p(1-p) collapses to zero. The observed NetHack pattern of early acceleration followed by lower asymptotic performance is exactly what this saturation would produce. The paper does not validate that the proxy preserves the ordering of tasks by true learnability, so the negative-transfer conclusion in §8 is not established.","section":"§7.3 and Supplementary D.1, D.3, D.4"},{"comment":"The paper's headline empirical claim—that 'popular curriculum learning methods are far less effective outside of the environments in which they were originally developed'—is drawn from experiments in which LP and SFL use the return-based proxy. The same section concedes: 'In their absence, value predictions might be a better approximation of competence than our return-based success metric defined in subsection 7.3. We leave a thorough investigation of this inconsistency as future work.' This admission applies directly to the central empirical contribution. The portability and reproduction claims are not affected, but the negative-transfer conclusion should be either re-derived using a validated binary success signal (e.g., actual task achievements or a clear task-completion predicate) or explicitly downgraded to a qualified observation about the proxy, not a general statement about the algorithms.","section":"§8, 'Overall' paragraph"},{"comment":"The statement that 'each of the libraries used in this paper have different design philosophies, software architectures, and multiprocessing implementations' is used to motivate the portability claim, but the paper does not quantify the integration effort (e.g., lines of code changed or time to integrate) beyond the illustrative code examples in Figure 2 and Supplementary F. A quantitative comparison of integration overhead against existing curriculum libraries (DCD, Minimax, JaxUED, TeachMyAgent) would substantially strengthen Contribution 1. As written, the 'few lines of code' claim is plausible but not directly measured.","section":"§7.3, last sentence"}],"minor_comments":[{"comment":"There is a typo: 'sequantual' should be 'sequential' in the sentence about the sequential curriculum.","section":"§5.5, 'sequential curricula'"},{"comment":"The phrase 'through it's synchronization wrappers' uses an incorrect apostrophe; it should be 'its synchronization wrappers'.","section":"Supplementary D.2, last paragraph"},{"comment":"The caption reads '95% Stratified Boostrapped Confidence Intervals'; 'Boostrapped' should be 'Bootstrap'.","section":"Figure 8 caption"},{"comment":"The mixing ratio hyperparameter is denoted p, which collides with the success probability p in the p(1-p) formula; using a distinct symbol such as ρ_mix would prevent confusion, especially since Supplementary D.5 later uses ρ for the SFL top-K sampling ratio.","section":"§5.5, SFL paragraph"},{"comment":"The sentence 'where R is the mean episodic return because our agents get close to but do not exceed 1000 mean episodic return' should specify whether R is the mean over all agents, over a single episode, or over a rollout; this is especially relevant because the proxy's saturation threshold determines whether the measured learning progress can ever become nonzero after the agent improves.","section":"Supplementary D.4"}],"recommendation":"major_revision","confidential_remarks":"The infrastructure contribution is solid and the reproductions are convincing, but the negative-transfer empirical conclusion is the manuscript's main advertised contribution and it rests on an unvalidated, hand-scaled success proxy. The authors explicitly concede in §8 that the proxy may be worse than value predictions, yet they still draw a general conclusion about the algorithms. This is a load-bearing issue that cannot be fixed by minor edits; the authors need to either validate the proxy against a binary success signal or substantially rephrase the contribution as a qualified report about the proxy. I would not reject outright because the library itself appears useful and the fix is within reach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"My take: the library is the real contribution. Syllabus is the first portable curriculum infrastructure that separates curriculum logic from RL training code and syncs curricula across processes through its own channel, and it demonstrably works with CleanRL, moolib, RLLib, and PufferLib. The reproductions of PLR on Procgen and LP/OMNI on Crafter match the reference implementations, which gives real credibility to the code. The first ACL baselines on NetHack and Neural MMO are genuinely new and useful to the community.\n\nThe soft spot is exactly where the stress-test note lands. The conclusion that LP and SFL \"do not transfer\" rests on a hand-scaled, clipped episodic-return proxy for what are supposed to be binary success signals. In NetHack the proxy is min(max(R/1000,0),1); in Neural MMO it is min(max(max_a(R)/10,0),1). Once the return crosses the hand-picked scale, every task's proxy saturates at 1, so LP's learning progress goes to zero and SFL's p(1-p) collapses. That matches the observed pattern of early acceleration followed by lower asymptotic performance. The paper concedes in Sec 8 that value predictions might be a better approximation than this return-based metric and leaves the investigation to future work, but the negative-transfer claim is already drawn from it. So the paper's main empirical contribution is not as solid as the library contribution. That needs to be fixed or softened before the claim is cited.\n\nSmaller issues: the Procgen subset of 10 environments is listed but not justified, and seed/map axes may not control difficulty in NetHack and Neural MMO. These are minor next to the proxy concern.\n\nOverall, the infrastructure is carefully engineered and the reproductions are honest. The negative-transfer conclusion is conditional on the success metric, and the paper mostly knows it. I would send this to a serious referee: the library deserves publication, and the empirical claims need scrutiny. I would cite the library, not the negative transfer claim.","headline":"The library and reproductions are a genuine contribution; the negative-transfer conclusions rest on a return-based success proxy that can saturate, and the paper itself flags this.","tokens_in":27264,"tokens_out":1566,"would_cite":true,"duration_ms":16206,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that curriculum learning can be separated from RL training code through a portable API, and that existing automatic curricula fail to transfer to NetHack and Neural MMO.","keywords":["curriculum learning","automatic curriculum learning","unsupervised environment design","portable RL infrastructure","reinforcement learning library","NetHack","Neural MMO","prioritized level replay"],"falsifier":"Run LP and SFL on NetHack and Neural MMO while replacing the manually scaled, clipped return proxy with the environments' native binary task-success events; if the methods then beat domain randomization or match their original-domain results, the paper's evidence that these methods do not transfer to complex environments would be an artifact of the proxy.","tokens_in":26265,"feed_emoji":"🧩","tokens_out":8849,"duration_ms":82062,"temperature":0.7,"pith_summary":"The paper is trying to establish that curriculum learning\\u2014deciding which tasks an agent practices\\u2014can be decoupled from the reinforcement-learning training loop and delivered as a portable layer. It claims that a small API plus a dedicated synchronization channel lets the same automatic-curriculum algorithms attach to almost any RL codebase with only a few lines of change, so researchers do not have to reimplement curricula for each library. To back this, the authors reimplement four popular methods, reproduce the results from the papers that introduced them, then run the same methods in two hard, previously unexplored environments. In those domains, none of the automatic curricula beat random task selection, and the paper reads this as evidence that existing curricula over level seeds do not transfer to complex long-horizon games.","feed_headline":"One small API adds auto-curricula to almost any RL training code","feed_subtitle":"Automatic task scheduling becomes portable across codebases; new NetHack and Neural MMO tests show where current methods fall short.","key_machinery":"The object that carries the argument is the Curriculum API together with its synchronization wrappers. A Curriculum maintains a sampling distribution and exposes update methods that can be called per step, per episode, or on demand; a TaskSpace maps arbitrary tasks into a uniform encoding; and a TaskWrapper lets an environment accept a new task at reset. The synchronization layer runs over its own communication channel, batching feedback and buffering task samples, so it does not depend on how the RL library passes messages between processes. That separation is what makes the same algorithm files portable across codebases, and it is also what lets the paper test one implementation of each method in several different training systems without rewriting the curriculum.","core_discovery":"The central discovery is a packaging claim with an empirical finding attached. The packaging claim: curriculum logic can live in a standalone Curriculum object that keeps a distribution over tasks, with environment wrappers that accept a new task at reset and a separate sender-receiver pathway that sends tasks outward and feedback back, so the RL training code only needs a few wrapper lines. The authors validate it by reproducing the behavior of Prioritized Level Replay, Learning Progress, and OMNI on their original benchmarks, then applying four automatic-curriculum methods\\u2014PLR, LP, OMNI, and Sampling for Learnability\\u2014to NetHack and Neural MMO as the first such baselines. The empirical finding: when curricula are restricted to selecting initial level seeds, none of the methods improve over domain randomization in those complex environments, and success-rate-based methods sometimes help early but hurt asymptotic performance. The paper concludes that curricula over initial environment conditions are not the right axis in long-horizon, multi-agent games.","pith_inferences":["A natural next experiment, not run in the paper, is to apply LP and SFL to Neural MMO using its built-in task predicates as the binary success signal rather than a clipped return; that would separate failure of the methods from failure of the proxy.","If the negative results generalize, the productive research direction is to make curricula select mid-episode objectives and opponent strategies rather than starting conditions, which the task-space abstraction in Syllabus is built to support.","The matching reproduction on original benchmarks suggests the portability claim is not bought at the price of correctness, so the library could become a neutral comparison layer for future curriculum-learning papers.","Because the paper tunes hyperparameters per environment with grid search, a cleaner default story might emerge from measuring how sensitive each method is to its hyperparameters, which the full-distribution variant of SFL already weakens."],"forward_implications":["Reproduction becomes cheap: any open-source training script can be turned into a curriculum experiment by wrapping the environment and attaching a curriculum object, with no changes to the policy optimizer.","Research comparisons become fairer: the same code for PLR, LP, OMNI, or SFL runs inside different RL codebases, so differences in results can be attributed to the training system or environment rather than to a reimplementation.","The negative NetHack and Neural MMO results challenge the assumption that curricula over level seeds are a general solution; new work should choose task axes like reward functions, objectives, or opponent skill.","The full-distribution variant of SFL, which removes two environment-dependent hyperparameters, is proposed as a strong default for exploring a new domain.","Syllabus supports sequential and joint task-and-opponent curricula, so a single training loop can graduate through staged tasks or simultaneously vary maps and co-players."],"supporting_citations":[{"why":"Introduces Prioritized Level Replay and its Procgen experimental setup, which Syllabus must reproduce before porting it.","marker":"[Jiang et al., 2021b]"},{"why":"Defines the Learning Progress curriculum whose success-rate feedback mechanism Syllabus reimplements.","marker":"[Kanitscheider et al., 2021]"},{"why":"Supplies the OMNI method and the Crafter benchmark code used to validate Syllabus's LP and OMNI implementations.","marker":"[Zhang et al., 2023]"},{"why":"Introduces Sampling for Learnability; Syllabus adds a full-distribution variant that removes its top-k hyperparameters.","marker":"[Rutherford et al., 2024]"},{"why":"Provides the NetHack Learning Environment, one of the two complex domains used to test portability and the transfer failure.","marker":"[K\\u00fcttler et al., 2020]"},{"why":"Provides the Neural MMO competition baseline and predicate task space that Syllabus integrates with.","marker":"[Suarez et al., 2024]"},{"why":"Exemplifies the single-file RL implementation style that motivates Syllabus's design philosophy and serves as an integration target.","marker":"[Huang et al., 2022]"},{"why":"Supplies the asynchronous distributed RL platform used to train NetHack agents with Syllabus.","marker":"[Mella et al., 2022]"}],"fun_headline_variants":["Portable auto-curricula for RL? Only in simple worlds, says Syllabus","Syllabus brings curriculum learning to any RL code, but real games push back","New RL library demystifies auto-curricula, revealing limits in complex games","Auto-curriculum portability in RL: easy API, but hard benchmarks humble it","Syllabus: universal curricula API, yet NetHack and MMO defy auto-scheduling"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"That a scaled and clipped episodic return is a fair stand-in for the binary task-success signal that Learning Progress and Sampling for Learnability are designed to consume.","fun_headline_variants_meta":{"raw":{"variants":["Portable auto-curricula for RL? Only in simple worlds, says Syllabus","Syllabus brings curriculum learning to any RL code, but real games push back","New RL library demystifies auto-curricula, revealing limits in complex games","Auto-curriculum portability in RL: easy API, but hard benchmarks humble it","Syllabus: universal curricula API, yet NetHack and MMO defy auto-scheduling"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000298,"raw_usage":{"total_tokens":1722,"prompt_tokens":942,"completion_tokens":780,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":673}},"tokens_in":558,"tokens_out":780,"duration_ms":8055,"temperature":1.0,"reasoning_tokens":673,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:39:46.042554+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run LP and SFL on NetHack and Neural MMO while replacing the manually scaled, clipped return proxy with the environments' native binary task-success events; if the methods then beat domain randomization or match their original-domain results, the paper's evidence that these methods do not transfer to complex environments would be an artifact of the proxy.","supporting_citations":[{"cited_title":"No regrets: Investigating and improving regret approximations for curriculum discovery","cited_arxiv_id":null,"evidence_quote":"Introduces Sampling for Learnability; Syllabus adds a full-distribution variant that removes its top-k hyperparameters."},{"cited_title":"Neural mmo 2.0: A massively multi-task addition to massively multi-agent learning","cited_arxiv_id":null,"evidence_quote":"Provides the Neural MMO competition baseline and predicate task space that Syllabus integrates with."}],"review_version":1}