{"id":"633e7380-2243-40e4-ad06-ca27f44a5df7","arxiv_id":"2505.07854","paper_version":1,"verdict":"REJECT","confidence":"LOW","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A curriculum learning method that co-evolves training tasks with agents improves cooperative multi-agent reinforcement learning under sparse rewards, according to experiments on MPE and Hide-and-Seek.","lead":"This paper introduces CCL, a training method that automatically creates easier practice tasks for teams of reinforcement learning agents and raises difficulty as they improve. It claims faster and more reliable learning in cooperative multi-agent games with sparse rewards, though the provided manuscript omits key results and equations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core algorithm and its reported results are both absent from the manuscript: the task encoding/crossover equations in Section 4 and Tables 1–2 in Section 5 are missing, so the claim of >95% success cannot be checked.","rationale":"The reader's REJECT is appropriate: the manuscript, as supplied, lacks the technical specification and the empirical tables needed to evaluate the central claim. My stress-test confirms the reader's weakest assumption about task encoding and extends it: Section 4 is not merely missing a proof of feasibility; the relevant equations themselves are absent. I do not see an internal contradiction in the described framework—there is too little formalism to contradict—but the absence of both the algorithm definition and the result tables makes 'CCL consistently outperforms competitors' untestable. No author misconduct is implied; the verdict is about insufficient evidence. The concern would be resolved by supplying the missing definitions and data, after which the paper should be re-reviewed.","tokens_in":6803,"tokens_out":3666,"duration_ms":37311,"concrete_test":"Independently re-derive Section 4 from the prose alone: specify the task vector space for MPE and HnS, the crossover operator (with the missing equations filled in only from stated text), and the KNN fitness formula. If any of these requires inventing an arbitrary encoding, direction rule, or distance metric, the algorithm is underspecified and the empirical claim cannot be attributed to CCL; if the derivation is uniquely determined, run the resulting algorithm for three seeds on 'simple propagation' and 'lock-back' and check whether median success matches the claimed >95%.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that CCL beats MAPPO, POET, GC, GoalGAN, and VACL, with over 95% success on the hardest tasks (Section 5.1). For that claim to hold, CCL must be a fully specified algorithm and the comparison must be supported by data. Section 4 is supposed to provide the specification, but it does not: §4.1 references 'the encoding method mentioned earlier' without defining the task vector space, gives placeholders where the crossover step/direction equations should be, and never states the sigmoid fitness formula; §4.2 describes a KNN fitness estimator but the distance formula and query/prototype definitions are missing. Consequently the variational evolutionary operator and the prototype evaluator are underdetermined: a reader cannot tell how a crossover vector maps to a valid MPE or HnS task, whether offspring tasks remain in the environment bounds, or what metric the KNN estimate uses. The empirical support is equally absent: Section 5.1 cites Tables 1 and 2, but the manuscript contains no table entries, no standard deviations, and only three seeds. The load-bearing condition—that CCL is a concrete, reproducible method with measurable gains—is not met by the text as provided.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Collaborative Curriculum Learning (CCL), a curriculum learning framework for cooperative multi-agent reinforcement learning with sparse rewards. It combines (i) a variational evolutionary operator with individual-perspective crossover to generate intermediate tasks, (ii) a sigmoid-shaped fitness function based on agent success rates, (iii) KNN-based prototype fitness evaluation to reduce computational cost, and (iv) MAPPO as the underlying MARL algorithm. The authors claim that CCL consistently outperforms MAPPO, POET, GC, GoalGAN, and VACL on five cooperative tasks in the MPE and Hide-and-Seek environments, achieving over 95% success on the most complex tasks, with additional ablations on the adaptive mutation step and the non-linear fitness function.","tokens_in":7091,"tokens_out":5434,"duration_ms":52376,"significance":"If substantiated, the paper would make a useful contribution to sparse-reward MARL: agent-specific task evolution, curricula centered on moderate difficulty, and cheap fitness estimation are plausible and relevant ideas. The explicit rationale for targeting a 50% success rate and the soft-selection strategy for retaining historical tasks are sensible design choices. However, the evaluation is impossible to carry out from the submitted text: the task encoding and crossover equations are missing, the KNN estimator is unspecified, and the experimental tables and figures are absent. The significance of the claimed results cannot be assessed until the missing technical content is supplied, so any assessment of the contribution must remain conditional.","major_comments":[{"comment":"The task encoding is never defined. The sentence \"based on the encoding method mentioned earlier\" refers to no earlier encoding, and the formulas for the crossover step size and direction are placeholders rather than equations. The sigmoid fitness function is described in prose but its mathematical form is absent. Since these components define how new tasks are generated and selected, the proposed CCL algorithm is under-specified and cannot be implemented or checked, which is load-bearing for the paper's central claim.","section":"§4.1, Variational Individual-perspective Evolutionary Operator"},{"comment":"The KNN fitness estimator is presented without the distance formula or explicit definitions of the prototype set, query set, and neighborhood index set. The text says \"can be calculated as shown below,\" but no equation follows. Without these definitions, the prototype-based evaluation step cannot be reproduced, and the computational-cost argument for this component cannot be verified.","section":"§4.2, Elite Prototype Fitness Evaluation"},{"comment":"Tables 1 and 2 are referenced to support the central claims of \"over 95% success\" and consistent outperformance, but the tables are not present in the manuscript. The results are averaged over only three seeds with no variance measures or statistical tests, which cannot support the phrase \"consistently outperforms\" for either learning speed or final performance. Baseline implementations are described only as \"same setup\" with no hyperparameters, so the comparison is not reproducible.","section":"§5.1, Main Result"},{"comment":"Figures 3 and 4, which are supposed to show the ablation results for the adaptive mutation step and the sigmoid fitness function, do not appear in the manuscript. The corresponding claims rest entirely on missing visual evidence and cannot be evaluated.","section":"§5.2, Ablation Studies"}],"minor_comments":[{"comment":"The method is called \"Collaborative Multi-dimensional Course Learning\" in the abstract, \"Collaborative Curriculum Learning\" in the title, and \"CMCL\" in Section 4; the acronym should be standardized.","section":"Abstract / Title"},{"comment":"Several equations in the problem formulation are not rendered, leaving blank spaces after \"where\" and before punctuation, which makes the RL and MARL objectives hard to follow.","section":"§2, Problem Statement"},{"comment":"The statement that an n-agent system yields 2^n direction combinations assumes each agent's direction is binary; this assumption should be stated explicitly when the direction sampling is introduced.","section":"§4.1, Variational Individual-perspective Crossover"},{"comment":"The environment descriptions are terse: \"simple/complex propagation\" and \"push-ball\" are not standard MPE task names, and the Hide-and-Seek task variants are not defined. Please provide environment definitions or citations with task specifications.","section":"§5.1, Main Result"},{"comment":"The reference list includes several entries numbered [44]-[51] that do not appear to support the surrounding technical claims; the authors should review whether these citations are necessary.","section":"References"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an incomplete draft: key equations in Section 4 and all result tables and figures in Section 5 are missing. The missing material is not a presentation issue but constitutes the entire evidence for the central claims. In addition, the reference list contains many recent preprints and self-citations that are not clearly connected to the content. I would advise the editor that this submission is not ready for review. I do not see the circularity concern as a difficulty, since using the agent's own success rate to define task difficulty is a standard adaptive-curriculum design; the block is the absence of specification and evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a plausible idea wrapped in an incomplete manuscript. The core equations for the crossover and the KNN estimator are missing from Section 4, and Tables 1 and 2 are empty placeholders, so the headline claim of over 95% success on the hard tasks is currently unsupported. I'd send it back for completion, not to reviewers.\n\nWhat is actually new: the combination of individual-perspective crossover (operating on each agent's subtask direction) with a prototype-based KNN fitness estimator for co-evolved curriculum tasks is not in the cited POET/GoalGAN/VACL line, as far as I can tell. The soft-selection step that retains a fraction of historical tasks to avoid catastrophic forgetting is sensible, and the ablation design (adaptive mutation step, sigmoid fitness) is reasonable. The problem is well motivated: sparse-reward cooperative MARL is a real headache, and adaptive task evolution is a legitimate response.\n\nWhere it falls down: the manuscript is not self-contained. Section 4.1 refers to 'the encoding method mentioned earlier' without ever defining the task vector space, and the formulas for crossover step and direction are absent. Section 4.2 describes a KNN fitness estimator but no distance metric or prototype/query definitions. That is not a minor omission; the method is underspecified to the point where a reader cannot reimplement it. The empirical section is equally bare: no actual results in Tables 1 and 2, no standard deviations, and only three seeds. The claim that CCL 'consistently outperforms' baselines cannot be checked. There is also an acronym inconsistency (CCL vs CMCL) and a block of unrelated self-citations [44-54] that should be cut. None of this is fatal to the underlying idea, but it is fatal to the paper as submitted.\n\nWho this is for: someone working on curriculum learning for multi-agent RL might find the conceptual combination worth a quick read, but they should wait for a completed version. The paper does not deserve a serious referee at this point; it should be returned to the authors to fill in the missing content. If the equations and tables are restored and the comparisons hold up with variance, then it becomes a reasonable incremental contribution worth reviewing.","headline":"Plausible co-evolutionary curriculum idea, but the manuscript omits the core equations and all result tables, so the headline claims are currently unverifiable.","tokens_in":7596,"tokens_out":3458,"would_cite":false,"duration_ms":32209,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A co-evolutionary curriculum is reported to lift multi-agent success past 95% on sparse-reward tasks.","keywords":["curriculum learning","multi-agent reinforcement learning","sparse rewards","co-evolutionary algorithms","task generation","evolutionary reinforcement learning","MAPPO","cooperative agents"],"falsifier":"Take an environment where the task-to-vector map is chosen adversarially, for example task parameters that do not preserve difficulty order under Euclidean distance, and run CCL; if the evolved task population contains a large fraction of infeasible or misordered tasks, or if KNN-predicted fitness disagrees sharply with measured success rates on held-out tasks, the encoding assumption is falsified.","tokens_in":6600,"feed_emoji":"🧩","tokens_out":5452,"duration_ms":52074,"temperature":0.7,"pith_summary":"Sparse-reward cooperative multi-agent tasks are hard because feedback arrives only at goal completion, so agents get little signal about what to try. CCL claims this bottleneck can be removed by evolving the training curriculum itself: a variational evolutionary operator generates intermediate tasks tailored per agent, a sigmoid-shaped fitness score keeps tasks at intermediate difficulty, and a prototype-plus-KNN estimator keeps the cost of evaluating a large task population low. Training agents with MAPPO on these evolved tasks is reported to reach over 95% success on the most complex tasks in MPE and Hide-and-Seek, while learning faster than MAPPO, POET, GC, GoalGAN, and VACL. If correct, automatic curriculum co-evolution replaces hand-designed reward shaping and manual task sequencing in cooperative sparse-reward settings.","feed_headline":"Co-evolved task curricula push sparse-reward teams past 95%","feed_subtitle":"CCL trains agents and tasks together, so intermediate challenges stay matched to current skill and success climbs faster.","key_machinery":"The load-bearing object is the co-evolutionary curriculum loop between a task population and a MAPPO-trained agent team. Within it, the 'variational individual-perspective crossover' forms child tasks by recombining pairs of parent tasks along per-agent directions, yielding $2^n$ direction combinations for $n$ agents so individual strategy imbalances drive task diversity. Fitness is assigned by a sigmoid function of the agents' success rate $r$ on a task, favoring tasks near $r=0.5$ and down-weighting near $0$ or $1$; a small set of prototype tasks is actually trained, and K-nearest-neighbor regression in task-vector space estimates fitness for all other tasks, keeping the population large without the cost of training on every task. This mechanism is what lets task difficulty track agent skill automatically.","core_discovery":"The paper's central claim is that task and agent should be trained as a co-evolving pair in sparse-reward cooperative MARL. The task population is encoded as vectors; crossover acts per-agent ('variational individual-perspective crossover') so that each agent's subtask direction contributes equally, and an adaptive mutation step adjusts task vectors to match current agent ability. Task fitness is a sigmoid of the agents' average success rate, peaking in usefulness at moderate difficulty, and fitness for unsampled tasks is estimated by KNN from a small set of 'prototype' tasks that are actually trained on. Across five cooperative tasks, the paper reports consistent outperformance in learning speed and final performance, with over 95% success on the hardest tasks, and ablations attribute the gain to each component.","pith_inferences":["The authors leave it implicit that CCL should transfer to any cooperative task whose goals are parameterizable; a direct test would be applying it to continuous-control cooperative manipulation with target-position task vectors.","Because task fitness is derived from a 0-1 success rate, adapting the sigmoid fitness to shaped or dense rewards is non-trivial; a testable extension would define the midpoint of the sigmoid from reward thresholds rather than success rate.","The KNN estimator assumes smooth fitness in task space; an experiment that compares estimated and true fitness for random query tasks would show how far the population size can be stretched.","Co-evolution may produce curriculum cycles where tasks stop getting harder as agents improve; logging difficulty trajectories would reveal whether the adaptive mutation step avoids such collapse."],"forward_implications":["Sparse-reward cooperative tasks no longer need hand-designed dense rewards or manual curriculum ordering; the evolution loop supplies both.","Training speed improves because agents begin on agent-specific easy tasks and are promoted only as success rates rise.","Large task populations become computationally affordable, since only prototype tasks are executed while KNN estimates the rest.","The per-agent crossover direction can balance strategy development when a single shared reward cannot tell who contributed what.","The framework transfers across MPE and Hide-and-Seek style tasks, suggesting applicability to other cooperative MARL benchmarks."],"supporting_citations":[{"why":"Supplies the MAPPO backbone used to train the agent team on each intermediate task.","marker":"[35]"},{"why":"Provides the MPE benchmark tasks (simple/complex propagation, Push-ball) used in the main experiments.","marker":"[40]"},{"why":"Provides the Hide-and-Seek environment with the ramp-passing and lock-back tasks.","marker":"[41]"},{"why":"POET baseline; the task-evolution lineage CCL extends and the comparison for learning speed and final performance.","marker":"[24]"},{"why":"GoalGAN baseline; automatic goal generation approach CCL is compared against.","marker":"[10]"},{"why":"GC baseline; improved task generation variant used as a comparison.","marker":"[36]"},{"why":"VACL baseline; variational curriculum method that CCL compares against.","marker":"[43]"}],"fun_headline_variants":["Co-evolving tasks and agents crack sparse-reward team games","Task-agent co-evolution pushes sparse-reward success past 95%","Curriculum co-design lifts sparse-reward multi-agent learning","Co-evolutionary curricula beat sparse rewards in MAS"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method leans on the belief that a task can be encoded as a vector and that crossover, mutation, and KNN distances on those vectors meaningfully interpolate between real training tasks; if the encoding is unrepresentative, the curriculum silently evolves invalid or misordered tasks and the reported gains do not follow.","fun_headline_variants_meta":{"raw":{"variants":["Co-evolving tasks and agents crack sparse-reward team games","Task-agent co-evolution pushes sparse-reward success past 95%","Curriculum co-design lifts sparse-reward multi-agent learning","Co-evolutionary curricula beat sparse rewards in MAS"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000663,"raw_usage":{"total_tokens":2957,"prompt_tokens":804,"completion_tokens":2153,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":420,"completion_tokens_details":{"reasoning_tokens":2085}},"tokens_in":420,"tokens_out":2153,"duration_ms":16296,"temperature":1.0,"reasoning_tokens":2085,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:17:29.770662+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take an environment where the task-to-vector map is chosen adversarially, for example task parameters that do not preserve difficulty order under Euclidean distance, and run CCL; if the evolved task population contains a large fraction of infeasible or misordered tasks, or if KNN-predicted fitness disagrees sharply with measured success rates on held-out tasks, the encoding assumption is falsified.","supporting_citations":[],"review_version":1}