{"id":"d85f52fb-4950-4c5d-84f6-61dd1a61848d","arxiv_id":"2605.24220","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Polar provides a scalable asynchronous RL rollout framework that works with arbitrary agent harnesses via black-box proxying and trajectory reconstruction, demonstrated by performance gains on SWE-Bench Verified across multiple coding harnesses.","lead":"Polar is a rollout framework that lets reinforcement learning run on any agent harness by treating the harness as a black box, proxying LLM calls, and reconstructing token-level trajectories. This decoupled design could reduce the engineering effort needed to apply RL to complex, long-running agent systems such as software-engineering agents.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"Reader's weakest assumption is the correct load-bearing point in principle, but the full manuscript's empirical results and ablations address it directly, removing the need to treat the assumption as unsecured. Verdict therefore stays UNVERDICTED only because of the original abstract-only limitation; the substance of the claim now has concrete backing.","tokens_in":1807,"tokens_out":259,"duration_ms":32033,"concrete_test":"Re-run the GRPO training loop on the Qwen Code harness using the exact reconstruction variant reported as best in the ablation; compare final SWE-Bench Verified score and trajectory-level log-probability match against the published numbers. A >2-point drop would indicate material signal loss.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on proxying LLM calls and reconstructing token-faithful trajectories inside arbitrary harnesses without material loss of training signal. The manuscript supplies empirical support via consistent gains on SWE-Bench Verified across four distinct harnesses plus explicit ablations of reconstruction strategies; these results directly test whether the black-box treatment suffices for GRPO. No internal inconsistency or unstated assumption that would invalidate the reported improvements is apparent from the provided text.","agreement_with_reader":"disagree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces Polar, a rollout framework for scalable asynchronous RL over arbitrary agent harnesses. It treats harnesses as black boxes by proxying LLM API calls, recording token-level interactions, and reconstructing token-faithful trajectories for training. Using GRPO, it reports improvements of 22.6, 4.8, 0.6, and 6.2 points on SWE-Bench Verified for Qwen3.5-4B across the Codex, Claude Code, Qwen Code, and Pi harnesses, respectively, along with demonstrations of offline data generation and ablations of reconstruction strategies.","tokens_in":1874,"tokens_out":308,"duration_ms":34063,"significance":"If the empirical results hold under rigorous evaluation, Polar would meaningfully lower the barrier to applying RL to complex, long-running agent systems by decoupling harness logic from training infrastructure. The multi-harness validation and explicit ablations directly test the viability of the black-box proxy approach, which is a practical strength.","major_comments":[{"comment":"Results section (and abstract): the reported point improvements on SWE-Bench Verified are presented without error bars, number of runs, statistical significance tests, baseline details, or controls for harness-specific variance. These omissions are load-bearing because the gains constitute the primary evidence that proxying and trajectory reconstruction preserve sufficient training signal for effective GRPO.","section":"Results"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback and for recognizing the potential of Polar to lower barriers for RL on complex agent systems. We address the major comment on results presentation below and commit to revisions that strengthen the empirical evidence.","responses":[{"response":"We agree that the absence of these details weakens the primary claims. In the revised manuscript we will: (1) report the number of independent training runs (minimum three seeds per harness), (2) add error bars (standard deviation across runs) to all reported point improvements, (3) include statistical significance tests (e.g., paired t-tests or Wilcoxon) comparing Polar-trained agents against the corresponding baselines, (4) expand baseline descriptions to clarify the exact harness configurations and evaluation protocols used, and (5) add explicit controls or discussion of harness-specific variance (e.g., by reporting per-harness variance and any normalization steps). These changes will appear in both the results section and the abstract.","revision_made":"yes","referee_comment":"[Results] Results section (and abstract): the reported point improvements on SWE-Bench Verified are presented without error bars, number of runs, statistical significance tests, baseline details, or controls for harness-specific variance. These omissions are load-bearing because the gains constitute the primary evidence that proxying and trajectory reconstruction preserve sufficient training signal for effective GRPO."}],"tokens_in":1346,"tokens_out":297,"duration_ms":18822,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"Polar treats the harness as a black box: it proxies LLM API calls, records the interactions, and rebuilds token-faithful trajectories for training. The decoupled rollout nodes handle prewarming, execution, reconstruction, and evaluation in parallel, then expose async endpoints for separate trainers. This setup is meant to work with any harness and any RL algorithm.\n\nWhat is new is the explicit black-box proxy plus reconstruction pattern that avoids rewriting the harness itself. They test it on four coding harnesses (Codex, Claude Code, Qwen Code, Pi) using simple GRPO on Qwen3.5-4B and report lifts of 22.6, 4.8, 0.6, and 6.2 points on SWE-Bench Verified. They also show offline data generation over custom harnesses and ablate reconstruction strategies. The registration as a NeMo Gym environment is a small practical signal.\n\nThe results are the strongest part. Consistent gains across distinct harnesses support the claim that the proxying preserves enough signal for GRPO to work. The parallel node design directly targets the scaling issues with long-running agent workloads.\n\nThe gains vary sharply by harness, so the benefit is not uniform. This work rewrites their earlier Prorl Agent system, so the advance is incremental rather than foundational. The abstract gives limited experimental detail, though the stress-test note indicates the full paper includes ablations that test the reconstruction step.\n\nThis paper is for engineers and researchers who already have complex agent harnesses and want to add RL without major rewrites. A reader focused on practical scaling of agent training would get concrete design ideas and benchmark numbers. It deserves a serious referee because it addresses a real integration bottleneck with working results on standard tasks.\n\nI would send it to peer review.","headline":"Polar gives a practical black-box way to run RL on top of existing agent harnesses by proxying calls and reconstructing trajectories, with uneven but positive gains on SWE-Bench.","tokens_in":2394,"tokens_out":446,"would_cite":false,"duration_ms":36388,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Polar lets any agent harness run scalable RL by proxying LLM calls and rebuilding token-faithful trajectories.","keywords":["agentic RL","rollout framework","black-box harness","trajectory reconstruction","SWE-Bench","asynchronous training","GRPO","language agents"],"falsifier":"A side-by-side comparison on the same harness and model where native RL integration produces higher final performance or faster convergence than Polar-mediated training.","tokens_in":2714,"feed_emoji":"🛠️","tokens_out":675,"duration_ms":26412,"temperature":0.7,"pith_summary":"Polar introduces a rollout framework that decouples agent harnesses from RL training so existing harnesses can be used without porting or modification. The system proxies LLM API calls inside the harness, records the interactions at token level, and reconstructs complete trajectories that trainers can consume asynchronously. This design improves compute utilization on long-running agent workloads while remaining agnostic to the specific harness, infrastructure, or RL algorithm. On software-engineering tasks, the approach yields measurable gains on SWE-Bench Verified when training a 4B model with simple GRPO across four different coding harnesses.","feed_headline":"Any agent harness works for scalable RL via API proxy","feed_subtitle":"Polar reconstructs trajectories from proxied calls, lifting a 4B model by up to 22.6 points on SWE-Bench Verified across four harnesses","key_machinery":"Black-box proxy of LLM API calls plus token-level trajectory reconstruction that turns an arbitrary harness into a source of training data without internal changes.","core_discovery":"Polar treats the agent harness as a black box: it proxies LLM API calls, records token-level model interactions, and reconstructs token-faithful trajectories for training. Each rollout node handles runtime prewarming, agent execution, trajectory reconstruction, and evaluation in parallel, exposing asynchronous service endpoints that independent trainers can consume at scale. Using this mechanism with GRPO, the framework improves Qwen3.5-4B by 22.6, 4.8, 0.6 and 6.2 points on SWE-Bench Verified with the Codex, Claude Code, Qwen Code and Pi harnesses respectively.","pith_inferences":["The approach could let researchers train agents on closed-source or proprietary harnesses where source access is unavailable.","If reconstruction fidelity holds, the same proxy pattern might apply to non-coding agent domains that already have mature harnesses.","Decoupling could reduce duplication of effort in the community by letting one harness serve both evaluation and RL training pipelines."],"forward_implications":["Existing coding harnesses can be used for RL without rewriting them as RL environments.","Rollout nodes can be scaled independently of trainers for better utilization on long-running tasks.","The same framework supports both online RL and offline data generation over custom harnesses.","Simple algorithms like GRPO become viable across multiple harnesses without harness-specific tuning."],"fun_headline_variants":["Polar scales RL on any agent harness via proxy","RL training works on arbitrary harnesses with Polar","Token trajectories from any harness enable scalable RL","Polar proxies calls for RL on diverse agent harnesses"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Proxying the LLM calls inside an arbitrary harness and reconstructing trajectories from those calls preserves every training signal required for effective RL.","fun_headline_variants_meta":{"raw":{"variants":["Polar scales RL on any agent harness via proxy","RL training works on arbitrary harnesses with Polar","Token trajectories from any harness enable scalable RL","Polar proxies calls for RL on diverse agent harnesses"]},"model":"grok-4.3","cost_usd":0.004479,"raw_usage":{"total_tokens":2277,"prompt_tokens":756,"num_sources_used":0,"completion_tokens":57,"cost_in_usd_ticks":44787000,"prompt_tokens_details":{"text_tokens":756,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1464,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":756,"tokens_out":57,"duration_ms":18752,"temperature":1.0,"reasoning_tokens":1464,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T14:35:09.899657+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A side-by-side comparison on the same harness and model where native RL integration produces higher final performance or faster convergence than Polar-mediated training.","supporting_citations":[],"review_version":1}