arxiv: 2604.18847 · v1 · submitted 2026-04-20 · 💻 cs.AI · cs.CL

Recognition: unknown

Human-Guided Harm Recovery for Computer Use Agents

Andi Peng, Andreea Bobu, Christy Li, Sky CH-Wang

Authors on Pith no claims yet

Pith reviewed 2026-05-10 04:09 UTC · model grok-4.3

classification 💻 cs.AI cs.CL

keywords harm recoverycomputer-use agentsreward modelhuman preferencespairwise judgmentsBackBenchagent safetyrecovery trajectories

0 comments

The pith

A reward model based on human pairwise preferences produces higher-quality harm recovery trajectories for computer-use agents.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to solve the problem of recovering from harmful actions by AI agents that control computer systems, rather than only preventing them. It conducts a user study to capture human preferences for recovery strategies through 1,150 pairwise comparisons, uncovering that people often favor pragmatic, context-specific fixes over broad long-term solutions. These insights inform a reward model that evaluates and selects among possible recovery plans generated by the agent. Evaluation on a new benchmark called BackBench with 50 tasks shows that this approach leads to recovery trajectories preferred by humans over those from unmodified agents or simpler rubric-based guidance. This matters for making agent systems safer in practice by addressing mistakes after they occur in ways that match human values.

Core claim

We formalize harm recovery as optimally steering an agent from a harmful state back to a safe one aligned with human preferences. A formative user study produces a natural language rubric from context-dependent preferences in 1,150 pairwise judgments. These are operationalized in a reward model that re-ranks candidate recovery plans at test time. On the BackBench benchmark of 50 computer-use tasks, human evaluation confirms that the reward model scaffold produces higher-quality recovery trajectories than base agents and rubric-based scaffolds.

What carries the argument

The reward model scaffold, which uses learned human preferences to re-rank multiple candidate recovery plans generated by an agent at test time.

Load-bearing premise

The context-dependent preferences identified in the 1,150 pairwise judgments will generalize to the diverse harmful states encountered in real computer-use scenarios and the 50 BackBench tasks.

What would settle it

A follow-up study applying the reward model to harmful states not represented in the original user study data and measuring whether human evaluators still rate its trajectories as superior to the baselines.

Figures

Figures reproduced from arXiv: 2604.18847 by Andi Peng, Andreea Bobu, Christy Li, Sky CH-Wang.

**Figure 1.** Figure 1: Harm Recovery for Computer Use Agents. An agent installs a seemingly routine software update that turns out to be malicious, leading to system compromise. Several recovery options (e.g., system restore, antivirus sweep, network quarantine) illustrate the challenge of weighting different attributes people consider (e.g. effectiveness, efficiency, communication) when choosing strategies that effectively re… view at source ↗

**Figure 2.** Figure 2: Agent Scaffold. Our agent scaffolds take a generate-and-verify approach whereby at test time LMgen generates n sample recovery plans and LMver evaluates the candidate recovery plans and selects one to be executed. (Top) The rubric-based verifier uses a frozen LM to perform pairwise A/B judgments on candidate plans according to a rubric of human preferences in harm remediation scenarios distilled. (Bottom) … view at source ↗

**Figure 3.** Figure 3: Backtracking in action. In this BACKBENCH scenario, the agent mistakenly shares a Google Sheets file containing sensitive employee information in the public general channel of a mock Slack interface instead of the intended private accounting-internal channel. To remediate, the agent deletes the misplaced message, verifies through both Slack and Google Sheets that only the accounting team retains access, a… view at source ↗

**Figure 4.** Figure 4: BackBench. BACKBENCH consists of 50 diverse computer use tasks where an agent begins in a harmful scenario and must backtrack and/or remediate various aspects of the starting scenario to return to a operational safe state. The tasks are spread across five macrocategories of harm—availability, financial, integrity, deliberate misuse, and security—and incorporate different step limits (15-step and 50-step) … view at source ↗

**Figure 5.** Figure 5: Human Evaluations. We compute Bradley-Terry ratings based on human-annotated A/B preference data for each method pairing between Reward model, Rubric-based, and Base model. We show the results of the evaluations over all tasks in BACKBENCH as well as for 15-step limit only and 50-step limit only tasks. We find that both scaffolds are strongly preferred over Base model, with Reward model achieving a 120-poi… view at source ↗

read the original abstract

As LM agents gain the ability to execute actions on real computer systems, we need ways to not only prevent harmful actions at scale but also effectively remediate harm when prevention fails. We formalize a solution to this neglected challenge in post-execution safeguards as harm recovery: the problem of optimally steering an agent from a harmful state back to a safe one in alignment with human preferences. We ground preference-aligned recovery through a formative user study that identifies valued recovery dimensions and produces a natural language rubric. Our dataset of 1,150 pairwise judgments reveals context-dependent shifts in attribute importance, such as preferences for pragmatic, targeted strategies over comprehensive long-term approaches. We operationalize these learned insights in a reward model, re-ranking multiple candidate recovery plans generated by an agent scaffold at test time. To evaluate recovery capabilities systematically, we introduce BackBench, a benchmark of 50 computer-use tasks that test an agent's ability to recover from harmful states. Human evaluation shows our reward model scaffold yields higher-quality recovery trajectories than base agents and rubric-based scaffolds. Together, these contributions lay the foundation for a new class of agent safety methods -- ones that confront harm not only by preventing it, but by navigating its aftermath with alignment and intent.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper carves out harm recovery as its own problem for computer-use agents, with a user-study rubric and BackBench benchmark, but the human eval's transfer from 1,150 judgments to the 50 tasks is the part that needs more checking.

read the letter

The main thing to know is that this work treats fixing harm after an agent has already acted on a computer as a distinct, solvable step rather than just hoping prevention works. They ran a formative study to surface what people actually value in recovery, turned the 1,150 pairwise judgments into a natural-language rubric that captures context shifts (pragmatic fixes sometimes beat comprehensive ones), trained a reward model on that, and used it to re-rank candidate plans at test time. They also released BackBench, a 50-task set focused on recovery scenarios, and report that human raters preferred the trajectories from their full scaffold over a plain agent or a rubric-only version.

Referee Report

2 major / 2 minor

Summary. The paper formalizes 'harm recovery' as the problem of steering LM-based computer-use agents from harmful post-execution states back to safe ones in alignment with human preferences. It reports a formative user study yielding 1,150 pairwise judgments that identify context-dependent recovery attributes and a natural-language rubric; these are used to train a reward model that re-ranks candidate recovery plans at test time. The authors introduce BackBench, a 50-task benchmark of harmful states, and present human evaluations claiming that the reward-model scaffold produces higher-quality recovery trajectories than base agents or rubric-only scaffolds.

Significance. If the empirical superiority claim holds under rigorous evaluation, the work would establish a new post-harm remediation paradigm for agent safety, complementing prevention-focused methods. The BackBench benchmark and the documented context-dependent preference shifts are potentially reusable contributions. The approach is grounded in human data rather than purely synthetic objectives, which strengthens its alignment claims relative to purely rule-based alternatives.

major comments (2)

[Human Evaluation] Human Evaluation section: the central superiority claim rests on human judgments of recovery quality, yet the manuscript provides no inter-rater agreement statistics, no power analysis or significance tests for the reported quality differences, and no explicit criteria or sampling procedure for the 50 BackBench tasks. Without these, selection bias or low reliability cannot be ruled out, directly undermining the load-bearing empirical result.
[§3 and §4] §3 (Formative Study) and §4 (Reward Model): the paper documents context-dependent preference shifts (pragmatic vs. comprehensive) across the 1,150 judgments, but does not report how the attribute weights or rubric were validated for transfer to the specific harmful states in BackBench. A mismatch between the judgment distribution and the benchmark distribution would make the re-ranking step misaligned, rendering observed gains non-generalizable.

minor comments (2)

[Abstract and §1] The abstract and introduction use the term 'Harm recovery' without an early formal definition or equation; a concise mathematical framing (e.g., as an optimization problem over trajectories) would improve clarity.
[Formative Study] Table or figure reporting the 1,150 judgments should include breakdown by context type and attribute to allow readers to assess the claimed shifts in preference.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback, which highlights important areas for strengthening the empirical rigor and generalizability of our claims. We address each major comment below and commit to revisions that will incorporate additional statistical reporting, task documentation, and validation analyses without altering the core contributions.

read point-by-point responses

Referee: [Human Evaluation] Human Evaluation section: the central superiority claim rests on human judgments of recovery quality, yet the manuscript provides no inter-rater agreement statistics, no power analysis or significance tests for the reported quality differences, and no explicit criteria or sampling procedure for the 50 BackBench tasks. Without these, selection bias or low reliability cannot be ruled out, directly undermining the load-bearing empirical result.

Authors: We agree that the absence of these details weakens the presentation of the human evaluation results. In the revised manuscript, we will add inter-rater agreement statistics (e.g., Fleiss' kappa or Krippendorff's alpha) computed over the multiple human judgments collected for each recovery trajectory. We will also include a post-hoc power analysis for the observed quality differences and report the results of appropriate statistical tests (such as paired Wilcoxon signed-rank tests) with effect sizes and confidence intervals. Finally, we will expand the BackBench section with explicit task selection criteria, sampling procedure, and a breakdown of task categories to allow readers to assess potential selection bias. revision: yes
Referee: [§3 and §4] §3 (Formative Study) and §4 (Reward Model): the paper documents context-dependent preference shifts (pragmatic vs. comprehensive) across the 1,150 judgments, but does not report how the attribute weights or rubric were validated for transfer to the specific harmful states in BackBench. A mismatch between the judgment distribution and the benchmark distribution would make the re-ranking step misaligned, rendering observed gains non-generalizable.

Authors: The formative study was intentionally broad to surface general, context-dependent recovery preferences rather than being tailored to a specific benchmark. That said, we acknowledge the value of demonstrating transfer. In the revision, we will add a new analysis subsection that compares the distribution of preferred attributes (pragmatic vs. comprehensive, etc.) from the 1,150 judgments against the attribute profiles of the 50 BackBench tasks. We will report any re-weighting or rubric adjustments applied at test time and discuss the degree of alignment, including limitations if mismatches exist. This will clarify the generalizability of the reward model. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical pipeline is self-contained

full rationale

The paper's chain proceeds from a formative user study (1,150 pairwise judgments yielding context-dependent preferences and a rubric) to a reward model that re-ranks agent-generated plans, followed by independent human evaluation on the newly introduced BackBench benchmark of 50 tasks. No equations, definitions, or self-citations equate any output (e.g., re-ranked trajectories or quality scores) to the study inputs by construction. The human evaluation step is external to the fitted reward model and does not reduce to a tautology or renamed fit. This is the standard non-circular structure for preference-learning papers.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The work is empirical and introduces the harm-recovery framing plus preference rubric as new constructs; relies on the assumption that pairwise human judgments capture stable, generalizable recovery preferences.

axioms (1)

domain assumption Human preferences elicited via pairwise judgments on recovery scenarios accurately reflect desired behavior in actual computer-use harm situations.
The paper grounds its reward model directly in the user-study rubric and judgments.

invented entities (1)

Harm recovery no independent evidence
purpose: Steering an agent from a harmful post-execution state back to a safe state aligned with human preferences
Newly formalized problem and solution class introduced in the paper; no independent evidence outside the study and benchmark.

pith-pipeline@v0.9.0 · 5516 in / 1323 out tokens · 34023 ms · 2026-05-10T04:09:16.726558+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

31 extracted references · 12 canonical work pages · 7 internal anchors

[1]

Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

Training a helpful and harmless assistant with reinforcement learning from human feedback , author=. arXiv preprint arXiv:2204.05862 , year=

work page internal anchor Pith review arXiv
[2]

Constitutional AI: Harmlessness from AI Feedback

Constitutional ai: Harmlessness from ai feedback , author=. arXiv preprint arXiv:2212.08073 , year=

work page internal anchor Pith review Pith/arXiv arXiv
[3]

Constitutional classifiers: Defending against universal jailbreaks across thousands of hours of red teaming

Constitutional classifiers: Defending against universal jailbreaks across thousands of hours of red teaming , author=. arXiv preprint arXiv:2501.18837 , year=

work page arXiv
[4]

Advances in neural information processing systems , volume=

Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=
[5]

Ethical and social risks of harm from Language Models

Ethical and social risks of harm from language models , author=. arXiv preprint arXiv:2112.04359 , year=

work page internal anchor Pith review arXiv
[6]

Research Paper, OpenAI , year=

Practices for governing agentic AI systems , author=. Research Paper, OpenAI , year=
[7]

, author=

An investigation into reactive planning in complex domains. , author=. AAAI , volume=
[8]

Artificial Intelligence , volume=

Robot task planning and explanation in open and uncertain worlds , author=. Artificial Intelligence , volume=. 2017 , publisher=

2017
[9]

Artificial Intelligence , volume=

Planning under time constraints in stochastic domains , author=. Artificial Intelligence , volume=. 1995 , publisher=

1995
[10]

Artificial intelligence , volume=

Planning and acting in partially observable stochastic domains , author=. Artificial intelligence , volume=. 1998 , publisher=

1998
[11]

Sayre and Ushnish Sengupta and Arthit Suriyawongkul and Ruby Thelot and Sofia Vei and Laura Waltersdorfer , title=

Gavin Abercrombie and Djalel Benbouzid and Paolo Giudici and Delaram Golpayegani and Julio Hernandez and Pierre Noro and Harshvardhan Pandit and Eva Paraschou and Charlie Pownall and Jyoti Prajapati and Mark A. Sayre and Ushnish Sengupta and Arthit Suriyawongkul and Ruby Thelot and Sofia Vei and Laura Waltersdorfer , title=. CoRR , volume=. 2024 , cdate=

2024
[12]

Advances in Neural Information Processing Systems , volume=

Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments , author=. Advances in Neural Information Processing Systems , volume=
[13]

WebArena: A Realistic Web Environment for Building Autonomous Agents

Webarena: A realistic web environment for building autonomous agents , author=. arXiv preprint arXiv:2307.13854 , year=

work page internal anchor Pith review arXiv
[14]

$\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains

Tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains , author=. arXiv preprint arXiv:2406.12045 , year=

work page internal anchor Pith review arXiv
[15]

Advances in Neural Information Processing Systems , volume=

Watch out for your agents! investigating backdoor threats to llm-based agents , author=. Advances in Neural Information Processing Systems , volume=
[16]

Os-harm: A benchmark for measuring safety of computer use agents

OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents , author=. arXiv preprint arXiv:2506.14866 , year=

work page arXiv
[17]

Openagentsafety: A comprehensive framework for evaluating real-world ai agent safety.arXiv preprint arXiv:2507.06134,

Openagentsafety: A comprehensive framework for evaluating real-world ai agent safety , author=. arXiv preprint arXiv:2507.06134 , year=

work page arXiv
[18]

Qualitative research in psychology , volume=

Using thematic analysis in psychology , author=. Qualitative research in psychology , volume=. 2006 , publisher=

2006
[19]

International Conference on Learning Representations (ICLR) , year=

React: Synergizing reasoning and acting in language models , author=. International Conference on Learning Representations (ICLR) , year=
[20]

Advances in neural information processing systems , volume=

Tree of thoughts: Deliberate problem solving with large language models , author=. Advances in neural information processing systems , volume=
[21]

arXiv preprint arXiv:2411.04468 , year=

Magentic-one: A generalist multi-agent system for solving complex tasks , author=. arXiv preprint arXiv:2411.04468 , year=

work page arXiv
[22]

WebGPT: Browser-assisted question-answering with human feedback

Webgpt: Browser-assisted question-answering with human feedback , author=. arXiv preprint arXiv:2112.09332 , year=

work page internal anchor Pith review arXiv
[23]

International conference on machine learning , pages=

Language models as zero-shot planners: Extracting actionable knowledge for embodied agents , author=. International conference on machine learning , pages=. 2022 , organization=

2022
[24]

Advances in Neural Information Processing Systems , volume=

Reflexion: Language agents with verbal reinforcement learning , author=. Advances in Neural Information Processing Systems , volume=
[25]

American Chess Journal , volume=

A comprehensive guide to chess ratings , author=. American Chess Journal , volume=
[26]

the method of paired comparisons , author=

Rank analysis of incomplete block designs: I. the method of paired comparisons , author=. Biometrika , volume=. 1952 , publisher=

1952
[27]

Llama guard 3 vision: Safe- guarding human-ai image understanding conversations,

Llama guard 3 vision: Safeguarding human-ai image understanding conversations , author=. arXiv preprint arXiv:2411.10414 , year=

work page arXiv
[28]

2025 , eprint=

Qwen3 Technical Report , author=. 2025 , eprint=

2025
[29]

AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents

Agentharm: A benchmark for measuring harmfulness of llm agents , author=. arXiv preprint arXiv:2410.09024 , year=

work page internal anchor Pith review arXiv
[30]

2025 , url =

Anthropic , title =. 2025 , url =

2025
[31]

2025 , url =

OpenAI , title =. 2025 , url =

2025