Pooled top-1 accuracy rankings in RCA benchmarks do not reliably identify per-subsystem winners, as pairwise comparisons across 11 subsystems show effects of both signs and leave-one-system-out selection incurs regret up to 24.8 pp.
Aiopslab: A holistic framework to evaluate ai agents for enabling autonomous clouds, 2025b
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
years
2026 4roles
background 1polarities
background 1representative citing papers
GSAR is a grounding-evaluation framework for multi-agent LLMs that uses a four-way claim typology, evidence-weighted asymmetric scoring, and tiered recovery decisions to detect and mitigate hallucinations.
Ambig-IaC detects structural disagreements in LLM-generated IaC candidates across three hierarchical axes to produce clarification questions, improving structure and attribute accuracy by 18.4% and 25.4% on a new 300-task benchmark.
OpsLLM's pipeline (HITL data curation, SFT, GRPO RL with a domain process reward model) improves LLM accuracy on software-operations QA and RCA, especially on in-distribution root-cause-analysis tasks.
citing papers explorer
-
Pooled Leaderboards Hide System-Specific Winners: A Reporting-Protocol Audit of Offline Root-Cause Analysis Benchmarks
Pooled top-1 accuracy rankings in RCA benchmarks do not reliably identify per-subsystem winners, as pairwise comparisons across 11 subsystems show effects of both signs and leave-one-system-out selection incurs regret up to 24.8 pp.
-
GSAR: Typed Grounding for Hallucination Detection and Recovery in Multi-Agent LLMs
GSAR is a grounding-evaluation framework for multi-agent LLMs that uses a four-way claim typology, evidence-weighted asymmetric scoring, and tiered recovery decisions to detect and mitigate hallucinations.
-
Ambig-IaC: Multi-level Disambiguation for Interactive Cloud Infrastructure-as-Code Synthesis
Ambig-IaC detects structural disagreements in LLM-generated IaC candidates across three hierarchical axes to produce clarification questions, improving structure and attribute accuracy by 18.4% and 25.4% on a new 300-task benchmark.
-
OpsLLM: Construction of Large Language Model for Software Operations with Multi-stage Learning
OpsLLM's pipeline (HITL data curation, SFT, GRPO RL with a domain process reward model) improves LLM accuracy on software-operations QA and RCA, especially on in-distribution root-cause-analysis tasks.