Pith. sign in

REVIEW 3 major objections 6 minor 37 references

Active RAG systems should be evaluated by utility frontiers, realized usage, threshold-transfer error, harm rates, and cost decompositions, not by a single adaptive-accuracy number.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:15 UTC pith:NNEIX3WM

load-bearing objection A genuinely useful evaluation protocol for Active RAG, with the one major caveat that the correctness labels are only weakly validated near the scoring threshold. the 3 major comments →

arxiv 2607.24010 v1 pith:NNEIX3WM submitted 2026-07-27 cs.LG

When Should Active RAG Retrieve? A Budget-Aware Evaluation of Utility, Calibration, and Cost

Pin Qian , Su Wang , Chong Peng , Junxian You , Lifei Liu , Haoran Yu , Yihang Chen , Xiaochong Jiang This is my paper
classification cs.LG
keywords active retrievalretrieval-augmented generationbudget-aware evaluationutility estimationthreshold calibrationretrieval harmcost accountingevaluation frontiers
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that evaluating active-retrieval systems—systems that decide when to look up external knowledge during generation—on a single accuracy number at a nominal budget is misleading. It recasts “when to retrieve” as budgeted utility estimation: retrieval is worth doing only when it changes a no-retrieval answer from wrong to right, and it can also turn right answers into wrong ones. The paper constructs three evaluation objects—exact usage-matched frontiers, deployable calibrated frontiers, and conservative budget-safe frontiers—plus harm audits and cost decompositions, and shows they often disagree. Across three multi-hop QA benchmarks and small open instruction models, retrieval harm is non-negligible, router rankings shift across datasets and budgets, nominal thresholds frequently miss their target usage on held-out inputs, and cheap uncertainty or retrieval-score baselines compete with learned routers. A sympathetic reader would take the central claim as: budget-aware evaluation should report these quantities alongside accuracy.

Core claim

At the paper's core is the claim that retrieval in active RAG should be treated as a decision under a budget, with each input assigned a utility of +1, 0, or −1 depending on whether retrieved evidence fixes a no-retrieval error, leaves it unchanged, or turns a correct answer into a wrong one. Under that framing, a trigger score is a ranking device, a threshold is a budget-enforcement device, and the two should be measured separately: exact top-k frontiers diagnose ranking quality at matched usage, deployable thresholds measure whether a calibration-split threshold transfers to future inputs, and conservative thresholds show the accuracy cost of discouraging budget violations. The paper repor

What carries the argument

The central machinery is the per-example utility decomposition u_i = correct(retrieved answer) − correct(no-retrieval answer), which turns active retrieval into utility estimation with three outcome classes: beneficial, neutral, and harmful. On top of it sit three evaluation objects: an exact top-k frontier that sorts held-out examples by a router score and takes the top fraction under a fixed usage budget; a deployable threshold frontier that fits a threshold on calibration data and applies it to held-out inputs; and a conservative frontier that adds finite-sample slack to the threshold to reduce budget violations. A cost model separates trigger computation (query features, no-retrieval gen

Load-bearing premise

Two load-bearing premises: automatic correctness labels for per-example retrieval utility (with only 40.0% validity on near-threshold rows in the paper's own audit) and the exchangeability assumption behind the finite-sample budget slack; if either fails, benefit/harm rates, frontier rankings, and violation rates all inherit the error.

What would settle it

Take two trigger routers and, on each random split, record adaptive accuracy, realized held-out usage, exact top-k accuracy, threshold-transfer error, and cost; if the router with higher adaptive accuracy at nominal 50% always also has higher exact usage-matched accuracy, lower threshold-transfer error, and lower cost on all three datasets, the paper's claim that single-point accuracy conflates ranking, calibration, and cost would be unsupported. A single such counterexample dataset would suffice to weaken the generalization.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A system that looks better at a nominal 50% evidence-usage budget can be worse once realized held-out usage is matched; reporting usage alongside accuracy prevents budget looseness from masquerading as retrieval skill.
  • Exact and deployable frontiers answer different questions, so a router can rank well but transfer poorly; both should be reported to separate ranking quality from threshold calibration.
  • Conservative thresholds with finite-sample slack nearly eliminate budget violations but cost accuracy, making budget safety a measurable trade-off rather than a free lunch.
  • Evidence-usage rate is not deployment cost: uncertainty-based and probe-retrieval triggers pay pre-decision computation that query-only routers avoid, so cost decompositions can change which router wins.
  • Retrieval harm is non-negligible even when retrieved evidence is relevant, so harm rates should be audited and reported rather than assumed zero.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This frontier protocol generalizes beyond RAG to any budgeted binary action under uncertainty—tool use, context extension, or multi-step reasoning—where a trigger score ranks actions and a threshold must enforce a resource budget.
  • Because cheap uncertainty and retrieval-score baselines rival learned utility routers in these experiments, any future learned router should be required to beat such references on the same frontiers, not just a full-pipeline accuracy number.
  • The audit's 40% validity on near-threshold EM/F1 labels implies that evaluations built on automatic correctness labels should report metric-threshold sensitivity, and suggests anchoring utility labels with a small human-validated sample.
  • One could make the budget a tunable Lagrangian penalty on expected usage rather than a post-hoc threshold, turning “how often to retrieve” into a single optimization that the frontier protocol could then evaluate.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a budget-aware evaluation protocol for Active RAG, treating retrieval triggers as policies over uncertain future utility under a budget. It defines utility as the marginal correctness change between retrieved-context and no-retrieval answers (Eq. 1), distinguishes exact top-k frontiers from deployable threshold frontiers and conservative DKW-style budgets (Eq. 4), and adds harm audits and cost decompositions. Experiments on HotpotQA, 2Wiki, and MuSiQue with small open models compare seven triggers; results show retrieval harm is non-negligible, router rankings shift across datasets and models, calibrated thresholds often miss held-out budgets, and simple baselines rival BUR. The paper argues that Active RAG evaluations should report frontiers, realized usage, threshold-transfer error, harm rates, and cost decompositions alongside accuracy.

Significance. If the measurement assumptions hold, the paper makes a useful methodological contribution: it exposes the operating-point ambiguity in Active RAG evaluations and provides concrete evaluation objects—exact vs. deployable frontiers, conservative calibration, harm audits, and cost decompositions—that future work can adopt. The experimental design is careful in several respects: five random splits, explicit sample sizes, manual harm annotation, an LLM-judge audit over 733 rows, and a referenced metric-threshold robustness check (though the numbers are not reported). The ordinal findings—retrieval is not monotonic, ranking and calibration separate, and simple uncertainty baselines are competitive—are plausible and align with recent literature. However, the central utility labels are automatic EM/F1 scores with a hard 0.80 threshold, and the paper's own audit shows near-threshold rows have only 40% validity; without a reported sensitivity analysis, the quantitative claims are not fully grounded.

major comments (3)
  1. [§4 (Models and retrieval), §5 RQ3, Eq. (1)] All utility labels u_i in Eq. (1) are computed from 'exact match or token F1 above 0.80.' The paper reports an LLM-judge audit with 93.3% validity for benefit labels and 83.8% for harm labels, but only 40.0% validity on near-threshold EM/F1 rows, and states 'we report metric-threshold robustness' without giving any numbers. Since benefit/harm rates (Table 1), BUR training targets, exact and deployable frontiers, and harm audits all inherit these labels, the threshold choice is load-bearing. Please report the metric-threshold robustness diagnostic (e.g., varying the F1 threshold over 0.7–0.9, or re-labeling near-threshold rows by the LLM judge) and show that the qualitative conclusions—retrieval harm non-negligible, router rankings, BUR vs. baselines—do not flip under plausible label noise.
  2. [§5 RQ2, Table 3] The title and abstract promise 'exact top-𝑘 utility frontiers' and 'deployable threshold frontiers,' but Table 3 and the text report only the 50% budget operating point; no frontier curves across the stated budgets {10, 25, 50, 75, 90}% are shown, and the conservative frontier is illustrated only with a single BUR-linear trade-off on HotpotQA. Without the full frontiers, the paper's central methodological claim—that frontier-based evaluation separates ranking quality from calibration—is not empirically demonstrated. Please include the frontier plots/tables for at least the main Qwen2.5-1.5B runs, or explicitly restrict the claims to single-point comparisons.
  3. [§5 RQ5, Eq. (3), Figure 1] The cost-accounting claim is supported only by a qualitative figure; no numeric token counts or cost decomposition table is provided, so the reader cannot verify the conclusion that 'BUR-linear can look attractive under an evidence-usage budget while paying both no-RAG generation and probe retrieval.' Please report the components of Eq. (3) numerically for the main settings, as the cost decomposition is one of the five promised evaluation dimensions.
minor comments (6)
  1. [§3] Typo: 'the distinction is between ascoreand anoperating point' should read 'a score and an operating point.'
  2. [Table 1] The caption lists 'Net 𝑢' but does not define it; state that it is the mean utility (benefit minus harm percentage).
  3. [Table 2] The 'Best@50' column labels 'Dense' ambiguously; clarify that this refers to the dense retriever (E5-small) as the trigger score family.
  4. [§4 (Datasets)] Stratification 'by utility class when possible' relies on the same automatic labels whose near-threshold behavior is noisy; the calibration-split construction may inherit that noise. Please discuss or provide a robustness check.
  5. [§5 RQ3] The 'metric-threshold robustness diagnostic' is mentioned twice but no results are shown. If it appears in a supplement, point to the specific table/figure; otherwise the claim is unverifiable.
  6. [Figure 1] The figure lacks description of axis units and the token-accounting assumptions; add a caption that defines 'token-equivalent cost' and the normalization denominator.

Circularity Check

0 steps flagged

No significant circularity: BUR routers are supervised on calibration-split utility labels and evaluated on held-out examples; reported metrics are measurements, not predictions that reduce to fitted inputs.

full rationale

The paper makes no claim to derive a first-principles result from its own assumptions. Eq. (1) defines utility labels from automatic correctness scoring; BUR routers are trained on calibration-split labels and evaluated on held-out test labels (Section 4), which is standard supervised evaluation rather than circular prediction. The exact top-k frontier uses oracle labels as an upper-bound diagnostic, and deployable/conservative frontiers measure threshold transfer on unseen data, so the central quantities are not forced by construction. The DKW-style slack in Eq. (4) is an external standard inequality and is explicitly treated empirically. The main concerns—automatic EM/F1 threshold sensitivity and the 40.0% LLM-judge validity on near-threshold rows—are measurement-quality limitations that the paper itself discloses through manual harm audits, an expanded LLM-judge audit, and a metric-threshold robustness diagnostic; they do not make any derivation equivalent to its inputs. I found no load-bearing self-citations and no renamed known result presented as a derivation. The paper's contribution is an evaluation protocol with empirical demonstrations, and its claims are supported by held-out and externally validated evidence.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The paper contributes an evaluation protocol rather than a new physical or mathematical theory. Its empirical claims rely on conventional supervised fitting (BUR), a standard statistical inequality (DKW), and a stated utility definition. The main uncharged input is the utility definition via automatic correctness scoring, which is partially validated by the authors' own audits; no new entities or mediators are postulated.

free parameters (5)
  • BUR-linear logistic regression weights = not reported
    Trained on calibration-split utility labels u_i ∈ {-1,0,+1} (Eq. 1); used as a reference learned router and compared against baselines.
  • BUR-GBM hyperparameters and weights = not reported
    Gradient-boosted variant fitted on the same calibration utility labels; hyperparameters are not given.
  • Correctness scoring threshold = F1 > 0.80 or exact match
    Hand-chosen threshold converting generated text to a binary correct/incorrect label; directly determines benefit/harm rates and all utility labels. The paper's LLM-judge audit shows near-threshold rows have only 40% validity.
  • Conservative slack confidence δ (Eq. 4) = not specified
    The DKW-style slack ε = sqrt(log(2/δ)/(2 n_cal)) requires choosing δ; the paper does not state the value used in experiments.
  • Router thresholds at each budget = calibration-selected
    For deployable and conservative frontiers, a threshold is selected on the calibration split for each router and budget; these are fitted operating points, though the fitted values are not tabulated.
axioms (5)
  • domain assumption Retrieval utility is fully captured by the marginal change in automatic answer correctness, u_i = correct(a_i^R,y_i) − correct(a_i^0,y_i) (Eq. 1).
    Defines the objective and all utility labels; excludes value from faithfulness, verifiability, provenance, or other non-accuracy criteria.
  • domain assumption Automatic EM/F1 scoring at 0.80 approximates human correctness.
    Used to assign all benefit/harm labels and to train BUR; the authors validate with a 60-example manual audit and an LLM-judge audit, but near-threshold rows have only 40% validity.
  • standard math Examples are exchangeable across calibration and held-out splits, and DKW-style slack (Eq. 4) controls population usage.
    Justifies conservative thresholds; the authors state they treat the rule empirically rather than as a strict guarantee.
  • domain assumption The active policy is binary (π(q) ∈ {0,R}); retrieval is a single BM25 top-5 retrieve-and-answer step.
    Excludes multi-step, FLARE-style within-generation retrieval, and official Self-RAG/FLARE reproductions, as acknowledged in Limitations.
  • domain assumption Evidence usage (fraction of examples using retrieved context) is the primary budget; token costs are a secondary simulated accounting.
    The core frontiers are defined in terms of evidence-usage budget; the cost model in Eq. 3 is introduced separately and remains a token-equivalent simulation rather than latency/dollar/energy measurement.

pith-pipeline@v1.3.0-alltime-deepseek · 11481 in / 10008 out tokens · 93622 ms · 2026-07-31T23:15:19.875963+00:00 · methodology

0 comments
read the original abstract

Active RAG systems decide when to retrieve external knowledge during generation, making them a budget-sensitive case of agentic RAG and self-adaptive retrieval. Yet evaluations often leave the operating point underspecified: two systems may both claim a 50% evidence-usage budget while realizing different held-out usage rates, so higher accuracy can reflect a looser budget rather than a better retrieval policy. We study budget-aware evaluation for Active RAG by recasting active retrieval as utility estimation, where retrieval is valuable only through its marginal correctness change over a no-retrieval answer. This view separates three questions that single-point evaluations conflate: whether trigger scores rank useful retrieval decisions, whether thresholds calibrated on past data meet future budgets, and how trigger-side computation changes deployment cost. We operationalize these questions with exact top-k utility frontiers, deployable threshold frontiers, conservative budget frontiers, harm audits, and cost decompositions. Across knowledge-intensive multi-hop QA datasets and open instruction models, retrieval harm is non-negligible, router rankings change across datasets and budgets, nominal thresholds can miss target usage, and simple uncertainty or retrieval-score baselines often rival learned utility routers. Budget-aware Active RAG evaluations should therefore report frontiers, realized usage, threshold-transfer error, harm rates, and cost decompositions alongside accuracy.

Figures

Figures reproduced from arXiv: 2607.24010 by Chong Peng, Haoran Yu, Junxian You, Lifei Liu, Pin Qian, Su Wang, Xiaochong Jiang, Yihang Chen.

Figure 1
Figure 1. Figure 1: Accuracy versus estimated token-equivalent cost [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 6 canonical work pages · 5 internal anchors

  1. [1]

    Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martín Blázquez, Guil- herme Penedo, Lewis Tunstall, Andrés Marafioti, Hynek Kydlíček, Agustín Pi- queres Lajarín, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan-Son Nguyen, Ben Burtenshaw, Clémentine Fourrier, Haojun Zhao, Hugo Larcher, Mathieu Morlon, Cyril Zakka, Colin Raffel, Leandro von ...

  2. [2]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. InThe Twelfth International Conference on Learning Representations. OpenRe- view.net, Vienna, Austria. https://openreview.net/forum?id=hSyW5go0v8

    2024

  3. [3]

    Qinyuan Cheng, Xiaonan Li, Shimin Li, Qin Zhu, Zhangyue Yin, Yunfan Shao, Linyang Li, Tianxiang Sun, Hang Yan, and Xipeng Qiu. 2024. Unified Active Retrieval for Retrieval Augmented Generation. InFindings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguis- tics, Miami, Florida, USA, 17153–17166. doi:10.1865...

    work page doi:10.18653/v1/2024.findings-emnlp.999 2024

  4. [4]

    Zhiyuan Cheng and Longying Lai. 2026. Energy-Efficient On-Device RAG on a Mobile NPU: System Design and Benchmark on Snapdragon X Elite. arXiv:2606.11257 [cs.CL] doi:10.48550/arXiv.2606.11257

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2606.11257 2026

  5. [5]

    Zhiyuan Cheng, Longying Lai, and Yue Liu. 2026. Resolving the Robustness- Precision Trade-off in Financial RAG through Hybrid Document-Routed Re- trieval. arXiv:2603.26815 [cs.CL] doi:10.48550/arXiv.2603.26815

    Pith/arXiv arXiv doi:10.48550/arxiv.2603.26815 2026

  6. [6]

    Zhiyuan Cheng, Longying Lai, Yue Liu, and Yu Sun. 2026. Toward Sustainable On-Device Intelligence: A Survey on Energy-Efficient RAG Systems with Small Language Models.A vailable at SSRN 6698538(2026). https://ssrn.com/abstract= 6698538

    2026

  7. [7]

    Yuzhuo Dang, Xin Zhang, Zhiqiang Pan, Yuxiao Duan, Wanyu Chen, Fei Cai, and Honghui Chen. 2025. MLLMRec: A Preference Reasoning Paradigm with Graph Refinement for Multimodal Recommendation.arXiv preprint arXiv:2508.15304 (2025). arXiv:2508.15304 [cs.IR] doi:10.48550/arXiv.2508.15304

    doi:10.48550/arxiv.2508.15304 2025

  8. [8]

    Yaxin Gao, Yao Lu, Zongfei Zhang, Jiaqi Nie, Shanqing Yu, and Qi Xuan

  9. [9]

    Granite Team, IBM. 2024. Granite-3.1-2B-Instruct Model Card. https:// huggingface.co/ibm-granite/granite-3.1-2b-instruct

    2024

  10. [10]

    Weinberger

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On Calibration of Modern Neural Networks. InProceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70). PMLR, Sydney, Australia, 1321–1330. https://proceedings.mlr.press/v70/guo17a.html

    2017

  11. [11]

    Xiao Han, Yao Xiao, Chenyu Wu, and Tongchen Zhang. 2026. How Early Is Early Enough? Design-Dependent Observation-Window Sufficiency in Subscription Churn Prediction. arXiv:2607.00473 [cs.LG] doi:10.48550/arXiv.2607.00473

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2607.00473 2026

  12. [12]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reason- ing Steps. InProceedings of the 28th International Conference on Computational Linguistics. International Committee on Computational Linguistics, Barcelona, Spain (Online), 6609–6625. doi:10.18653/v1/2020.colin...

    doi:10.18653/v1/2020.coling-main.580 2020

  13. [13]

    Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong Park

  14. [14]

    Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi- Yu, Yiming Yang, Jamie Callan, and Graham Neubig

    Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi- Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active Retrieval Augmented Generation. InProceedings of the 2023 Conference on Empirical Meth- ods in Natural Language Processing. Association for Computational Linguistics, Singapore, 7969–7992. doi:10.18653/v1/2023.emnlp-main.495

    doi:10.18653/v1/2023.emnlp-main.495 2023

  15. [15]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval- Augmented Generation for Knowledge-Intensive NLP Tasks. InAdvances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., Online,...

    2020

  16. [16]

    Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, and Huan Liu. 2025. From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Associ...

    doi:10.18653/v1/2025.emnlp-main.138 2025

  17. [17]

    Yuqi Li, Kuiye Ding, Chuanguang Yang, Szu-Yu Chen, and Yingli Tian. 2026. Distilling Time Series Foundation Models for Efficient Forecasting. InICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 4631–4635. doi:10.1109/ICASSP55912.2026.11460474

    arXiv 2026

  18. [18]

    Yanhang Li, Zhichao Fan, and Zexin Zhuang. 2026. SafetyRepro: Configuration-Conditional Rank Instability on Alignment Benchmarks. arXiv:2605.25492 [cs.LG] doi:10.48550/arXiv.2605.25492

    Pith/arXiv arXiv doi:10.48550/arxiv.2605.25492 2026

  19. [19]

    Yuqi Li, Chuanguang Yang, Hansheng Zeng, Zeyu Dong, Zhulin An, Yongjun Xu, Yingli Tian, and Hao Wu. 2025. Frequency-Aligned Knowledge Distillation for Lightweight Spatiotemporal Forecasting. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision (ICCV). 7262–7272. arXiv:2507.02939 [cs.LG] https://openaccess.thecvf.com/content/ICCV2025...

    Pith/arXiv arXiv 2025

  20. [20]

    Jiayuan Liu, Tianqin Li, Shiyi Du, Xin Luo, Haoxuan Zeng, Emanuel Tewolde, Tai Sing Lee, Tonghan Wang, Carl Kingsford, and Vincent Conitzer. 2026. The Memory Curse: How Expanded Recall Erodes Cooperative Intent in LLM Agents. arXiv preprint arXiv:2605.08060(2026). arXiv:2605.08060 [cs.CL] doi:10.48550/ arXiv.2605.08060

    Pith/arXiv arXiv doi:10.48550/arxiv.2605.08060 2026

  21. [21]

    Viktor Moskvoretskii, Maria Marina, Mikhail Salnikov, Nikolay Ivanov, Sergey Pletenev, Daria Galimzianova, Nikita Krayko, Vasily Konovalov, Irina Nikishina, and Alexander Panchenko. 2025. Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Vol...

    doi:10.18653/v1/2025.acl- 2025

  22. [22]

    Hrishikesh Paranjape, Abhishek Mandal, and Xian Sun. 2026. Optimality of Sequential Filtering Under Independent Cost and Selectivity Models. In2026 IEEE International Conference on Electro/Information Technology (EIT). IEEE. arXiv:2606.07589 [cs.LG] doi:10.48550/arXiv.2606.07589 Accepted at EIT 2026; IEEE proceedings metadata pending

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2606.07589 2026

  23. [23]

    Qwen Team. 2025. Qwen2.5 Technical Report. arXiv:2412.15115 [cs.CL] doi:10. 48550/arXiv.2412.15115

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.15115 2025

  24. [24]

    Weihang Su, Yichen Tang, Qingyao Ai, Zhijing Wu, and Yiqun Liu. 2024. DRAGIN: Dynamic Retrieval Augmented Generation based on the Real-time Information Needs of Large Language Models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Bangkok, Thaila...

    2024

  25. [25]

    Yiyun Su, Huiying Zhu, Yu Tian, Changruo Zhao, Zujun Peng, Yuting Liu, Liang Fan, Baihua Li, and Luyan Zhang. 2026. Agentic-SQL Taxonomy: A Survey of Autonomous and Interactive Text-to-SQL with LLMs.Authorea Preprints(March 2026). doi:10.22541/au.177430005.57777158/v2

    arXiv 2026

  26. [26]

    Xian Sun, Yingshuo Wang, Wei Gao, Lingdong Kong, Zexin Zhuang, Zhichao Fan, Wenlong Dong, Hrishikesh Paranjape, and Zhiyuan Zheng. 2026. Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluation. InTrustworthy AI for Good (AI4GOOD) Workshop at ICML 2026. https://openreview.net/forum?id=6OfjuNWxJd

    2026

  27. [27]

    Yicheng Tao, Yiqun Wang, Xiangchen Song, Xin Luo, Kai Liu, and Jie Liu. 2026. GRASP: Plan-Guided Graph Retrieval with Adaptive Fusion and Reranking on Semi-Structured Knowledge Bases.arXiv preprint arXiv:2605.30237(2026). arXiv:2605.30237 [cs.IR] doi:10.48550/arXiv.2605.30237

    Pith/arXiv arXiv doi:10.48550/arxiv.2605.30237 2026

  28. [28]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal

  29. [29]

    Yingshuo Wang, Xian Sun, Lingdong Kong, Wei Gao, Yanhang Li, Zhichao Fan, and Zexin Zhuang. 2026. Do Time Series Foundation Model Benchmarks Hide Regime-Dependent Failures? Evidence from Traffic Speed Forecasting. arXiv:2606.18367 [cs.LG] doi:10.48550/arXiv.2606.18367 Accepted at the Work- shop on Forecasting as a New Frontier of Intelligence, ICML 2026

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2606.18367 2026

  30. [30]

    Chenyu Wu. 2026. Class Weighting versus Amount Conditioning in Credit-Card Fraud Detection: A Dollar-Metric Study with a Temporal Explanation Audit. arXiv:2607.14686 [cs.CE] doi:10.48550/arXiv.2607.14686

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2607.14686 2026

  31. [31]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium, 2369–2...

    doi:10.18653/v1/d18- 2018

  32. [32]

    Zhang, Gaoyuan Du, Qianyi Sun, Shiqi Wang, Jiaxuan Li, and Xian Sun

    Johnny R. Zhang, Gaoyuan Du, Qianyi Sun, Shiqi Wang, Jiaxuan Li, and Xian Sun. 2026. Carbon-Aware Compute–Power Scheduling for AI Data Centers with Microgrid Prosumer Operations. arXiv:2605.03751 [cs.CE] doi:10.48550/arXiv. 2605.03751

    Pith/arXiv arXiv doi:10.48550/arxiv.2605.03751 2026

  33. [33]

    Changruo Zhao, Zujun Peng, Yu Tian, Yuting Liu, Yiyun Su, Hui-Ying Zhu, Luyan Zhang, and Heming Zeng. 2026. Agentic-SQL Revisited: Autonomy-Based Tax- onomy and Empirical Benchmark Analysis for LLM Text-to-SQL.ResearchGate preprint(July 2026). doi:10.13140/RG.2.2.20023.48809

    arXiv 2026

  34. [34]

    Zexin Zhuang, Yanhang Li, and Zhichao Fan. 2026. Pre-Registering the Detectable Effect: A Paired-MDE Budget for 4-bit Quantization Benchmarks, with a Pilot Audit. arXiv:2605.28873 [cs.LG] doi:10.48550/arXiv.2605.28873 7

    Pith/arXiv arXiv doi:10.48550/arxiv.2605.28873 2026

  35. [2022]

    Transactions of the Association for Computational Linguistics10 (2022), 539–554

    MuSiQue: Multihop Questions via Single-hop Question Composition. Transactions of the Association for Computational Linguistics10 (2022), 539–554. doi:10.1162/tacl_a_00475

    doi:10.1162/tacl_a_00475 2022

  36. [2024]

    InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)

    Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Association for Computational Linguistics, Mexico City, Mexico, 7036–7050. doi:10.18...

    doi:10.18653/v1/2024.naacl-long 2024

  37. [2025]

    InConference on Language Modeling

    SmolLM2: When Smol Goes Big—Data-Centric Training of a Fully Open Small Language Model. InConference on Language Modeling. OpenReview.net, Montreal, Canada. https://openreview.net/forum?id=3JiCl2A14H