Pith. sign in

REVIEW 4 major objections 6 minor 62 references

Adapting to Non-Stationary Environments: Multi-Armed Bandit Enhanced Retrieval-Augmented Generation on Knowledge Graphs

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Treating each retrieval method as a bandit arm and updating on per-query feedback beats single-retriever, ensemble, and static-router RAG in stationary and non-stationary settings.

desk verdict Good engineering idea for adaptive KG-RAG retriever selection, but the non-stationary experiments lack online adaptive baselines; the central claim needs that control before it holds. read the letter →

arxiv 2412.07618 v2 pith:7AH53Q6N submitted 2024-12-10 cs.AI cs.CL

Xiaqiang Tang , Jian Li , Nan Du , Sihong Xie This is my paper · ORCID
classification cs.AIcs.CL
keywords retrieval-augmentedgenerationknowledgegraphsmulti-armedbanditcontextualGeneralizedGiniIndexnon-stationaryenvironmentsgraphquestionansweringonlinelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Knowledge-graph retrieval-augmented generation systems can draw on several retrieval methods with different strengths — dense embeddings are fast, SPARQL generation is precise, and graph agents handle multi-hop reasoning — but most deployments either commit to one method or use a static router that requires fully labeled data. This paper tries to establish that treating each retrieval method as an arm in a deep contextual multi-armed bandit, and updating the router after every query with real-time feedback, makes the system adapt to non-stationarity of two kinds: shifting query domains and backend retriever upgrades. On the WebQSP and ComplexWebQuestions benchmarks, the paper reports that this GGI-MAB router outperforms single-retriever, ensemble, and static-router baselines in stationary settings and holds or improves accuracy when the environment changes. The reason this matters is that real deployments face exactly these shifts, and current RAG designs lack an explicit mechanism to keep learning from user responses.

What carries the argument

The load-bearing component is a deep contextual multi-armed bandit whose arms are the retrieval methods themselves — DECAF dense retrieval, ChatKBQA SPARQL generation, and Reason-on-Graph KG-agent reasoning — with a DistilBERT query encoder producing arm-selection probabilities and an epsilon-greedy rule balancing exploration with exploitation. What makes the bandit multi-objective is the Generalized Gini Index (GGI), an inequality metric used here to aggregate three losses — mean-squared error against hit, mean-squared error against recall, and KL divergence against a retrieval-delay distribution — into a single scalar loss $Loss_{GGI}(\theta) = \sum_{i=1}^D w_i (l_i)_\tau$, where the weights favour balanced performance so that no single objective dominates. The training procedure is offline pretraining on a static dataset followed by online per-query updates from partial-information feedback, and that online-update loop is the mechanism intended to give the system its adaptability.

What would settle it

Run the retriever-upgrade and domain-shift scenarios again with the binary hit label delayed by a fixed number of queries (for example, ten) or with 20% of labels randomly dropped, and check whether the reported gains over static routers survive; if accuracy falls back to the static-router level, the claim that online per-query feedback carries the result is disproved.

Watch

Extended reading notes

Core claim

The paper's central claim is that the system does not need to know in advance which retrieval method is best for a query: a router that selects among dense retrieval, SPARQL-based retrieval, and a knowledge-graph agent can learn the right choice per query and keep revising it as conditions change. The router is a DistilBERT encoder that maps a query to a distribution over retrieval arms, selects with an epsilon-greedy rule, and updates after each query using the observed hit, recall, and retrieval delay of the chosen arm, which means the feedback is partial-information because unselected arms are not observed. On WebQSP, the method reports an 86.64 hit rate and 75.60 recall, compared with 86.20 and 75.03 for the best static NN-Router, and on CWQ it reports 79.35 and 72.02 versus 78.53 and 71.52. In the non-stationary retriever-upgrade scenario it reports 84.80 hit and 72.24 recall versus 83.74 and 67.52 for the retrieval ensemble, and in the domain-shift scenario 76.35 and 69.47 versus 67.93 and 68.01. The paper's assertion is that these gains come from the online adaptive selection mechanism rather than from any single retriever or from a static combination of retrievers.

Load-bearing premise

The load-bearing premise is that a correct/incorrect signal, called the hit label, is available immediately after every query and can stand in for real user feedback; if that signal is delayed, sparse, or noisy, the per-query online updates that drive the reported non-stationary gains cannot be applied as described.

Editorial extensions

If this is right

  • A KG-based RAG system can be deployed without committing to a single retriever; the router will learn which retrieval method fits each query while the system is running.
  • Backend retriever upgrades need not trigger a full data-labeling and fine-tuning campaign, because per-query updates let the router discover the improved arm on its own.
  • Accuracy and responsiveness can be optimized together, since retrieval delay enters the same GGI loss as hit and recall.
  • Static neural routers that require complete labeled data and periodic fine-tuning are unnecessary in this setting, because partial-information feedback suffices for online adaptation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same GGI-style multi-objective bandit could route among any set of modular LLM tools — search APIs, code interpreters, or external databases — wherever per-call latency and answer quality compete.
  • Because the reward in the experiments is a dataset-derived hit label, a practical deployment would need to convert noisy, delayed, or implicit user signals into per-query rewards; the reported gains should be read as an upper bound on what a real feedback loop would deliver.
  • The non-stationary comparisons pit an online-updating router against offline baselines, so a periodically retrained static router would be a sharper control for isolating the value of online adaptation.
  • The action space is limited to three retrieval arms; testing with many more arms would show whether epsilon-greedy exploration plus GGI updates scale, or whether structured exploration is needed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes GGI-MO-MAB, a DistilBERT-based contextual multi-armed bandit router that selects among dense retrieval, ChatKBQA-style SPARQL generation, and Reason-on-Graph KG-agent retrieval for KG-based RAG. The router is pretrained offline using accuracy and efficiency losses and is then updated per query with hit, recall, and delay feedback, with the Generalized Gini Index used to aggregate the multiple objectives. Experiments on WebQSP and CWQ report gains over static routers and ensembles in stationary settings (Table 1) and over offline/static baselines in two simulated non-stationary settings: backend retriever upgrade and domain shift (Table 2). The central claim is that online multi-objective bandit adaptation yields significant robustness gains in non-stationary environments while retaining state-of-the-art stationary performance.

Significance. If the non-stationary claim were cleanly established, the paper would be a useful practical contribution: it demonstrates a lightweight router that can trade off accuracy, recall, and latency across heterogeneous KG retrievers, ships code and data, evaluates with multiple LLM generators, and includes an additional degradation experiment (Table 4). The use of the GGI for reward aggregation is principled, and the stationary comparison is informative. However, the experimental design in Table 2 currently conflates the value of the online adaptive algorithm with the informational advantage of test-time label access, so the headline non-stationary conclusion is not yet supported.

major comments (4)
  1. [§3.2, Table 2] The non-stationary comparison does not isolate the proposed adaptation mechanism. GGI-MAB is the only method that receives per-query hit, recall, and delay feedback during testing and updates its parameters; Retrieval Ensemble, Offline MO-MAB, NN Router, and LLM Router are static or offline and therefore cannot exploit this information. The observed gains, such as 76.35 versus 69.57 hit under domain shift, may reflect the extra label access rather than the bandit or GGI machinery. The paper should include online adaptive baselines under the identical per-query-update protocol, for instance LinUCB, Thompson Sampling, and SO-Deep-MAB, which already appear in Table 3, plus an ablated version of GGI-MAB that is never updated online, to separate adaptation from label exploitation.
  2. [§3.1] The online environment is simulated by assuming that a binary hit label is immediately available for every test query, stated as "simulate an online environment with a hit value (0 or 1) to approximate binary user feedback." Real user feedback is delayed, sparse, and noisy, and dataset-derived hit labels are not equivalent to it. Because the non-stationary results depend on per-query updates with these labels, the reported numbers should be framed as an oracle-feedback upper bound, with additional experiments under delayed, missing, or corrupted rewards to test robustness.
  3. [§2.2, Eqs. (2)–(3)] The update rule does not clearly implement partial-information feedback. Equations (2) and (3) define l1 = MSE(max(fθ(x)), h) and l2 = MSE(max(fθ(x)), rc), so the loss depends only on the maximum predicted value, not on the value of the arm actually selected during epsilon-greedy exploration. When an exploratory arm is chosen, the observed hit or recall is attributed to the maximum arm rather than to the selected arm, and when the selected arm is not the maximum the model receives no gradient for that arm from these terms. The loss should be rewritten in terms of the selected action's predicted reward, for example fθ(x)[a], to be consistent with the stated partial-feedback setup.
  4. [§3.2, Algorithm 1] The non-stationary protocol is underspecified. The text says training uses Think-on-Graph and testing switches to Reason-on-Graph, and that training on WebQSP is followed by testing on CWQ, but it does not state how many test queries are used for online updates, whether updates are applied in a single chronological pass, what epsilon schedule is used, how often evaluation is performed, or how the mean and standard deviation over ten seeds are computed when the model changes during testing. Without this information, the reader cannot determine whether the reported gains come from the adaptation procedure or from a particular test-time update protocol. Please provide the exact protocol for both non-stationary scenarios.
minor comments (6)
  1. [Algorithm 1 line 2 and Eq. (5)] The algorithm initializes "equal initial weights" for w, but the GGI definition requires w1 > w2 > ... > wd; equal weights make the ordering τ irrelevant and collapse the GGI to a plain weighted sum. Clarify whether the equal initialization is temporary or whether the strict ordering is enforced during training.
  2. [Tables 1–4] The method name is inconsistent: "GGI-MAB" is used in Tables 1 and 2, while "GGI-MO-MAB" is used in Tables 3 and 4. Please choose one name and use it throughout.
  3. [References] The citation "Chen et al. (????)" appears in the reference list as "Training for Stable Explanation for Free" without a year or venue, and "Chen et al." is used in Section 6.6 without a full citation. This reference needs to be completed or removed.
  4. [§3.1, Tables 1 and 3] The paper claims at least ten independent rounds with different seeds, but Table 1 reports no standard deviations. Please report variability consistently across all tables and provide confidence intervals or significance tests for the "significantly outperforms" claims.
  5. [§2.2, Eq. (1)] The notation for delay is inconsistent: the text writes d_i and "di" interchangeably, while Eq. (1) uses d_i and d_j. Please unify the notation.
  6. [References] The citation "lla 2024" and the LlamaIndex selector documentation entry are inconsistent; provide the proper author, title, and venue information.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation is self-contained; the non-stationary comparison is an experimental-control concern, not a reduction of the claim to its inputs.

full rationale

The paper's derivation chain starts from defined multi-objective losses (Eqs. 2-5), a defined Generalized Gini aggregation, and a per-query gradient update (Algorithm 1); nothing in this chain assumes the reported performance numbers as inputs. The GGI weights are underspecified (Algorithm 1 says 'equal initial weights' while Eq. 5 requires w1 > w2 > ... > wd), and the non-stationary experiments give only GGI-MAB access to per-query test hit labels while offline baselines do not receive such feedback—these are correctness and experimental-design issues, not circularity. The only self-citation is the malformed reference 'Chen, C.; ... Xie, S. ???? Training for Stable Explanation for Free', used peripherally in Section 6.6 as '(Chen et al.)'; it is not load-bearing and does not make the central claim equivalent to an input. The central claim is empirically testable against the listed static and offline baselines, even if the comparison is not fully controlled.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The system relies on two openly acknowledged free parameters (epsilon and GGI weights) plus implicit domain assumptions about feedback availability, delay observability, and online learning stability. The ad hoc reward regression target in Eqs. (2)-(3) is the most paper-specific assumption. No new physical or conceptual entities are introduced.

free parameters (2)
  • epsilon in epsilon-greedy = not stated
    Exploration rate in arm selection; a hyperparameter whose value is never specified in the paper.
  • GGI weights w = initialized equal; final values not reported
    Weights in Eq. (5) define the aggregation; Algorithm 1 says 'Set equal initial weights' but Eq. (5) requires decreasing weights for the Gini property, creating ambiguity.
assumptions (5)
  • domain assumption The binary hit label is a valid proxy for real-time user feedback.
    Section 3.1: 'we simulate an online environment with a hit value (0 or 1) to approximate binary user feedback'; the entire online adaptation relies on this.
  • domain assumption The delay distribution sigma(d_i) can be computed for all arms, including unselected ones, using observed delays.
    Eq. (1) defines sigma(d_i) over all K retrievers, but only the selected arm's delay is observed per query; the paper does not explain how unselected delays are obtained.
  • domain assumption Online fine-tuning with gradient descent on recent queries adapts to non-stationarity without catastrophic forgetting.
    Section 2.2 states the model is updated per query, but no mechanism (e.g., replay) is described to prevent overwriting useful prior behavior.
  • ad hoc to paper The reward regression target max(fθ(x)) in Eqs. (2)-(3) is a valid training signal for a contextual bandit.
    This choice is not derived from a bandit objective; it is an ad hoc way to tie the selected arm's probability to the observed reward.
  • standard math Generalized Gini Index properties from Weymark 1981 are assumed as background.
    The fairness and efficiency properties in Section 6.1 are invoked without proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adapting to Non-Stationary Environments: Multi-Armed Bandit Enhanced Retrieval-Augmented Generation on Knowledge Graphs." pith.science (2026). https://pith.science/paper/7AH53Q6N

@misc{pith2026241207618,
  author       = {Pith},
  title        = {Pith review of: Adapting to Non-Stationary Environments: Multi-Armed Bandit Enhanced Retrieval-Augmented Generation on Knowledge Graphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7AH53Q6N}},
  note         = {Machine review of arXiv:2412.07618}
}
read the original abstract

Despite the superior performance of Large language models on many NLP tasks, they still face significant limitations in memorizing extensive world knowledge. Recent studies have demonstrated that leveraging the Retrieval-Augmented Generation (RAG) framework, combined with Knowledge Graphs that encapsulate extensive factual data in a structured format, robustly enhances the reasoning capabilities of LLMs. However, deploying such systems in real-world scenarios presents challenges: the continuous evolution of non-stationary environments may lead to performance degradation and user satisfaction requires a careful balance of performance and responsiveness. To address these challenges, we introduce a Multi-objective Multi-Armed Bandit enhanced RAG framework, supported by multiple retrieval methods with diverse capabilities under rich and evolving retrieval contexts in practice. Within this framework, each retrieval method is treated as a distinct ``arm''. The system utilizes real-time user feedback to adapt to dynamic environments, by selecting the appropriate retrieval method based on input queries and the historical multi-objective performance of each arm. Extensive experiments conducted on two benchmark KGQA datasets demonstrate that our method significantly outperforms baseline methods in non-stationary settings while achieving state-of-the-art performance in stationary environments. Code and data are available at https://github.com/FUTUREEEEEE/Dynamic-RAG.git

Figures

Figures reproduced from arXiv: 2412.07618 by the authors.

Figure 1
Figure 1. An online KG-based RAG system facing challenges from non-stationary environments and the need to balance [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Proposed MAB-enhanced RAG framework. The input query undergoes feature extraction (e.g., multi-entity query), [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of retrieval methods for the query, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Confusion matrices comparing retrieval methods [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: MAB enhanced RAG systems with LLM variants [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: SPARQL-Based Retriever gives the most accurate [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: KG Agent gives the most accurate context [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 31 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Define Selector Module for Routing --- Llama Index Documentation

    2024. Define Selector Module for Routing --- Llama Index Documentation. https://docs.llamaindex.ai/en/stable/examples/retrievers/router_retriever/#define-selector-module-for-routing. Accessed: 2024-08-07

    work page 2024

  4. [4]

    Agrawal, S.; and Goyal, N. 2013. Thompson sampling for contextual bandits with linear payoffs. In International conference on machine learning, 127--135. PMLR

    work page 2013

  5. [5]

    Y.; Ayd n, \"O .; and Karaarslan, E

    Alan, A. Y.; Ayd n, \"O .; and Karaarslan, E. 2024. A RAG-based Question Answering System Proposal for Understanding Islam: MufassirQAS LLM. Available at SSRN 4707470

    work page 2024

  6. [6]

    Auer, P. 2002. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research, 3(Nov): 397--422

    work page 2002

  7. [7]

    Auer, P.; Cesa-Bianchi, N.; and Fischer, P. 2002. Finite-time analysis of the multiarmed bandit problem. Machine learning, 47: 235--256

    2002

  8. [8]

    F.; and Saffari, A

    Baek, J.; Aji, A. F.; and Saffari, A. 2023. Knowledge-augmented language model prompting for zero-shot knowledge graph question answering. arXiv preprint arXiv:2306.04136

    arXiv 2023

Show all 62 references
  1. [9]

    Bang, Y.; Cahyawijaya, S.; Lee, N.; Dai, W.; Su, D.; Wilie, B.; Lovenia, H.; Ji, Z.; Yu, T.; Chung, W.; et al. 2023. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023

    2023 arXiv

  2. [10]

    BehnamGhader, P.; Miret, S.; and Reddy, S. 2023. Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Model. In Findings of the Association for Computational Linguistics: EMNLP 2023, 15492--15509

    2023

  3. [11]

    D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al

    Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877--1901

    2020

  4. [12]

    Bubeck, S.; Cesa-Bianchi, N.; et al. 2012. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends in Machine Learning , 5(1): 1--122

    2012

  5. [13]

    Busa-Fekete, R.; Sz \"o r \'e nyi, B.; Weng, P.; and Mannor, S. 2017. Multi-objective bandits: Optimizing the generalized Gini index. In International Conference on Machine Learning, 625--634. PMLR

    2017

  6. [14]

    ???? Training for Stable Explanation for Free

    Chen, C.; Guo, C.; Chen, R.; Ma, G.; Zeng, M.; Liao, X.; Zhang, X.; and Xie, S. ???? Training for Stable Explanation for Free. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

  7. [15]

    W.; Sutton, C.; Gehrmann, S.; et al

    Chowdhery, A.; Narang, S.; Devlin, J.; Bosma, M.; Mishra, G.; Roberts, A.; Barham, P.; Chung, H. W.; Sutton, C.; Gehrmann, S.; et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240): 1--113

    2023

  8. [16]

    Collier, M.; and Llorens, H. U. 2018. Deep contextual multi-armed bandits. arXiv preprint arXiv:1807.09809

    2018 arXiv

  9. [17]

    Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  10. [18]

    Du, W.; Chen, J.; Zhang, X.; Ma, Z.; and Liu, S. 2024. Molecule joint auto-encoding: trajectory pretraining with 2D and 3D diffusion. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23. Curran Associates Inc

    2024

  11. [19]

    Du, Z.; Qian, Y.; Liu, X.; Ding, M.; Qiu, J.; Yang, Z.; and Tang, J. 2021. Glm: General language model pretraining with autoregressive blank infilling. arXiv preprint arXiv:2103.10360

    2021 arXiv

  12. [20]

    Ennen, P.; Freddi, F.; Lin, C.-J.; Kung, P.-N.; Wang, R.; Yang, C.-Y.; Shiu, D.-s.; and Bernacchia, A. 2023. Hierarchical Representations in Dense Passage Retrieval for Question-Answering. In Proceedings of the Sixth Fact Extraction and VERification Workshop (FEVER), 17--28

    2023

  13. [21]

    Gamage, G.; Mills, N.; De Silva, D.; Manic, M.; Moraliyage, H.; Jennings, A.; and Alahakoon, D. 2024. Multi-Agent RAG Chatbot Architecture for Decision Support in Net-Zero Emission Energy Systems. In 2024 IEEE International Conference on Industrial Technology (ICIT), 1--6. IEEE

    2024

  14. [22]

    S.; and Chang, Y

    Guo, S.; Zou, L.; Chen, H.; Qu, B.; Chi, H.; Yu, P. S.; and Chang, Y. 2024. Sample Efficient Offline-to-Online Reinforcement Learning. IEEE Transactions on Knowledge and Data Engineering, 36(3): 1299--1310

    2024

  15. [23]

    V.; Laurent, T.; LeCun, Y.; Bresson, X.; and Hooi, B

    He, X.; Tian, Y.; Sun, Y.; Chawla, N. V.; Laurent, T.; LeCun, Y.; Bresson, X.; and Hooi, B. 2024. G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering. arXiv preprint arXiv:2402.07630

    2024 arXiv

  16. [24]

    Jenkins, S. 2017. The measurement of income inequality. In Economic inequality and poverty, 3--38. Routledge

    2017

  17. [25]

    J.; Madotto, A.; and Fung, P

    Ji, Z.; Lee, N.; Frieske, R.; Yu, T.; Su, D.; Xu, Y.; Ishii, E.; Bang, Y. J.; Madotto, A.; and Fung, P. 2023. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12): 1--38

    2023

  18. [26]

    Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D

    Jiang, A. Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D. S.; Casas, D. d. l.; Bressand, F.; Lengyel, G.; Lample, G.; Saulnier, L.; et al. 2023 a . Mistral 7B. arXiv preprint arXiv:2310.06825

    2023 arXiv

  19. [27]

    X.; and Wen, J.-R

    Jiang, J.; Zhou, K.; Dong, Z.; Ye, K.; Zhao, W. X.; and Wen, J.-R. 2023 b . Structgpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645

    2023 arXiv

  20. [28]

    S.; Wu, L.; Edunov, S.; Chen, D.; and Yih, W.-t

    Karpukhin, V.; Oguz, B.; Min, S.; Lewis, P. S.; Wu, L.; Edunov, S.; Chen, D.; and Yih, W.-t. 2020. Dense Passage Retrieval for Open-Domain Question Answering. In EMNLP (1), 6769--6781

    2020

  21. [29]

    N.; and Veinott Jr, A

    Katehakis, M. N.; and Veinott Jr, A. F. 1987. The multi-armed bandit problem: decomposition and computation. Mathematics of Operations Research, 12(2): 262--268

    1987

  22. [30]

    Langford, J.; and Zhang, T. 2007. The epoch-greedy algorithm for multi-armed bandits with side information. Advances in neural information processing systems, 20

    2007

  23. [31]

    Lee, S.; Seo, Y.; Lee, K.; Abbeel, P.; and Shin, J. 2022. Offline-to-online reinforcement learning via balanced replay and pessimistic q-ensemble. In Conference on Robot Learning, 1702--1712. PMLR

    2022

  24. [32]

    u ttler, H.; Lewis, M.; Yih, W.-t.; Rockt \

    Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal, N.; K \"u ttler, H.; Lewis, M.; Yih, W.-t.; Rockt \"a schel, T.; et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33: 9459--9474

    2020

  25. [33]

    Li, L.; Chu, W.; Langford, J.; and Schapire, R. E. 2010. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, 661--670

    2010

  26. [34]

    LlamaIndex. 2024. Prompt Engineering for RAG. Accessed: 2024-05-16

    2024

  27. [35]

    Luo, H.; Tang, Z.; Peng, S.; Guo, Y.; Zhang, W.; Ma, C.; Dong, G.; Song, M.; Lin, W.; et al. 2023 a . Chatkbqa: A generate-then-retrieve framework for knowledge base question answering with fine-tuned large language models. arXiv preprint arXiv:2310.08975

    2023 arXiv

  28. [36]

    Luo, L.; Ju, J.; Xiong, B.; Li, Y.-F.; Haffari, G.; and Pan, S. 2023 b . Chatrule: Mining logical rules with large language models for knowledge graph reasoning. arXiv preprint arXiv:2309.01538

    2023 arXiv

  29. [37]

    Luo, L.; Li, Y.-F.; Haffari, G.; and Pan, S. 2023 c . Reasoning on graphs: Faithful and interpretable large language model reasoning. arXiv preprint arXiv:2310.01061

    2023 arXiv

  30. [38]

    Mahajan, A.; and Teneketzis, D. 2008. Multi-armed bandit problems. In Foundations and applications of sensor management, 121--151. Springer

    2008

  31. [39]

    Mehrotra, R.; Xue, N.; and Lalmas, M. 2020. Bandit based optimization of multiple objectives on a music streaming platform. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, 3224--3233

    2020

  32. [40]

    OpenAI. 2024. ChatGPT. https://openai.com/chatgpt. Accessed: 2024-05-20

    2024

  33. [41]

    OpenAI, R. 2023. Gpt-4 technical report. arxiv 2303.08774. View in Article, 2(5)

    2023 arXiv

  34. [42]

    Pan, S.; Luo, L.; Wang, Y.; Chen, C.; Wang, J.; and Wu, X. 2024. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering

    2024

  35. [43]

    Petroni, F.; Piktus, A.; Fan, A.; Lewis, P.; Yazdani, M.; De Cao, N.; Thorne, J.; Jernite, Y.; Karpukhin, V.; Maillard, J.; et al. 2020. KILT: a benchmark for knowledge intensive language tasks. arXiv preprint arXiv:2009.02252

    2020 arXiv

  36. [44]

    K.; Griffin, D.; Le, F.; and Rio, M

    Reis, J.; Rocha, M.; Phan, T. K.; Griffin, D.; Le, F.; and Rio, M. 2019. Deep Neural Networks for Network Routing. In 2019 International Joint Conference on Neural Networks (IJCNN), 1--8

    2019

  37. [45]

    Sanh, V.; Debut, L.; Chaumond, J.; and Wolf, T. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108

    2019 arXiv

  38. [46]

    Sawarkar, K.; Mangal, A.; and Solanki, S. R. 2024. Blended RAG: Improving RAG (Retriever-Augmented Generation) Accuracy with Semantic Search and Hybrid Query-Based Retrievers. arXiv preprint arXiv:2404.07220

    2024 arXiv

  39. [47]

    Shi, Q.; Xiao, F.; Pickard, D.; Chen, I.; and Chen, L. 2023. Deep neural network with linucb: A contextual bandit approach for personalized recommendation. In Companion Proceedings of the ACM Web Conference 2023, 778--782

    2023

  40. [48]

    Slivkins, A. 2011. Contextual Bandits with Similarity Information. In Kakade, S. M.; and von Luxburg, U., eds., Proceedings of the 24th Annual Conference on Learning Theory, volume 19 of Proceedings of Machine Learning Research, 679--702. Budapest, Hungary: PMLR

    2011

  41. [49]

    Sun, J.; Xu, C.; Tang, L.; Wang, S.; Lin, C.; Gong, Y.; Shum, H.-Y.; and Guo, J. 2023. Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph. arXiv:2307.07697

    2023 arXiv

  42. [50]

    Sun, W.; Qin, Z.; Li, D.; Shen, X.; Qiao, Y.; and Zhong, Y. 2024 a . Linear Attention Sequence Parallelism. arXiv preprint arXiv:2404.02882

    2024 arXiv

  43. [51]

    Sun, W.; Qin, Z.; Sun, W.; Li, S.; Li, D.; Shen, X.; Qiao, Y.; and Zhong, Y. 2024 b . CO2: Efficient distributed training with full communication-computation overlap. arXiv preprint arXiv:2401.16265

    2024 arXiv

  44. [52]

    Talmor, A.; and Berant, J. 2018. The web as a knowledge-base for answering complex questions. arXiv preprint arXiv:1803.06643

    2018 arXiv

  45. [53]

    Tekin, C.; and Tur g ay, E. 2018. Multi-objective contextual multi-armed bandit with a dominant objective. IEEE Transactions on Signal Processing, 66(14): 3799--3813

    2018

  46. [54]

    Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  47. [55]

    T.; Liu, Y.; Williams, J

    Wanigasekara, N.; Liang, Y.; Goh, S. T.; Liu, Y.; Williams, J. J.; and Rosenblum, D. S. 2019. Learning Multi-Objective Rewards and User Utility Function in Contextual Bandits for Personalized Ranking. In IJCAI, volume 19, 3835--3841

    2019

  48. [56]

    Weymark, J. A. 1981. Generalized Gini inequality indices. Mathematical Social Sciences, 1(4): 409--430

    1981

  49. [57]

    H.; Shi, P.; Zhong, R.; Scholak, T.; Yasunaga, M.; Wu, C.-S.; Zhong, M.; Yin, P.; Wang, S

    Xie, T.; Wu, C. H.; Shi, P.; Zhong, R.; Scholak, T.; Yasunaga, M.; Wu, C.-S.; Zhong, M.; Yin, P.; Wang, S. I.; et al. 2022. UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models. In Proceedings of the 2022 Conference on Empiric...

    2022

  50. [58]

    J.; Guevara, M.; Wang, T.; Deshpande, M.; Wang, X.; and Li, Z

    Xu, Z.; Cruz, M. J.; Guevara, M.; Wang, T.; Deshpande, M.; Wang, X.; and Li, Z. 2024. Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering. arXiv preprint arXiv:2404.17723

    2024 arXiv

  51. [59]

    Yih, W.-t.; Richardson, M.; Meek, C.; Chang, M.-W.; and Suh, J. 2016. The value of semantic parse labeling for knowledge base question answering. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 201--206

    2016

  52. [60]

    H.; Wang, J.; Hu, Y.; Wang, W.; Wang, Z.; and Xiang, B

    Yu, D.; Zhang, S.; Ng, P.; Zhu, H.; Li, A. H.; Wang, J.; Hu, Y.; Wang, W.; Wang, Z.; and Xiang, B. 2022. Decaf: Joint decoding of answers and logical forms for question answering over knowledge bases. arXiv preprint arXiv:2210.00063

    2022 arXiv

  53. [61]

    Zhang, P.; Xiao, S.; Liu, Z.; Dou, Z.; and Nie, J.-Y. 2023. Retrieve anything to augment large language models. arXiv preprint arXiv:2310.07554

    2023

  54. [62]

    Zhou, D.; Li, L.; and Gu, Q. 2020. Neural contextual bandits with ucb-based exploration. In International Conference on Machine Learning, 11492--11502. PMLR

    2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.