{"id":"9e888fee-515b-4638-ba76-10f2ca473aa8","arxiv_id":"2412.07618","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A multi-armed bandit router that selects knowledge-graph retrieval methods and adapts online to non-stationary conditions improves KGQA accuracy and latency over static baselines.","lead":"This paper proposes a multi-armed bandit system that chooses among knowledge-graph retrieval methods for every question, learning from feedback to adapt as queries and backends change. In tests, the adaptive router beats static routers and single-retriever baselines on two question-answering benchmarks, particularly when the retrieval backend is upgraded or the question domain shifts.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Non-stationary gains conflate test-time label access with bandit adaptation; missing online adaptive baseline leaves central claim unsupported.","rationale":"The reader's weakest assumption (real-time binary feedback availability) is a legitimate deployment concern, but the more immediately load-bearing issue is the absence of an online adaptive baseline in the non-stationary experiments. The paper's own claim is that its system 'leverage[s] partial information during testing' and that this drives the gains; yet the baselines are offline and never see this information, so the comparison is asymmetric. This is an internal-validity threat that could overturn the central claim even under the paper's stated assumptions. The reader's rationale mentions the missing online bandit baselines but did not elevate it to the weakest assumption; I partially agree with the reader's identification. My proposed test — an online fine-tuned router given identical feedback — would settle whether the bandit mechanism per se adds value. The verdict remains CONDITIONAL (unchanged) because the issue is addressable with additional controlled baselines and significance testing.","tokens_in":14577,"tokens_out":5066,"duration_ms":44991,"concrete_test":"Add an online adaptive baseline to Table 2: e.g., update the NN-Router (or a linear contextual bandit such as LinUCB) after each query using the same (hit, recall, delay) feedback and the same epsilon-greedy exploration, in both the retriever-upgrade and domain-shift scenarios. Recompute hit/recall/delay across the same ten seeds; if this baseline matches GGI-MAB within error bars (or exceeds it), the claimed superiority of the GGI-MAB adaptation mechanism is unsupported. Also report paired bootstrap confidence intervals on the differences between GGI-MAB and this online baseline.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 (RQ2) attributes large non-stationary improvements (e.g., 76.35 vs 69.57 hit under domain shift, 84.80 vs 83.74 under retriever upgrade) to the model's ability to \"leverage partial information during testing\" (Section 3.2, RQ2). Inspection of Table 2 shows that GGI-MAB is the only method that performs per-query updates using the test-set hit/recall/delay feedback. The baselines — Retrieval Ensemble, Offline MO-MAB, NN Router, LLM Router — are static or offline and never receive test-time labels. The reported advantage therefore conflates the value of the proposed bandit and GGI mechanism with the informational advantage of seeing the ground-truth feedback during evaluation. In the retriever-upgrade scenario, the action space changes from Think-on-Graph to Reason-on-Graph; in the domain-shift scenario, the query distribution changes from WebQSP to CWQ. Any online learner that can adapt to these changes should benefit from the same per-query labels, but no such controlled comparison is provided. Even granting the paper's explicit assumption that binary hit labels are available per query (Section 3.1), the experimental design does not isolate the contribution of the adaptive router. The central claim of significant superiority in non-stationary environments is therefore not established by the reported experiments.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GGI-MO-MAB, a DistilBERT-based contextual multi-armed bandit router that selects among dense retrieval, ChatKBQA-style SPARQL generation, and Reason-on-Graph KG-agent retrieval for KG-based RAG. The router is pretrained offline using accuracy and efficiency losses and is then updated per query with hit, recall, and delay feedback, with the Generalized Gini Index used to aggregate the multiple objectives. Experiments on WebQSP and CWQ report gains over static routers and ensembles in stationary settings (Table 1) and over offline/static baselines in two simulated non-stationary settings: backend retriever upgrade and domain shift (Table 2). The central claim is that online multi-objective bandit adaptation yields significant robustness gains in non-stationary environments while retaining state-of-the-art stationary performance.","tokens_in":14824,"tokens_out":5606,"duration_ms":48507,"significance":"If the non-stationary claim were cleanly established, the paper would be a useful practical contribution: it demonstrates a lightweight router that can trade off accuracy, recall, and latency across heterogeneous KG retrievers, ships code and data, evaluates with multiple LLM generators, and includes an additional degradation experiment (Table 4). The use of the GGI for reward aggregation is principled, and the stationary comparison is informative. However, the experimental design in Table 2 currently conflates the value of the online adaptive algorithm with the informational advantage of test-time label access, so the headline non-stationary conclusion is not yet supported.","major_comments":[{"comment":"The non-stationary comparison does not isolate the proposed adaptation mechanism. GGI-MAB is the only method that receives per-query hit, recall, and delay feedback during testing and updates its parameters; Retrieval Ensemble, Offline MO-MAB, NN Router, and LLM Router are static or offline and therefore cannot exploit this information. The observed gains, such as 76.35 versus 69.57 hit under domain shift, may reflect the extra label access rather than the bandit or GGI machinery. The paper should include online adaptive baselines under the identical per-query-update protocol, for instance LinUCB, Thompson Sampling, and SO-Deep-MAB, which already appear in Table 3, plus an ablated version of GGI-MAB that is never updated online, to separate adaptation from label exploitation.","section":"§3.2, Table 2"},{"comment":"The online environment is simulated by assuming that a binary hit label is immediately available for every test query, stated as \"simulate an online environment with a hit value (0 or 1) to approximate binary user feedback.\" Real user feedback is delayed, sparse, and noisy, and dataset-derived hit labels are not equivalent to it. Because the non-stationary results depend on per-query updates with these labels, the reported numbers should be framed as an oracle-feedback upper bound, with additional experiments under delayed, missing, or corrupted rewards to test robustness.","section":"§3.1"},{"comment":"The update rule does not clearly implement partial-information feedback. Equations (2) and (3) define l1 = MSE(max(fθ(x)), h) and l2 = MSE(max(fθ(x)), rc), so the loss depends only on the maximum predicted value, not on the value of the arm actually selected during epsilon-greedy exploration. When an exploratory arm is chosen, the observed hit or recall is attributed to the maximum arm rather than to the selected arm, and when the selected arm is not the maximum the model receives no gradient for that arm from these terms. The loss should be rewritten in terms of the selected action's predicted reward, for example fθ(x)[a], to be consistent with the stated partial-feedback setup.","section":"§2.2, Eqs. (2)–(3)"},{"comment":"The non-stationary protocol is underspecified. The text says training uses Think-on-Graph and testing switches to Reason-on-Graph, and that training on WebQSP is followed by testing on CWQ, but it does not state how many test queries are used for online updates, whether updates are applied in a single chronological pass, what epsilon schedule is used, how often evaluation is performed, or how the mean and standard deviation over ten seeds are computed when the model changes during testing. Without this information, the reader cannot determine whether the reported gains come from the adaptation procedure or from a particular test-time update protocol. Please provide the exact protocol for both non-stationary scenarios.","section":"§3.2, Algorithm 1"}],"minor_comments":[{"comment":"The algorithm initializes \"equal initial weights\" for w, but the GGI definition requires w1 > w2 > ... > wd; equal weights make the ordering τ irrelevant and collapse the GGI to a plain weighted sum. Clarify whether the equal initialization is temporary or whether the strict ordering is enforced during training.","section":"Algorithm 1 line 2 and Eq. (5)"},{"comment":"The method name is inconsistent: \"GGI-MAB\" is used in Tables 1 and 2, while \"GGI-MO-MAB\" is used in Tables 3 and 4. Please choose one name and use it throughout.","section":"Tables 1–4"},{"comment":"The citation \"Chen et al. (????)\" appears in the reference list as \"Training for Stable Explanation for Free\" without a year or venue, and \"Chen et al.\" is used in Section 6.6 without a full citation. This reference needs to be completed or removed.","section":"References"},{"comment":"The paper claims at least ten independent rounds with different seeds, but Table 1 reports no standard deviations. Please report variability consistently across all tables and provide confidence intervals or significance tests for the \"significantly outperforms\" claims.","section":"§3.1, Tables 1 and 3"},{"comment":"The notation for delay is inconsistent: the text writes d_i and \"di\" interchangeably, while Eq. (1) uses d_i and d_j. Please unify the notation.","section":"§2.2, Eq. (1)"},{"comment":"The citation \"lla 2024\" and the LlamaIndex selector documentation entry are inconsistent; provide the proper author, title, and venue information.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the missing controlled comparison for non-stationary adaptation. If the authors add online adaptive baselines with identical per-query label access, clarify the update protocol, and address the loss-formulation concern in Eqs. (2)–(3), the paper may become acceptable. I would not reject on the stationary results alone, but the non-stationary headline must be made credible before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The punchline: this is a sensible engineering paper that applies deep contextual multi-armed bandits to choose among KG retrieval methods in RAG, using GGI to balance hit, recall, and latency. The stationary experiments are reasonably convincing, and the idea of treating each retriever as a bandit arm with per-query feedback is a genuine application gap. But the non-stationary comparison, which is the paper's main selling point, does not include any online adaptive baseline. The claimed advantage is not actually isolated, and that needs to be fixed before the central claim can be taken seriously.\n\nWhat's new: previous KG-RAG systems either pick a single retriever or use a static router; the concurrent Blended RAG does not do online adaptation. The paper also includes a degradation scenario (Section 6.5) where one retriever fails, which is a nice touch. Code is available, and they report mean ± std over ten seeds, so the empirical work is reproducible in principle.\n\nWhere it's soft: the non-stationary experiments (Table 2) pit the online GGI-MAB against offline or static baselines only. The baselines never see test-time labels, while GGI-MAB updates per query on exactly those labels. The stress-test note is right: the result conflates the bandit mechanism with the informational advantage of oracle feedback. The fix is straightforward—run online baselines such as LinUCB, Thompson Sampling, and SO-Deep-MAB under the same non-stationary protocols. The paper already uses those methods in Table 3, so this is a missing control, not a missing concept. Without it, the non-stationary claim is unsupported. The stationary gains over the best static router are small (about 0.4–0.8 points in hit/recall) and no significance test is reported; the standard deviations for Table 1 are not given, so we cannot tell whether the difference is meaningful. The GGI loss is defined but not derived, and the weights w are initialized equal with no sensitivity analysis—minor. The assumption that a binary hit label is available per query at test time is explicit but may not hold in real deployments; the authors should acknowledge delayed or sparse feedback more prominently, since that assumption is load-bearing for the online adaptation story.\n\nWho this is for: anyone building RAG serving systems with multiple retrievers and wanting automatic adaptation. It deserves a serious referee, but the referee should demand the missing baselines and significance tests. My recommendation: send to peer review with clear instructions that the non-stationary claim needs a controlled comparison against online adaptive baselines. If that comes back empty, the paper reduces to a modest stationary improvement plus a system description.","headline":"Good engineering idea for adaptive KG-RAG retriever selection, but the non-stationary experiments lack online adaptive baselines; the central claim needs that control before it holds.","tokens_in":15371,"tokens_out":2767,"would_cite":false,"duration_ms":24688,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Treating each retrieval method as a bandit arm and updating on per-query feedback beats single-retriever, ensemble, and static-router RAG in stationary and non-stationary settings.","keywords":["retrieval-augmented generation","knowledge graphs","multi-armed bandit","contextual bandit","Generalized Gini Index","non-stationary environments","knowledge graph question answering","online learning"],"falsifier":"Run the retriever-upgrade and domain-shift scenarios again with the binary hit label delayed by a fixed number of queries (for example, ten) or with 20% of labels randomly dropped, and check whether the reported gains over static routers survive; if accuracy falls back to the static-router level, the claim that online per-query feedback carries the result is disproved.","tokens_in":14335,"feed_emoji":"🎯","tokens_out":14676,"duration_ms":108710,"temperature":0.7,"pith_summary":"Knowledge-graph retrieval-augmented generation systems can draw on several retrieval methods with different strengths — dense embeddings are fast, SPARQL generation is precise, and graph agents handle multi-hop reasoning — but most deployments either commit to one method or use a static router that requires fully labeled data. This paper tries to establish that treating each retrieval method as an arm in a deep contextual multi-armed bandit, and updating the router after every query with real-time feedback, makes the system adapt to non-stationarity of two kinds: shifting query domains and backend retriever upgrades. On the WebQSP and ComplexWebQuestions benchmarks, the paper reports that this GGI-MAB router outperforms single-retriever, ensemble, and static-router baselines in stationary settings and holds or improves accuracy when the environment changes. The reason this matters is that real deployments face exactly these shifts, and current RAG designs lack an explicit mechanism to keep learning from user responses.","feed_headline":"Bandit router adapts KG retrieval per query and beats static routers","feed_subtitle":"Reported hit and recall gains over static routers, plus adaptation to query shifts and retriever upgrades.","key_machinery":"The load-bearing component is a deep contextual multi-armed bandit whose arms are the retrieval methods themselves — DECAF dense retrieval, ChatKBQA SPARQL generation, and Reason-on-Graph KG-agent reasoning — with a DistilBERT query encoder producing arm-selection probabilities and an epsilon-greedy rule balancing exploration with exploitation. What makes the bandit multi-objective is the Generalized Gini Index (GGI), an inequality metric used here to aggregate three losses — mean-squared error against hit, mean-squared error against recall, and KL divergence against a retrieval-delay distribution — into a single scalar loss $Loss_{GGI}(\\theta) = \\sum_{i=1}^D w_i (l_i)_\\tau$, where the weights favour balanced performance so that no single objective dominates. The training procedure is offline pretraining on a static dataset followed by online per-query updates from partial-information feedback, and that online-update loop is the mechanism intended to give the system its adaptability.","core_discovery":"The paper's central claim is that the system does not need to know in advance which retrieval method is best for a query: a router that selects among dense retrieval, SPARQL-based retrieval, and a knowledge-graph agent can learn the right choice per query and keep revising it as conditions change. The router is a DistilBERT encoder that maps a query to a distribution over retrieval arms, selects with an epsilon-greedy rule, and updates after each query using the observed hit, recall, and retrieval delay of the chosen arm, which means the feedback is partial-information because unselected arms are not observed. On WebQSP, the method reports an 86.64 hit rate and 75.60 recall, compared with 86.20 and 75.03 for the best static NN-Router, and on CWQ it reports 79.35 and 72.02 versus 78.53 and 71.52. In the non-stationary retriever-upgrade scenario it reports 84.80 hit and 72.24 recall versus 83.74 and 67.52 for the retrieval ensemble, and in the domain-shift scenario 76.35 and 69.47 versus 67.93 and 68.01. The paper's assertion is that these gains come from the online adaptive selection mechanism rather than from any single retriever or from a static combination of retrievers.","pith_inferences":["The same GGI-style multi-objective bandit could route among any set of modular LLM tools — search APIs, code interpreters, or external databases — wherever per-call latency and answer quality compete.","Because the reward in the experiments is a dataset-derived hit label, a practical deployment would need to convert noisy, delayed, or implicit user signals into per-query rewards; the reported gains should be read as an upper bound on what a real feedback loop would deliver.","The non-stationary comparisons pit an online-updating router against offline baselines, so a periodically retrained static router would be a sharper control for isolating the value of online adaptation.","The action space is limited to three retrieval arms; testing with many more arms would show whether epsilon-greedy exploration plus GGI updates scale, or whether structured exploration is needed."],"forward_implications":["A KG-based RAG system can be deployed without committing to a single retriever; the router will learn which retrieval method fits each query while the system is running.","Backend retriever upgrades need not trigger a full data-labeling and fine-tuning campaign, because per-query updates let the router discover the improved arm on its own.","Accuracy and responsiveness can be optimized together, since retrieval delay enters the same GGI loss as hit and recall.","Static neural routers that require complete labeled data and periodic fine-tuning are unnecessary in this setting, because partial-information feedback suffices for online adaptation."],"supporting_citations":[{"why":"supplies the retrieval-augmented generation paradigm the system extends.","marker":"(Lewis et al. 2020)"},{"why":"defines the Generalized Gini Index used for multi-objective reward aggregation.","marker":"(Weymark 1981)"},{"why":"provides the deep contextual multi-armed bandit model the router builds on.","marker":"(Collier and Llorens 2018)"},{"why":"supplies the DistilBERT query encoder that produces arm-selection probabilities.","marker":"(Sanh et al. 2019)"},{"why":"ChatKBQA is one of the three retrieval arms and a stationary baseline.","marker":"(Luo et al. 2023a)"},{"why":"Reason-on-Graph is one of the arms and the upgraded retriever in the non-stationary retriever-upgrade test.","marker":"(Luo et al. 2023c)"},{"why":"Think-on-Graph is a stationary baseline and the training-phase retriever for the upgrade scenario.","marker":"(Sun et al. 2023)"},{"why":"DECAF is the dense-retrieval arm in the router's action space and a stationary baseline.","marker":"(Yu et al. 2022)"},{"why":"WebQSP is one of the two knowledge-graph question-answering benchmarks.","marker":"(Yih et al. 2016)"},{"why":"ComplexWebQuestions is the second, harder benchmark used for evaluation.","marker":"(Talmor and Berant 2018)"}],"fun_headline_variants":["Bandit router adapts retrieval per query, beats static in changing data","Online multi-armed bandit learns best KG retriever per query","Adaptive RAG router uses partial feedback to survive retriever shifts","Bandit router outperforms static routers in non-stationary KG QA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a correct/incorrect signal, called the hit label, is available immediately after every query and can stand in for real user feedback; if that signal is delayed, sparse, or noisy, the per-query online updates that drive the reported non-stationary gains cannot be applied as described.","fun_headline_variants_meta":{"raw":{"variants":["Bandit router adapts retrieval per query, beats static in changing data","Online multi-armed bandit learns best KG retriever per query","Adaptive RAG router uses partial feedback to survive retriever shifts","Bandit router outperforms static routers in non-stationary KG QA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001244,"raw_usage":{"total_tokens":5160,"prompt_tokens":1061,"completion_tokens":4099,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":677,"completion_tokens_details":{"reasoning_tokens":4020}},"tokens_in":677,"tokens_out":4099,"duration_ms":26021,"temperature":1.0,"reasoning_tokens":4020,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:39:03.303717+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the retriever-upgrade and domain-shift scenarios again with the binary hit label delayed by a fixed number of queries (for example, ten) or with 20% of labels randomly dropped, and check whether the reported gains over static routers survive; if accuracy falls back to the static-router level, the claim that online per-query feedback carries the result is disproved.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the Generalized Gini Index used for multi-objective reward aggregation."},{"cited_title":"Deep Contextual Multi-armed Bandits","cited_arxiv_id":"1807.09809","evidence_quote":"provides the deep contextual multi-armed bandit model the router builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"WebQSP is one of the two knowledge-graph question-answering benchmarks."}],"review_version":1}