Learning Query-Aware Budget-Tier Routing for Runtime Agent Memory

Bowen Jin; Chengwei Qin; Haodong Yue; Haozhen Zhang; Jianzhu Bao; Jiaxuan You; Quanyu Long; Tao Feng; Weizhi Zhang; Wenya Wang

REVIEW 4 major objections 5 minor 10 cited by

A lightweight RL router that picks a memory-processing budget per query can beat offline memory systems on both accuracy and cost, across three benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:00 UTC pith:AGHHOCHF

load-bearing objection Useful modular testbed, but the router's causal role is untested — missing a fixed-tier control; needs a major revision before the headline frontier claims are believable. the 4 major comments →

arxiv 2602.06025 v3 pith:AGHHOCHF submitted 2026-02-05 cs.CL cs.AIcs.LG

Learning Query-Aware Budget-Tier Routing for Runtime Agent Memory

Haozhen Zhang , Haodong Yue , Tao Feng , Quanyu Long , Jianzhu Bao , Bowen Jin , Weizhi Zhang , Xiao Li

show 3 more authors

Jiaxuan You Chengwei Qin Wenya Wang

This is my paper

classification cs.CL cs.AIcs.LG

keywords runtime memorybudget-tier routingquery-aware memory extractionreinforcement learningLLM agentsperformance-cost trade-offPPOmemory-augmented generation

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that runtime, query-aware memory extraction can outperform the usual 'build once, use always' offline memory pipelines, while giving system builders explicit control over the accuracy–cost trade-off. The claim is that by splitting memory processing into modules—filter, entity, temporal, topic, summary—and letting a small learned router choose a LOW/MID/HIGH budget tier for each module per query, an agent gets better answers and better accuracy per dollar than existing memory-augmented systems. A sympathetic reader would care because this turns memory from a fixed preprocessing cost into a controllable runtime expense, which is exactly what real deployments need when latency and price matter. The paper further claims that the same router can transfer to a different backbone model without retraining, and that the three tiering strategies behave differently across budget regimes.

Core claim

BudgetMem claims that a modular runtime memory pipeline, where each module exposes three budget tiers and a shared reinforcement-learned router selects tiers query-by-query, delivers higher answer quality and better accuracy–cost frontiers than offline, query-agnostic memory systems such as MemoryBank, A-MEM, LightMem, and others. Concretely, on LoCoMo, LongMemEval, and HotpotQA, all three BudgetMem variants—implementation tiering, reasoning tiering, and capacity tiering—surpass strong baselines in the performance-first regime, and tracing lambda from 0 to 0.9 yields smooth Pareto frontiers that envelope the baselines at both low and high cost.

What carries the argument

The central object is the budget-tiered modular pipeline: a fixed sequence of modules (filter, entity, temporal, topic, summary), each exposing a uniform LOW/MID/HIGH interface. A shared lightweight neural router, trained with PPO under a cost-aware reward that combines task accuracy and normalized extraction cost, selects a tier for every module invocation based on the query and intermediate states. The three tiering strategies instantiate the tiers along orthogonal axes—implementation (heuristics to BERT to LLM), reasoning (direct to CoT to reflection), and capacity (small to large backbone)—so the same framework can compare which axis yields the best trade-offs at different budgets.

Load-bearing premise

The claimed accuracy–cost dominance assumes that a dollar of cost means the same thing for BudgetMem as for each baseline—cheap non-LLM tiers are counted as negligible and offline construction costs of baselines are not included, so the frontier comparisons depend on that cost-accounting choice.

What would settle it

Recompute the cost column and the Pareto curves with (a) amortized serving or hardware costs for the non-LLM LOW tiers of BudgetMem, and (b) the offline summarization/indexing costs of MemoryBank, A-MEM, LightMem, etc., included for all methods; if BudgetMem's frontier no longer dominates under that accounting, the trade-off claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

If correct, memory extraction no longer needs offline preprocessing: deferring all computation to query time can beat offline pipelines on both accuracy and cost.
The same budget-tier interface can be dropped onto other modular agent pipelines—retrieval, planning, tool use—turning any multi-stage process into a controllable cost knob.
A router trained on one backbone transfers to another without retraining, so deployments can swap the underlying LLM while keeping the budget policy intact.
Implementation and capacity tiering span wider cost ranges, while reasoning tiering gives fine-grained quality gains in a narrow band—so a builder can pick the tiering axis by their operating budget regime.
Module-level tier selection ratios show the router responds predictably to cost pressure, qualifying BudgetMem as a controllable performance–cost mechanism rather than a fixed trade-off point.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The cost accounting treats non-LLM tiers as negligible; a stricter accounting that includes serving infrastructure or GPU rental for those LOW tiers would likely compress the low-cost end of the reported frontier (my inference, not the paper's).
The transfer result hints that the routing policy captures query difficulty rather than backbone-specific quirks; an untested extension is to train the router on a small mixed-corpus set and measure cross-domain transfer directly.
The query-aware tier choices could be inspected to produce cheap-to-check policy explanations or a 'cost contract' for API users—an interpretive step the paper leaves implicit.
Because the paper shows retrieval size is a separate cost knob, a natural follow-up is to let the router also decide how many chunks to retrieve, coupling evidence coverage with tier choice.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Useful modular testbed, but the router's causal role is untested — missing a fixed-tier control; needs a major revision before the headline frontier claims are believable.

read the letter

BudgetMem is a genuinely reusable idea—module-level budget tiers with an RL-trained router for runtime memory extraction—and the paper evaluates it broadly. But the central claim is undercut by a missing control: the authors never compare the learned router against a fixed-tier policy (all-LOW, all-MID, all-HIGH, or random). Every experiment pits the full system (router + pipeline) against external baselines, so you cannot tell whether the router's query-aware decisions add anything beyond the modular pipeline itself. The only ablation (reward-scale alignment) shows that without alignment the router collapses to LOW tiers; that is a training-stability result, not a routing-effectiveness result. This is the load-bearing gap, and it makes the headline 'better accuracy–cost frontiers' over-stated.

What the paper does well: the three tiering axes (implementation, reasoning, capacity) are cleanly instantiated under one interface, and the analysis of when each axis pays off is genuinely helpful—reasoning tiering acts as a narrow-band quality knob while capacity tiering pushes the frontier outward. The transfer of a trained router to a different backbone (Qwen without retraining) is a nice generality check, and the retrieval-size sensitivity analysis is a useful practical datapoint. The modular abstraction itself is a contribution: future memory systems can plug in their own tiers and reuse the routing infrastructure.

Soft spots beyond the missing ablation: the cost model assigns zero cost to non-LLM tiers and excludes baselines' offline construction costs, so the cost comparisons are not like-with-like. That said, including offline costs would make the baselines more expensive, so this concern would likely favor BudgetMem rather than hurt it—but it is still a fairness problem in reporting. There are no error bars or significance tests on the headline numbers; the gaps are large, so they'd probably survive, but the paper doesn't show it. And Appendix Table 6 has a corrupted row: the per-category Judge scores for BudgetMem-REA at λ=0.1 do not average to the stated average. Careless errors like that undercut trust in the rest of the tables.

On the 'trade-off curves are by construction' worry: that is not a real flaw. The reward explicitly trades off task reward against cost, so the frontier is the training objective, not an independent prediction. The external validity rests entirely on the baseline comparison—which brings us back to the missing fixed-tier control.

Who this is for: agent-memory researchers who want a testbed for budget-aware runtime memory. The framework deserves serious referee time, but the revision must add an all-LOW/all-MID/all-HIGH baseline to isolate the router, correct the cost accounting, and fix the table error. If the code is released, that would also help. My recommendation: send to peer review; expect major revision.

Referee Report

4 major / 5 minor

Summary. The paper proposes BudgetMem, a runtime agent-memory framework in which memory extraction is decomposed into a modular pipeline (filter → entity/temporal/topic → summary) and each module exposes three budget tiers (LOW/MID/HIGH). A shared lightweight router selects a tier for each module at query time; it is trained with PPO under a cost-aware reward that combines task reward and a normalized cost reward (Eq. 7). Three tiering strategies are instantiated — implementation (rules → BERT → LLM), reasoning (direct → CoT → reflection), and capacity (small → medium → large LLM) — and evaluated on LoCoMo, LongMemEval, and HotpotQA against seven memory-augmented agent baselines. The paper claims that BudgetMem outperforms strong baselines in the performance-first setting and traces better accuracy–cost frontiers when the cost weight λ is varied, with additional analyses of reward-scale alignment, module-level tier selection, and retrieval-size sensitivity.

Significance. If the claims hold, BudgetMem would be a practical step toward explicit performance–cost control in runtime agent memory, and the systematic comparison of three tiering axes is a useful contribution. The paper's strengths include a broad evaluation across three benchmarks and two base LLMs, a cross-backbone transfer test (LLaMA-trained router applied to Qwen without retraining), a clean PPO formulation with robust cost normalization and reward-scale alignment, and released code. However, the central causal claim about the learned router is not isolated from the modular pipeline, and the cost comparisons rest on an accounting choice that treats non-LLM computation as free. These issues are fixable with additional experiments, but they are load-bearing for the paper's headline conclusions.

major comments (4)

[Sec. 6.1/6.2, Table 1, Fig. 2] The paper attributes its reported gains to the learned query-aware router, yet every comparison in Table 1 and Fig. 2 uses the full BudgetMem system (router + filter/entity/temporal/topic/summarization modules) against external memory baselines. There is no fixed-tier control (e.g., always-LOW, always-MID, always-HIGH, or random routing) and no condition that removes the router. The only ablation, Fig. 3, toggles reward-scale alignment and does not compare routing against no routing. Add fixed-tier baselines to establish the router's causal contribution; without them, the claimed accuracy–cost frontier could be produced by the modular pipeline alone.
[Sec. 4.3, Eq. (8)–(9); Sec. 5.1; Tables 1, 5–10] Costs are not necessarily compared on a like-for-like basis. Eq. (8) assigns zero cost to non-LLM tiers, and Sec. 5.1 computes baseline costs by summing API token usage at service pricing. If baselines include offline LLM construction costs while BudgetMem's non-LLM components (heuristics, BERT-based extractors, retrieval) are treated as free, the Pareto comparisons in Fig. 2 and the cost columns in Tables 1, 5–10 are distorted. State the cost boundary explicitly and include a sensitivity analysis that assigns nonzero costs (e.g., local GPU-hours or amortized inference cost) to non-LLM tiers for both BudgetMem and baselines.
[Appendix Table 6] The BudgetMem-REA row at COE=0.1 is internally inconsistent: the per-category Judge scores 18.24, 6.07, 11.93, 5.67 average to roughly 10.5, not the listed 55.25; the per-category F1 scores 45.17, 33.14, 40.63, 28.46 average to roughly 36.9, not 40.52. This error undermines confidence in the appendix data used for the trade-off analysis in Sec. 6.2 and must be corrected or explained.
[Sec. 6.2, Fig. 2; Eq. (7)] The smooth performance–cost frontiers are partly a direct consequence of the training objective: Eq. (7) explicitly penalizes cost as λ increases, so the downward cost trend with increasing λ is by construction. The empirical contribution should therefore be framed as the accuracy achieved at each cost point relative to baselines, not as the existence of a monotone trade-off curve. Please clarify this framing so that the frontier claims emphasize the accuracy-at-cost comparison rather than the shape of the curve.

minor comments (5)

[Sec. 5.2] The cross-backbone transfer (LLaMA-trained router → Qwen) is reported only as aggregated scores. It would be informative to analyze whether the router's tier selections remain semantically sensible under the new backbone (e.g., tier-selection agreement or per-module routing distributions).
[Eq. (9)] If all raw costs in the sliding window are zero (e.g., when all selected tiers are non-LLM), then Q5 = Q95 and the denominator in the normalization is zero. A numerical safeguard or a small floor on c_raw should be specified.
[Fig. 2] The figure shows smooth curves but does not label which λ values correspond to which points. Please add explicit λ labels or a color gradient so readers can interpret the trade-off direction.
[Appendix C] Several prompt templates contain formatting typos: '7{9' and '1{3' should be '7–9' and '1–3'. While this does not affect the results, it makes the prompts look unpolished.
[Table 1] The '†' footnote states 'no training using this base model' but does not specify whether all hyperparameters (including λ values for the trade-off curves) are identical across backbones. Clarify the transfer protocol.

Circularity Check

0 steps flagged

No significant circularity: BudgetMem's accuracy–cost results are comparative against external baselines, and the self-citations are not load-bearing.

full rationale

The paper's trade-off curves are generated by explicitly varying the cost weight λ in the reward r = r_task + λ·α·r_cost (Eq. 7), so the existence of a smooth accuracy–cost frontier is a direct consequence of the training objective, not an independent prediction. However, the paper's central claims are comparative: BudgetMem outperforms external memory baselines in the performance-first setting (Table 1) and its frontier is claimed to dominate baseline points (Fig. 2). These comparisons are not forced by the reward definition; they depend on the actual quality of the modular pipeline and the measured costs. The only self-citations (Router-R1 for sliding-window normalization, GraphRouter/Jin et al. as related work) are used for a normalization detail and background, not as load-bearing evidence for the main results. The absence of a fixed-tier baseline (e.g., all-LOW/all-HIGH) is a validation gap regarding the router's causal contribution, but it is not a circular reduction of the paper's claims to its inputs. No equation is self-referential in a way that makes the reported findings tautological relative to the stated assumptions.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 0 invented entities

No new physical or conceptual entities are introduced; 'budget tiers' and 'router' are abstractions over existing LLM components. The central claim is empirical, so the main burden is experimental: λ, top-K, chunk sizes, capacity choices, cost-normalization quantiles, and the cost model itself. The assumptions above are the load-bearing domain choices that the experimental results are conditional on.

free parameters (7)

Cost weight λ (COE) = Swept over 0, 0.05, 0.1, 0.3, 0.5, 0.7, 0.9
Eq. 7: controls the performance-cost trade-off. The paper's trade-off curves are generated by sweeping this knob, so the curves are a property of the reward design rather than an independent discovery.
Reward-scale alignment α = std(rtask)/(std(rcost)+ε), updated online
Eq. 10: adaptive scaling used during PPO training; it changes the effective reward balance and is tuned to training dynamics.
Cost-normalization quantiles Q5/Q95 = 5th and 95th percentiles of sliding window over √cost
Eq. 9: maps raw cost to [0,1]. The choice of quantiles affects the cost reward and therefore the learned routing policy.
Retrieval top-K = 5
Sec. 5.2 and 6.4: the number of chunks passed to the memory pipeline. The sensitivity analysis selects 5 as the best balance, so it is a data-dependent choice that bounds the evidence available to all modules.
Chunk sizes = 256 tokens (LoCoMo, LongMemEval); 1024 tokens (HotpotQA)
Sec. 5.2: fixed segmentation choice; affects what the retriever can retrieve and hence the whole downstream pipeline.
Capacity tier model choices = Llama 3B/8B/70B; Qwen 7B/32B/80B
Appendix A.4.3: defines LOW/MID/HIGH for the capacity strategy. These model identities directly set the cost and quality ladder for one of the three main comparisons.
PPO hyperparameters = lr=3e-4, batch size 32, ≤600 steps, entropy coef 0.01
Appendix A.3: standard RL training choices; the reported policy behavior depends on them, though they are not fitted to specific datasets.

axioms (6)

domain assumption Retrieved chunks C_q from Contriever contain the evidence needed to answer q
Eq. 1: the entire runtime pipeline operates on C_q. If retrieval misses key chunks, no module-level budget tier can recover them.
domain assumption LLM-as-a-judge scores are a faithful measure of answer quality
Used as both r_task during training and the reported 'Judge' metric. If the judge is biased or noisy, both the optimization and the evaluation shift.
domain assumption Token-usage × API price is a faithful proxy for extraction cost
Sec. 4.3/5.1: the cost axis in every table and frontier is this number. Non-LLM tiers are assigned zero cost, so the proxy is incomplete for implementation tiering.
ad hoc to paper Budget tiers preserve the module input-output contract
Sec. 4.2 and Appendix A.4: attribution of performance differences to tier choice assumes only internal compute changes, not the interface or downstream expectations.
ad hoc to paper A router trained on the LLaMA backbone transfers to Qwen without retraining
Sec. 5.2: the Qwen rows in Table 1 rely on this transfer. State-distribution shift between backbones is not analyzed, yet the same policy is used.
standard math PPO is a valid optimization algorithm for this routing MDP
Appendix A.3 relies on standard RL theory; the paper does not prove convergence or optimality, which is normal for this line of work.

pith-pipeline@v1.3.0-alltime-deepseek · 32314 in / 14309 out tokens · 139267 ms · 2026-08-03T04:00:53.648485+00:00 · methodology

0 comments

read the original abstract

Memory is increasingly central to Large Language Model (LLM) agents operating beyond a single context window, yet most existing systems rely on offline, query-agnostic memory construction that can be inefficient and may discard query-critical information. Although runtime memory utilization is a natural alternative, prior work often incurs substantial overhead and offers limited explicit control over the performance-cost trade-off. In this work, we present \textbf{BudgetMem}, a runtime agent memory framework for explicit, query-aware performance-cost control. BudgetMem structures memory processing as a set of memory modules, each offered in three budget tiers (i.e., \textsc{Low}/\textsc{Mid}/\textsc{High}). A lightweight router performs budget-tier routing across modules to balance task performance and memory construction cost, which is implemented as a compact neural policy trained with reinforcement learning. Using BudgetMem as a unified testbed, we study three complementary strategies for realizing budget tiers: implementation (method complexity), reasoning (inference behavior), and capacity (module model size). Across LoCoMo, LongMemEval, and HotpotQA, BudgetMem surpasses strong baselines when performance is prioritized (i.e., high-budget setting), and delivers better accuracy-cost frontiers under tighter budgets. Moreover, our analysis disentangles the strengths and weaknesses of different tiering strategies, clarifying when each axis delivers the most favorable trade-offs under varying budget regimes.

Figures

Figures reproduced from arXiv: 2602.06025 by Bowen Jin, Chengwei Qin, Haodong Yue, Haozhen Zhang, Jianzhu Bao, Jiaxuan You, Quanyu Long, Tao Feng, Weizhi Zhang, Wenya Wang, Xiao Li.

**Figure 1.** Figure 1: BudgetMem overview. Given a user query q, we retrieve raw chunks Cq from a chunked history (without offline memory preprocessing) and process them with a modular pipeline (filter → entity/temporal/topic → summary). Each module exposes LOW/MID/HIGH budget tiers instantiated by one of three strategies (implementation, reasoning, capacity). A shared lightweight router selects tiers module-wise based on the q… view at source ↗

**Figure 2.** Figure 2: Performance–cost trade-offs across tiering strategies on LoCoMo. By varying the cost weight λ, BudgetMem traces smooth, controllable frontiers that shift toward higher performance as budget increases, and envelop baselines in both low- and high-cost regimes. three variants: BudgetMem-IMP, BudgetMem-REA, and BudgetMem-CAP. Sec. 6.2 varies λ to trace performance– cost curves, Sec. 6.3 ablates cost modeling a… view at source ↗

**Figure 4.** Figure 4: Budget-tier selection ratios. Module-wise LOW/MID/HIGH routing ratios on LongMemEval under varying cost weights λ using the capacity tiering strategy. chunks on LoCoMo, evaluated under all three tiering strategies. Increasing the retrieval size predictably raises cost due to longer inputs and additional processing, and it often improves Judge score by providing more supporting evidence, reflecting the st… view at source ↗

**Figure 3.** Figure 3: Ablation of reward-scale alignment under capacity tiering strategy on LoCoMo. 6.4. Discussion Budget tier selection ratio [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

**Figure 5.** Figure 5: Retrieval-size sensitivity on LoCoMo. Cost and Judge versus the number of retrieved raw chunks, evaluated under all three tiering strategies. 7. Conclusion We present BudgetMem, a runtime agent memory framework for explicit performance–cost control in on-demand memory extraction. BudgetMem equips each module in a modular memory pipeline with LOW/MID/HIGH budget tiers and learns a lightweight router to se… view at source ↗

Review history (2 revisions) →

discussion (0)

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

LLM Agents Are Latent Context Managers: Eliciting Self-Managed Context via State Proprioception
cs.CL 2026-06 conditional novelty 7.0

Exposing per-block token, recency, and access metadata with lossless archive/recovery elicits latent context management in untrained LLM agents and roughly doubles LOCA-Bench success under pressure.
LLM Agents Are Latent Context Managers: Eliciting Self-Managed Context via State Proprioception
cs.CL 2026-06 conditional novelty 7.0

A training-free, model-agnostic dashboard that exposes per-block context state (tokens, age, budget) with lossless archive/recovery improves long-horizon tool-agent performance on LOCA-Bench, BrowseComp-Plus, and GAIA.
Learning What to Remember: Observability-Safe Memory Retention via Constrained Optimization for Long-Horizon Language Agents
cs.AI 2026-06 unverdicted novelty 7.0

OSL-MR is a learning-augmented framework that casts memory retention as constrained stochastic optimization under partial observability and outperforms heuristic baselines on LoCoMo and LongMemEval.
LLM Agents Are Latent Context Managers: Eliciting Self-Managed Context via State Proprioception
cs.CL 2026-06 unverdicted novelty 6.0

VISTA supplies LLM agents with a visible proprioceptive dashboard of typed context blocks, enabling untrained self-management that lifts performance on long-horizon tool-use benchmarks across multiple model scales.
LLM Agents Are Latent Context Managers: Eliciting Self-Managed Context via State Proprioception
cs.CL 2026-06 conditional novelty 6.0

Giving LLM agents a visible dashboard of their context blocks, plus lossless archive and recovery, sharply improves long-horizon tool-agent performance without any training.
Learning What to Remember: Observability-Safe Memory Retention via Constrained Optimization for Long-Horizon Language Agents
cs.AI 2026-06 unverdicted novelty 6.0

OSL-MR applies constrained stochastic optimization and learning to memory retention in long-horizon agents, outperforming recency and heuristic baselines on LoCoMo and LongMemEval under tight budgets.
EMBER: Efficient Memory via Budgeted Evidence Retention for Long-Horizon Agents
cs.CL 2026-06 conditional novelty 5.0

EMBER learns to retain budgeted, source-backed evidence capsules so long-horizon agents recover answer-relevant facts without rereading the full history.
EMBER: Efficient Memory via Budgeted Evidence Retention for Long-Horizon Agents
cs.CL 2026-06 unverdicted novelty 5.0

EMBER learns to retain source-backed evidence capsules under a fixed token budget, improving F1, Retain-Recall, and Read-Recall on LongMemEval-RR over budgeted baselines.
Exploring Cross-Scenario Generality of Agentic Memory Systems: Diagnostics and a Strong Baseline
cs.AI 2026-06 unverdicted novelty 5.0

An agentic harness letting the LLM self-manage flat text-file storage via tool calls outperforms eight prior memory systems on cross-scenario generality across QA, chat, trajectory, stress-test, and long-horizon tasks.
Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering
cs.SE 2026-04 accept novelty 5.0

LLM agent progress depends on externalizing cognitive functions into memory, skills, protocols, and harness engineering that coordinates them reliably.

Reference graph

Works this paper leans on

14 extracted references · cited by 5 Pith papers

[1]

Correctness: - Is the model answer factually consistent with ANY of the correct answers? - Does it avoid contradictions or introducing false information?
[2]

Relevance: - Does the answer address the question directly without unnecessary content?
[3]

explanation

Completeness: - Does the answer include all essential information needed to fully answer the question?,→ - Partial answers are allowed but should receive lower scores. Scoring Rules: - Score = 1.0 if the answer is fully correct. - Score = 0.5 if the answer is partially correct but incomplete or slightly inaccurate.,→ - Score = 0.0 if the answer is incorre...
[5]

N" [date_time=

A`<memories>`section containing individual`<memory>`elements. Each memory is formatted as: ``` <memory index="N" [date_time="..." session_id="..." dia_id="..."]> memory content text </memory> ``` Where: -`index`is the memory's position in the list. -`date_time`,`session_id`,`dia_id`are optional metadata attributes. - The text between the tags is the memor...
[7]

N" [date_time=

A`<memories>`section containing individual`<memory>`elements. Each memory is formatted as: ``` <memory index="N" [date_time="..." session_id="..." dia_id="..."]> memory content text </memory> ``` Where: -`index`is the memory's position in the list. -`date_time`,`session_id`,`dia_id`are optional metadata attributes. - The text between the tags is the memor...
[8]

A`<query>`section containing the user's question
[9]

N" [date_time=

A`<memories>`section containing individual`<memory>`elements. Each memory is formatted as: ``` <memory index="N" [date_time="..." session_id="..." dia_id="..."]> memory content text </memory> ``` Where: -`index`is the memory's position in the list. -`date_time`,`session_id`,`dia_id`are optional metadata attributes. - The text between the tags is the memor...

2023
[13]

A`<Topic Relations>`section containing one`<topic>`tag per topic relationship.,→ **Synthesis Guidelines:** - Do **not** answer the query directly. - Explain what information is available and how it should be used to formulate an answer.,→ **Output Format:** Your entire response must end with the following line: `<answer>your summary text here</answer>` Th...
[16]

A`<Temporal Relations>`section containing one`<temporal>`tag per temporal fact.,→
[17]

- Do **not** answer the query directly

A`<Topic Relations>`section containing one`<topic>`tag per topic relationship.,→ **Synthesis Guidelines:** - **Integrate** relevant entity, temporal, and topic facts into a coherent structure.,→ - **Extract** key information that directly supports or constrains the answer.,→ - **Reorganize** content for clarity and logical flow. - Do **not** answer the qu...
[18]

A`<query>`defining the subject and scope
[19]

An`<Entity Relations>`section containing one`<entity>`tag per relationship string.,→
[20]

An`<Temporal Relations>`section containing one`<temporal>`tag per temporal fact.,→
[21]

A`<Topic Relations>`section containing one`<topic>`tag per topic relationship.,→ **Synthesis Guidelines:** - Do **not** answer the query directly. - Explain what information is available and how it should be used to formulate an answer.,→ To complete the task systematically, please follow the steps reasoning framework outlined below:,→ **Reasoning Steps:*...

[1] [1]

Correctness: - Is the model answer factually consistent with ANY of the correct answers? - Does it avoid contradictions or introducing false information?

[2] [2]

Relevance: - Does the answer address the question directly without unnecessary content?

[3] [3]

explanation

Completeness: - Does the answer include all essential information needed to fully answer the question?,→ - Partial answers are allowed but should receive lower scores. Scoring Rules: - Score = 1.0 if the answer is fully correct. - Score = 0.5 if the answer is partially correct but incomplete or slightly inaccurate.,→ - Score = 0.0 if the answer is incorre...

[4] [5]

N" [date_time=

A`<memories>`section containing individual`<memory>`elements. Each memory is formatted as: ``` <memory index="N" [date_time="..." session_id="..." dia_id="..."]> memory content text </memory> ``` Where: -`index`is the memory's position in the list. -`date_time`,`session_id`,`dia_id`are optional metadata attributes. - The text between the tags is the memor...

[5] [7]

N" [date_time=

A`<memories>`section containing individual`<memory>`elements. Each memory is formatted as: ``` <memory index="N" [date_time="..." session_id="..." dia_id="..."]> memory content text </memory> ``` Where: -`index`is the memory's position in the list. -`date_time`,`session_id`,`dia_id`are optional metadata attributes. - The text between the tags is the memor...

[6] [8]

A`<query>`section containing the user's question

[7] [9]

N" [date_time=

A`<memories>`section containing individual`<memory>`elements. Each memory is formatted as: ``` <memory index="N" [date_time="..." session_id="..." dia_id="..."]> memory content text </memory> ``` Where: -`index`is the memory's position in the list. -`date_time`,`session_id`,`dia_id`are optional metadata attributes. - The text between the tags is the memor...

2023

[8] [13]

A`<Topic Relations>`section containing one`<topic>`tag per topic relationship.,→ **Synthesis Guidelines:** - Do **not** answer the query directly. - Explain what information is available and how it should be used to formulate an answer.,→ **Output Format:** Your entire response must end with the following line: `<answer>your summary text here</answer>` Th...

[9] [16]

A`<Temporal Relations>`section containing one`<temporal>`tag per temporal fact.,→

[10] [17]

- Do **not** answer the query directly

A`<Topic Relations>`section containing one`<topic>`tag per topic relationship.,→ **Synthesis Guidelines:** - **Integrate** relevant entity, temporal, and topic facts into a coherent structure.,→ - **Extract** key information that directly supports or constrains the answer.,→ - **Reorganize** content for clarity and logical flow. - Do **not** answer the qu...

[11] [18]

A`<query>`defining the subject and scope

[12] [19]

An`<Entity Relations>`section containing one`<entity>`tag per relationship string.,→

[13] [20]

An`<Temporal Relations>`section containing one`<temporal>`tag per temporal fact.,→

[14] [21]

A`<Topic Relations>`section containing one`<topic>`tag per topic relationship.,→ **Synthesis Guidelines:** - Do **not** answer the query directly. - Explain what information is available and how it should be used to formulate an answer.,→ To complete the task systematically, please follow the steps reasoning framework outlined below:,→ **Reasoning Steps:*...